Versions Compared

Old Version 1

changes.mady.by.user Tran Tang (Unlicensed)

Saved on

compared with

New Version 2

changes.mady.by.user Tran Tang (Unlicensed)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Regression testing for CONNECT Gateway Release 45.7 3 consists of SoapUI tests found in the RegressionSuite directory, Direct testing, System Administration Module testing, X12 testing and Large Payload testing. The regression tests are designed to ensure that previously existing functionality still works in the current release.

In Release 45.73, Regression projects are updated with JMX scripting to support WildFly application server.  Event Logging Regression projects are re-written to validate some of JSON elements for each service: Event Logging - Standard and Event Logging - Passthrough. 

Some Direct QOS test cases were automated and the automated test cases are denoted in the /wiki/spaces/CONNECTWIKI/pages/24576016QOS test cases table. Automated Direct test cases are run as part of the CONNECT nightly CI but were also manually executed as part of 45.7 3 release testing. NOTE: Since Bluebutton trustbundle got expired, Direct tests are limited to Trust Anchor and Anchor test cases.

The SoapUI regression test suite, Direct testing, SAM and large payload (DS, DR) were successfully executed with CONNECT 4.7 RC1.  X12 LargePayload was successfully executed with CONNECT 4.7 RC25.3.

SoapUI Regression Test Suite

...

Info
titleJMX error

Occasionally when executing manual regression tests, we see "JBREM000200: Remote connection failed: java java.io.IOException: An established connection was aborted by the software in your host machine" in server.log first time when we manually executes jmx project.  This only happens when soapUI terminates the JMX connection client before the server sends a response. It is an environment issue and not code related and doesn't affect CONNECT functionalities. The only remedy is to wait for few mins after the server starts before executes any JMX projects.

...

234710121719343536DocQueryXDSErrors38414445464849535456586061666768
Regression Test Project NameNightly CITesting ModeCommentsResult
1AdapterPatientDiscoveryAsyncReqYesBimodal
PASS
AdapterPatientDiscoveryAsyncReqErrorYesBimodal
PASS
AdapterPatientDiscoveryAsyncRespYesBimodal
PASS
AdapterPolicyEngineSAMLConformanceTestsYesBimodal
PASS
5AdapterRegistryErrorListMissingTestYesBimodal
PASS6
ValidateDQDRMultispecSupportTestYesNOBimodalThis test was not found in CI Nightly output. Executed Manually in Standard, Passthrough modes and it is PASS.PASSmanually-testedPASS
ValidatePDCountryCodeTestYesBimodal
PASS
8ValidateDQAdapterMultipleEventCodesTestYesBimodal
PASS
9ConnectionManagerTestExchangeManagerTestsYesBimodal
PASS
DocRegistryAndRepositoryYesBimodal
PASS
11DocumentRetrieveXDSErrorValidationYesBimodal
PASS
ValidatePDCommunicationErrorHandlingTestYesBimodalPASS13QueryByStateYesBimodal
PASS
14XDRAdapterTestsYesBimodal
PASS
15ValidateSAMLResourceURIAttributeTestNoYesBimodal


PASS
16ValidateQDMessageSpecVersionTestNoYesBimodal
PASS
ValidateAuditQueryLogServiceNoYesBimodal
PASS
18ValidateRDMessageSpecVersionTestNoYesBimodal
PASS
DocQueryAdditionalStoredQueriesValidateWSA-ActionSoapMustUnderstandTestNoManualIn Manual verification script it says to verify <was:ReplyTo> in NHIN Inbound message (response msg) . But clarified with Deepthi, and if it is not present in response message that is fine since IHE only requires <Wsa:ReplyTo> to be present in the request. So in synchronous request message, deferred request(req) , deferred response(req) it should always have <Wsa:ReplyTo> and it is present.PASS
20ValidateWSAToFieldInSoapHeaderTestNoManual

In Nhin Outbound message <To> tag as below:
<To xmlns="http://www.w3.org/2005/08/addressing">https://localhost:8181/Gateway/DocumentSubmission/2_0/NhinService/XDRRequest_Service</To>

In Nhin Inbound message <To> tag as below:
<To xmlns="http://www.w3.org/2005/08/addressing">http://www.w3.org/2005/08/addressing/anonymous</To>

Verified and the above entries are as expected.

PASS
21PurposeForOfUse - Passthru - CommunityNoManualThe hl7 data in the requests do not match with what is expected in the Manual test steps. Passthrough_g1_community_True_2.2true has hl7 with code="PSYCHOTHERAPY" when "PUBLILC HEALTH" is expected. -→ I think it is issue with manual test step issue. FHAC-1169 should correct this.PASS
22PurposeForOfUse - Passthru - GatewayNoManualPASS23PurposeForOfUse - Standard - CommunityNoManualPASS24PurposeForOfUse - Standard - GatewayNoManualPASS25RelatesToListNoManualPASS26RequestResponseContentTypeTest - Admin DistributionNoManualPASS27RequestResponseContentTypeTest - Docucment RetrieveNoManualPASS
28RequestResponseContentTypeTest - Document SubmissionNoManualManul test step 2 under DS Defer Req is old and details needs to be updated in FHAC-1169. PASS
29RequestResponseContentTypeTest - Document QueryNoManualPASS30RequestResponseContentTypeTest - Patient DiscoveryNoManualPASS31TransactionTestNoManualPASS32TransactionTimeoutNoManualPASS33X12ErrorHandlingTestYesPassthroughPASSYesPassthrough
PASS
DocQueryRetrieveSelfTestYesPassthrough
PASS
DocQueryXDSErrorsYesPassthrough
PASS
EventLogging-PassthroughYesPassthrough
PASS
NHINCReceivingXDRAsyncRequestTestsYesPassthrough


PASS
NHINCReceivingXDRAsyncResponseTestsYesPassthrough


PASS
AuditLogging-PassthroughNoPassthroughAlso make sure to use dn=localhost as self-sign cert since some tests will use the value to make assertion. Disable JMX call on the AuditLogging when running DDS on WAS8.5PASS
DocQueryErrorCodesPassthruYesPassthrough
PASS
Admin-Distribution-PassthroughYesPassthrough
PASS
NHINSAMLSubjectConfirmationTestDocQueryAdditionalStoredQueriesYesPassthrough
PASS
X12ErrorHandlingTestDocQueryRetrieveSelfTestYesPassthrough
PASS37
AdapterSpringInjectionTestYesPassthrough
PASS
DocQueryErrorCodesPassthruEventLogging-PassthroughYesPassthroughPASS39NHINCReceivingXDRAsyncRequestTestsNoPassthroughPASS40NHINCReceivingXDRAsyncResponseTestsNoPassthroughPASSAuditLogging-PassthroughNoPassthrough
PASS42DocQueryErrorCodesPassthruNoPassthroughPASS
43Admin-Distribution-StandardYesNoStandardPASS test passed if audit logging doesn't run into timeout-issue; timeout issue is resolved by configuring webserviceproxy.retryattempts and webserviceproxy.retrydelay in gateway.properties  filesPASS
AuditLogging-StandardYesStandardPASS 
PASS
DocQueryErrorCodesStandardYesStandard
PASS
DocQueryFanoutTestTargetYesStandard
PASS
47DocQueryOnDemandYesStandard
PASS
DocQuerytestsforUDDIYesStandard
PASS
EntityDocQueryYesStandard
PASS
50EntityDocQuerySecuredInterfaceTestYesStandard
PASS
51EntityDocRetrieveYesStandard
PASS
52EntityDocRetrieveSecuredInterfaceTestYesStandard
PASS
EntityPolicyEngineTestYesStandardPASS 
PASS
EntityXDRTestsYesStandard
PASS PASS
55EventLogging-StandardYesStandardPASS 
PASS
ExchangeManagerYesStandard
PASS
KeepAliveHeaderYesStandard
PASS
NHINCPatientDiscoveryMAAYesStandard
PASS
57PatientCorrelationYesStandardPASS 
PASS
PatientDiscoveryDBYesStandard
PASS PASS
59PatientDiscoveryInternalErrorMessageIdTestYesStandard
PASS PASS
PDSAMLIssuerWithX509SubjectNameSAMLIssuerWithX509SubjectNameYesStandardThe Project name appears in the Nightly build as "PDSAMLIssuerWithX509SubjectNameTest"PASS 
PASS
ValidateConnectionManagerForAllServicesYesStandard
PASS PASS
62XDRAsyncRequestInbound-TestsYesStandard
PASS
63XDRAsyncRequestOutbound-TestsYesStandard
PASS
64TransactionIDTestNoYesStandard
PASS
65EntitySendingXDRAsyncReqTestsNoYesStandard
PASS
EntitySendingXDRAsyncResponseTestsNoYesStandardPASS 
PASS
Fanout-TestNoYesStandard
PASS
ValidationSuiteValidation suiteYesNoStandardExecute Validation suite with internet off and on. This test the functionality of tickets fixed CONN-1625 and CONN-1632PASS 

System Administration Module (SAM) Testing

...

StandardAlso executed test manually. Tester need to switch between standard and passthrough manually.PASS

Admin GUI Testing

The Admin GUI has been /wiki/spaces/CONNECTWIKI/pages/24576018

SAM test results

on the four supported application servers for the CONNECT 5.3 release

Test NameTest Cases Covered
Comments
Result
Login, GatewayStatus, Cross-Gateway Query Client, Accountmanagement and ConnectionManagementExecute all the test cases from wiki : https://connectopensource.atlassian.net/wiki/x/UAB3AQ
RC1
PASS
Direct
Properties ConfigurationExecute all the test cases from Wiki:
https
 Gateway Properties, Adapter Properties, Audit Properties and Internal Endpoint PropertiesPASS
Audit ViewerExecute all the test cases from Wiki: /wiki/
connectopensource.atlassian.net/wiki/x/TgB3AQRC1

PASS

Properties Configuration
spaces/CONNECTWIKI/pages/58490927PASS
Certificate Manager Testing Execute all the test cases from Wiki
: https
: /wiki/
connectopensource.atlassian.net/wiki/x/LoAtAgRC1PASSAudit Viewer
spaces/CONNECTWIKI/pages/675446792PASS
Failure Logging TestingExecute all the test cases from Wiki: /wiki/spaces/CONNECTWIKI/pages/
58490927
671711305
RC1
PASS

...

Test Data Loader Testing

...

Execute all the test cases from Wiki: /wiki/spaces/CONNECTWIKI/pages/

...

672235568PASS

Direct Testing

As described on the Direct testing page, some Direct tests have been automated. The following is an aggregate listing of all Direct test results.

 Basic test results

...

Unable to see following events:

...

END_INBOUND_MDN_DISPATCHED

Highlight
Used .net and .org pollers to run this test and I was able to see both Begin/End inbound MDN Dispatched event.

...

Unable to see following events:

...

END_INBOUND_MDN_DISPATCHED

Highlight
Used .net and .org pollers to run this test and I was able to see both Begin/End inbound MDN Dispatched event.

...

  • BEGIN_INBOUND_MDN_DISPATCHED
  • END_INBOUND_MDN_DISPATCHED

QOS test results

...

1

...

no longer used and work as is from CONNECT 4.7.

Large Payload and X12 Testing

X12 has been regression tested for the CONNECT 45.7 3 release by executing the the /wiki/spaces/CONNECTWIKI/pages/24576043.  Tested Large  Large Payload has been tested by following the configuration steps at:https:/wiki/connectopensource.atlassian.net/wiki/x/KwB3AQspaces/CONNECTWIKI/pages/24576043.

Large Payload Test Results

Test NameTesting ModeCommentsResult
Document Submission (Deferred)PassthroughTest using 1GB payloadPASS
Document RetrievePassthroughTest using 1GB payloadPASS


X12 Test Results

https://cgiinterop.atlassian.net/browse/FHAC-1162  – Fixed and tested 4.7.0-RC2https://cgiinterop.atlassian.net/browse/FHAC-1162 – Fixed and tested on 4.7.0-RC2
Test NameTesting ModeCommentsResultTickets
CAQH Core X12 Batch RequestPassthroughTest using 20MB, 200MB and 1GB payload PASSPASS
CAQH Core X12 Batch ResponsePassthroughTest using 20MB, 200MB and 1GB payloadPASS PASS

Performance Test Results

WildFly - 8.2.1 To  WildFly-8.2.1

Below metrics captured by configuring AuditRepositoryProxyConfig.xml to auditrepositorynoop, auditrepositorywssecured and auditrepositoryjava with audit on and off modes

#ServiceAudit On (Per Min)Audit Off (Per Min)
Noop ImplSecured ImplJava ImplNoop ImplSecured ImplJava Impl
1PD13006002300290016003000
2DQ9006002600180013003100
3DR11006202700270017002900