Performance Test Plan

Owned by Tran Tang (Unlicensed)
Last updated: Jul 26, 2019 by Patrick Lobre (Unlicensed)

Version History:

Version#
Date
Modified By
Description of Modification
1.02/10/2016Sailaja AdusumilliInitial Version

Overview

The objective of this document is to define Performance Load Test Plan for CONNECT 4.6  gateway and provide results from its execution. Adherence to this test plan will ensure that a quality product that meets all requirements has been assembled as per the standard quality policy. This test plan will also identifies required resources such as equipment, disk space and tools to perform test activities. 

Performance Test Requirements

There are two goals for Performance Testing: one is to determine how many concurrent messages CONNECT 4.6 can handle in one minute and the other is to prove that CONNECT 4.6 can handle 1600 concurrent messages per minute.  As the goal is to only test for the CONNECT gateway processing performance and not the reference adapters, lightweight adapters were used for both initiating and responding configurations. 

Approach

Important note regarding UDDIConnectionInfo.xml and internalConnectionInfo.xml

As of CONNECT 5.1, both files have been replaced with the more versatile exchangeInfo.xml and internalExchangeInfo.xml files. For users of CONNECT 5.0 and earlier versions, substitute references to these new files with the original uddiConnectionInfo.xml and internalConnectionInfo.xml files

  • Testing will be restricted to ensure results reflect gateway capabilities and not tertiary components, such as hardware configuration or adapter performance.
  • The gateways will be setup in an ideal/optimized configuration to test upper limitations of CONNECT, similar to what was done in the 4.0 performance testing.
  • Tested services will include: PD, QD, RD, DS and X12.
  • Two rounds of testing will be performed to test each of the initiating (entity, message proxy) and receiving (NHIN) interfaces.
  • All initiating machines will be HCID 1.1 and all responding machines will be HCID 2.2. 
  • Metrics will be captured bidirectionally on each CONNECT supported application servers.
  • One exchangeInfo.xml is to be duplicated on all servers.

Hardware and Software Requirements for Testing

CONNECT 4.6 resources and environment setup details for Performance Test are as follows:

  • 1 Linux server with CONNECT on WebLogic 12.1.1 
  • 1 Solaris server with CONNECT on WebSphere 8.5.5.3 
  • 2 Windows servers with CONNECT on Wildfly 8.2.1
  • 1 Linux server with CONNECT on JBoss 7
  • 1 Linux server with CONNECT and as a loadUI control machine 
  • 1 Linux server with CONNECT on Wildfly 
  • minimum 8GB of RAM on each machine
  • minimum 50 GB of hard disk space
  • Processor details ( if required need to update )
  • MySql(5.1), Java SDK 1.6
  • self-signed certificates shared among all gateways
  • CONNECT 4.6 deployed on each application server that includes PerformanceTest jar.
OS/AS
IP
Application server
Paired Application Server
Solaris192.168.36.3WebSphere 8.5.5.3WildFly 8.2.1
Windows192.168.18.4WildFly 8.2.1
Linux192.168.35.4WebLogic 12.1.1WildFly 8.2.1
Linux192.168.35.63WildFly 8.2.1
Linux192.168.35.20JBoss 7WildFly 8.2.1
Windows192.168.34.2WildFly 8.2.1
Linux192.168.35.8WildFly 8.2.1 & LoadUIWildFly 8.2.1

Test Framework

The testing framework will be built using SoapUI and LoadUI. Metrics will be obtained through LoadUI for response times, while JConsole will be used for CPU and memory usage. The test framework will contain following test components:

  • SoapUI scripts will be used to send requests to CONNECT Gateway.
  • LoadUI projects will be used to perform the actual load test. This includes,  It will be used to capture and report response time. 
  • JConsole will be used to capture and report the CONNECT gateway JVM metrics, CPU utilization and Memory usage. nd to simulate concurrent messaging.
  • Custom adapters will be used to quickly respond to service requests to eliminate the processing time that a full adapter would require. The adapters are available at : Plugins/PerformanceTestTools
  • Test Scripts are available at : //shared-folder/Software/Performance 4.6/ConnectPerformanceTest. Follow Configuration instructions for test setup and execution.

Test Cases

#
Test Case
Description
LoadUI Adapter
1Patient Discovery
  1. Test will be performed with an overloaded initiator and an overloaded responder. The goal is to determine how many concurrent Patient Discovery messages CONNECT can handle in one minute and prove that Patient Discovery can handle 1600 concurrent messages per minute.
  2. Execute below scenarios four times bidirectionally (Ex: JBoss as initiator and JBoss as responder) on individual CONNECT supported application server and capture metrics and provide a summary including average throughput rate.
    1. Audit logging on using audit proxy secure Implementation.
    2. Audit logging on using audit proxy javaImplementation.
    3. Audit logging off with any audit proxy implementation.

CONNECTLoadTestAdapter - gov.hhs.fha.nhinc.patientdiscovery.loadtest.PatientDiscoveryBO

2Document Query
  1. Test will be performed with an overloaded initiator and an overloaded responder. The goal is to determine how many concurrent Document query messages CONNECT can handle in one minute and prove that Document Query can handle 1600 concurrent messages per minute.
  2. Execute below scenarios four times bidirectionally (Ex: JBoss as initiator and JBoss as responder) on individual CONNECT supported application server and capture metrics and provide a summary including average throughput rate.
    1. Audit logging on using audit proxy secure Implementation.
    2. Audit logging on using audit proxy javaImplementation.
    3. Audit logging off with any audit proxy implementation.

CONNECTLoadTestAdapter - gov.hhs.fha.nhinc.documentquery.loadtest.DocumentQueryBO

3Document Retrieve
  1. Test will be performed with an overloaded initiator and an overloaded responder. The goal is to determine how many concurrent Document Retrieve messages CONNECT can handle in one minute and prove that Document Retrieve can handle 1600 concurrent messages per minute.
  2. Execute below scenarios four times bidirectionally (Ex: JBoss as initiator and JBoss as responder) on individual CONNECT supported application server and capture metrics and provide a summary including average throughput rate.
    1. Audit logging on using audit proxy secure Implementation.
    2. Audit logging on using audit proxy javaImplementation.
    3. Audit logging off with any audit proxy implementation.

CONNECTLoadTestAdapter - gov.hhs.fha.nhinc.documentret

4Document Submission
  1. Test will be performed with an overloaded initiator and an overloaded responder. The goal is to determine how many concurrent Document Submission messages CONNECT can handle in one minute and prove that Document Submission can handle 1600 concurrent messages per minute.
  2. Execute below scenarios four times bidirectionally (Ex: JBoss as initiator and JBoss as responder) on individual CONNECT supported application server and capture metrics and provide a summary including average throughput rate.
    1. Audit logging on using audit proxy secure Implementation.
    2. Audit logging on using audit proxy javaImplementation.
    3. Audit logging off with any audit proxy implementation.

CONNECTLoadTestAdapter - gov.hhs.fha.nhinc.docsubmission.loadtest.DocumentSubmissionBO

5X12 RealTime
  1. Test will be performed with an overloaded initiator and an overloaded responder. The goal is to determine how many concurrent X12 RealTime messages CONNECT can handle in one minute and prove that X12 RealTime can handle 1600 concurrent messages per minute.
  2. Execute below scenarios four times bidirectionally (Ex: JBoss as initiator and JBoss as responder) on individual CONNECT supported application server and capture metrics and provide a summary including average throughput rate.
    1. Audit logging on using audit proxy secure Implementation.
    2. Audit logging on using audit proxy javaImplementation.
    3. Audit logging off with any audit proxy implementation.

CONNECTLoadTestAdapter - gov.hhs.fha.nhinc.x12.loadtest.CORE_X12DSRealTimeBO


6X12 BatchRequest
  1. Test will be performed with an overloaded initiator and an overloaded responder. The goal is to determine how many concurrent X12 BatchRequest messages CONNECT can handle in one minute and prove that X12 BatchRequest can handle 1600 concurrent messages per minute.
  2. Execute below scenarios four times bidirectionally (Ex: JBoss as initiator and JBoss as responder) on individual CONNECT supported application server and capture metrics and provide a summary including average throughput rate.
    1. Audit logging on using audit proxy secure Implementation.
    2. Audit logging on using audit proxy javaImplementation.
    3. Audit logging off with any audit proxy implementation

CONNECTLoadTestAdapter - gov.hhs.fha.nhinc.x12.batch.request.loadtest.CORE_X12DSBatchRequestBO

7X12 BatchResponse
  1. Test will be performed with an overloaded initiator and an overloaded responder. The goal is to determine how many concurrent X12 BatchResponse messages CONNECT can handle in one minute and prove that X12 BatchResponse can handle 1600 concurrent messages per minute.
  2. Execute below scenarios four times bidirectionally (Ex: JBoss as initiator and JBoss as responder) on individual CONNECT supported application server and capture metrics and provide a summary including average throughput rate.
    1. Audit logging on using audit proxy secure Implementation.
    2. Audit logging on using audit proxy javaImplementation.
    3. Audit logging off with any audit proxy implementation.

CONNECTLoadTestAdapter - gov.hhs.fha.nhinc.x12.batch.response.loadtest.CORE_X12DSBatchResponseBO