Overview

Document Submission, Retrieve Documents, and CAQH Core X12 Document Submission services all require the ability to transfer large payloads and have support for the SOAP Message Transmission Optimization Mechanism (MTOM) specification. This page provides information on how to set up and test large payload transfers between two independent gateways for the following services:

1. Document Submission
2. Document Retrieve
3. CAQH Core X12 Generic Batch Request and Response

All services have been tested with payloads of 1GB.

Prerequisites

1. Two CONNECT gateways, configured to communicate with each other, on separate hosts - see Gateway Setup below.
   
   1. Both servers must have the CONNECT EAR deployed with all services that will be tested; for best results, use the latest CONNECT release binary.
   2. The system time on the initiating and responding gateway machines should be the same, or as close to as possible.
   3. For this document, the Initiating Gateway will use Home Community ID (HCID) 1.1 and the Responding Gateway will use HCID 2.2.
   4. For both Initiating and Responding Gateways, import the Remote Gateway's public certificate into the the Local Gateway's truststore:
      1. For example, keytool -import -v -trustcacerts -alias remote -file remote.cer -keystore C:/glassfish3/glassfish/domains/domain1/config/cacerts.jks
          
2. SoapUI 4.5.1 or higher, with ValidationSuite loaded.

Gateway Setup

These gateway setup steps are common to all test cases. All property file changes should happen in the gateway properties directory:

Common Setup (both Initiating and Responding Gateway)

1. Temp Files / Folders
   1. Create a directory for temporary files (e.g., /tmp/largepayload), then create three sub-directories: cache, inbound and outbound.
   2. Different test cases call for different test file sizes.  Here are some examples of how to generate these files:
      
      1. In Solaris: /usr/sbin/mkfile 20m /tmp/largepayload/outbound/test.txt
      2. In Linux: fallocate -l 1g /tmp/largepayload/outbound/test.txt
      3. In Windows (with elevated privileges): fsutil file createnew test.txt 1000000000
   3. Some test cases will require you to set the Payload property value in the SoapUI test case to the base64-encoded path of the large payload file in the initiating gateway's system (e.g., file:///c:/largepayload/outbound/test.txt would become ZmlsZTovLy9jOi9sYXJnZXBheWxvYWQvb3V0Ym91bmQvdGVzdC50eHQ=).  Encode this path in advance to make testing easier.
      1. https://www.base64encode.org/ is one of many free Base64 tools available.
2. JVM Properties:
   1. Add or uncomment: -Dorg.apache.cxf.io.CachedOutputStream.OutputDirectory=/tmp/largepayload/cache
   2. Remove or comment out: -Dcom.sun.xml.ws.transport.http.HttpAdapter.dump=true
   3. Remove or comment out: -Dcom.sun.xml.ws.transport.http.client.HttpTransportPipe.dump=true
   4. Remove or comment out: -Djavax.enterprise.resource.xml.webservices.security.level=FINE
   5. Remove or comment out: -Djavax.enterprise.resource.webservices.jaxws=FINE
3. gateway.properties changes:
   
   1. Set webserviceproxy.timeout to at least 480000 milliseconds (8 minutes).
   2. Set ParsePayloadAsFileURIOutbound to true - this allows the outbound message payload to be converted to a file URI and streamed as an object.
   3. Set SavePayloadToFileInbound to true - this allows the inbound streamed payload to be saved as a file.
   4. Set PayloadSaveDirectory to the inbound directory from step 1 (e.g., C:/largepayload/inbound) - this specifies where to save the inbound streamed payload file.
   5. Set all services to run in Passthrough mode.
   6. If there are timeout issues while running any of the tests, adjust the following two properties for DocumentSubmission and AdminDistribution:
      1. Set TimeStampTimeToLive to at least 600 seconds (10 minutes)
      2. Set FutureTimeToLive to at least 600 seconds (10 minutes) 
         
         OR for CORE_X12, adjust the following properties:
         
         
      3. CoreX12GenericBatchTimeStampStrict=true
      4. Set CoreX12GenericBatchTimeStampTimeToLive to at least 600 seconds.
      5. Set CoreX12GenericBatchFutureTimeToLive to at least 600 seconds.

Initiating Gateway Setup

1. exchangeInfo.xml changes:
   
   1. Add an entry for HCID 2.2. updating endpoints to use the hostname or IP address of the responding gateway.

Responding Gateway Setup

1. gateway.properties changes:
   1. Update the HCID to 2.2
2. adapter.properties changes:
   
   1. Update the HCID to 2.2
3. internalExchangeInfo.xml changes:
   1. Add or update the entry for the HCID 2.2
4. exchangeInfo.xml changes:
   1. Update the endpoints for HCID 1.1 to use the hostname or IP address of the initiating gateway.

NOTE: While executing the SoapUI test cases, clear the "GatewayPropDir" property.  If this is not possible, make the internal and uddi endpoint changes in ValidationSuite/[uddi|internal]ConnectionInfo_[g0|g1].xml.

CONNECT 4.6 Large Payload Testing

• Large payload test has been done on Glassfish app server and Wildfly app server using CONNECT 4.6 RC1 tag using 18.4 box as initiator and 18.6 as responder 

Wildfly Configuration:

While running Large Payload test on wildfly app server we need below configuration in standalone.xml (under Appserver/.../standalone/configuration). These below changes allow wildfly to handle large bandwidth/data to transfer across the other gateway.

• • <https-listener name="https" socket-binding="connect" security-realm="ApplicationRealm" verify-client="REQUIRED" max-post-size="1124000000"/>

  • <buffer-cache name="default" buffer-size="2048" buffers-per-region="2048" max-regions="10"/>

Final Steps

1. If either server has a virus scanner, disable it until all tests are complete.
2. Make sure to leave the "GatewayPropDir" value blank in SoapUI to use the internalExchangeInfo.xml and uddiConnectInfo.xml from the

Document Submission (Deferred)

CONNECT 4.x supports large file streaming via the the Document Submission Deferred services. File location for streaming as well as SAML timestamp and timeout settings are configurable, as mentioned in the Common Setup instructions. 

Execution

1. In SoapUI, load the ConnectValidation project and open the g1 test suite.
2. Under the Document Submission Deferred Req test case, open the Doc Submission Deferred Req SOAP message.
3. Replace the <urn5:Document id="Document01"> property under <urn:ProvideAndRegisterDocumentSetRequest> in the SOAP request message with the Base64-encoded path to your large payload test file (file:///C:/delete_my/largepayload/outbound/test.txt), e.g.:
   
   1. <urn5:Document id="Document01">ZmlsZTovLy9DOi9kZWxldGVfbXkvbGFyZ2VwYXlsb2FkL291dGJvdW5kL3Rlc3QudHh0</urn5:Document>
4. Execute the Document Submission Deferred Req test case. 
5. Once the SoapUI test has passed, verify that the payload message has been successfully transferred to the responding gateway's inbound folder.

Document Retrieve

CONNECT 4.x also supports the retrieval of large payloads.  The setup is mostly the same as in the Document Submission (Deferred) test case, but additional configuration is required to point the document repository adapter to the file system for streaming.

Preparation

Common Setup (both Initiating and Responding Gateway)

1. DocumentRetrieveProxyConfig.xml changes:
   1. Where alias="adapterdocumentrepository", set name="adapterdocumentrepositorybean" - this will change the default doc repository adapter to the custom adapter designed for large payload testing.
   2. Where bean id="adapterdocumentrepositorybean" and property name="document", set value to the unencoded full path of the large payload test file to be returned in the response (e.g., file:///c:/largepayload/outbound/test.txt).

Execution

1. In SoapUI, load the ConnectValidation project and open the g1 test suite.
2. Execute the Document Retrieve test case.
3. Once the SoapUI test has passed, verify that the payload message has been successfully transferred to the initiating gateway's inbound folder

CAQH Core X12 Batch Request and Batch Response

In Release 4.4, the CONNECT team added a new service for NwHIN CAQH Core X12 Document Submission service. As part of this implementation we will have 3 services - RealTime request, Generic Batch request, and Generic Batch response. The Generic Batch request and Generic Batch response services are expected to support large payload transfer functionality. The large payload feature includes file streaming and added timestamp functionality to account for longer file transfer durations. This testing effort involves transferring large files of various sizes from one Gateway(Separate Server) to another Gateway (Separate Server) and making sure the large file is also included in the response.

Test Scenarios

As part of CAQH Core X12 Generic Batch Request/Response services the users should be able to send large payload utilizing MTOM. The X12 batch Request/Response transaction services can have a payload as part of the SOAP request as well as the SOAP response. Note - this testing is done only for Generic Batch request and response and not for Real Time, since MTOM is not applicable for Real Time.

NOTE - As part of this testing, the following three test cases are executed for X12 Document Submission Deferred Request transaction (CAQH Core X12 Generic Batch Submission and Response) and then should be repeated for X12 Document Submission Deferred Response transaction (CAQH Core X12 Generic Batch Submission and Response) - 

Preparation

Common Setup (both Initiating and Responding Gateway)

1. CORE_X12DSGenericBatchProxyConfig.xml changes:
   1. Where alias="adaptercore_x12dsgenericbatchrequest", set name="adaptercore_x12dsgenericbatchrequestproxybean" - this will change the default X12 Batch Request adapter to custom adapter to test large payload.
   2. Where alias="adaptercore_x12dsgenericbatchresponse", set name="adaptercore_x12dsgenericbatchresponseproxybean" - this will change the default X12 Batch Response adapter to custom adapter to test large payload.
   3. Where bean id="adaptercore_x12dsgenericbatchrequestproxybean" and property name="payload", set value to the full path of the unencoded full path of the large payload test file to be returned in the response (e.g., file:///C:/largepayload/outbound/test.txt).
   4. Where bean id="adaptercore_x12dsgenericbatchresponseproxybean" and property name="payload", set value to the full path of the unencoded full path of the large payload test file to be returned in the response (e.g., file:///C:/largepayload/outbound/test.txt).

Execution

1. In SoapUI, load the ConnectValidation project and open the g0 test suite.
2. Enable the BatchSubmit RequestTransactionTestCase test case.
3. Under the BatchSubmit Request TransactionTestCase test case, open the BatchSubmitTransaction SOAP message.
4. Replace the <Payload> property under <cor:COREEnvelopeBatchSubmission> in the SOAP request message with the Base64-encoded path to your large payload test file.
5. Execute the test case.
6. Repeat Steps 1-5 for BatchSubmit Response TransactionTestCase test case.

Note:

If you see Socket Timeout on SoapUI, try to adjust Timeout property (under BatchSubmit Request TransactionTestCase → TestCase → BatchSubmitTransaction) to a higher value. Recommended value for 1 GB is 9999999.

Related articles