Multi-exchange certificate, TLS and SAML support

Overview

Initial support for multiple exchanges has been determined and implemented. CONNECT is currently capable of supporting multiple exchanges from a directory perspective with its ability to download organizational data from any FHIR STU 3 compliant healthcare provider directory. Multi-exchange certificate, TLS and SAML support extends the ability to support multiple exchanges with on CONNECT instance by providing the following:

As an initiating gateway, select an appropriate certificate (based on certificate alias) to be used at the TLS layer and the corresponding private key to sign XML messages
As an initiating gateway, include a Server Name Indication (SNI) value to describe the intended exchange (assuming all participants of a single exchange are required to use the same domain value)
As a responding gateway, select the following:
- A certificate to present, based on SNI, during the server hello
- A public key, based on SNI, to use for signature validation

Requirement for SNI approach	Details
Wildfly 15 or higher	WildFly 15 supports server side SNI on its HTTPS listeners. This allows a WildFly instance listening on a single socket but with multiple virtual hosts associated with that listener to provide a different server certificate depending on what SNI name the client requests. For Wildfly 15 SNI configuration, refer to Multi-exchange certificate, TLS and SAML support .
WebSphere 8.5 or higher	Websphere 8 seems supporting SNI with its own IBM SDK. For more information, please see https://www.ibm.com/support/knowledgecenter/SSYKE2_7.0.0/com.ibm.java.security.component.70.doc/security-component/whats_new/security_changes_70/security_whatsnew.html
Other Application servers	Not all application servers have built-in SNI support. In order to reap benefits from multiple exchange feature of CONNECT, adopters can employ Apache server. See section Multi-exchange certificate, TLS and SAML support for details.

Multi-exchange support with SNI

An assumption is made that CONNECT implementers participating in multiple exchanges will obtain separate SSL certificates for each exchange, thereby requiring the management of multiple private keys. This requires CONNECT to understand which private key to associate with which exchange. The certificate alias field in exchangeInfo.xml is used to load appropriate certificate for signing the SAML assertion. If there is no certificateAlias is defined in exchangeInfo.xml, CONNECT will pick the certificateAlias from system property CLIENT_KEY_ALIAS defined in server configuration.

exchangeInfo.xml

        <exchange type="uddi">
            <name>eHealthExchange</name>
            <certificateAlias>gateway</certificateAlias>
            <sniName>ehealth</sniName>
            <url>http://YourExchangeUrl/</url>
            <disabled>false</disabled>
        </exchange>

TLS Handshake

An assumption is made that each exchange will incorporate the use of SNI and all participants of a single exchange are required to use the same domain value. It is also assumed that this domain value will be included in each exchange's directory.

CONNECT, will read the SNI from exchangeInfo.xml file and initiates an SSL connection with the responding gateway. Based on SNI from initiating gateway, responding gateway will pick up the appropriate certificate to present for SSL handshake. After the SSL handshake is successful, its business as usual. If there are no SNI specified in exchangeInfo, CONNECT, as initiator, will not send the SNI during SSL handshake. At the responding gateway server, a default SNI needs to be defined in this case.

CONNECT as Initiator

1.Client initiates TLS with the SNI name defined in the exchangeInfo.xml

exchangeInfo.xml

        <exchange type="uddi">
            <name>eHealthExchange</name>
            <certificateAlias>gateway</certificateAlias>
            <sniName>ehealth</sniName>
            <url>http://YourExchangeUrl/</url>
            <disabled>false</disabled>
        </exchange>

2.Client authenticates certificate

3.Based on <exchangeName> defined in entity request, CONNECT finds the <certificateAlias> to use to select the private key for signing the SAML assertion and XML header. CONNECT then sends a request with correct DN value in SAML assertion.

CONNECT as Responder

1.Server knows which certificate to present based on SNI value sent during client hello.

2.Server presents appropriate certificate.

3. Server validate client cert and if trust, TLS is completed and ready to accept soap message.

4. Server receives request and validate Soap Message to make sure it can be trusted. At this point, it is expected for implementation to extract assertion block in NHIN message to route to an appropriate adapter and return responding message back to the initiating gateway.

SNI configuration in Wildlfy 15

For demonstration purposes, we are using 2 exchange ehealth and carequality. Below is a sample SNI configuration on wildfly 15.0.0

<subsystem xmlns="urn:wildfly:elytron:5.0" final-providers="combined-providers" disallowed-providers="OracleUcrypto">
...
<tls>
                <key-stores>
                    <key-store name="ehealth">
                        <credential-reference clear-text="changeit"/>
                        <implementation type="JKS"/>
                        <file path="/modules/system/layers/base/org/connectopensource/configuration/main/gateway.jks" relative-to="jboss.home.dir"/>
                    </key-store>
                    <key-store name="carequality">
                        <credential-reference clear-text="changeit"/>
                        <implementation type="JKS"/>
                        <file path="/modules/system/layers/base/org/connectopensource/configuration/main/gateway.jks" relative-to="jboss.home.dir"/>
                    </key-store>
                    <key-store name="carequalityTS">
                        <credential-reference clear-text="changeit"/>
                        <implementation type="JKS"/>
                        <file path="/modules/system/layers/base/org/connectopensource/configuration/main/cacerts.jks" relative-to="jboss.home.dir"/>
                    </key-store>
                    <key-store name="ehealthTS">
                        <credential-reference clear-text="changeit"/>
                        <implementation type="JKS"/>
                        <file path="/modules/system/layers/base/org/connectopensource/configuration/main/cacerts.jks" relative-to="jboss.home.dir"/>
                    </key-store>
                </key-stores>
                <key-managers>
                    <key-manager name="ehealthKM" key-store="ehealth" alias-filter="gateway">
                        <credential-reference clear-text="changeit"/>
                    </key-manager>
                    <key-manager name="carequalityKM" key-store="carequality" alias-filter="carequality">
                        <credential-reference clear-text="changeit"/>
                    </key-manager>
                </key-managers>
                <trust-managers>
                    <trust-manager name="ehealthTM" key-store="ehealthTS"/>
                    <trust-manager name="carequalityTM" key-store="carequalityTS"/>
                </trust-managers>
                <server-ssl-contexts>
                    <server-ssl-context name="ehealthSSC" need-client-auth="true" key-manager="ehealthKM" trust-manager="ehealthTM"/>
                    <server-ssl-context name="carequalitySSC" need-client-auth="true" key-manager="carequalityKM" trust-manager="carequalityTM"/>
                </server-ssl-contexts>
                <server-ssl-sni-contexts>
                    <server-ssl-sni-context name="connectSNI" default-ssl-context="ehealthSSC">
                        <sni-mapping host="connect.carequality.com" ssl-context="carequalitySSC"/>
                        <sni-mapping host="connect.ehealth.com" ssl-context="ehealthSSC"/>
                    </server-ssl-sni-context>
                </server-ssl-sni-contexts>
            </tls>
</subsystem >
....
<subsystem xmlns="urn:jboss:domain:undertow:8.0" default-server="default-server" default-virtual-host="default-host" default-servlet-container="default" default-security-domain="other">
....
 <https-listener name="https" socket-binding="connect" ssl-context="connectSNI" enable-http2="true"/>
</subsystem>

Servers that don't support SNI

Adopters, when limited by their application server platform, can use Apache (proxy) server and multiple ports (binding each exchange to a unique port) to implement multiple-exchange feature. Initiating gateway will send the SNI during SSL handshake. The SSL handshake will happen between Initiating gateway and Apache (proxy) server. Once the handshake is successful, Apache (proxy) server will direct the request to appropriate port based on the SNI sent in SSL handshake.

Setup Details for Wildfly-8.2.1

Configuration tested using three instances; Initiating gateway (Wildfly-15.0.0), Apache proxy instance, and Responding gateway (Wildfly-8.2.1)

Install Apache Proxy server

How to install Apache Proxy

[linux:apache -- directory]
#Config location
/etc/httpd/conf.d
#Server logs
/var/log/httpd

[linux-permission issue with httpd]
$ sudo setsebool -P httpd_can_network_connect 1

[apache--installation]
#installation page
#note: url--https://www.liquidweb.com/kb/how-to-install-apache-on-centos-7/
$ sudo yum clean all
$ sudo yum -y update
$ sudo yum -y install httpd
$ sudo yum install mod_ssl

#firewall: no firewall-cmd
#$ sudo firewall-cmd --permanent --add-port=80/tcp
#$ sudo firewall-cmd --permanent --add-port=443/tcp
#$ sudo firewall-cmd --reload

#start-apache
$ sudo systemctl start httpd
$ sudo systemctl enable httpd
$ sudo systemctl status httpd
>>> Active: active (running)

#Verify the test page comes up by visiting the apache server's IP address. You should see a "Testing 123" landing page.
#$ sudo systemctl stop httpd

$ sudo systemctl restart httpd.service

#verify-proxy-config -- https://www.digitalocean.com/community/tutorials/how-to-use-apache-as-a-reverse-proxy-with-mod_proxy-on-centos-7
$ httpd -M

#You should see a number of modules listed. Ensure you see the following:
# proxy_module (shared)
# lbmethod_byrequests_module (shared)
# proxy_balancer_module (shared)
# proxy_http_module (shared)
# ssl_module (shared)


#you will need to have permission for the following directories
$ sudo chmod -R 777 /etc/httpd/conf/
$ sudo chmod -R 777 /var/log/httpd
$ sudo chmod -R 777 /etc/httpd/conf.d/
$ sudo chmod -R 777 /etc/pki/tls/

#self-signed certificate for apache
#https://www.akadia.com/services/ssh_test_certificate.html

#certificates: localhost_i1 and localhost_i2

$ sudo chmod -R 777 /etc/pki/tls/

#apache-certificate configuration
$ keytool -importkeystore -srckeystore gateway.jks -destkeystore localhost_i1.p12 -deststoretype PKCS12
$ keytool -importkeystore -srckeystore gateway.jks -destkeystore localhost_i2.p12 -deststoretype PKCS12

#remove-i2 from localhost_i1.p12 and remove-i1 from localhost_i2.p12

#i1-certficate/key
$ openssl pkcs12 -in localhost_i1.p12 -nokeys -clcerts -out i1_localhost_cert.pem
$ openssl pkcs12 -in localhost_i1.p12 -nocerts -out i1_localhost_keypass.pem 
$ openssl rsa -in i1_localhost_keypass.pem -out i1_localhost_key.pem
$ cat i1_localhost_key.pem i1_localhost_cert.pem > i1_localhost_pair.pem


#i2-certficate/key
$ openssl pkcs12 -in localhost_i2.p12 -nokeys -clcerts -out i2_localhost_cert.pem
$ openssl pkcs12 -in localhost_i2.p12 -nocerts -out i2_localhost_keypass.pem 
$ openssl rsa -in i2_localhost_keypass.pem -out i2_localhost_key.pem
$ cat i2_localhost_key.pem i2_localhost_cert.pem > i2_localhost_pair.pem


#apache:ssl-engine
$ sudo systemctl restart httpd.service

Apache (proxy) Server Configuration to Wildfly (application) server

When the the apache server receives traffic on port 443 with SNI of "ehex.org", it will present ehealth cert to initiator gateway...Once TLS handshake is successful, it will route message to port 8181(https) or 9191(https) depending on Apache's proxy configuration.

Configure SNI in Apache Proxy

Make the following configuration changes in $<Apache Install Dir>/httpd/conf.d/default-site.conf:

Add two SNI domain name entries pointing to 8181, 9191 ports individually:

default-site.conf

#apache httpd.conf/default-site.conf

<VirtualHost *:443>
    #The domain name for the virtual server you wish to connect to, "ehex.org" in this case
    ServerName ehex.org

    SSLEngine on
    SSLCertificateFile /etc/pki/tls/[eHex-Cert.pem]
    SSLCertificateKeyFile /etc/pki/tls/[eHex-privateKey.pem]
    SSLProxyMachineCertificateFile /etc/pki/tls/[eHex-keyPair.pem]
    ProxyPreserveHost On
    SSLProxyEngine On

    #for development purpose/assume your self-certificate are valid
    SSLProxyVerify none 
    SSLProxyCheckPeerCN off
    SSLProxyCheckPeerName off
    SSLProxyCheckPeerExpire off

    #LogLevel debug

    ProxyPass / https://[Responding gateway ip]:8181/
    ProxyPassReverse / https://[Responding gateway ip]:8181/
</VirtualHost>

<VirtualHost *:443>
    #The domain name for the virtual server you wish to connect to, "carequality.org" in this case
    ServerName carequality.org

    SSLEngine on

    SSLCertificateFile /etc/pki/tls/[careQuality-Cert.pem]
    SSLCertificateKeyFile /etc/pki/tls/[careQuality-privateKey.pem]
    SSLProxyMachineCertificateFile /etc/pki/tls/[careQuality-keyPair.pem]

    ProxyPreserveHost On
    SSLProxyEngine On

    #for development purpose/assume your self-certificate are valid
    SSLProxyVerify none 
    SSLProxyCheckPeerCN off
    SSLProxyCheckPeerName off
    SSLProxyCheckPeerExpire off

    #LogLevel debug

    ProxyPass / https://[Responding gateway ip]:9191/
    ProxyPassReverse / https://[Responding gateway ip]:9191/
</VirtualHost>

Connect as Initiating Gateway

In order for us to test the "ehex.org" and "carequality.org" domains, we want to add a mapping in the /etc/hosts file to point to our Apache Server. Failure to do so may result in a virtual host confusion attack, and Apache may block the connection from being established if the IP address is used in place of a domain name combined with an SNI name.

<apache server ip>   ehex.orgz carequality.orgz

Connect as Responding Gateway

CONNECT requires the use of multiple SSL ports to support secure messaging with multiple exchanges (certificates).

Port 8181 for inbound/outbound SSL requests using certificate 1
Port 9191 for inbound/outbound SSL requests using certificate 2

Make the following configuration changes to $WildFly_HOME/standalone/configuration/standalone.xml for WildFly-8.2.1:

Standalone.xml

	</security-realms>        
		...........
    	<security-realm name="ApplicationRealm">
                <server-identities>
                    <ssl>
                        <keystore path="modules/system/layers/base/org/connectopensource/configuration/main/gateway.jks" relative-to="jboss.home.dir" keystore-password="changeit" alias="gateway"/>
                    </ssl>
                </server-identities>
                <authentication>
                    <truststore path="modules/system/layers/base/org/connectopensource/configuration/main/cacerts.jks" relative-to="jboss.home.dir" keystore-password="changeit"/>
                    <local default-user="$local" allowed-users="*" skip-group-loading="true"/>
                    <properties path="application-users.properties" relative-to="jboss.server.config.dir"/>
                </authentication>
                <authorization>
                    <properties path="application-roles.properties" relative-to="jboss.server.config.dir"/>
                </authorization>
            </security-realm>
			<security-realm name="ApplicationRealm2">
				<server-identities>
					<ssl>
						<keystore path="modules/system/layers/base/org/connectopensource/configuration/main/gateway.jks" relative-to="jboss.home.dir" keystore-password="changeit" alias="gateway_b"/>
					</ssl>
				</server-identities>
				<authentication>
					<truststore path="modules/system/layers/base/org/connectopensource/configuration/main/cacerts.jks" relative-to="jboss.home.dir" keystore-password="changeit"/>
					<local default-user="$local" allowed-users="*" skip-group-loading="true"/>
					<properties path="application-users.properties" relative-to="jboss.server.config.dir"/>
				</authentication>
				<authorization>
					<properties path="application-roles.properties" relative-to="jboss.server.config.dir"/>
				</authorization>
		</security-realm>
        .........
	</security-realms>
	.............
    <subsystem xmlns="urn:jboss:domain:undertow:1.2">
            <buffer-cache name="default"/>
            <server name="default-server">
                <http-listener name="default" socket-binding="http"/>
                <https-listener name="https" socket-binding="connect" security-realm="ApplicationRealm" verify-client="REQUIRED"/>
				<https-listener name="https2" socket-binding="connect2" security-realm="ApplicationRealm2" verify-client="REQUIRED"/>
			.........
			</server>
	............
	</subsystem>
	.............
   	<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
      	............
  		<socket-binding name="connect" port="8181"/>
		<socket-binding name="connect2" port="9191"/>
		............
   	</socket-binding-group>

Test Via Connect - Validation Suite

Update endpoint URL for Organization 2 from "localhost:8181" to "carequality.org" in exchangeInfo-g0.xml
Update the SNI Name and Certificate Alias for the exchange to point to the correct certificate alias and "carequality.org" from your hosts file as the SNI name
Update endpoint URL for Organization 2 from "localhost:8181" to "ehex.org" in exchangeInfo-g1.xml
Update the SNI Name and Certificate Alias for the exchange to point to the correct certificate alias and "ehex.org" from your hosts file as the SNI name

G0 Suite should use the carequality certificate and G1 should use the exhex.org certificate.

exchangeInfo_g1.xml

<exchangeInfo xmlns="urn:gov:hhs:fha:nhinc:exchange" xmlns:ns2="urn:gov:hhs:fha:nhinc:exchange:directory">
  ..........
  <defaultExchange>Exchange 1</defaultExchange>
  <exchanges>
    <exchange type="uddi">
      <name>Exchange 1</name>
	  <certificateAlias>gateway</certificateAlias>
	  <sniName>ehex.org</sniName>
 	  ...........
     <organizationList>
        <organization>
          <ns2:name>Gateway 1</ns2:name>
          <ns2:hcid>urn:oid:1.1</ns2:hcid>
          <ns2:endpointList>
            <ns2:endpoint>
              <ns2:name>QueryForDocuments</ns2:name>
              <ns2:endpointConfigurationList>
                <ns2:endpointConfiguration>
                  <ns2:url>https://localhost:8181/Gateway/DocumentQuery/2_0/NhinService/RespondingGateway_Query_Service/DocQuery</ns2:url>
                  <ns2:version>2.0</ns2:version>
                </ns2:endpointConfiguration>
              </ns2:endpointConfigurationList>
            </ns2:endpoint>
		    ............
        </ns2:endpointList>
		.............
        </organization>
        <organization>
          <ns2:name>Gateway 2</ns2:name>
          <ns2:hcid>urn:oid:2.2</ns2:hcid>
          <ns2:endpointList>
            <ns2:endpoint>
              <ns2:name>QueryForDocuments</ns2:name>
              <ns2:endpointConfigurationList>
                <ns2:endpointConfiguration>
                  <ns2:url>https://ehex.org/Gateway/DocumentQuery/2_0/NhinService/RespondingGateway_Query_Service/DocQuery</ns2:url>
                  <ns2:version>2.0</ns2:version>
                </ns2:endpointConfiguration>
              </ns2:endpointConfigurationList>
            </ns2:endpoint>
			............
          </ns2:endpointList>
			............
        </organization>
      </organizationList>
    </exchange>
  </exchanges>
</exchangeInfo>