...
...
...
...
...
Version#
...
Date
...
Modified By
...
Description of Modification
...
Eric McDonald
...
Table of Contents maxLevel 3 indent 20px separator disc
Background
With GFE environment going away, there was a need to:
...
To solve this, CONNECT team came up with an AWS based solution, that will host both the continuous integration (CI) and testing platform for CONNECT. For security purposes, CONNECT team set up an OpenVPN for accessing the CI and Configurable CONNECT. This guide is written specifically with AWS cloud in mind. However any cloud that provide Infrastructure as a Service (IaaS) can easily be replace AWS.
Configurable CONNECT
Configurable CONNECT allows testers and developers to spun a test environment instance with any CONNECT version deployed on any supported server. User also has the option of selecting gateway to gateway/loopback and standard/passthrough modes. Below is a screenshot of Configurable CONNECT:
Design Approach
Configurable CONNECT is built using AWS infrastructure, and employs a number of AWS cloud tools/utilities and NodeJs. Listed below are the details of each Configurable CONNECT component:
AMI Default Image
Create a default EC2 machine/instance with the following software installed:
...
This default image will be used to create EC2 instances for Configurable CONNECT.
Repository for CONNECT ears
Use S3 Browser or any ftp site to host CONNECT ears.
Pre-configured CONNECT config files
Configure internalExchangeInfo.xml / internalConnectionInfo.xml for gateway to gateway setup
Configure exchangeInfo.xml / uddiConnectionInfo.xml for gateway to gateway setup
Configure gateway.properties for standard and pass-through modes
NodeJs
ConfigurableCONNECT uses node js scripts to create a GUI where user can select CONNECT version, server type, gateway mode( service type: standard/passthrough), gateway to gateway configuration and to deploy Admin GUI. NodeJs uses ssh to create EC2 instance from AMI Default image, copy the user selected ear from S3 browser, copy connect properties, run CONNECT release specific database scripts and pre-configured files for different connect setups. Once EC2 instance is created, NodeJs deploy selected CONNECT.ear. The instance is now ready to use for testing. User can also re-deploy an EAR, reset CONNECT database, run ValidationSuite and View node js logs.
Launch CONNECT Manually
Once Configurable CONNECT successfully launches an EC2 machine, user can log in that box using software like Putty or Mobaxterm to view application logs, start and stop the server, deploy and un-deploy CONNECT ear.
Continuous Integration (CI)
CONNECT employs Jenkins to build and verify core CONNECT functionality for its continuous integration purposes. CONNECT is using AWS EC2 instances for Jenkins. There are 5 EC2 Jenkins servers that are available 24x7:
- Jenkins master(runs Wildfly server)
- Jenkins slave JBoss
- Jenkins slave WebSphere
- Jenkins slave Weblogic
- Jenkins slave SonarQube
...
- Github Pull Request Validator
- CONNECT_CI Stage_1
- CONNECT_CI_LINUX_JBOSS
- CONNECT_CI_LINUX_WEBLOGIC
- CONNECT_CI_LINUX_WEBSPHERE
- CONNECT_CI_MERGE
- CONNECT_CI_NOADMINWS
- CONNECT_COMMON_TYPES_CI
- CONNECT_WEBSERVICES_CI
- CONNECT_NIGHTLY
- CONNECT_TAG
CONNECT Merge Build Process And Associated Jenkins Jobs
Step 1: Developer pushes changes to a *_integration branch within the Github repository at https://github.com/CONNECT-Solution/CONNECT (currently CONNECT_integration branch is the only one being watched by CI).
...
Step 3: CONNECT_CI Stage_1 builds on Jenkins master. It executes Parallel Integration Testing on Jenkins Slaves
- Triggered by: PR merge on Github.
- Executes: mvn clean install -P weblogic,was,PD,DQ,DR,DS,DDS,AD,X12,DB,admingui . Copy the binary files generated in this step to appropriate Jenkins slave machines and then in parallel runs the various app server validation jobs (CONNECT_CI_LINUX_JBOSS, CONNECT_CI_LINUX_WEBLOGIC, and CONNECT_CI_LINUX_WEBSPHERE). Each of these jobs run all the soapui Validation Suite tests against the applicable application server. Refer to Figure 2.
...
- If the failure can be fix immediately then it is OK to submit another pull request to resolve the issue.
- If the failure requires any amount of time to resolve, then a "revert" commit needs to be submitted ASAP to back out the failing changes.
- It is important that failures are acted upon ASAP so that other branches can continue to be merged. A failed merge build will continue to fail until the issue is resolved, not even separate PRs can be merged if a previous one is failing.
Figure 2
CONNECT_COMMON_TYPES_CI
Jenkins runs the CONNECT_COMMON_TYPES_CI build whenever a PR is merged for CommonTypes repository. When successful, the CommonTypes snapshot is updated in sonaType(https://oss.sonatype.org/content/repositories/snapshots/org/connectopensource/) and it also triggers a build on CONNECT_WEBSERVICES_CI.
CONNECT_WEBSERVICES_CI
Jenkins runs this job when a commit is made against CONNECT-Solution/CONNECT-Webservices, the code is pulled, built, and the resulting jars are uploaded to Sonatype OSSRH (https://oss.sonatype.org/content/repositories/snapshots/org/connectopensource/).
CONNECT_NIGHTLY
Jenkins runs the CONNECT_NIGHTLY build every night after midnight. This build runs the entire suite of static code analysis tools (Findbugs, SonarQube and CheckStyle) as well as all of the automated regression suites. This build is run at off peak times due to the large amount of time to execute.
If the Build Fails
The team will be emailed about the failure during the execution of CONNECT_NIGHTLY and the fix will be put in according to team processes.
CONNECT_TAG
The CONNECT_TAG build is a manual build which can be kicked off to create a tag in github and optionally generate binaries. This build is parameterized and takes the following inputs:
...