...
Older versions of CONNECT provided default values that could pass SAML validation on most responding gateways but CONNECT 5.1 and later versions encourage implementors to provide more appropriate values for parameters such as subject info since CONNECT does not actually confirm user authenticity.
DNS Certificate issues
CONNECT uses the Direct JAVA RI for retrieving certificates via DNS. The logic used there is that it checks for an individual certificate first, in your case it would be admin.mail.testserver.com, and if this CERT record isn’t found then it will look for the organization certificate at mail.testserver.com.
The PKIX path errors are because one machine doesn't trust the other?
...