Google fails endpointID Validation in bc161?

[goochjj]

This code:

                try {
                        new URL("https://google.com").openStream(); // force a CA certificate lookup
                }
                catch (Exception e) {
                        e.printStackTrace();
                        System.exit(1);
                }

Throws this:

INFO: Client raised fatal(2) certificate_unknown(46) alert: Failed to read record
org.bouncycastle.tls.TlsFatalAlert: certificate_unknown(46)
	at org.bouncycastle.jsse.provider.ProvSSLSocketDirect.checkServerTrusted(Unknown Source)
	at org.bouncycastle.jsse.provider.ProvTlsClient$1.notifyServerCertificate(Unknown Source)
	at org.bouncycastle.tls.TlsUtils.processServerCertificate(Unknown Source)
	at org.bouncycastle.tls.TlsClientProtocol.handleServerCertificate(Unknown Source)
	at org.bouncycastle.tls.TlsClientProtocol.handleHandshakeMessage(Unknown Source)
	at org.bouncycastle.tls.TlsProtocol.processHandshakeQueue(Unknown Source)
	at org.bouncycastle.tls.TlsProtocol.processRecord(Unknown Source)
	at org.bouncycastle.tls.RecordStream.readRecord(Unknown Source)
	at org.bouncycastle.tls.TlsProtocol.safeReadRecord(Unknown Source)
	at org.bouncycastle.tls.TlsProtocol.blockForHandshake(Unknown Source)
	at org.bouncycastle.tls.TlsClientProtocol.connect(Unknown Source)
	at org.bouncycastle.jsse.provider.ProvSSLSocketDirect.startHandshake(Unknown Source)
	at org.bouncycastle.jsse.provider.ProvSSLSocketDirect.startHandshake(Unknown Source)
	at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1301)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
	at java.net.URL.openStream(URL.java:1041)
	at container.main(container.java:6)
Caused by: java.security.cert.CertificateException: No subject alternative name found matching IP address 172.217.12.142
	at org.bouncycastle.jsse.provider.HostnameUtil.checkHostname(Unknown Source)
	at org.bouncycastle.jsse.provider.ProvX509TrustManager.checkEndpointID(Unknown Source)
	at org.bouncycastle.jsse.provider.ProvX509TrustManager.checkEndpointID(Unknown Source)
	at org.bouncycastle.jsse.provider.ProvX509TrustManager.checkExtendedTrust(Unknown Source)
	at org.bouncycastle.jsse.provider.ProvX509TrustManager.checkExtendedTrust(Unknown Source)
	at org.bouncycastle.jsse.provider.ProvX509TrustManager.checkTrusted(Unknown Source)
	at org.bouncycastle.jsse.provider.ProvX509TrustManager.checkServerTrusted(Unknown Source)
	... 19 more

On oracle java 8u201, with:

security.provider.1=org.bouncycastle.jce.provider.BouncyCastleProvider
security.provider.2=org.bouncycastle.jsse.provider.BouncyCastleJsseProvider
security.provider.3=sun.security.provider.Sun
security.provider.4=sun.security.rsa.SunRsaSign

Same code works fine in bc160.

[peterdettman]

Earlier versions did not support endpoint identification. We have now added support, and tried to be as compatible with SunJSSE as possible. Unfortunately, when using HttpsURLConnection SunJSSE uses some magic (reflection and/or internal API) to tell the socket about the "original hostname" used for the connection ("google.com" here), and we can't use that same magic.

You will need to use one of the following two workarounds to allow the endpoint validation to work:

1. Set the system property jdk.tls.trustNameService to true. However, note this advice from the 8u51 release notes:

If an application does need to perform reverse name lookup for raw IP
addresses in SSL/TLS connections, and encounter endpoint identification
compatibility issue, System property "jdk.tls.trustNameService" can be
used to switch on reverse name lookup. Note that if the name service is
not trustworthy, enabling reverse name lookup may be susceptible to MITM
attacks.

2. The second (and recommended) alternative is to set a customized SSLSocketFactory on the
   HttpsURLConnection, then intercept the socket creation call and manually set the SNI host_name on the created socket. Please see the example code below. This is a general pattern that people typically use anyway when they want to e.g. adjust what ciphersuites are enabled.

{
    SSLContext sslContext = ...;
    final URL url = ...;
    HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
    conn.setSSLSocketFactory(new
org.bouncycastle.jsse.util.CustomSSLSocketFactory(sslContext.getSocketFactory())
    {
        @Override
        protected Socket configureSocket(Socket s)
        {
            if (s instanceof SSLSocket)
            {
                SSLSocket ssl = (SSLSocket)s;

                SNIHostName sniHostName = getSNIHostName(serverURL);
                if (null != sniHostName)
                {
                    SSLParameters sslParameters = new SSLParameters();

                    sslParameters.setServerNames(Collections.<SNIServerName>
singletonList(sniHostName));
                    ssl.setSSLParameters(sslParameters);
                }
            }
            return s;
        }
    });
}

private SNIHostName getSNIHostName(URL url)
{
    String host = url.getHost();
    if (null != host && host.indexOf('.') > 0
        && !org.bouncycastle.util.IPAddress.isValid(host))
    {
        try
        {
            return new SNIHostName(host);
        }
        catch (RuntimeException e)
        {
        }
    }
    return null;
}

[goochjj]

Option #2 is great for software I control.... I guess I don't know how to predict what will happen running something else. (Tomcat, Lucee, etc)

Is option #1 any less secure than bc160 was?

[Gilis95]

Hello, I am experiencing mostly the same issue. Instead of HTTPS connection I'm using LDAPS, which means the identification algorithm specified in SSL parameters is LDAPS instead of HTTPS, which results in mostly the same code. I investigate a little bit in what's actually going on.

• First thing I noticed is that, no matter whether I am providing an IP address or hostname(DNS entry) inside HostnameUtil IP address is always entering

• Second thing, I've took a look how it's implemented by you and JDK. Inside JDK on Server Hello requestedServerNames are being set inside SSLSession. This can be seen inside ClientHandshaker#serverHello. Right after this it comes server certificate message, which result in checking whether server hostname matches server certificate CN or SAN. This is the reason why it's filled in properly inside JDK. On the other hand i cannot see such thing to be done inside TlsClientProtocol, which is your code actually and it's the one that guides the whole handshake process, correct if I'm wrong. What I'm actually wondering is how IP address is being filled inside server names.

[peterdettman]

@Gilis95 There were further fixes in this area in our beta releases. Please try using the latest beta version from https://downloads.bouncycastle.org/betas/ and if you are still seeing the issue we can investigate further.

[Gilis95]

Ok, I've downloaded the beta jar and test it. Again to me it seems like IP address is entering HostnameUtil, although I've provided DNS entry. I am attaching stack trace:

"thread":"Ldap Readiness Thread","level":"WARN","loggerName":"com.axway.hada.ldap.health.BaseLdapHealthCheckThread","message":"Connection to pesho domain cannot be established","thrown":{"commonElementCount":0,"localizedMessage":"HADA-DC.dc.hada.lab.sofi.axway.int:636","message":"HADA-DC.dc.hada.lab.sofi.axway.int:636","name":"javax.naming.CommunicationException","cause":"commonElementCount":0,"localizedMessage":"certificate_unknown(46)","message":"certificate_unknown(46)","name":"org.bouncycastle.tls.TlsFatalAlert","cause":{"commonElementCount":0,"localizedMessage":"No subject alternative name found matching IP address 10.232.4.47","message":"No subject alternative name found matching IP address 10.232.4.47","name":"java.security.cert.CertificateException","extendedStackTrace":"java.security.cert.CertificateException: No subject alternative name found matching IP address 10.232.4.47
 org.bouncycastle.jsse.provider.HostnameUtil.checkHostname(HostnameUtil.java:69) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.jsse.provider.ProvX509TrustManager.checkEndpointID(ProvX509TrustManager.java:293) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.jsse.provider.ProvX509TrustManager.checkEndpointID(ProvX509TrustManager.java:278) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.jsse.provider.ProvX509TrustManager.checkExtendedTrust(ProvX509TrustManager.java:246) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.jsse.provider.ProvX509TrustManager.checkExtendedTrust(ProvX509TrustManager.java:225) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.jsse.provider.ImportX509TrustManager_5.checkServerTrusted(ImportX509TrustManager_5.java:58) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.jsse.provider.ProvSSLSocketDirect.checkServerTrusted(ProvSSLSocketDirect.java:141) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.jsse.provider.ProvTlsClient$1.notifyServerCertificate(ProvTlsClient.java:263) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.tls.TlsUtils.processServerCertificate(TlsUtils.java:3727) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.tls.TlsClientProtocol.handleServerCertificate(TlsClientProtocol.java:553) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.tls.TlsClientProtocol.handleHandshakeMessage(TlsClientProtocol.java:433) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.tls.TlsProtocol.processHandshakeQueue(TlsProtocol.java:545) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.tls.TlsProtocol.processRecord(TlsProtocol.java:463) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.tls.RecordStream.readRecord(RecordStream.java:224) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.tls.TlsProtocol.safeReadRecord(TlsProtocol.java:686) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.tls.TlsProtocol.blockForHandshake(TlsProtocol.java:324) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.tls.TlsClientProtocol.connect(TlsClientProtocol.java:82) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.jsse.provider.ProvSSLSocketDirect.startHandshake(ProvSSLSocketDirect.java:398) ~[bctls-jdk15on-162b11.jar:1.62.0.11.0]
 org.bouncycastle.jsse.provider.ProvSSLSocketDirect.startHandshake(ProvSSLSocketDirect.java:379)

[peterdettman]

How are you creating the socket? i.e. which constructor of ProvSSLSocketDirect gets called, and with what arguments?

[Gilis95]

It's inside Java LDAP implementation, but here it's:
Class com.sun.jndi.ldap.Connection.createSocket

createSocket = socketFactoryClass.getMethod("createSocket",
new Class<?>[]{String.class, int.class});

This refers to SocketFactory, which I am providing:

    @Override
    public Socket createSocket(String host, int port) throws IOException, UnknownHostException {
        return addHostnameVerificationParameterToSocket(getParentSocketFactory().createSocket(host, port));
    }

...

    private Socket addHostnameVerificationParameterToSocket(Socket socket) {
        if (socket instanceof SSLSocket) {
            SSLSocket sslSocket = (SSLSocket) socket;

            SSLParameters sslParameters = sslSocket.getSSLParameters();
            sslParameters.setEndpointIdentificationAlgorithm("LDAPS");

            sslSocket.setSSLParameters(sslParameters);

            return sslSocket;
        }

        return socket;
    }

...

    public TrustedSocketFactoryWithHostnameVerification() {
        try {
            LdapSSLConfiguration sslConfiguration = Factory.getInstance().getLdapDomainProvider().getLdapConfiguration()
                    .getSSLConfiguration();

            if (sslConfiguration != null && sslConfiguration.isSecureConnection()) {
                SSLSocketFactory localSocketFactory = mSocketFactory;
                if (localSocketFactory == null) {
                    synchronized (TrustedSocketFactoryWithHostnameVerification.class) {
                        localSocketFactory = mSocketFactory;
                        if (localSocketFactory == null) {
                            mSocketFactory = Factory.getInstance().getSecurityManager()
                                    .createSSLContext(sslConfiguration.getSecurityProtocol(),
                                            sslConfiguration.getTrustStoreLocation(),
                                            sslConfiguration.getTrustStorePassword().toCharArray())
                                    .getSocketFactory();
                        }
                    }
                }
            }
        } catch (SSLException | LdapConfigurationNotInitializedException e) {
            sLogger.error("Failed to initialize SSL Context", e);
        }
    }
...

    /**
     * {@inheritDoc}
     */
    @Override
    protected SSLSocketFactory getParentSocketFactory() {
        return mSocketFactory;
    }

[peterdettman]

Firstly, your example code can easily be made to work by adding:

sslParameters.setServerNames(Collections.<SNIServerName>singletonList(sniHostName));

as is shown in the workaround given above, and I would recommend that.

However it's not clear to me in your example why it wouldn't be working automatically. Is the 'host' argument to createSocket a qualified name (i.e. host.domain) and not something like "localhost"?

You could debug in the method ProvTlsClient.getSNIServerNames to see what, if any gets auto-added as the SNI hostname. Also you could debug in ProvSSLSocketDirect.notifyConnected and see what ends up in the peerHostSNI and peerHost members. If you report the results here, it should be easy to explain what's going on.

It remains true that BCJSSE is unable to work exactly as SunJSSE for the reasons given in my first response (which is why I recommend the explicit option).

[Gilis95]

The hostname of machine is not something like localhost, it's something like cn.dn.int. However I think there are also problem if hostname is not persisted into DNS, but in /etc/hosts instead. I observe this problem even in SUN security provider.
SUN Provider behavior in case of hostname filled in only in /etc/hosts:
First HostnameChecker always enters in IP case
Second even though certificate has SAN extension with server hostname filled in, hostname verification doesn't pass.
I suspect that BC provider hits the same issue, but I am going to debug, described by you classes in Monday.

[Gilis95]

Ok, inside ProvTlsClient#getSNIServerNames sslParameters has filled in server names. Because of the fact hostname contains . , sniServerNames is never being filled in. In the end the method returns null. The client never enters in ProvSSLSocketDirect#notifyConnected , I haven't search why is this happening.

[minQueue]

Hi @peterdettman ,

I am currently using JDK1.7 with Bouncy Castle FIPS provider.
Since I also have the same issue, I tried to follow your second alternative to set a customized SSLSocketFactory on the HttpsURLConnection.
But the problem is JDK1.7 does not provide SNIHostName class and setServerNames method in SSLParameters class.

Do you have any idea or alternative for JDK1.7 to solve this issue?

Thanks

[peterdettman]

From v1.70 if you want to use option 1 please prefer
-Dorg.bouncycastle.jsse.client.assumeOriginalHostName=true
instead of
-Djdk.tls.trustNameService=true

See #1202 .