HOWTO: Solve the “Server certificate was rejected by the verifier” exception

November 30, 2012 | Tomas Knopp

This blogpost applies to Rebex components since version 2012 R3. It's an updated version of a blogpost we publised in 2009.

If you wrote an application using Rebex FTP/SSL or Rebex Secure Mail component and get the “Server certificate was rejected by the verifier because…” error when connecting to a server secured using the TLS/SSL protocol, this HOWTO is just what you need. In order to understand the problem, at least basic understanding of public key certificates is helpful. If this is still a gray area for you, just check out our Introduction to Public Key Certificates.

The actual exception message varies, but the reason is similar – either something is wrong with the certificate or it is not trusted by the client.

The following exceptions can occur as a result of failed certificate verification:

Server certificate was rejected by the verifier because the certificate's common name '…' does not match the hostname '…'.
Server certificate was rejected by the verifier because it has expired.
Server certificate was rejected by the verifier because it is revoked.
Server certificate was rejected by the verifier because of an unknown certificate authority.
Server certificate was rejected by the verifier because it is bad.
Server certificate was rejected by the verifier because of other problem.
Server certificate was rejected by the verifier because it is unsupported.

In general, when any of these errors occur, you have the following options:

a) Solve the problem. These error messages indicate that something is wrong with the certificate or certificate infrastructure. Fixing the problem is the proper thing to do, although it might not be always possible. See the instructions below for a detailed descriptions of various error conditions.

b) If this is the “common name” mismatch problem (no. 1 in the list), see the instructions below for more information.

c) Implement a custom certificate validation handler - check out the appropriate section of our FTP/SSL tutorial or Secure Mail TLS/SSL tutorial for details. Also, check out the WinFormClient sample (in Rebex FTP/SSL) or Pop3Browser/ImapBrowser samples (in Secure Mail) - these include a certificate validation handler that asks the user whether to accept or reject a certificate if in doubt, and it even has a capability to automatically add the certification authority certificate into the appropriate store if the problem occurred because of an unknown certification authority (no. 4) - you can simply re-use this verifier in your application or modify it to suit your needs, because the source files that implement the verifier (Verifier.cs/.vb and VerifierForm.cs/.vb) are independent of the rest of the samples.

d) Implement a custom certificate validation handler that accepts or reject certificates based on their hash code (also known as fingerprint) – this value uniquely identifies a certificate, making it possible to be used as its ID:

C#:

// This method gets called during the SSL handshake
// process when the certificate chain is received from the server.
private void validateCertificate(object sender, SslCertificateValidationEventArgs e)
{
    // get a string representation of the certificate's fingerprint
    string fingerprint = e.Certificate.Thumbprint;
    // check whether the fingerprint matches the desired fingerprint
    bool ok = ...;
    if (ok)
        e.Accept();
    else
        e.Reject();
}
...
// To use the certificate validation handler for an FTP session:
Ftp ftp = new Ftp();
// Register to the ValidatingCertificate event
// to handle the certificate validation on your own.
ftp.ValidatingCertificate += new EventHandler(validateCertificate);
// Connect securely using explicit SSL.
ftp.Connect(hostname, SslMode.Explicit);
...

VB.NET:

' This method gets called during the SSL handshake
' process when the certificate chain is received from the server.
Private Sub validateCertificate(sender As Object, e As SslCertificateValidationEventArgs)
    ' get a string representation of the certificate's fingerprint
    Dim fingerprint As String = e.Certificate.Thumbprint
    ' check whether the fingerprint matches the desired fingerprint
    Dim ok As Boolean = ...
    If ok Then
        e.Accept()
    Else
        e.Reject()
    End If 
End Sub 
...

' To use the certificate validation handler for an FTP session: 
Dim ftp As New Ftp() 
' Register to the ValidatingCertificate event 
' to handle the certificate validation on your own. 
AddHandler ftp.ValidatingCertificate, AddressOf validateCertificate 
' Connect securely using explicit SSL. ftp.Connect(hostname, SslMode.Explicit) 
...

In practice, an application would have to keep a database of acceptable IDs and validate certificates according to it. However, this approach deviates from a standard public key infrastructure practices and should only be used if other solutions are unsuitable.

e) Use SslAcceptAllCertificates option - this will undoubtedly make the certificate accepted, but doing this in production environment is highly discouraged, as it effectively disables server authentication, bypassing one of the key features of TLS/SSL. Even though this will make it possible for you to connect to the server, there will be no proof that you are actually connected to the desired server.

C#:

// Accept any certificate (don't do this in production environment!)
ftp.Settings.SslAcceptAllCertificates = true;
// Connect to the server.
ftp.Connect(hostname, SslMode.Explicit);
...

VB.NET:

' Accept any certificate (don't do this in production environment!)
ftp.Settings.SslAcceptAllCertificates = true 
' Connect to the server
ftp.Connect(hostname, SslMode.Explicit) 
...

Now, let’s look at the different error conditions.

1. Common name does not match the hostname

This one is usually easy to solve. The server hostname you used in the Connect method call’s serverName argument has to match the server name that is embedded in the certificate’s common name attribute (part of certificate subject string). If they don’t match, it means, for example, that the server you know as “www.rebex.net” is presenting a certificate that belongs to “server01.rebex.net”, and this is an equivalent of one guy presenting another’s passport at the airport – while the document may be entirely valid, the two just don’t belong together.

Possible solutions:

a) Try using the proper server name present in the certificate to connect to the server. If the certificate says the server is “server01.rebex.net” when you connect to “www.rebex.net”, then perhaps it is in fact its proper name and you wish to connect to “server01.rebex.net” instead of “www.rebex.net” which is just an alias.

b) If the common name in the certificate cannot be used to connect to the desired server (for example, it is a local domain-less name such as “server01”), the proper thing to do is to issue a new certificate for the server that contains the correct name. If this is not possible, try the solution below.

c) If the common name in the certificate cannot be used to connect to the desired server, but you are unable to issue a new certificate (for example if the server is not under your control), use the Settings's SslServerName property to specify the expected common name:

C#:

// Specify the expected common name
ftp.Settings.SslServerName = "server01";
// Connect to the server
ftp.Connect("www.rebex.net", SslMode.Explicit);
...

VB.NET:

' Specify the expected common name
ftp.Settings.SslServerName = "server01"
' Connect to the server
ftp.Connect("www.rebex.net", SslMode.Explicit)
...

d) One of our users also encountered a server that presented a different certificate with different common name each time they connected. In this situation, the only workaround is to write a custom certificate verification handler using the instructions in our tutorial.

2. Server certificate was rejected by the verifier because it has expired.

This happens when the validity period of the server certificate is over. When the certificate expires, the proper thing to do is to replace it with a new certificate. If this can’t be done (if you don’t have control over the server and its administrators are refusing to solve the problem), the solution is to write a custom certificate verification handler using the instructions in our tutorial.

3 Server certificate was rejected by the verifier because it is revoked.

This is a problem that is worth paying attention to. It means that the certificate signature has been revoked by its signer and that the certificate should no longer be used and trusted. A custom certificate validation handler can also override this problem, but doing so is strongly discouraged.

4 Server certificate was rejected by the verifier because of an unknown certificate authority.

This error message means that the certificate authority that issued the server certificate is not in the list of trusted authorities. For this reason, the identity of the server could not have been verified. This problem can be solved by adding the certificate of the root CA that signed the server certificate into the list of trusted certification authorities as described by the previous post about public key certificates, or using Rebex CertificateStore class - check out the Verifier.cs/.vb file in the WinFormClient or Pop3Browser/ImapBrowser samples for information on how to do this in a custom certificate verification handler.

5, 6 and 7…

The following three possible errors still remain:

5. Server certificate was rejected by the verifier because it is bad.
6. Server certificate was rejected by the verifier because of other problem.
7. Server certificate was rejected by the verifier because it is unsupported.

If you get one of these, please let us know and mail us the communication log produced using the built-in LogWriter with LogLevel.Debug and we will solve it quicky.

That’s all for today – if you are still stuck with an unusable certificate after trying all the tricks above, please let us know!