Introduction to Public Key Certificates

Once every week or two, one of our customers using Rebex FTP/SSL or Rebex Secure Mail components encounters the “Server certificate was rejected by the verifier because…” error when connecting to a server secured using the TLS/SSL protocol. And even though the workings of TLS/SSL - the communication protocol using certificates and public key infrastructure to authenticate servers and clients - are sufficiently explained in our FTP/SSL tutorial and Secure Mail TLS/SSL Tutorial, we often have to provide a quick introduction into public key certificates in order to explain why the error occurred and how to solve it. Publishing this on our blog looks like a proper thing to do.

Public-key cryptography

Public-key cryptography, also known as asymmetric key cryptography, is a method for secret communication between two parties without requiring an initial exchange of secret keys. It can be used to create digital signatures, to encrypt messages using public key encryption, or to compute a shared secret key. The most common public-key cryptography algorithms used to enable secure transmission of information over the Internet are RSA, DSA and Diffie-Hellman.

Public-key cryptography is known as asymmetric because a key owner has a pair of two cryptographic keys – a public key and a private key. The public key may be freely distributed and published, while the private key must be kept secret and only accessible to the owner. The keys are related mathematically, but the private key cannot be feasibly derived from the public key.

To compute a digital signature of a message, the private key is needed. However, a digital signature can be verified using the public key alone. This makes it possible for the receiver of a message and its signature to verify that message was signed by an entity that has access to the corresponding private key – the claimed sender.

For more information, check out the Wikipedia article on public-key cryptography.

Public key certificate

A certificate is a document that uses a digital signature to bind a public key with an identity (such as a person, an organization or a server). A certificate can be freely distributes (just like the public key) and does not contain any secret information – basically, all it contains is the public key, identity information, validity conditions, additional attributes, information about its signer and a signature. However, for each certificate, there is a private key that corresponds to the certificate’s public key. The private key is not a part of the certificate and must be kept secret by the identity that owns the certificate.

Check out the picture on the right – it shows the contents of Google’s certificate used by https://www.google.com to secure user authentication.

Public key certificates can be used for variety of purposes – the ones our users are most interested in are:

• Secure communication using TLS/SSL protocol, where a server presents its certificate to the client to make it possible for the client to verify that it is really connected to the desired server. Optionally, a client can also present his own certificate to authenticate himself to the server. TLS/SSL is used by HTTPS (HTTP over SSL), a well-known protocol used to secure websites, but many other protocols such as FTP, SMTP, POP3 or IMAP can also utilize it.

• E-mail signature and encryption using S/MIME. This makes it possible to sign e-mail (using sender’s secret private key), enabling others to verify the signature using the signer certificate’s public key. On the other hand, to encrypt e-mail, the recipient certificate’s public key is used – once encrypted, the e-mail can only be decrypted by the recipient (using his corresponding private key). Check out our S/MIME tutorial for more information.

Certification Authorities (CAs) and Public Key Infrastructure (PKI)

The certificate is usually signed using a private key of another identity (known as certification authority) and the actual data being signed is the certificate content - public key, identity, validity conditions, additional attributes and information about the signer. This ensures that the certificate information cannot be forged or tampered with and that the signer (the certification authority) attests that the public key and the identity belong together. Therefore, if one trusts the signer (usually called the certificate issuer), all the certificates it signs (issues) are also trusted. This arrangement (using a CA to bind a public key with the respective identities) is called public key infrastructure.

In the X.509 public key infrastructure, the public key of a certificate authority itself is also a part of another certificate that is either signed by yet another certification authority or by the private key that corresponds to the certificate’s public key itself (that is a self-signed certificate, also known as root certificate). This forms a certificate chain of trust – certificate of every identity chains up to a root CA certificate. If a root certificate authority is trusted, all certificates it (or one of its sub-ordinate CAs) issues are also implicitly trusted if all the certificates in their chain are known and valid. The validity check evaluates the chain, making sure each link is signed by the next link, that the signature is valid, that each certificate has can be used for a specific purpose at the specific time and that the last certificate in the chain is a trusted CA certificate.

The Google’s certificate above was issued and signed by Thawte, whose certificate was in turn issued and signed by Verisign using its root CA certificate. The picture on the right shows this certificate chain.

Trusted root certification authorities

Microsoft Windows and most web browsers and SSL libraries come with a list of trusted certification authorities. These are usually commercial certification authorities that have to undergo annual security audits by independent organizations in order to be included in these lists. Rebex TLS/SSL library also uses Windows certificate infrastructure and its certificate stores to validate certificates, which makes it easy to manage using well-known tools. The same infrastructure is utilized by Windows components and many Windows applications that use CryptoAPI to perform security operations. For example, Internet Explorer uses it to validate server certificates when browsing a website protected using HTTPS (HTTP over SSL) protocol. You can even use Internet Explorer to manage the certificate stores (alternatively, use the “Certificates” MMC snap-in).

There are many commercial certification authorities (CAs) that charge for their services. Some of these are authorities that have been added to the aforementioned list by the OS and browser vendors, others are local certification authorities that are trusted by government institutions and are often not trusted by Microsoft Windows and other operating systems out-of-the-box. Some CAs that issue certificates to the public at no cost are also not trusted unless explicitly added to the list of trusted CAs.

To add a certificate of a certification authority into the “Trusted Root Certification Authorities” store of the current user, simply right-click the .cer/.der or .p7b file downloaded from the CA’s web site or start Internet Explorer and follow the instructions in the picture on the right. Only self-signed certificates belong into this store – certificates of non-root CAs have their own store - “Intermediate certification authorities”.

Please note that each Windows user has a separate set of certificate stores. There are also local machine certificate stores that are used by Windows Services and other similar applications. This can be managed using Microsoft Management Console’s “Certificates” snap-in that is a part of every Windows installation.

What to do when a server certificate is not trusted?

A server certificate will usually fall into one of these categories:

• The certificate is signed by a certificate authority that is trusted by Windows certificate infrastructure. This is the desired state.
• The certificate is signed by a certificate authority that is not trusted by Windows certificate infrastructure. For example, this can be a third-party CA or a corporate CA. Add the root CA certificate into “Trusted Root Certification Authorities” store and add each intermediate CA certificate into “Intermediate certification authorities” store to make the server certificate trusted.

• The certificate is self-signed. This is what many FTP and mail servers generate by default. Either replace the default certificate with a certificate issued by a real CA, or make it trusted by Windows certificate infrastructure by adding it into the “Trusted Root Certification Authorities” store, because – in theory and in practice – it is both a server certificate and a CA certificate.

To be continued…

Tomorrow, we will look into the most common certificate problem encountered by our customers – the “Server certificate was rejected by the verifier because…” exception.