How to make Apache HttpClient trust Let's Encrypt Certificate Authority

29. 2. 2016

Apache HttpClient is a popular library that many other frameworks build upon. Naming one for all - you can find it in Spring Framework RestTemplate. It's quite suprising that its not easy to find compherensible walkthroughs how to make it trust server certificates that are not validable by certificate chains baked in standard Java installation. Spending a few hours searching and tweaking the code I've decided to document easy - step by step solution how to do this for emerging Let's Encrypt certificate authority (but it's applicable to any other CA too).

Do you want to save several hours of yours? Read on :)

Let's make one thing clear in the beginning - this walkthrough guides you how to make certain CA trusted only by your application that you are developing. If you can and want to make CA trusted by entire Java installation (ie. all JVM instances running from it), you can add certificate to global Java truststore by following walkthrough. The reason I didn't want to go this way is that I didn't want to add certificate to truststore of all environments my application runs on (it's quite impractical).

If you want to keep this internal to your project, you need to ...

1) Download certificate of CA

In case of Let's Encrypt authority you'll find all certificates on this page. Namely you need this one:

https://letsencrypt.org/certs/isrgrootx1.pem

Download it, place it in your src/main/resources/somedirectory path as letsencrypt.cer (naming is not important here).

2) Download backup certificate of IdenTrust CA

You may also want to include IdentTrust root chain in your trust store - in case something goes wrong with LetsEncrypt root certificate:

https://www.identrust.com/certificates/trustid/root-download-x3.html

Download it, place it in your src/main/resources/somedirectory path as identrust.cer (naming is not important here).

3) Create truststore in a file on your classpath

Now we need to create new truststore file where we'll import this CA certificate. You can do this by running keytool in the directory where you saved letsencrypt.cer in previous step:

[source]keytool -keystore letsencrypt-truststore -alias isrgrootx -importcert -file letsencrypt.cer
keytool -keystore letsencrypt-truststore -alias identrust -importcert -file identrust.cer[/source]

File naming is again not important - but to successfully finish this walkthrough stick with these names. You can always rename files once your unit test is green. When executing this command you'll be asked for truststore passphrase. Enter passcode letsencrypt and hit enter. Next you'll be asked to confirm that you trust letsencrypt.cer file and you need to again confirm this.

If you want to verify that CA certificate is safely in your truststore run:

[source]keytool -list -v -keystore letsencrypt-truststore[/source]

If you want to play more with trustore read official documentation here.

Note aside: Do you ever wonder how truststore differs from keystore when Java code always uses only KeyStore class? Read more detailed description here but in short - trust store is only name convention for file where you store external certificates you trust compared to keystore where you store digital keys you own.

4) Exclude file from Maven filtering

We've created nice binary file on classpath that will be probably immediatelly corrupted by wonderful Maven feature, that is called filtering. So let's tell Maven not to touch our files on classpath:


<build>
   <resources>
      <resource>
         <directory>src/main/resources</directory>
         <filtering>false</filtering>
      </resource>
   </resources>
</build>

We can be more specific and let Maven filter some files while others not - but that's a task you can solve on your own.

5) Implement TrustManagerDelegate

Now let's do some coding. We need to teach Java to use our own truststore on classpath but fallback to its default trustore preinstalled with Java (we want to trust to every CA Java does and add a new CA on top of it). So we create delegating TrustManager that will ask main TrustManager managing our classpath truststore if certificate chain can be trusted and if not it delegates decision to fallback TrustManager which is the main Java one:


import javax.net.ssl.X509TrustManager;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
public class TrustManagerDelegate implements X509TrustManager {
   private final X509TrustManager mainTrustManager;
   private final X509TrustManager fallbackTrustManager;
   public TrustManagerDelegate(X509TrustManager mainTrustManager, X509TrustManager fallbackTrustManager) {
      this.mainTrustManager = mainTrustManager;
      this.fallbackTrustManager = fallbackTrustManager;
   }
   @Override
   public void checkClientTrusted(final X509Certificate[] x509Certificates, final String authType) throws CertificateException {
      try {
         mainTrustManager.checkClientTrusted(x509Certificates, authType);
      } catch(CertificateException ignored) {
         this.fallbackTrustManager.checkClientTrusted(x509Certificates, authType);
      }
   }
   @Override
   public void checkServerTrusted(final X509Certificate[] x509Certificates, final String authType) throws CertificateException {
      try {
         mainTrustManager.checkServerTrusted(x509Certificates, authType);
      } catch(CertificateException ignored) {
         this.fallbackTrustManager.checkServerTrusted(x509Certificates, authType);
      }
   }
   @Override
   public X509Certificate[] getAcceptedIssuers() {
      return this.fallbackTrustManager.getAcceptedIssuers();
   }
}

6) Persuade HttpClient to use our truststore

Now we have to wire it all together. Let's create Spring FactoryBean for constructing HttpClient instance that will be used in our RestTemplate. Note several important points in this class:

getKeyStore - this method loads contents of our truststore file from classpath and parses them into KeyStore memory object - it uses passphrase to open the file
createSslContext - creates two TrustManagers - javaDefaultTrustManager that contains multiple CA certificates that Java trusts by default, customCaTrustManager that contains only single CA certificate that we imported to our file; these two managers are composed together by TrustManagerDelegate decribed above


import org.apache.commons.io.IOUtils;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLInitializationException;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.springframework.beans.factory.FactoryBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ClassPathResource;
import org.springframework.stereotype.Service;
import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.TrustManagerFactory;
import javax.net.ssl.X509TrustManager;
import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.security.KeyManagementException;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.cert.CertificateException;
@Service
public class RestTemplateFactory implements FactoryBean&lt;CloseableHttpClient&gt; {
   private CloseableHttpClient client;
   private final SecureRandom secureRandom = new SecureRandom();
   @PostConstruct
   public void init() throws Exception {
      final SSLContext sslContext = createSslContext();
      SSLConnectionSocketFactory sslSocketFactory = new SSLConnectionSocketFactory(sslContext);
      PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(
            RegistryBuilder.&lt;ConnectionSocketFactory&gt;create()
                  .register("http", PlainConnectionSocketFactory.getSocketFactory())
                  .register("https", sslSocketFactory)
                  .build()
      );
      client = HttpClients.custom()
                     .setConnectionManager(cm)
                     .build();
   }
   @Override
   public CloseableHttpClient getObject() throws Exception {
      return client;
   }
   @Override
   public Class&lt;?&gt; getObjectType() {
      return CloseableHttpClient.class;
   }
   @Override
   public boolean isSingleton() {
      return true;
   }
   @PreDestroy
   public void close() throws Exception {
      client.close();
   }
   private SSLContext createSslContext() throws KeyStoreException, IOException, CertificateException {
      final SSLContext sslContext;
      try {
         sslContext = SSLContext.getInstance("TLS");
         final TrustManagerFactory javaDefaultTrustManager = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
         javaDefaultTrustManager.init((KeyStore)null);
         final TrustManagerFactory customCaTrustManager = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
         customCaTrustManager.init(getKeyStore());
         sslContext.init(
               null,
               new TrustManager[]{
                     new TrustManagerDelegate(
                           (X509TrustManager)customCaTrustManager.getTrustManagers()[0],
                           (X509TrustManager)javaDefaultTrustManager.getTrustManagers()[0]
                     )
               },
               secureRandom
         );
      } catch (final NoSuchAlgorithmException ex) {
         throw new SSLInitializationException(ex.getMessage(), ex);
      } catch (final KeyManagementException ex) {
         throw new SSLInitializationException(ex.getMessage(), ex);
      }
      return sslContext;
   }
   private KeyStore getKeyStore() throws KeyStoreException, IOException, NoSuchAlgorithmException, CertificateException {
      KeyStore ks = KeyStore.getInstance("JKS");
      InputStream is = new ClassPathResource("META-INF/lib_rest_api/certificate/letsencrypt-truststore").getInputStream();
      try {
         ks.load(is, "letsencrypt".toCharArray());
      } finally {
         IOUtils.closeQuietly(is);
      }
      return ks;
   }
}

7) Write the test

Now we have to write test to prove that we are able to communicate with TEST server using Let's Encrypt SSL certificate as well as PRODUCTION server using SSL certificate of some other trusted certificate authority:

import org.apache.http.HttpHost;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.junit.Assert;
import org.junit.Test;
public class RestTemplateFactoryTest {
   private RestTemplateFactory tested;
   @Test
   public void shouldContactTestApi() throws Exception {
      tested = new RestTemplateFactory();
      tested.init();
      final CloseableHttpClient client = tested.getObject();
      final CloseableHttpResponse response = client.execute(new HttpHost("test.mysecureddomain.com", 443, "https"), new HttpGet("/api/test"));
      Assert.assertNotNull(response);
   }
   @Test
   public void shouldContactProductionApi() throws Exception {
      tested = new RestTemplateFactory();
      tested.init();
      final CloseableHttpClient client = tested.getObject();
      final CloseableHttpResponse response = client.execute(new HttpHost("www.mysecureddomain.com", 443, "https"), new HttpGet("/api/test"));
      Assert.assertNotNull(response);
 }
}

Oh wait! Shouldn't we started with the test in the first place? :D

Happy coding ...

Credits: thanks for corrections from v6ak

Komentáře

Otec Fura 18.4.2016 16:12

You were right. I’ve corrected the article. Thanks for help.

Vít Šesták 'v6ak' 28.3.2016 15:40

„Adding the DST certificate is more future-proof, because LE might switch from one intermediate CA to another one in case of some unfortunate event.“

Well, even a fortunate event can cause change of the intermediate CA, which breaks the code that adds the single intermediate CA to the truststore. This is no longer a theoretical issue: The introduced WinXP support has caused this for new certs: https://community.letsencrypt.org/t/upcoming-intermediate-changes/13106 (via http://www.root.cz/clanky/let-s-encrypt-funguje-v-xp-veci-se-ale-mohou-rozbit/ ).

I can confirm having a new cert issued by Let’s Encrypt Authority X3 (not X1!). Any code relying on X1 will break after a certificate renewal. Because the renewal needs to be done at least every 90 days (but is usually done about every 60 days), thing will break soon.

One way to address this might be adding the X3 (and maybe also X4) to the truststore, but this might break in a similar way in the future. As I have written, a more future-proof way is adding the DST root: https://www.identrust.com/certificates/trustid/root-download-x3.html

novoj 8.3.2016 09:29

Ok, I must study this a little bit deeper. I’m just scratching the surface here - but I’ve never needed to solve such issues so far. Thanks!

novoj 8.3.2016 09:27

Chápu - v mém případě byla geneze taková, že pro produkci jsem pro správnou funkcionalitu REST clienta nemusel dělat nic - produkční servery měly certifikát od “standardních” autorit.

Stejný kód na testu / preprodukci ovšem padal, protože tam byl certifikát od Let’s Encrypt. Tj. chtěl jsem jen ošéfovat funkcionalitu pro test.

Ale dobrý argument a budu na něj myslet!

Filip Jirsák 8.3.2016 08:57

Ano, ale to je záměr :-) Nechci důvěřovat certifikátům ze systémového úložiště, protože nevím, které certifikáty tam jsou, a které tam budou s jinou verzí JRE.

Chápu, že někomu to takhle může vyhovovat, ale osobně to nedoporučuju – jste pak závislí na tom, jaké certifikáty se dodavatel JRE zrovna rozhodne přibalit. Proto si raději udělám vlastní úložiště s certifikáty, které potřebuju – a u kterých mám jistotu, že je tam budu mít i za měsíc, až vyjde aktualizace JRE.

novoj 7.3.2016 17:31

Toto bohužel nefunguje - takto nahozený SSLContext správně zvaliduje pouze certifikáty uvnitř certFile v metodě .loadTrustMaterial, ale není tam už ten fallback na defaultní certifikáty v Java truststore.

Totot jsem si teď prakticky znovu ověřil pomocí testů.

Vít Šesták 'v6ak' 5.3.2016 12:48

Few notes:

First, the issue is missing DST certificate in trust store. Adding the DST certificate is more future-proof, because LE might switch from one intermediate CA to another one in case of some unfortunate event. Note that the issue is related to Java rather than to Apache HttpClient: https://community.letsencrypt.org/t/will-the-cross-root-cover-trust-by-the-default-list-in-the-jdk-jre/134/2

Second, there is an alternative (at least for Debian): You can install ca-certificates-java. Your Java CAs will be imported from OS then. (OK, these alternatives have their pros and cons. If adding a file to $JAVA_HOME/jre/lib/security/cacerts is too complicated, then ca-certificates-java will be probably also too complicated.)

Filip Jirsák 5.3.2016 08:17

Tohle se používá na serveru při přihlášení klientským certifikátem. Když server vyžaduje přihlášení klientským certifikátem, může klientovi poslat seznam autorit, jejichž certifikáty akceptuje. Typicky webový prohlížeč dává uživateli na výběr, který certifikát se má pro přihlášení použít, a podle tohohle seznamu certifikáty vyfiltruje.

V tomhle je to rozhraní hloupě navržené, protože míchá dohromady klientský a serverový kód. Ale bacha při implementaci, doporučuju implementovat všechny metody, i když implementujete jen klienta a nějaká metoda by měla být určená jen pro server. Měl jsem to implementované tak, že serverové metody vyhazovaly nějakou unsupported výjimku, a pak myslím s Javou 7 najednou aplikace začala padat, protože v rámci jakéhosi bezpečnostního vylepšení začali volat i na klientovi některé serverové metody.

Filip Jirsák 5.3.2016 08:11

Novou certifikační autoritu lze přidat k těm systémovým (ať už úpravou toho systémového úložiště, nebo jeho kopií, přidáním nové CA a odkázáním přes tu systémovou proměnnou).

Já v tom vidím jiný problém – touhle úpravou měním globálně certifikační autority pro celou Javu, a když používám systémové úložiště autorit, musí všechny části aplikace důvěřovat těm samým autoritám. To se mi z hlediska bezpečnosti nelíbí. Když má jedna služba komunikovat s nějakým HTTPS serverem, předám jí truststore s certifikátem jedné CA nebo přímo certifikátem serveru, a nechci tam mít povolené žádné jiné autority. Druhá služba pak bude komunikovat s jiným HTTPS serverem, a bude mít zase jiný truststore pouze s certifikáty, které potřebuje.

Mám třeba ne moc významnou službu, která komunikuje se serverem, který má certifikát vydaný privátní autoritou provozovatele té služby. OK, pro tu službu mi to nevadí, ta firma může vydáním špatného certifikátu hacknout maximálně sama sebe. Ale nemůžu jejich CA důvěřovat i pro ostatní služby, protože pak by mohla hacknout i ty.

Filip Jirsák 4.3.2016 09:23

Pro jistotu ještě importy:

import org.apache.http.impl.client.HttpClients; import org.apache.http.ssl.SSLContexts;

Filip Jirsák 4.3.2016 09:22

V aktuálních verzích Apache HTTP Client není nutné vytvářet vlastní implementace TrustManageru apod., vše už je implementováno:

File certFile = …;

SSLContext sslContext = SSLContexts .custom() .useProtocol(“TLSv1”) .loadTrustMaterial(certFile) .build();

this.httpClient = HttpClients .custom() .setSSLContext(sslContext) .build();

Otec Fura 1.3.2016 22:19

Dobrá připomínka - tady asi narážíme na to, že Chef si u nás spravují kluci z Operations a je to pro mě komplikovanější varianta, než výše popsaná věc.

Otec Fura 1.3.2016 22:17

No jestli správně chápu: http://stackoverflow.com/questions/5871279/java-ssl-and-cert-keystore

“javax.net.ssl.trustStore - Location of the Java keystore file containing the collection of CA certificates trusted by this application process (trust store). If a trust store location is not specified using this property, the SunJSSE implementation searches for and uses a keystore file in default Java locations.”

Tak tím sice vyřešíš Let’s Encrypt, ale odřízneš si cestu k těm zapečeným CA uvnitř Java trust storu ne?

A druhý argument je opět to, že bys to musel nasetupovat na všech prostředích + tím ovlivníš celou JVM, což ne vždycky chceš.

A navíc se to blbě testuje :)

Má to ale tu výhodu, že to můžeš hodit na Operations :)

Augi (@AugiCZ) 1.3.2016 22:08

No ale kdo dnes obchází stroje a ručně na nich něco mění? :-) To stačí zadat jednou do Puppetu/Ansiblu/Chefu a hotovo, ne? :-)

Augi (@AugiCZ) 1.3.2016 22:06

A proč raději nevyužít -Djavax.net.ssl.trustStore= a -Djavax.net.ssl.trustStorePassword= ?

Otec Fura 1.3.2016 21:42

Asi by bylo možná vhodné udělat join polí z obou TrustManagerů - nicméně nikde jsem nenašel, k čemu se tato metoda používá (v HttpClientu vůbec) a evidentně to nemá na funkci rostlináře žádný vliv. :)

Michal Nikodim 1.3.2016 21:38

Vsechno jasny az na:

public X509Certificate[] getAcceptedIssuers() { return this.fallbackTrustManager.getAcceptedIssuers(); }

tady nepotrebujes slozeninu z mainTrustManageru a fallbackTrustManageru ?

Otec Fura 1.3.2016 10:19

Přesně tohle píšu v třetím odstavci článku. Protože pokud máte cluster, dev, test a prod prostředí, tak to musíte obejít všechno. Tento způsob se dá jednoduše přibalit do aplikace, pokrýt testy a mít jistotu :)

Michal Hlavac 1.3.2016 10:17

Preco nestaci vlozit certifikat do $JAVA_HOME/jre/lib/security/cacerts ?