; Date: Wed Apr 12 2017

Tags: HTTPS »»»» SSL »»»» Lets Encrypt

The search engines and browser makers are telling us to encrypt all websites. A driving factor is to protect everyone from not only miscreants wanting to hijack the web for nefarious goals, but the government security agencies who are snooping into everything. If everything on the Web is encrypted, then we'll all be better off. Until Let's Encrypt came along, the requirement to encrypt carried with it a high cost of paying for SSL certificates, and therefore many website owners would be unable to keep going. The free Lets Encrypt service opens HTTPS up to regular folk, allowing all website owners to encrypt their web traffic irregardless of how deep their pockets are. With that in mind, let's look into what it takes to set up HTTPS using Let's Encrypt.

My immediate goal is to add HTTPS to the web service operated by my client. It is a Spring/Hibernate based website running under Tomcat, and using Bootstrap for the front-end. This website has been operating for 2+ years, and we now want to add HTTPS.

To start, let's set up a simple website and work our way towards the end goal.

Overview

Reviewing the Let's Encrypt documentation we learn that one uses an "ACME client" to access their service. Through that client one retrieves SSL certificates after validating you own the domain in question.

"ACME" stands for Automatic Certificate Management Environment (ACME) and is a proposed IETF protocol that's to be used to "automate the process of verification and certificate issuance." The problem is current certification authorities (the companies from which you buy SSL certificates) use a variety of authentication means. The ad-hoc nature isn't such a good idea, and instead it's better to have a well organized process to procure SSL certificates. Hence, the Let's Encrypt project is part of a larger effort to simplify/streamline SSL certificate procurement.

When an ACME client requests an SSL certificate from an ACME server, the server needs to know whether the requestor actually owns the domain in question. There are several algorithms that can be used, such as installing a cryptographically secure document in a well-known location on the domain, or adding a cryptographically secure record to the DNS entry for the domain.

The client makes the request, the server issues the challenge, the client arranges things to satisfy the requirement, and if all goes well the ACME server issues the SSL certificate.

Setup

For my test I set up a simple Ubuntu 16.04 server on Digital Ocean ( https://digitalocean.com). The lowest cost VPS is $5 per month which is more than adequate to run a simple Apache instance and the ACME client.

CERTBOT The leading ACME client is Certbot. It is available on some systems through the official package managment system. However, it initially seemed that Ubuntu 16.04 didn't have Certbot in its regular repository. Therefore, I installed certbot-auto as so:

$ wget https://dl.eff.org/certbot-auto
$ chmod a+x ./certbot-auto
$ ./certbot-auto --help

I eventually found there's a PPA available allowing Certbot to be installed on Ubuntu:

$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt-get update
$ sudo apt-get install certbot

If you're using a different system see: https://certbot.eff.org/docs/install.html

Apache For the initial learning, let's not do anything fancy. We're instead going to use a "spare domain" and a bare simple website of a couple simple HTML pages. That's enough to get our feet wet.

$ sudo apt-get update
$ sudo apt-get install apache2

Then we can install MySQL and PHP fairly simply:

$ sudo apt-get install mysql-server php5-mysql
$ sudo mysql_install_db
$ sudo mysql_secure_installation
$ sudo apt-get install php7.0 php7.0-cgi php7.0-curl php7.0-gd  php7.0-json php7.0-mysql php7.0-sqlite3
$ sudo apt-get install libapache2-mod-php7.0
$ sudo apt-get install php7.0-mcrypt
$ sudo service apache2 restart

In /etc/apache2/mods-enabled/dir.conf, change the DirectoryIndex line so index.php appears first, like so:

<IfModule mod_dir.c>
    DirectoryIndex index.php index.html index.cgi index.pl index.xhtml index.htm
</IfModule>

Virtual Hosting Out of the box, the Ubuntu config for Apache gives us a 000-default.conf website. We want to explore configuration for multiple domains using Apache's virtual hosting support.

First, make the WEBROOT directory that will contain the HTML files:

$ sudo mkdir /var/www/DOMAIN-NAME.TLD
$ sudo cp -r /var/www/html/* /var/www/DOMAIN-NAME.TLD

The second line simply copies the simple HTML demo website provided by Ubuntu. We can also create a phpinfo.php script in the directory containing: <?php phpinfo(); ?> because this will be useful to explore the server setup when we have multiple domains on the site.

Next we copy the 000-default.conf file to make a file describing our website.

$ sudo cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/DOMAIN-NAME.TLD.conf

We need to edit that file to change a few settings:

<VirtualHost *:80>
    ServerAdmin webmaster@DOMAIN-NAME.TLD
    ServerName DOMAIN-NAME.TLD
    DocumentRoot /var/www/DOMAIN-NAME.TLD
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Make sure the settings at least match the above.

Then enable the site: $ sudo a2ensite DOMAIN-NAME.TLD

You should have already configured the DNS for DOMAIN-NAME.TLD to have A record(s) pointing at this server.

Once the DNS change settles out through the network, you can visit http://DOMAIN-NAME.TLD to verify it connects to the correct place. Visiting http://DOMAIN-NAME.TLD/phpinfo.php will give you certainty because it shows the pathnames for files and such.

Notice that we're visiting http:// because we've not set up HTTPS yet.

You can repeat the above for as many domains as you wish to experiment with. I have several spare domains, and did it with three of them to make sure I had the process down.

Using Certbot with Apache authentication

The simplest method is to use: # ./certbot-auto run

This runs through the entire process of validating ownership of the domain you enter, to installing certificates, and then configuring Apache with the right config file. Looking in /etc/apache2/sites-enabled you'll see a second config file: DOMAIN-NAME.TLD-le-ssl.conf and that changes were made to DOMAIN-NAME.TLD.conf

In DOMAIN-NAME.TLD.conf you'll find this added:

RewriteEngine on
RewriteCond %{SERVER_NAME} =DOMAIN-NAME.TLD
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

This causes the browser to automatically switch to the https:// server if the http:// server is accessed.

In DOMAIN-NAME.TLD-le-ssl.conf you'll find these settings:

<IfModule mod_ssl.c>
<VirtualHost *:443>
    ServerName DOMAIN-NAME.TLD
    ServerAdmin webmaster@DOMAIN-NAME.TLD
    DocumentRoot /var/www/DOMAIN-NAME.TLD
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
    SSLCertificateFile /etc/letsencrypt/live/DOMAIN-NAME.TLD/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/DOMAIN-NAME.TLD/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

It's also possible to use these commands:

# ./certbot-auto certonly --apache  -w /var/www/DOMAIN-NAME.TLD -d DOMAIN-NAME.TLD
# ./certbot-auto install --apache

The options to use for the install command aren't entirely clear.

There are two tests to check whether you have this correctly set up. Visit https://DOMAIN-NAME.TLD/phpinfo.php and look in the browser location bar to ensure it has the green lock icon. If there's a problem might warn you of an insecure setup, and require you agree to an exception. Once you've accessed phpinfo.php look at its output to ensure the file came from the correct directory, it understand the domain correctly, and so forth.

Using Certbot with the Webroot authentication method

Certbot offers several authentication methods. With the Apache method, it automatically set up the configuration files for us. Since my goal is setting up HTTPS with a Tomcat-based Spring website, I need a different authentication mechanism.

The Webroot method involves writing a challenge file to a well-known directory within the Webroot. The goal is so that the Let's Encrypt server can request a well-known URL to verify you could upload the data file it provided.

The URL requested might be: http://DOMAIN-NAME.TLD/.well-known/acme-challenge/wopbIsTwVp472g33z0MxtqG6IyERIOFkM-nL_mo-7ZU

Preparation Set up a new site on your server using the http:// config shown above. Verify that it works for HTTP requests.

Procure SSL certificate Then run this command:

# ./certbot-auto certonly --webroot -w /var/www/DOMAIN-NAME.TLD  -d DOMAIN-NAME.TLD

Notice we use --webroot rather than --apache. This is how one selects between the authentication algorithms. If the challenge is successful your certificates will land in /etc/letsencrypt/live/DOMAIN-NAME.TLD/

It is now possible to manually set up the Apache config for the HTTPS version of the website.

$ sudo cp /etc/apache2/sites-available/000-default-le-ssl.conf /etc/apache2/sites-available/DOMAIN-NAME.TLD-le-ssl.conf

Then edit the latter file as appropriate, and enable the site: a2ensite DOMAIN-NAME.TLD-le-ssl

You then run the same tests as before, visit https://DOMAIN-NAME.TLD/phpinfo.php and look at the details.

Summary While this didn't get an SSL certificate with a Tomcat-based app, it got us a step closer. The verification occurs within a directory on the file system that corresponds to a URL. All that's required is to implement that connection in a Tomcat-based app.

Setting up Tomcat on Ubuntu with Let's Encrypt SSL certificates

I'm not ready to offer a solution for this, but we have the sketch of it in our hands.

At community.letsencrypt.org t how-to-use-the-certificate-for-tomcat 3677 39 we see that a Tomcat Connector can be defined easily with the SSL certificates we get from Let's Encrypt.

Most of the guides for SSL with Tomcat say it's required to convert the PEM certificate from Let's Encrypt into a PKCS12 keystore. For example at dzone.com articles spring-boot-secured-by-lets-encrypt we're told to do this:

$ openssl pkcs12 -export -in fullchain.pem \
                 -inkey privkey.pem \
                 -out keystore.p12
                 -name tomcat \
                 -CAfile chain.pem \
                 -caname root

This would be done while in the /etc/letsencrypt/live/DOMAIN-NAME.TLD directory. However, beginning with Tomcat v8.5.3 this step is not required, according to the discussion on the community.letsencrypt.org post linked above. Instead you can now refer to the PEM files directly.

That would secure the application once you have the certificates. But how do you get the certificates?

The tutorials I've found suggest to use the standalone authentication scheme in Certbot. That method creates a tiny webserver that handles the protocol and snarfs down the certificates. That should work, but then how do you handle renewals? It's also possible to configure Apache or Nginx as a reverse proxy, and then use Certbot against those webservers.

The option I see is that in our xyzzy-servlet.xml file we have this line:

<mvc:resources location="/resources/" mapping="/static/**"></mvc:resources>

This causes a directory, resources to be served at the /static path. Therefore we could define another instance so that some known directory is served at the /.well-known path. At that point it should be possible to use the webroot method as shown above.

But, I don't have that all worked out yet. I'll update this when I work it out.