Using Letsencrypt for TLS

In previous posts I pointed outwhy TLS is important, how to configure Apache toscore an A+ and how totune HTTP headers. All this is dependent on getting an SSL cert.

Some jargon explained

Before we delve into a “how to”, some basic jargon should be explained:

SSL/TLS

TLS (“Transport Layer Security”) is the successor to SSL (“Secure Socket Layer”). SSL was created by Netscape in the mid 90s (I remember installing “Netscape Commerce Server” in 1996). When you hear people talk about SSL and TLS they pretty much use the terms interchangeably. You can think of it as a progression SSL2, SSL3, TLS1.0, TLS1.1, TLS1.2 with each version better than the previous.

SSL/TLS is what makes the connection to the server secure.

SSL Certificate

This is a file you place on your server and this basically says “ I am server x.y.z ”. The server for this website says “ I am server www.sweharris.org ”. There are other things this certificate can say (” I can also be called sweharris.org ”). Different certificates can also be used for other reasons (e.g. signing code), but I won’t go into that here.

The SSL certificate is signed by a trusted party (the “Certificate Authority” - CA), so when your browser goes to a web site it can verify the certificate is valid. Your browser has a list of CAs that the browser creator trusts. If I tried to create a certificate for “google.com” then no CA would sign it, and your browser would say “Nope!”

Domain Validation

In order to get a certificate signed the CA needs to know that you are who you claim you are. “Domain Validation” (DV) is the weakest and easiest method of validating ownership. This may be done by sending an email to a standard address (eg postmaster or hostmaster ). It might be done by requiring you to put a specific file on the web server. Or changing a DNS entry… the idea is to require the requester to prove they have access to a controlled resource associated with the domain.

This techique is used elsewhere; e.g. if you put your website into Google Webmaster Tools then it will require you to put a specific entry on your server (eg /googleSOMESTRING.html ). If you want to use Google Apps for your mail then it may want a DNS entry (eg googleSOMESTRING IN CNAME google.com. ). These are both “domain validation” examples.

Extended Verification

Extended Validation (EV) certs take a deeper approach to validating the request; they also check the owner claims. So, for example, I might be able to get a DV cert for chas3.com , but I would not be able to get an EV cert saying “this is owned by JPMorgan Chase”. An EV cert is meant to help reduce phishing attacks by helping the web user spot if an EV cert is in use.

Compare my site to Chase’s:

We both have the green padlock, showing the cert’s are good, but the Chase entry has more details about the owner next to the padlock.

For a site like mine a DV cert is sufficient. You need to decide if you want a DV or EV for your site.

Getting a cheap cert

When I first started out in 1996 getting a cert was a hard tedious process. We had to fax (fax!) details to one of the very few CAs that were around, and it took weeks to get the cert. Painful. It was effectively an EV process, long before such a thing really existed.

As time went on the DV process became easier to get, but they still cost money. It wasn’t really worth it for most people.

In 2012 I noticed that StartCom were providing free DV certs, under the brand StartSSL. This was an Israeli company and their CA was a common one. It made sense.

The StartSSl process was pretty painful. It was totally manual and involved using a web site with cut’n’paste. The DV process expired after 30 days so every renewal required revalidation. But it was free, and I only had 7 certs with them, so I could jump through the hoops every so often.

Unfortunately, it’s not clear if StartSSL is still trustworth , and may now operate from China .

Letsencrypt

Fortunately Letsencrypt started up. Their goal was to automate a lot of the manual steps, while still providing free DV certs. They even provide a client so you can basically say “encrypt my site” and their software will create the necessary cert files, do the DV process (stick a file on the server), retrieve the signed result, reconfigure your webserver daemon and, boom, in seconds your site is now SSL enabled. The certificate is only valid for 90 days, but that’s not a problem; the same software can automatically renew the cert before it expires.

Very impressive.

Unfortuntately it doesn’t work for me. My site is replicated across multiple providers (one copy at Linode in Texas; one copy at Panix in New York; one copy at OVH in Canada). I manage content on my home machine and rsync the results out. I also manage the webserver configurations via ansible .

If I ran their client on one machine then I’d have to copy the resulting certs to the other two… it’s doable, but messy.

However there was another way of doing the DV certification, and that’s via DNS. Now my DNS is also replicated (hidden master server at home; public masters at the three sites). So this means I could do the cert DV validation via DNS from my home machine, then push the resulting cert out to the web servers via ansible .

The original Letsencrypt client couldn’t do DNS validation, but Lukas Schaur created an alternative shell based client called deydrated . This is a much simpler program, but allowed DNS mode to be used, and had a call out so I could modify how the DNS change ran.

How it works

Last week one of my certs needed renewing, so I went through the renew process:

$ cd etc/localnet/Letsencrypt/ $ ./refresh + Valid till Dec 6 23:35:00 2016 GMT (Less than 8 days). Renewing! + Signing domains... + Generating private key... + Generating signing request... + Requesting challenge for linode.spuddy.org... + Responding to challenge for linode.spuddy.org... + Challenge is valid! + Requesting certificate... + Done! + Creating fullchain.pem... /home/sweh/etc/localnet/PlayBooks/Files/Postfix/Per-host/linode

The refresh command calls ./dehydrated -c to do all the hard work of talking to the Letsencrypt servers. It noticed that the linode cert needed renewing, and got a new signed cert. My script then compared all the managed certs and copied any changed ones into the ansible tree. In this case the single cert for the linode Postfix server (also used by INN) was updated.

Since a change was done I could then use ansible to push it out:

$ cd ../PlayBooks/ $ ansible-playbook -l linode postfix-inn.yml PLAY [Set up postfix config on mail servers] ********************************** [...] TASK: [send machine specific certs] ******************************************* changed: [linode] => (item=privkey.pem) changed: [linode] => (item=fullchain.pem) NOTIFIED: [restart postfix] *************************************************** changed: [linode] PLAY [Set up INN] ************************************************************* [...] TASK: [send INN privkey] ****************************************************** changed: [linode] TASK: [send INN cert chain] *************************************************** changed: [linode] [...] PLAY RECAP ******************************************************************** linode : ok=21 changed=4 unreachable=0 failed=0

And that’s it! I can also verify the new certs are in place and working

$ cd ../Letsencrypt $ ./check_remote nntp linode.spuddy.org linode.spuddy.org NNTP: OK notAfter=Feb 27 16:31:00 2017 GMT $ ./check_remote smtp linode.openvpn linode.spuddy.org linode.spuddy.org SMTP: OK notAfter=Feb 27 16:31:00 2017 GMT How to know when to renew a cert

It would be bad to allow a cert to expire, so you need ample warning of when a cert needs to be refreshed. Because of the tree used by the dehydrated code is consistent this is very easy to script:

#!/bin/ksh -p # Warn if cert will expire in this many days let DAYS=${DAYS:-7} let warn=DAYS*86400 typeset -R25 h bad="" cd /home/sweh/etc/localnet/Letsencrypt/certs || exit for a in */cert.pem do b=$(openssl x509 -checkend $warn -in $a) if [ "x$b" = "xCertificate will expire" ] then b=$(openssl x509 -noout -dates -in $a | sed -n 's/^notAfter=//p') h=${a%/*} bad="$bad\nletsencrypt $h: $b" fi done if [ -n "$bad" ] then print Following certificate expiration dates: print "$bad" fi

So I can now see what certs are due to expire:

$ checkssl $ DAYS=20 checkssl $ DAYS=25 checkssl Following certificate expiration dates: letsencrypt bofh.spuddy.org: Dec 28 23:46:00 2016 GMT letsencrypt gallery.spuddy.org: Dec 28 23:47:00 2016 GMT letsencrypt games.spuddy.org: Dec 28 23:47:00 2016 GMT letsencrypt home.spuddy.org: Dec 28 23:47:00 2016 GMT letsencrypt mercury7.spuddy.org: Dec 28 23:48:00 2016 GMT letsencrypt panix-news.spuddy.org: Dec 28 23:48:00 2016 GMT

This is easy to put into a cron job. Indeed

1 1 * * * /home/sweh/bin/checkssl

In the morning I can refresh any certs necessary while drinking my coffee :-)

Automation

So far I’ve only automated 90% of the job. I still have to run the refresh and ansible-playbook commands. Partly this is because it’s so easy, partly because it gives me positive feedback (I know the work was done and the push successful), but mostly because this is “good enough” for my use case.

When I started using this process I would occassionally get errors with the DV step; I’m not sure if that was due to DNS propagation issues or caching at the LetsEncrypt side or something else. I haven’t had a problem for quite a while now, but I like to keep my eye on it.

But we can see, quite easily, how a daily “refresh/push” process could be created so the overnight “checkssl” job could become a “check and refresh” job.

Currently I have 12 certificates managed this way, and it takes less than 5 minutes a month to manage.

Summary

Where DV certs are sufficient, a tooling around Letsencrypt is perfectly adequate and sensible. It can even be used for internal web sites, as long as DNS challenge/response can be handled.

There’s little excuse to not use SSL on web sites any more!