david wong

Hey! I'm David, cofounder of zkSecurity and the author of the Real-World Cryptography book. I was previously a crypto architect at O(1) Labs (working on the Mina cryptocurrency), before that I was the security lead for Diem (formerly Libra) at Novi (Facebook), and a security consultant for the Cryptography Services of NCC Group. This is my blog about cryptography and security and other related topics that I find interesting.

Let's Encrypt Overview posted June 2015

letsencrypt

What is Let's Encrypt?

Basically, it's a way to get a quick x509 certificate for your server without knowing much about what is a x509 certificate:

You have a website. You want people to be able to log in on it from starbucks without the guy sitting at a near table reading the passwords in clear from the packets you're sending everyone around you. So you google a few random keywords like "secure website https" and you end up with a bunch of links and services (which may not be free) and you suddenly have to understand what are certificates, PKI, x509, public-key cryptography, RSA, FQDN, haaa.... Well, worry no more, Let's Encrypt was thought so that you wouldn't have to go through all that trouble and destroy your server on the way.

Oh and yeah. It's for free. But is hasn't been released yet.

How does it work?

You can learn more reading their technical overview, or some of their videos... or read my tl;dr:

  1. You download their program on your server that has the address www.example.com:
sudo apt-get install lets-encrypt
  1. You run it as sudo telling it you want to get a certificate for your domain
lets-encrypt example.com

And voila.

Behing the curtains this is what happens:

ACME

  1. lets-encrypt will generate a pair of RSA private key/public key and contact the CA with your public key.

  2. The CA will register your public key

  3. The program will then ask the CA to verify your domain.

  4. The CA will answer with a set of challenges. These are some tasks you can complete to prove to the CA you own that domain. One of the common one is to upload a certain file at a certain address of that domain.

  5. The program you installed will then do that for you and poll the CA for a confirmation

  6. The CA will tell you "OK man, all is good".

  7. The program will then generate another long term pair of private key/public key, generate a CSR (Certificate Signing Request) with the new public key and send that CSR to the CA.

  8. The CA will extract the information, create a beautiful x509 certificate, sign it and send it back to you.

  9. lets-encrypt will install the certificate on your server and set certain options (or not) to force https

By the way, the certificate you will get will be a DV certificate, meaning that they only verified that you owned the domain, nothing more. If you want an EV certificate this is what you will have to go through (according to wikipedia):

  • Establish the legal identity as well as the operational and physical presence of website owner;
  • Establish that the applicant is the domain name owner or has exclusive control over the domain name; and
  • Confirm the identity and authority of the individuals acting for the website owner, and that documents pertaining to legal obligations are signed by an authorised officer.

But how does it really work?

So! The lets-encrypt program you run on your server is open sourced here: https://github.com/letsencrypt/lets-encrypt-preview. It is called lets-encrypt-preview I guess because it isn't done yet. It's written in Python and has to be run in sudo so that it can do most of the work for you. Note that it will install the certificates on your server only if you are using Apache or Nginx. Also, the address of the CA is hardcoded in the program so be sure to use the official lets-encrypt.

The program installed on the CA is also open sourced! So that anyone can publicly review and audit the code. It's called Boulder and it's here: https://github.com/letsencrypt/boulder and written in Go.

Lets-encrypt and Boulder both use the protocol ACME for Automated Certificate Management Environment specified here as a draft: https://letsencrypt.github.io/acme-spec/

ACME

ACME spec

ACME is written like a RFC. It actually wants to become an RFC eventually! So if you've read RFCs before you should feel like home.

The whole protocol is happening over TLS. As a result the exchanges are encrypted, you know that you are talking to the CA you want to talk to (eventhough you might not use DNSSEC) and replay attacks should be avoided.

The whole thing is actually a RESTful API. The client, you, can do GET or POST queries to certains URI on the CA webserver.

Registration

The first thing you want to do is register. Registration is actually done by generating a new pair of RSA keys and sending them the public key (along with your info).

ACME specifies that you should use JWS for the transport of data. Json Web Signature. It's basically Json with authentication (so that you can sign your messages). It actually uses a variant of JWS called Jose that doesn't use a normal base64 encoding but that's all you should know for now. If you really want to know more there is an RFC for it.

So here what a request should look like with JWS (your information are sent unencrypted in the payload field (but don't worry, everything is encrypted anyway because the exchange happens over TLS)):

POST /acme/new-registration HTTP/1.1
Host: example.com

{
 "payload":"<payload contents>",
 "signatures":[
  {"protected":"<integrity-protected header 1 contents>",
   "header":<non-integrity-protected header 1 contents>,
   "signature":"<signature 1 contents>"}
}

Boulder will check the signature with the public key you passed, verify the information you gave and eventually add you to its database.

Once you are registered, you can perform several actions:

  • Update your infos
  • Get one domain authorized (well actually as many as you'd like)

The server will authenticate you because you will now send your public key along AND you will sign your requests. This all runs on top of TLS by the way. I know I already said that.

Boulder's guts

boulder

Boulder is separated in multiple components, this makes the code clearer and ensure that every piece of code does what it is supposed to do and nothing more.

One of the components is called the Web-Front-End (WFE) and is the only one accepting queries from the Client. It parses them, verifies them and passes them to the Registration Authority (RA) that combines the other authorities together to produce a response. The response is then passed back to the WFE and to the client over the ACME protocol. Are you following?

TWe'll see what other authorities the RA has access to in the next queries the client can do. But just to talk about the previous query, the new registration query, the RA talks to the Storage Authority that deals with the database (Which is currently SQLlite) to register your new account.

All the components can be run from a single machine (for the exception of the CA that runs on another library), but they can also be run seperately from different machines that will communicate on the same network via AMQP.

New Authorization

Now that you are registered, you have to validate a domain before you can request a certificate.

You make a request to a certain URI.

Well to be exact you make a POST request to /new-authz, and the response will be 201 if it works. It will also give you some information about where you can go next to do stuff (/authz)

Here's the current list of API calls you can do and the relevant answers.

API calls

The server will pass the info to the Policy Authority (PA) and ask it if it is willing to accept such a domain. If so, it will then answer with a list of challenges you can complete to prove you own the domain, along with combinations of accepted challenges to complete. For now they only have two challenges and you can complete either one:

  • SimpleHTTPS

  • DVSNI

If you choose SimpleHTTPS the lets-encrypt client will generate a random value and upload something at the address formed by a random value the CA sent you and the random value you generated.

If you choose DVSNI, the client will create a TLS certificate containing some of the info of the challenge and with the public key associated to his account.

The client then needs to query the CA again and the CA's Validation Authority (VA) will either check that the file has been uploaded or will perform a handshake with the client's server and verify that specific fields of the certificates have been correctly filled. If everything works out the VA will tell the RA that will tell the WFE that will tell you...

After that you are all good, you can now make a Certificate Signing Request :)

New Certificate

The agent will now generate a new pair of private key/public key. And create a CSR with it. So that your long term key used in your certificate is not the same as your let's encrypt account.

CSR

a CSR example, containing the identifier and the public key

After reception of it, the Web-Front-End of Boulder will pass it to the Registration Authority (RA) which will pass it to the Certificate Authority (CA) that will do all the work and will eventually sign it and send it back to the chain.

1: Client ---new-cert--> WFE
2:                       WFE ---NewCertificate--> RA
3:                                                RA ---IssueCertificate--> CA
4:                                                                          CA --> CFSSL
5:                                                                          CA <-- CFSSL
6:                                                RA <------return--------- CA
7:                       WFE <------return------- RA
8: Client <------------- WFE

Oh and also. the CA is talking to a CFSSL server which is CloudFlare's PKI Toolkit, a nice go library that you can use for many things and that is used here to act as the CA. CFSSL has recently pushed code to be compatible with the use of HSM which is a hardware device that you HAVE to use to sign keys when you are a CA.

After reception the lets-encrypt client will install the fresh certificate along with the chain to the root on your server and voila!

You can now revoke a certificate in the same way, but interestingly you won't need to sign the request with your account key, but with the private key associated to your certificate's public key. So that even if you lose your agent's key you can still revoke the certificate.

Other stuff

There are a bunch of stuff that you will be able to do with the lets-encrypt client but that haven't been implemented yet:

  • Renew a certificate
  • Read the Terms of Service
  • Query OCSP requests (see if a certificate has been revoked) ...

Moar

This post is a simplification of the protocol. If you want to know more and don't want to dig in the ACME specs right now you can also take a look at Boulder's flow diagrams.

key ceremony

If you've followed the news you should have seen that Let's Encrypt just generated its root certificate along with several other certificates: https://letsencrypt.org/2015/06/04/isrg-ca-certs.html

This is because when you are a CA you are suppose to keep the root certificate offline. So you sign a (few) certificate(s) with that root, you lock that root and you use the signed certificate(s) to sign other certificates. This is all very serious because if something goes wrong with the root certificate, you can't revoke anything and well... the internet goes wrong as a result (until vendors start removing these from their list of trusted roots).
So the keys for the certificate have to be generated during a "ceremony" where everything is filmed and everyone must authenticate oneself at least with two different documents, etc... Check Wikipedia's page on Key Ceremony it's "interesting".

Also, I received a note from Seth David Schoen and I thought that was an interesting anecdote to share :)

the name "Boulder" is a joke from the American children's cartoon Looney Tunes:
https://en.wikipedia.org/wiki/Wile_E._Coyote_and_The_Road_Runner
This is because the protocol that Boulder implements is called ACME, which was also the name of the company that made the products unsuccessfully used by the coyote to attempt to catch the roadrunner. https://en.wikipedia.org/wiki/Acme_Corporation
Two of those products were anvils (the original name of the CA software) and boulders.

Well done! You've reached the end of my post. Now you can leave a comment or read something else.

Comments

Freddy

I am perturbed that I must install this "lets-encrypt" application on my server, I am perturbed that it must be run as root, and I am perturbed that it may start affecting my Apache and/or nginx configurations.

While I do wish for HTTPS everywhere, so much about this perturbs me that I feel so very uneasy!

david

@Freddy: it seems like it's a global concern. But all the process is pretty open, made by people who know what they're doing (and supported by many trusted companies/organizations).

It is also aimed toward people who don't know much and might do way worse than that if they try doing it themselves. All in all, this wouldn't work without being run as root. And this seems like the best way to make _more_ people setup https on their server, and to avoid collateral damage by letting novices trying to setup https on their server.

You are free to not use this service. As I am. But if someone who doesn't know much about all of this comes to me and ask me for advice, I think I'll be more reassured sending him towards lets encrypt than letting him do everything by himself.

Marc

What if I host multiple domains on one server, perhaps even managed by Plesk or something similar - would lets-encrypt handle that without problems?

Freddy

@david: I do think the intent of this initiative is good natured; I truly do! But like other things Mozilla has done lately, the good intent is negated by how it is implemented.

I believe that the Australis user interface changes to Firefox were meant to improve it. But they didn't. They made the Firefox user experience worse.

I believe that we are seeing the same thing with Let's Encrypt. The intent is good, but the hoops through which we must jump to get to the end result are just not acceptable in so many ways!

Mozilla's push to deprecate HTTP (citation: [1]) is one more example of this at work. Encouraging the use of HTTPS is a very excellent thing to encourage, but forcing it upon unwilling users is not beneficial!

The real users are being forgotten by these software systems and initiatives. Firefox's user interface was changed to improve the experience for some mythical "average" user; but usability suffers for all. Let's Encrypt is designed to be usable by "people who don't know much", yet these people aren't running their own web servers; they're using managed hosting of some sort.

We need to see these initiatives done in a way that achieves their worthy goals, but without so much pain and suffering in order to get to the final destination.

[1] https://blog.mozilla.org/security/2015/04/30/deprecating-non-secure-http/

Freddy

To clarify my previous comment, when I wrote "Mozilla's push to deprecate HTTP" I am referring to unencrypted nonsecure HTTP, not HTTP altogether. I am talking of URLs starting with http:// not the protocol!

alex

@Freddy: It needs to run as root because you need to prove to the CA that you control ports<1024 on the relevant domain. You can't just allow every user to get a valid certificate.

Svein

@Freddy: You don't need to run it at all. The protocol is open; the program only automates the steps, but you can do them manually if you want, and it'll be no harder than getting a certificate is right now.

HÃ¥kan

I don't find this reference(?) implementation in itself to be the most interesting part or, for that matter, that we should judge the whole initiative based on it. This implementation really comes across as more of a proof of concept than anything else.

Having the protocol in place and a CA that works with it, however, could lead to pretty cool stuff!

The messy part that previous comments complain about seem to just be a result of the lets-encrypt program being an entirely independent thing from the program that actually needs the certificate. What if, for instance, instead you had a libacme (or whatever name makes sense and is actually available) that for instance an nginx module could use and then you'd just enable some acme-auto-certificate option for the relevant vhosts and you'd be done. Or, better yet, what if it was on by default unless you configured a certificate of your own?

To my understanding the goal of the project is not so much about helping people that do not know how to set up TLS to do so (as has been pointed out, they generally do not directly deal with this stuff anyway), but rather to change the general mindset from "do I really need TLS?" to "why wouldn't I use TLS?".

stano

As a hosting provider i am interested if i will be able to use it to create automatically certificates for our customers, so when they buy hosting they will have signed ssl for free immediately. I am talking about thousands of existing domains, and 10..100 new per day.
How will be common/subjaltname ? only domain.com/www.domain.com or it will be possible for wildcard domain.com/*.domain.com ?

fanboy

Hakan hit the nail on the head. Freddy, Apache, Nginx and Bob's-favorite-JDK-project can all implement their own version of the Let's Encrypt client. Hopefully they will. The awesome point of this is to have a free CA with a clean, modern API that will grant certificates automatically based on appropriate validation so HTTPS can be the default for people who can barely use WordPress and

Also, this opens the door for rapid rotation of certs. If the maintenance of the cert is automatice, why not have the program rotate it 15 days before expiration and expire the cert every 2-3 months? Until 'something new' replaces the flawed PKI infrastructure we rely on now, this is a huge step forward, IMESHO.

fanboy

@stano: from what's been said in their chat room they will support SANs so if a hosting provider can work with SNI they will be able to cover multiple domains on a single cert and IP. They won't be doing wildcards. It sounds like they are absolutely planning to work with hosting providers so it can be integrated. I'd ping the leadership of the project so they can plan with you before you do a massive rollout, though.

david

@fanboy: exactly, I thought about expiring certs after a week. But being far from that kind of discussion I have no real idea if that's actually a good idea for PKI in general...

schoen

@Freddy: the reason for updating your web server configuration is that we've been told that most users want that, often because they don't know how to do it! If you know how to do it and you don't want that, you don't have to have the client update your web server configuration; for example, you can use standalone mode and save the cert in a file that you can then use as you wish.

From what we've been told, it's extremely common for sysadmins to wish for a tool that automates obtaining, deploying, and renewing the cert -- so we're developing such a tool. Automating the process will inherently require running code as a privileged user in order to change configurations. Sysadmins are also welcome to use less-automated and more manual options to get and deploy our certs.

Anonymous

Why does this only work for Nginx/Apache? Is there not a way to distribute certificates to a more general audience?

Anonymous

Oh never-mind, I misunderstood. The certs are only installed for Nginx/apache. That only implies that one using software other than Nginx/Apache would have to install/configure their server to use the certificates themselves, rather than use the automated service. Sorry for my previous misleading post! The product looks great. :)

Anonymous

Arriving Mid-2015? Their web site says September. That's stretching the mid bit quite a bit!

Martijn

I'm grateful for the work by the Let's Encrypt team. An automated and free CA is much needed and their open approach is the best way to do it. I think we'll see lots of implementations of the protocol into various frameworks soon, making it ever easier to get a certificate installed.

Benjamin

So once I have these certs encrypting my apache webpages, can I use the same certificate for my Dovecot IMAP server? I am currently using snakeoil certs and think let's encrypt would be much better.

Dean

Question about the renewal process: Do we need to request and complete challenges when renewing a Let's Encrypt certificate?

I read carefully:
1. The "How it Works" page:
https://letsencrypt.org/how-it-works/
2. This post
3. The "Boulder Flow Diagrams":
https://github.com/letsencrypt/boulder/blob/master/DESIGN.md

and still I don't see the answer documented anywhere. Do we need to prove domain ownership when renewing a Let's Encrypt certificate once our agent proves that it has control over the domain? If not, is there an expiration period (probably), after which the agent again has to request and complete challenges proving domain ownership.

Right now, I have a working/completing a script for requesting and installing Multiple Domain (SAN) certificates in Exchange server. The script works fine (it's really cool and effortless), and it's tempting to configure it every time to clear the vault (it's based on ACME POSH) and go through the authorization process from scratch; actually, even perform a new client registration, but that doesn't make sense.

Could you document better the renewal process or if I've missed it, make this information easier to find?

Paul Jaros

Thanks for the detailed information.

Now (2017-09-27) "Let's Encrypt" is quite established and very useful for "the rest of us": We are running our hackerspace-webpage with it and I use it to secure my https-access for my home hosted owncloud.

I'm a programmer gettings my hands dirty with system- and network-enginering. Thanks again for your insightful post.

Greetings from Switzerland.

Matteo

I agree with Dean, documentation about renewal could be improved.
I'm worried about a webserver that has only 443 port opened where I installed a Let's encrypt certificate: to do that I had to open temporarely port 80 otherwise ACME was unable to complete the checks.
Certificate's renewal process will use the same port 80 for its controls (forcing me to re-open everytime temporarely port 80) or will it try to work over port 443?
Thank you mate, greetings from Italy

ssl issues

Your article is great, I've read it many times and still have many feelings. I think that will visit your site more. This is a really great and informative post.

leave a comment...