Acme, a brief history of one of the protocols which has changed the Internet

Source: Hacker News

Preamble

I would like to share with you this article I wrote about the ACME protocol, which I “fell in love with” about ten years ago. It is for me a way to give back to this fantastic Free Software and Open Protocols developers community.

This article is about the roots, the conception, the standardization, the relation with its ecosystem and the evolution challenges faced by the ACME protocol.

To write this article, I had the privilege of interviewing several people who have been involved in the creation and the evolution of ACME: Aaron Gable, Sarah Gran, Jacob Hoffman‑Andrews and J.C. Jones (more below).

Thank you so much to all of you for your time and support! 💚

Internet and Network Protocols

Open and Standardized Protocols at the Heart of the Internet’s Success

During the 1990s, computing underwent a true revolution driven by the rise and global spread of the Internet. The Internet fulfilled the promise embodied in Sun Microsystems’ slogan “The Network is the Computer”.

By interconnecting individual computers, the Internet enabled its users to communicate without limits and without worrying about borders.

This unrestricted interconnection emerged at a pivotal moment in modern history: the opposition between the West and the Eastern Bloc led by the USSR had—albeit temporarily, as we now know—faded away, China was becoming the world’s factory, and the movement and collaboration between people were much freer and open than ever.

The Internet supported a kind of utopia of instant communication and sharing, previously unknown. This utopia was made possible by a set of open and standardized protocols. This was the key to enabling all kinds of different systems to cooperate and communicate seamlessly.

There were, of course, isolationist or monopolistic temptations from certain manufacturers or software editors. But open and standardized protocols ultimately prevailed, enabling unprecedented expansion. Built on top of IP, TCP, UDP, and DNS, among others, the HTTP and HTML duo would propel the Web as the Internet’s preferred communication platform for the next 30 years.

Limited Use of Encryption

The success of this communication utopia was achieved without much concern for ensuring authentication, integrity, and confidentiality of exchanges.

In 2015, only ~40% of websites used encryption. The consequences of this negligence in addressing security risks were confirmed by Edward Snowden’s revelations in 2013: our data was exposed to anyone who wanted and could intercept and collect it.

Let’s Encrypt is coming

When asked about the main obstacles to the widespread adoption of encryption, J.C. Jones, one of the architects of Let’s Encrypt and now one of its site reliability engineers after leading Firefox’s cryptographic team, responds:

“More and more information was flowing across the Web, and most data being transferred did not have integrity or confidential protections from TLS. The biggest stumbling block to using TLS everywhere was obtaining and managing server‑side certificates, and so: Let’s Encrypt” – J.C. Jones

Obtaining a certificate was the main obstacle, and this was the priority to address.

This view was shared by a group of partners who, starting in 2013, pooled resources to establish Let’s Encrypt, an automated and free certificate authority. Sarah Gran, VP of Advancement at Let’s Encrypt, shares:

“Early collaborators included people from Mozilla, Electronic Frontier Foundation, Akamai, Cisco, and the University of Michigan” – Sarah Gran

And that’s how Let’s Encrypt was born.

In the Web ecosystem, certificate authorities are organizations from which you can obtain a certificate for a domain after proving you control it.

Since 2015, Let’s Encrypt has been a certificate authority that delivers free (as in free beer) TLS server certificates. On the legal/administrative side, Let’s Encrypt operates for the public’s benefit and is a service provided by the Internet Security Research Group (ISRG), a California public benefit corporation.

Regarding Let’s Encrypt results ten years after its birth, they are impressive (over 700 M active certificates, over 60% of all public TLS server certificates). As Sarah Gran points out, global HTTPS usage has also risen dramatically:

“When we started issuance, only about 39% of website visits were HTTPS. Today, it’s nearly 95% in the United States, and over 83% globally. We still have work to do, but we are proud of the progress we’ve made over the last ten years” – Sarah Gran

Let’s Encrypt delivers certificates in an automated manner using the ACME protocol, which implies no manual action from the site owner nor the certificate authority. So, let’s speak now a little about the automation aspect!

Automation: The Core of the Operation

From the mid‑2020s perspective, the automation at the heart of Let’s Encrypt might seem obvious, but in the first half of the 2010s, it was far from the norm. The ecosystem of public certificate authorities issuing server certificates was no exception.

At first glance, automation appears to be there to help website managers reliably deploy the TLS protocol on their sites, but it was first and foremost an absolute prerequisite for the very viability of the Let’s Encrypt project.

As Aaron Gable, tech lead of Boulder—the software at the core of Let’s Encrypt—, confirms:

“Automation was always going to be critical to Let’s Encrypt’s success. From the very beginning, we knew that there was no way we could scale manual validation on a non‑profit’s budget” – Aaron Gable

Indeed, Let’s Encrypt has operated on an Internet scale from the start with a small team of about fifteen engineers, or even fewer at launch. For this team, automation was the only viable way to fulfill the immense mission they had set for themselves.

ACME

The Open and Automated Protocol That Powers Let’s Encrypt

When we talk about automation in relation to Let’s Encrypt, we are talking about ACME (Automated Certificate Management Environment).

This protocol allows client software to prove to an ACME‑compatible certificate authority that it controls the domain for which it is requesting a certificate.

Sarah Gran clarifies an important point:

“An important aspect of how Let’s Encrypt works is that we verify control over a domain, not ownership” – Sarah Gran

Control vs. ownership of a domain—a nuance everyone should keep in mind.

The proof of control involves the client responding to a challenge issued by the ACME‑compatible certificate authority. The challenge can be an HTTP, DNS, or TLS challenge, depending on the client’s choice and the authority’s support. Completing the challenge requires the ACME client to place a value provided by the ACME server—in a standardized HTTP path, a DNS zone, or a TLS response, respectively. All of these operations involve cryptography.

The key point with ACME is that this entire dialogue between the client and the ACME server is executed without any human intervention, enabling the automatic issuance of certificates and their deployment.