WolfSSL Sucks Too, So Now What?

Source: Hacker News

Cryptographic Library Landscape – Quick Thoughts

• OpenSSL – Generally reliable, but can feel heavyweight for some projects.
• BoringSSL (Google) – Tailored to Google’s internal needs; not always a perfect fit for external use cases.
• AWS‑LC (Amazon) – Optimized for Amazon’s services; may lack broader community focus.
• GnuTLS – Historically solid, though personal experiences with it have been mixed.
• LibreSSL – A commendable effort, but still missing some features compared to its peers.

Note: Each library has its own design goals and trade‑offs. Choosing the right one depends on your specific requirements, threat model, and ecosystem constraints.

What happened now?

Last year an article from HAProxy highlighted how terribly slow OpenSSL had become. The story circulated a few times, and I decided to help enable FreeBSD to package a variant of HAProxy built against wolfSSL.

This seemed like an easy way to give wolfSSL wider exposure, since most Linux distributions are unlikely to ship such a build. In practice, the only people running a wolfSSL‑backed HAProxy are those who know what they’re doing and compile it themselves. I haven’t checked whether Arch, Gentoo, Nix, etc., have similar packages, but they would be the most straightforward candidates for a haproxy‑wolfssl package.

The bug

I built HAProxy with wolfSSL, ran it in a few environments, and then hit a bug. I reported the issue, forgot about it, and moved on—until I encountered the same problem again. Motivated to resolve it, I reopened the bug report and dug through the code until I identified the root cause.

TLS 1.3 and middleboxes

TLS 1.3 is defined in RFC 8446. It works quite differently from TLS 1.2, which introduced a host of compatibility problems. The specification even notes:

“The design of TLS 1.3 was constrained by widely deployed non‑compliant TLS middleboxes.”

Ah, the infamous middleboxes—those invisible pieces of network infrastructure that can tamper with traffic. You often don’t know they exist until they cause you grief, and they certainly will.

Middlebox Hell

Middleboxes were practically invented in “hell,” and no amount of wish‑casting can make them disappear. (Perhaps a few Etsy witches could lend some luck, but that’s another story…)

The problem is that we want stronger security guarantees than TLS 1.2 provides, yet many middleboxes only understand TLS 1.2. Because TLS 1.3 would be broken by these devices, it must be able to pretend to be TLS 1.2.

The RFC‑defined workaround

The TLS 1.3 specification includes a Middlebox Compatibility Mode (see RFC 8446, Appendix D.4). The idea is simple but costly:

1. ClientHello – The client includes a non‑empty Session ID. This tricks middleboxes that expect a TLS 1.2 handshake.
2. Dummy ChangeCipherSpec records – Both client and server exchange placeholder ChangeCipherSpec messages that are ignored by TLS 1.3 but satisfy the expectations of legacy devices.
3. Proceed with TLS 1.3 – After the dummy exchange, the normal TLS 1.3 handshake continues.

What this means

• Compatibility – The connection can traverse middleboxes that only understand TLS 1.2.
• Latency penalty – The extra Session ID and dummy ChangeCipherSpec records add round‑trip time to the handshake.
• Security – Once past the middleboxes, the session enjoys the full security guarantees of TLS 1.3.

In short, Middlebox Compatibility Mode is a pragmatic (if inefficient) bridge that lets modern TLS 1.3 connections survive in an environment dominated by legacy middleboxes.

The Upside Down

The RFC is pretty clear about how this is all meant to play out.

Compatibility mode

• This mode is partially negotiated:

  • The client may provide a session ID (or omit it).
  • The server must echo the session ID if it is present.

• If the client sends a non‑empty session ID, the server MUST send the ChangeCipherSpec as described in the appendix.

---

WolfSSL’s stance

WolfSSL requires the library to be compiled with

-DWOLFSSL_TLS13_MIDDLEBOX_COMPAT

to enable middlebox‑compatibility functionality. This flag forces the library to be either always in this mode or never in it—there is no intermediate option.

Consequences

1. TLS 1.3 clients cannot be trusted to interoperate reliably with WolfSSL.
2. Compatibility now depends entirely on how forgiving the client implementation is, which does not inspire confidence.
3. A GitHub issue comment (linked at the end of the original text) suggests that the WolfSSL team is not prioritising RFC compliance for this feature.

There is no “middle ground” or alternative implementation for middlebox compatibility in WolfSSL: it is either RFC‑compliant or it isn’t, and the current implementation falls into the latter category.

The Plaintiff

At the moment I’ve only identified one victim of this decision, but there are likely more.

Erlang/OTP ships with its own SSL library implementation, and you can reasonably assume that the team took Joe Armstrong’s advice to heart when adding TLS 1.3 support:

“Make it work, then make it beautiful, then if you really, really have to, make it fast.” – Joe Armstrong

To protect themselves, the Erlang/OTP developers enabled middlebox_comp_mode by default (see the source line here).

• If you want maximum speed and you know it’s safe, you can turn this option off.

Unfortunately, with the default setting every Elixir/Erlang (and related) HTTP client now fails to connect to a WolfSSL HTTPS server when TLS 1.3 is available.

Where Do We Go From Here?

OpenBSD was probably right: we should focus on LibreSSL and stop trying to juggle the other TLS libraries. As HAProxy noted, it isn’t a victim of the OpenSSL 3.0 “screw‑ups” because it forked earlier, but it does miss some optimizations. That’s a fair trade‑off, and the gaps will be filled in due time.

So don’t be like me. My hubris led me to think I could get faster TLS termination for my sites, only to waste a lot of time learning something I didn’t need and then writing this blog post. You’ve been warned.

Elixir PoC

A proof‑of‑concept for Elixir 1.17.3 (compiled with Erlang/OTP 26) is as simple as the script below.

#!/usr/bin/env elixir

url = "https://some-wolfssl-endpoint"
url = String.to_charlist(url)

{:ok, _} = Application.ensure_all_started(:inets)
{:ok, _} = Application.ensure_all_started(:ssl)

:logger.set_application_level(:ssl, :debug)

http_options = [
  ssl: [
    verify: :verify_peer,
    cacerts: :public_key.cacerts_get(),
    depth: 2,
    customize_hostname_check: [
      match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
    ],
    versions: [:"tlsv1.2", :"tlsv1.3"],
    middlebox_comp_mode: true   # ← toggle this
  ]
]

options = [body_format: :binary]

:httpc.request(:get, {url, []}, http_options, options)

Expected error (when middlebox_comp_mode: true)

11:00:44.996 [warning] Description: ~c"Failed to assert middlebox server message"
     Reason: [missing: {:change_cipher_spec, 1}]

11:00:45.014 [notice] TLS :client: In state :hello_middlebox_assert at ssl_gen_statem.erl:821 generated CLIENT ALERT: Fatal - Unexpected Message
 - {:unexpected_msg,
    {:internal,
      {:encrypted_extensions,
        %{
          elliptic_curves: {:supported_groups,
            [:secp521r1, :secp384r1, :secp256r1, :x25519, :ffdhe2048]}
        }}}}

When you see this output, you know you’ve been “thrown to the wolves.”

Fix: set middlebox_comp_mode to false in http_options and the request will succeed as expected.