I Replaced 500 Lines of cert-manager Config With a 50-Line Blueprint

Source: Dev.to

The 3 AM Certificate Expiry Crisis

Have you experienced this? Your phone explodes with PagerDuty alerts. A production API is down. Not because of bad code. Not because of a DDoS attack. But because an internal service certificate had expired.

The irony? This happens even with “automated” certificate management via cert‑manager, even if there is a 90‑day rotation, monitoring, and runbooks.

This crisis is repeated all too often.

My experience: a 6‑hour restoration process

• Emergency cert renewal – 45 minutes
• Restarting pods to pick up new certs – 30 minutes
• Root cause analysis – 2 hours
• Writing a post‑mortem – 1 hour
• Updating runbooks so this “never happens again” – 2 hours

Three weeks later, a different service. Same issue. Different certificate.

What did I learn? Automating certificate issuance isn’t the same as eliminating certificate problems.

The real question isn’t “how do we rotate certs faster?” but “why are we using certificates at all for internal east‑west traffic?”

Here’s What “Automated” Certificate Management Actually Looks Like

Before: cert‑manager

150 + lines of Certificate YAML

• Issuer configuration
• CertificateRequest management
• Secret rotation scripts
• Monitoring alerts for expiry
• Runbooks for emergency renewal

≈ 200 lines of config
≈ 30 % of platform team’s time managing PKI

After: Lane7 Blueprint

# This is the entire deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app-pod-1
spec:
  # ... standard Kubernetes deployment
  # Workload Security Proxy (WoSP) sidecar handles all networking, identity, and encryption

50 lines total. Deploy in 10 minutes. Credentials rotate every communication session between Service A and Service B (any two services) with identity‑trust verification.

No certificates. No PKI. No 3 AM alerts.

The False Security Blanket of Automated PKI

What I thought “automating certificates” meant:

• ✅ Auto‑renewal every 90 days
• ✅ Automated distribution to pods
• ✅ Monitoring for expiry
• ✅ Zero manual intervention

Sounds like Zero Trust, right?

Wrong. It’s just “Zero Touch” issuance.

The Problems That Automation Doesn’t Solve

1. The “Secret Zero” Problem – How do you securely deliver the first secret that fetches the certificate? You bootstrap trust with… another secret. It’s turtles all the way down.
2. Certificate Authority = Single Point of Failure – If that CA is compromised, misconfigured, or has a bug, your entire security model collapses.
3. Every cert rotation is an entirely new identity – Apps lose their history of trusted interactions when a rotation occurs. It’s like getting a new driver’s license with a different picture, address, and signature every 90 days.
4. Handshake Overhead – Each connection requires certificate validation, chain verification, revocation checking (OCSP/CRL), and key‑exchange negotiation, adding latency and CPU load to every internal service call.
5. Static Identity Window – Even with 90‑day rotation, the certificate sits on disk for 90 days. If an attacker obtains it on Day 1, they have 89 days to use it.
6. Expiry Anxiety – No matter how good your automation, there’s always the fear:
   • What if the rotation script fails?
   • What if the webhook times out?
   • What if the monitoring alert is missed?

The Realization

I realized that “automated PKI” wasn’t actually Zero Trust—it was just faster legacy security. PKI works well for root domains because they are vetted by a third party. Vetting of workloads submitting a CSR doesn’t happen. The interest is PKI for fast TLS connections—not trust.

The chain of trust ended at the Certificate Authority, not at the workload itself.

I needed a way to verify the identity of the service without relying on files sitting on disk.