The Scaling Gauntlet, Pt. 3: Cache Rules Everything Around Me

Source: Dev.to

It starts, like all false dawns, with good news.

Postgres Pete is calm.
The team celebrates. Someone makes a meme.

But something’s off.

• Not “the app is down” bad. Just… squishy.
• Latency spikes for logged‑out users.
• Report generation still eats a chunk of the CPU.
• Every time someone lands on the homepage, your servers run the same query for the 400,000th time this week.

You’re not slow anymore. You’re wasteful.

Welcome to Chapter 3: where your architecture is finally fast enough to reveal just how much duplicated work you’re doing.

---

War Story: The Leaderboard That Melted Everything

A few years back we launched a gamified campaign: Leaderboard, Badges, Daily rewards.

The catch?
We recalculated the leaderboard on every request.

• Every hit triggered a massive sort, join, and filter across millions of rows.
• At launch it didn’t matter—maybe 100 users an hour.
• When it went viral? 90,000 hits in a day.

Our DB didn’t crash, but performance was terrible. Adding a 60‑second Redis cache dropped response times from 912 ms → 38 ms and query load by 99.7 %.

Postgres Pete wrote us a thank‑you note.

---

Chapter 3: Caching Isn’t Cheating

• At small scale, caching is optional.
• At large scale, it’s everything.

The illusion is that if you make your database fast and your queries efficient, you’re good.
When 1,000 users hit the same route and you generate 1,000 identical responses—congratulations, you’ve optimized waste.

Caching is how you stop being a short‑order cook … and start being a chef with a prep station.

---

The Caching Stack

Let’s break it down by layer, not tool.
You don’t need Redis just because someone read a blog post. You need the right kind of cache for the right job.

1. Page & Fragment Caching

2. Query Result Caching

3. Object Caching

4. Edge Caching & CDNs

---

Redis Example: Cache‑Aside Pattern

// Basic cache-aside pattern
const getCachedUser = async (userId) => {
  const cached = await redis.get(`user:${userId}`);
  if (cached) return JSON.parse(cached);

  const user = await db.users.findById(userId);
  await redis.setex(`user:${userId}`, 300, JSON.stringify(user));
  return user;
};

---

Concrete Example: Homepage Latency

Caching just three components (featured products, blog teasers, testimonials) made the biggest impact.

---

Cache Busting: The Forgotten Half of Caching

Writing to a cache is easy.
Invalidating it correctly separates grown‑up systems from hopeful experiments.

+------------------------+
|   Does the data        |
|   change often?        |
+------------------------+
          |
+----------------+----------------+
|                |                |
Yes              No
|                |
+--------------+  +--------------------+
| Can you hook |  | Use long TTL with  |
| into writes? |  | fallback refresh   |
+------+-------+  +--------------------+
       |Yes
       |
+--------------+
| Event-driven |
| invalidation |
+--------------+
       |No
       |
+--------------+
| Use low TTL  |
| w/ polling   |
+--------------+

Cache Busting Methods

Time‑Based (TTL)

• Easy to reason about.
• Accepts a bit of staleness.
• “Good enough” for dashboards, stats, pricing.

Event‑Driven

• Invalidate cache when data updates.
• Use hooks in your ORM or a pub/sub system.
• Harder to manage but more accurate.

// In your product update handler:
await redis.del(`product:${product.id}`);
await redis.del(`category:${product.category}:featured`);

Dependency‑Tracking (Advanced)

• Track what data powers what cache entry.
• Rebuild only what’s affected.
• Requires discipline and tooling (or you’ll hate yourself).

---

Signs Your Cache Is (or Isn’t) Working

---

Cache Debugging: What to Watch

• Hit‑rate metrics (Redis INFO stats → keyspace_hits / keyspace_misses).
• Latency of cache reads vs DB reads.
• TTL distribution – are keys expiring too soon or never?
• Invalidation logs – ensure every write that should bust a key actually does.
• Memory pressure – watch used_memory and eviction policies.

---

TL;DR

• Cache at the right layer (page, query, object, edge).
• Pick the right TTL or event‑driven strategy for each data type.
• Monitor hit rates and latency; adjust before things go stale or explode.

With a disciplined caching strategy you’ll turn wasteful “fast enough” into truly efficient, scalable performance.

# Cache Health Checklist

## ✅ Check INFO STATS
- Look for `keyspace_hits` vs. `keyspace_misses`.

## 📈 Log Slow / Missed Lookups
- Tag routes where the cache failed silently.

## ⚡️ Detect Cache Stampedes
- Identify when many users request the same uncached item at once.  
- Consider locking or **stale‑while‑revalidate** strategies.

## ⏰ Track TTL Expirations
- Verify that TTLs align with actual usage patterns.

---

# Advanced Patterns (For When You’re Ready)

## Stale‑While‑Revalidate
- **Serve stale data immediately.**  
- In the background, fetch fresh data and replace it in the cache.  
- Reduces wait time and perceived latency for users.  
- **Implementation options:**  
  - HTTP headers: `Cache-Control: stale-while-revalidate=60`  
  - Custom middleware in your application.

## Soft TTL + Refresh
- Items have a TTL, but when accessed near expiry, refresh in the background.  
- Prevents cold starts and keeps hot items alive.  
- Great for data that’s accessed frequently but updated infrequently.  
- **Implementation options:**  
  - Async job queues  
  - Middleware hooks

## Sharded or Namespaced Caches
- Use key prefixes to separate cache scopes.  
  - Example: `tenant-42:user:profile`, `locale-en:settings`  
- Prevents key collisions and simplifies bulk invalidation.  
- Adopt structured key‑naming conventions to support future automation.

---

# Caching Anti‑Patterns

- **Caching user‑specific or sensitive data globally** – a GDPR violation waiting to happen.  
- **Hard‑coding long TTLs for data that changes daily** – fast, but wrong.  
- **Caching everything without a purge strategy** – you end up with a second, unmanaged database.

---

# TL;DR – Cache with Intent

- **Don’t optimize slow things** – avoid doing them repeatedly.  
- Choose the right cache for the right problem.  
- Design your cache‑busting strategy **before** you go live.  
- Monitor **hit rates**, not just cache size.  
- Caching isn’t cheating; it’s how systems scale.  
- Your app isn’t just fast now – it’s efficient.  
- But don’t relax too long… you’ll need to start splitting your read and write workloads.

*Stay tuned.*  
**Next up:** Scaling Reads Without Sharding Your Soul