Caching — Core Concepts

Interview Relevance: Very High — Caching is the #1 tool for scaling reads. Know every layer, every policy, and every trade-off.

Caching stores copies of data in a faster storage layer so future requests for that data are served faster without hitting the origin (database or API).

---

The Cache Hierarchy — All 5 Layers

---

Layer 1 — Client-Side Cache (Browser)

How HTTP Cache Headers Work

Key Cache Headers

Header	Example	Meaning
Cache-Control: max-age=86400	86400 seconds	Cache for 24 hours
Cache-Control: no-cache	—	Must revalidate before serving
Cache-Control: no-store	—	Never cache (sensitive data)
Cache-Control: private	—	Only browser can cache (not CDN)
Cache-Control: public	—	CDN and browser can both cache
ETag: "abc123"	Hash of content	Used for conditional requests (304)
Last-Modified: Mon, 01 Jan 2025	Timestamp	Alternative to ETag

---

Layer 2 — CDN Cache

What to cache on CDN:

• ✅ Static assets (JS, CSS, images, fonts)
• ✅ Public API responses (product listings, news feeds)
• ✅ Video/audio streams
• ❌ Authenticated user data (use Cache-Control: private)
• ❌ Real-time or frequently updated data

---

Layer 4 — Application Cache (Redis & Memcached)

This is the layer most discussed in FAANG interviews.

Redis vs. Memcached

Feature	Redis	Memcached
Data structures	String, Hash, List, Set, Sorted Set, Stream	String only
Persistence	✅ RDB snapshots + AOF log	❌ In-memory only
Replication	✅ Master-replica + Sentinel	❌ No native replication
Clustering	✅ Redis Cluster (hash slots)	✅ Client-side sharding
Pub/Sub	✅ Built-in	❌ No
Atomic operations	✅ INCR, LPUSH, ZADD	Limited
Memory efficiency	Slightly higher overhead	✅ Slightly more efficient
Use case	Leaderboards, sessions, queues, rate limiting	Simple key-value cache only

Interview answer: "I'd default to Redis in almost every case — the richer data structures, persistence, and native clustering make it far more versatile than Memcached."

---

Cache Eviction Policies

When the cache is full, the eviction policy decides which item to remove.

LRU in Action — Step by Step

Cache capacity: 3 slots

Request sequence: A, B, C, A, D

Step 1: GET A → MISS → Cache: [A]
Step 2: GET B → MISS → Cache: [B, A]
Step 3: GET C → MISS → Cache: [C, B, A]
Step 4: GET A → HIT  → Cache: [A, C, B]  ← A moved to front (recently used)
Step 5: GET D → MISS → Cache full!
                        EVICT B (least recently used, at the tail)
                        Cache: [D, A, C]

LRU vs LFU — Which to Use?

Scenario	Best Policy	Reason
URL Shortener redirects	LRU	Recent URLs are likely still hot
Trending products	LFU	Most popular products should stay cached
Session data	TTL	Sessions have a natural expiry time
News feed	LRU	Recent content is what users want
Music streaming	LFU	Popular songs accessed millions of times
Rate limiting counters	TTL	Counters reset after a time window

All Redis Eviction Policies

Policy	Evicts	Best For
allkeys-lru	LRU across all keys	General-purpose cache
volatile-lru	LRU among keys with TTL only	Mix of cache + persistent data
allkeys-lfu	LFU across all keys	Frequency-based (trending content)
allkeys-random	Random key	When access pattern is uniform
volatile-ttl	Key with shortest TTL first	Prioritize expiring data
noeviction	❌ Returns error when full	When data loss is unacceptable

---

Cache Invalidation Strategies

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

Strategy 1 — Cache-Aside (Lazy Loading) ⭐ Most Common

The application manages the cache explicitly. On a miss, it loads from DB and populates the cache.

✅ Cache only what's actually needed (lazy)
✅ Cache failures don't break the app (fallback to DB)
✅ Simple to implement
❌ Cache miss on first request (cold start)
❌ Potential stale data between write and invalidation (race condition)

---

Strategy 2 — Write-Through

Every write goes through the cache synchronously before returning success.

✅ Cache is always consistent with the database
✅ No stale reads
❌ Write latency = cache write + DB write
❌ Cache fills with data that may never be read (write-heavy apps waste memory)

---

Strategy 3 — Write-Behind (Write-Back)

Writes go to cache immediately; the cache asynchronously flushes to the database.

✅ Extremely fast writes (returns before DB write)
✅ Batching reduces DB write pressure
❌ Risk of data loss if cache crashes before flushing
❌ Complex to implement correctly
❌ Not suitable for financial transactions

---

Strategy 4 — Refresh-Ahead (Proactive Refresh)

The cache predicts which keys will expire soon and pre-fetches them before they go stale.

✅ Zero cache misses for hot data (best user experience)
✅ Smooth for predictable access patterns
❌ May prefetch data that's never accessed (wasted DB load)
❌ Complex implementation

Strategy Comparison

Strategy	Write Latency	Read Latency	Consistency	Complexity	Best For
Cache-Aside	Low	High on miss	Eventual	Low	General read-heavy apps
Write-Through	High (synchronous)	Low	Strong	Medium	Read-heavy, write-rare
Write-Behind	Very Low	Low	Eventual	High	Write-heavy, loss-tolerant
Refresh-Ahead	N/A	Very Low	Near-real-time	High	Predictable hot data

---

Cache Pitfalls & Solutions

Pitfall 1 — Cache Stampede (Thundering Herd)

Pitfall 2 — Cache Penetration

Requests for non-existent keys bypass the cache and hammer the DB every time.

Pitfall 3 — Cache Avalanche

Many cache keys expire at the same time, causing a sudden spike of DB queries.

❌ Problem:
  All product pages cached with TTL=3600s at midnight
  At 1:00 AM, all 100,000 keys expire simultaneously
  → 100,000 DB queries hit at once → DB crashes

✅ Fix: TTL Jitter
  Instead of TTL = 3600
  Use TTL = 3600 + random(0, 600)  ← adds up to 10min of jitter
  Keys now expire spread across a 10-minute window
  → DB load smoothed out ✅

---

Worked Example — Caching in the URL Shortener

Cache decisions for the URL Shortener:

Decision	Choice	Reason
Cache strategy	Cache-Aside	Simple, lazy-loads only accessed URLs
Eviction policy	allkeys-lru + TTL	Recent URLs are hot; TTL mirrors URL expiry
Invalidation	DEL on URL update/delete	Immediately removes stale entry
Stampede protection	Redis SETNX mutex lock	Prevents duplicate DB fetches on popular expiries
Penetration protection	Return 404 and cache null for 60s	Prevents DB hits for invalid short codes
TTL jitter	expiry ± random(0, 300s)	Prevents mass expiry of URLs created in bulk

---

Interview Cheat Sheet

One-Line Summaries

Cache-Aside:    App checks cache → on MISS load DB → populate cache
Write-Through:  Every write → cache + DB synchronously
Write-Behind:   Write to cache only → async flush to DB later
Refresh-Ahead:  Pre-fetch hot keys before they expire

LRU: Evict the least recently accessed item
LFU: Evict the least frequently accessed item
TTL: Evict after a fixed time window

Stampede:    Many simultaneous misses → fix with mutex lock
Penetration: Non-existent keys bypass cache → fix with Bloom Filter
Avalanche:   Mass simultaneous expiry → fix with TTL jitter

The Interview Phrase

"For this read-heavy system, I'd use Cache-Aside with Redis in front
 of the database. The eviction policy would be allkeys-lru since recency
 predicts access probability. I'd add TTL jitter of ±10% to prevent
 cache avalanche, and a Redis SETNX mutex to handle cache stampede on
 popular key expiry. For cache penetration, I'd cache null results for
 60 seconds and optionally add a Bloom Filter at the app layer."

---

Red Flags vs. Green Flags

🔴 Red Flag	🟢 Green Flag
"Just add Redis" with no further detail	Specify strategy, eviction policy, TTL, and invalidation
Ignore cache invalidation	Address it explicitly — it's the hardest problem
Use Write-Through for write-heavy apps	Match strategy to workload (Write-Behind for writes)
Forget cache stampede / avalanche	Always mention TTL jitter and mutex lock
Cache authenticated user data on CDN	CDN only for Cache-Control: public responses
Treat Redis and Memcached as identical	Know Redis's richer features and when each fits

IMPORTANT

Cache invalidation is the hardest problem in caching. Always address it explicitly. Say: "When a URL is updated or deleted, I immediately DEL the cache key so the next read fetches fresh data from the DB."

TIP

Mentioning cache stampede, cache penetration, and cache avalanche — and their mitigations — is a strong senior signal. Most candidates only talk about hit rate and eviction policies.