wtf( )unctionsystem design, drawn
← all problemsCachingHard

The instant one key expires

One cached key serves the homepage. It has a 5-minute TTL, and the site takes thousands of requests per second.

The moment it expires, every in-flight request misses at once and they all go to the database together. The database stalls, requests pile up, and when the value is finally written the whole cycle re-arms itself for five minutes' time.

Insert the gate that collapses a thundering herd into a single database read.
Components — tap one, then tap a slot on the diagram
!The hot key just expired and every request missed at the same instant.

Boundaries, outermost first: App tier: Cache (key expired), an empty slot for the on the miss path Outside every boundary: Request, Request, Request, Database (one slow query; FAILED: stalling) Connections: Request calls Cache — all miss at once (step 1) Request calls Cache Request calls Cache Cache calls on the miss path (step 2) on the miss path calls Database — exactly one read (step 3)

Request
Cachekey expired
Request
Request
Databaseone slow querystalling