wtf( )unctionsystem design, drawn
← all problemsDistributed SystemsHard

Ten thousand tenants, and one of them is on fire

A B2B product serves thousands of independent business customers on one public address. Today they all share one stack, and last month one customer's traffic took the whole thing down for everyone.

The rebuild repeats the entire product per group of customers. Copy 1 is drawn as the pattern and copy 2 is nearly complete. Three components are missing, and every one of them is a component the copies SHARE or a component that decides what a copy may lose.

  1. R1One public address serves every customer, and something has to decide which copy answers a request by reading the request itself.
  2. R2That deciding layer is shared by everyone, so it must keep nothing of its own that survives a restart. Losing it must cost nothing but the time to start it again.
  3. R3The customer-to-copy assignment is one artifact, fetched whole on a timer and held in memory. It is never queried per request, and it must outlive any restart of the layer that reads it.
  4. R4Each copy absorbs its own repeated reads instead of sending them to its records every time. Whatever absorbs them may vanish at any moment without the copy giving a wrong answer.
Compose what sits in front of the copies. Tier 1 is the shared front and what it reads, tier 2 is what a copy is allowed to lose.
Components — tap one, then tap a slot on the diagram
?Anything you place in front of the copies is shared by every customer, which makes it the one thing that can take all of them down at once.

Boundaries, outermost first: COPY 1: Cache (copy 1 only), App (copy 1), Records (copy 1 only), Balancer (copy 1) COPY 2: Records (copy 2 only), App (copy 2), Balancer (copy 2), an empty slot for the soaks up repeated reads, safe to lose, tier 2 Outside every boundary: DNS (one address), Customers (one public name), an empty slot for the picks a copy from the request itself, tier 1, an empty slot for the which customer belongs to which copy, tier 1 Connections: Customers calls DNS DNS calls picks a copy from the request itself picks a copy from the request itself controls which customer belongs to which copy — fetched on a timer picks a copy from the request itself calls Balancer picks a copy from the request itself calls Balancer Balancer calls App App calls Records App calls Cache Balancer calls App App calls Records App calls soaks up repeated reads, safe to lose

DNSone address
Recordscopy 2 only
Appcopy 2
Cachecopy 1 only
Appcopy 1
Recordscopy 1 only
Customersone public name
Balancercopy 1
Balancercopy 2