wtf( )unctionsystem design, drawn
← all problemsMicroservicesHard

Analytics got slow, and nobody could check out

Analytics is not on the critical path for anything. Checkout calls it at the end of an order to record a funnel event, and the code that does it already ignores the error, because a failed funnel event is not a failed order.

On Thursday analytics went from twenty milliseconds to eight seconds. Checkout then served nothing at all for eleven minutes — including requests that never reach the analytics call. Payments was healthy. Inventory was healthy. Checkout's own process was running the whole time and its health check said so.

  1. R1While an instance is saturated it must stop receiving new requests. Whatever distributes work across checkout's instances has to choose on what each one is currently holding — a rotation hands a parked instance the same share as an idle one, and a check that the process is running answered yes throughout the incident.
  2. R2A request that never calls analytics must still be served while analytics is slow. Checkout has a fixed number of workers and an eight-second call holds one for eight seconds, so what must be bounded is the SHARE of checkout's capacity any single dependency may occupy — not how long one call may take, and not how many calls may be in flight altogether, since either of those leaves one slow dependency holding everything.
  3. R3Awarding points and recording funnel events do not change the answer the customer is given, so they must leave the request path entirely. What they are handed to has to survive their service being down at the moment of the order, and has to survive checkout being redeployed, which happens about twenty times a day.
Compose the order path. Tier 1 is what distributes work to checkout's instances and what stands between checkout and the calls it must make, tier 2 is what takes the work the answer does not depend on.
Components — tap one, then tap a slot on the diagram
?Ignoring the error does not make the call free. It still occupies a worker for eight seconds, and workers are the resource every request shares — including the ones that were never going to call analytics.

Outside every boundary: Shoppers (placing orders), Payments (cannot come off the path), Checkout (a fixed pool of workers), Analytics (a funnel event; FAILED: 8 seconds), an empty slot for the chooses which checkout instance gets it, tier 1, an empty slot for the stands between checkout and the calls it must make, tier 1, an empty slot for the takes the work the answer does not depend on, tier 2 Connections: Shoppers calls chooses which checkout instance gets it chooses which checkout instance gets it calls Checkout Checkout calls stands between checkout and the calls it must make stands between checkout and the calls it must make calls Payments Checkout publishes to takes the work the answer does not depend on takes the work the answer does not depend on publishes to Analytics — when it is up again Checkout must NOT reach Analytics — never on the answer's path

Shoppersplacing orders
Paymentscannot come off the path
Checkouta fixed pool of workers
Analyticsa funnel event8 seconds