wtf( )unctionsystem design, drawn
← all problemsWebSocketsHard

The connection outlived the process that accepted it

Ten thousand chat rooms, and at any moment most of them have nobody typing. The current design keeps a server process per room and the bill is for memory nobody is using.

The platform being moved to holds the connections itself: it accepts the socket, and it invokes your code only when a message arrives. Between messages your code is not running — and on some platforms is not in memory at all, while the clients stay connected. Four sentences describe what that costs you.

  1. R1When a message arrives, the code handling it must be able to look up which connections belong to that room. It cannot remember them from last time, because nothing was running last time, and it must still be right after the platform replaces every instance.
  2. R2Sending to a room means sending to each connection individually, by identifier, through a separate call. Each of those calls must happen exactly once — a connection that receives the same message twice has a duplicate on screen, and this work is distributed across whatever instances are awake.
  3. R3That same lookup must survive eviction. In-memory state does not: the platform is explicit that it is reset, and it resets it the moment a room goes quiet.
  4. R4Connections must be kept alive during long idle periods without waking anything up. If keeping a connection alive requires the application to receive and answer something, then an idle room costs exactly what a busy one costs and the whole arrangement has bought nothing.
Compose what has to exist when nothing is running. Tier 1 is who is connected and how a message reaches them, tier 2 is what survives an eviction.
Components — tap one, then tap a slot on the diagram
?There is no process between messages, so there is no in-process anything. Every map, every set, every counter you would have kept in a variable has to live somewhere else.

Boundaries, outermost first: HELD BY THE PLATFORM: Connection front door (holds every socket), an empty slot for the keeps sockets up without waking anything, tier 1 Outside every boundary: Members (connected, idle), Message handler (runs only on a message), an empty slot for the which connections are in this room, tier 1, an empty slot for the one send per connection, done once, tier 2 Connections: Members holds an open connection to Connection front door Connection front door controls keeps sockets up without waking anything — answered below the app Connection front door calls Message handler — on a message Message handler calls which connections are in this room Message handler publishes to one send per connection, done once one send per connection, done once sends outbound traffic to Connection front door — send to one id

Membersconnected, idle
Message handlerruns only on a message
Connection front doorholds every socket