Every request brings its own connection
Under load the logs fill with "too many connections". The app runs 40 instances and each opens a connection per request; the database allows 500 and each one costs it memory and a backend process.
The database is not out of CPU. It is out of connections, which is a different limit with a different fix.
Put the right thing between the app tier and the database.
Components — tap one, then tap a slot on the diagram
!The database is refusing new connections. It has plenty of CPU left.
Boundaries, outermost first: App tier: Instance 2, Instance 40, Instance 1 Outside every boundary: Database (max 500 connections; FAILED: no connections), an empty slot for the between the tiers Connections: Instance 1 calls between the tiers — many clients (step 1) Instance 2 calls between the tiers Instance 40 calls between the tiers between the tiers calls Database — few connections (step 2)
Instance 2
Instance 40
Instance 1
Databasemax 500 connectionsno connections