Monitoring that cannot fire for another two months
A credit model decides who gets an instant approval. Whether the decision was right — did the account actually default — is known 60 to 90 days later, so the team's monitoring computes accuracy against those outcomes.
An upstream service started sending income in cents on Tuesday. Every applicant now looks a hundred times richer than they are. The accuracy dashboard for this quarter is blank, because the labels for it do not exist yet.
Boundaries, outermost first: Serving path: Applications (live traffic), Risk model (in production; FAILED: income now in cents), Scoring service, Decision (approve or decline) Outside every boundary: Served features (what the model saw), Outcomes (60-90 days later), Accuracy monitor (needs labels; FAILED: nothing to score yet), an empty slot for the fires without labels Connections: Applications calls Scoring service (step 1) Scoring service calls Risk model — feature vector (step 2) Risk model calls Decision (step 3) Scoring service controls Served features (step 2) Served features calls fires without labels — against the training window (step 4) Decision publishes to Outcomes (step 5) Outcomes calls Accuracy monitor (step 6)