Mistake Clinic

Module-Specific Mistake Radar

Mistake	Signal	Repair
Weights are the version	Prompt/index mismatch	Version the complete system
Mean latency only	Users see long tails	Measure percentiles and queues
Cost per token only	Retries and review hidden	Cost successful outcomes
Logs without correlation	Regression source unknown	Trace component versions
Rollback on paper	Old dependencies unavailable	Rehearse rollback

Practice Mistake Checks

Verify bounds, idempotency, backpressure, version correlation, privacy, tail latency, unit cost, quality signals, abort criteria, fallback, rollback, and ownership.

Required evidence

Submit reproducible deployment configuration, correlated telemetry, load and failure results, user-impact and unit-cost calculations, one rejected alternative, and the executed response. A dashboard screenshot without raw query and version context does not pass.

Oral defense

Trace one request across boundaries, explain a tail-latency or quality regression, demonstrate containment and rollback, and defend the residual operational risk.

Source backbone

Use Building Secure and Reliable Systems, Google SRE books, OpenTelemetry specifications, and NIST AI RMF.