Skip to main content

Reliability assessment and calibration

Attempt independently. Score 0 absent/incorrect, 1 prompted/incomplete, 2 independently correct evidence with limits, 3 justified fresh transfer. All six categories require at least 2.

Required categories​

  • RE1 — Population: define user-visible good/bad events, eligibility, latency boundary and overlapping error handling; reconcile actual request counts with collection.
  • RE2 — Arithmetic: hand-calculate unequal-volume burn windows, distinguish budget from burn and state future-traffic assumptions before projecting exhaustion.
  • RE3 — Evaluator: implement the archive and add independent onset/recovery/missing-collection cases. Explain insufficiency and exact threshold equality.
  • RE4 — Alert integration: implement justified fast/slow rules in the actual monitoring setup, with owner, action and missing-telemetry policy. Exercise firing, recovery and low traffic; the reference alone does not satisfy this gate.
  • RE5 — Containment: retain original saturation and graceful-overload runs with offered load, raw latency, throughput, rejection, errors, utilization and queue depth. Show a measured tradeoff and a load-generator limitation.
  • RE6 — Incident: run the local dependency/cache failure exercise, preserve detection/diagnosis/mitigation/recovery timeline and verify an owned follow-up action. State what the experiment cannot establish.

Calibration​

At target 99%, 2/100 bad gives burn 2. Pooling (10,1) and (100,1) gives burn 20/11, about 1.82, not 5.5. A pair with short burn below threshold stays quiet even if long burn is high. Exact equality alerts in this lab. Missing selected collection or insufficient counts returns insufficient; it must not be presented as proof of health.

A higher rejection rate can make completed-request latency look better while user outcomes worsen. Use the agreed SLI and offered-load accounting. Burn is detection evidence, not root cause. Clearing an alert does not replace service recovery checks or validate every failure mode.

Delayed transfer​

After at least 48 hours, evaluate an unseen traffic mix with collector loss and a different target. Defend the response policy without the reference. Preserve mistakes, repairs and assistance disclosures. Author fixtures are not measured service reliability.