Test the claim with an independent oracle

What This Concept Is

An oracle decides whether an observed result satisfies the contract. A test that repeats the implementation algorithm can repeat the same defect.

Why It Matters Here

The original risk-based matrix must explain why each test would detect the important failure, not simply list test types or coverage percentages.

Concrete Example

For reservations, compare results to a small independently computed ledger of capacity, successful unique operations and cancellations. Preserve both accepted and rejected attempts. Intentionally remove the capacity check in a disposable test branch/copy and show the relevant test fails. Then restore the correct implementation and verify it passes.

Common Confusion / Misconception

Mutation detection supports test sensitivity but is not proof of correctness. High line coverage does not imply a meaningful assertion. A deliberately faulty variant is not a real production defect and must be labeled.

How To Use It

Map each highest-risk claim to raw inputs, expected result, independent derivation and a fault it should detect. Use a small exhaustive domain where feasible, then test representative real-system integration behavior.

Check Yourself

Could your oracle share the same bug? Does a failure distinguish the intended defect? What untested domain remains?

Mini Drill or Application

Add one negative control and one positive control to the capstone invariant test. Preserve the faulty variant result without merging it into the working system.

Read This Only If Stuck

Building Secure and Reliable Systems provides selective operational reasoning. Use the module sources below for current tool behavior; do not read a whole book before attempting the exercise.

SRE testing chapter; Google code review. Prior semester testing and concurrency labs remain the backbone.