Preserve the failing schedule before repairing a defect

What This Concept Is

A reproduction packet makes a failure observable with enough environmental and ordering information to investigate. Timing-dependent failures need more than a screenshot of a red test.

Why It Matters Here

The defect clinic requires a genuine diagnosis and minimal fix. Repeated retries until green hide rather than explain flakiness.

Concrete Example

Two workers read the last seat before either writes. A barrier-controlled test can force that interleaving. Preserve seed, request IDs, versions, datastore setup and before/after records. After the fix, the same schedule should enforce the invariant. A separate stress run can explore other schedules but does not replace the controlled case.

Common Confusion / Misconception

Increasing timeouts does not establish the cause. A passing test on a developer laptop does not reproduce a CI environment. Removing the assertion is not a repair.

How To Use It

Separate product defect, test isolation failure and environmental failure. Retain failing evidence, hypothesis, discriminating experiment, fix and regression. Explain which schedules remain unexplored.

Check Yourself

Which observation distinguishes a race from a slow dependency? What state leaks between tests? Can another engineer reproduce it?

Mini Drill or Application

Use the original real defect, or transparently seeded training defect if none exists yet; never present a seeded defect as a discovered production incident.

Read This Only If Stuck

Building Secure and Reliable Systems provides selective operational reasoning. Use the module sources below for current tool behavior; do not read a whole book before attempting the exercise.

SRE testing chapter; Google code review. Prior semester testing and concurrency labs remain the backbone.