Skip to main content

Differential Oracles and Mutation Evidence

What This Concept Is​

A differential test compares old and new implementations on the same controlled experiment. It supports equivalence for those observations. An independent specification oracle checks a different question: whether the chosen behavior meets a stated model.

Why It Matters Here​

Two versions can share the same bug. A copied expected expression can repeat the implementation's mistake. The kit uses old/new trace comparison alongside exact rational rounding and a scalar grid model so those evidence sources remain distinguishable.

Concrete Example​

Two invoice lines each costing 0.335 round individually to 0.34, producing 0.68 before the deadline discount. Rounding their sum instead produces 0.67. A test using only whole-cent lines misses the difference.

The supplied checker uses Fraction arithmetic as an independent oracle and compares ordered events for failure cases. It tries a clock at deadline-1, deadline and deadline+1. No random seed or wall-clock sleep is needed.

Common Confusion / Misconception​

A mutation is deliberately faulty code used to challenge a checker. Rejecting ten faults does not prove the suite catches every error. A surviving mutation may expose missing assertions, an equivalent transformation or a fault outside the contract. Investigate before counting it as a test failure.

How To Use It​

Freeze the baseline. Define the comparison boundary and exclude nondeterminism explicitly. Add boundary and failure inputs. Seed a plausible fault, confirm a meaningful assertion rejects it, and retain a small witness. Keep behavior-repair tests separate from preservation tests.

Check Yourself​

What does agreement with a buggy baseline establish? Why can sorting both outputs hide an ordering regression?

Mini Drill or Application​

Replace < with <= in a private invoice copy. Find the smallest failing deadline case and name the assertion. Then propose a mutation that changes only private helper names. Feedback: the former should fail; the latter should not fail a public behavior checker.

Read This Only If Stuck​

Use the Refactoring book index for small-step discipline and the reading guide for supporting primary documentation. Continue with the contracts and labs.