Mistake Clinic

Module-Specific Mistake Radar

Mistake	Detection signal	Repair
Dataset-first problem framing	No named actor or action	Write the decision contract
Straw-person baseline	Baseline is knowingly unusable	Reproduce current workflow
Random split on temporal data	Future or repeated entities cross splits	Group and time-bound the split
One aggregate metric	Important harms are invisible	Add slices, calibration, and costs
Test-set tuning	Final score changes repeatedly	Quarantine and log access
Notebook-only result	Clean run cannot reproduce	Create one-command harness

Practice Mistake Checks

Before submission, ask: Can another learner rerun it? Is the baseline credible? Was the final set untouched? Is every feature available at decision time? Are failures sampled? Does the conclusion state uncertainty and a stop condition?

Repair loop

Choose one failed check, write the mechanism causing it, repair the pipeline, rerun every affected comparison, and append—not overwrite—the corrected evidence.