Guided Labs

Lab 1: Reproducible Baseline Harness

Build one command that loads a versioned dataset, creates a group-aware or temporal split, trains a rule baseline and a statistical baseline, and writes metrics plus a data fingerprint. Start from an empty output directory. A second learner must reproduce the result.

Injected failures: a post-outcome field, duplicate records across splits, and an unseeded step. Detect and remove all three.

Lab 2: Metric and Threshold Review

Given an imbalanced classifier and a fixed review budget, produce precision-recall, calibration, subgroup, and capacity evidence. Choose a threshold and an abstention region. Then respond to a surprise 40% capacity cut without retraining.

Lab 3: Error Taxonomy Clinic

Blindly sample 50 errors. Define failure categories before slicing the full set. Quantify the three most consequential slices with uncertainty, propose one data remedy and one product fallback, and identify a finding that should not be generalized.

Exit evidence

Submit commands, environment lock, dataset card, immutable result bundle, decision memo, peer reproduction record, and AI-use declaration. Screenshots without runnable evidence do not pass.