Exact paired randomization and honest testing
What this concept is
A randomization test compares the observed statistic with assignments allowed by the experimental design under a null hypothesis. Here independent fair assignment swaps A/B within each matched pair, and the sharp null says treatment changes no unit’s outcome. Under that null the observed within-pair differences can have their signs flipped according to the assignment mechanism.
Why it matters here
This gives a finite, inspectable inference example and reinforces why resampling must respect the actual unit and assignment scheme.
Concrete example
For four pair differences [1,2,3,4], use the absolute sum as the two-sided statistic. The observed value is 10. Among 16 equally likely sign patterns, only all-plus and all-minus have absolute sum at least 10, so p=2/16=0.125. The observed mean effect is 2.5 units. A small study can have a large descriptive effect yet coarse p-value resolution.
Common confusion / misconception
This p-value is not P(null is true), nor a probability the result is “just chance.” Failure to reject does not establish equivalence. Arbitrarily flipping signs in observational pairs requires additional null-distribution symmetry assumptions; the randomized design used here must not be silently presumed.
How to use it
Prespecify the statistic, two-sided tail rule, practical effect threshold and sample count. Enumerate all 2^n assignments for small n, counting ties with ≥ and retaining multiplicities even when some differences are zero. The exact calculation includes the observed assignment, so do not apply a Monte Carlo correction to this full enumeration. The lab uses integer measurements so tail comparisons are exact. Library two-sided conventions can differ; always state yours.
Check yourself
Why do [0,0,0] give p=1? Why is a one-sided result not interchangeable with this prespecified two-sided test?
Mini drill or application
For [1,2,3,0], observed absolute sum is 6. Four of 16 assignments are at least as extreme, so p=1/4. Explain the duplicated sign patterns from the zero. If the pairs were chosen after looking at treatment outcomes, what part of the argument is no longer justified?
Read this only if stuck
Primary source support. See the extension guide for practice, scope and evidence requirements. Local examples and lab code are original.