Use sensitivity analysis to choose the next experiment

What This Concept Is

Sensitivity analysis asks which conclusions change when assumptions vary. Its purpose is to identify fragile decisions and useful measurements, not to decorate a design with many unsupported decimal places.

Why It Matters Here

The 45-minute design exercise produces a hypothesis. The later review-ready document should show what was measured, which assumptions remain uncertain and what would overturn the choice. Prior architecture decisions supply constraints; capacity reasoning tests them under workload pressure.

Concrete Example

Seven replicas satisfy the fictional 900-unit rate, 200-unit capacity, 3/4 policy and one-failure calculation. If representative capacity is only 150, the requirement becomes nine replicas. If 60% of requests target one indivisible key, the balanced-replica assumption may fail entirely; adding replicas does not automatically divide that key's work.

Increasing payload size by 100x may first exhaust network or memory while the request rate is unchanged. Tightening freshness from minutes to seconds may remove a batching optimization and increase downstream operations. Recalculate the changed mechanism instead of multiplying every number by the same factor.

Common Confusion / Misconception

A regional failure is not necessarily one failed replica. It can remove many replicas and shared dependencies at once. A fleet total also does not establish capacity in each failure domain. A small successful benchmark cannot prove linear scaling to the proposed size.

How To Use It

Vary demand, work size, skew, failure scope and service capacity independently, then examine a plausible combined scenario. Identify where the recommendation changes. Run the cheapest discriminating experiment on a representative implementation and retain raw observations, environment and limits. Separate calculations from measurements in the review packet.

Check Yourself

Which assumption does the hot-key example violate? Why is a region-loss exercise different from N+1 arithmetic? What result would justify revisiting the architecture rather than merely increasing a replica count?

Mini Drill or Application

Use the original system design to make a baseline, a 10x-traffic case, a 100x-object case and a tighter-freshness case. Execute a small representative load/failure experiment and compare predicted versus observed behavior. Explain one discrepancy and revise only affected design decisions. Keep the full write/read trace and peer-review findings.

Read This Only If Stuck

Fundamentals of Software Architecture — evidence-based architecture decisions. SRE Classroom supplies optional NALSD practice; use one exercise selectively after completing your own design and sensitivity analysis.