Skip to main content

Averages hide burst admission and backlog

What This Concept Is​

A backlog is unfinished admitted work. A finite admission limit bounds how much work may remain in the system; excess offered work must be rejected, deferred elsewhere or handled by another explicit policy. Equal average arrival rates do not imply equal behavior.

Why It Matters Here​

A queue can absorb a short burst without increasing sustainable throughput. It can also hide overload until work becomes useless. The original failure walkthrough needs a stated admission and recovery policy, not a queue box labeled “scales.”

Concrete Example​

In the lab, arrivals enter at the start of each one-second slot, then service runs. With arrivals [6,0,0], service budgets [2,2,2] and total capacity limit 4, results are (admitted=4, served=2, dropped=2, remaining=2), then (0,2,0,0), then (0,0,0,0). Two service opportunities in the last slot are idle; unused capacity cannot travel backward to accept the first burst.

Spread the same six arrivals as [2,2,2] and no work drops. The totals and average offered rate are identical. At limit zero this model drops everything even with service available: it models total in-system capacity, not a zero-wait queue with direct handoff. Choosing a different event order changes the model and must change its contract.

Common Confusion / Misconception​

Backlog divided by service rate is not a measured p95 latency. Bulk slots omit arrival timing within the slot, individual costs and deadlines. Retrying every drop introduces new arrivals and can amplify overload; it is not part of the reference model.

How To Use It​

Predict a short trace by hand before simulating. Check conservation: offered = served + dropped + remaining, starting empty. Test an outage interval, recovery and a hot partition separately. State what happens to rejected work and whether the client sees success, failure or uncertainty. Do not hide dropped work from the demand ledger.

Check Yourself​

Why does equal average demand produce different drops? Where do unused service budgets go? What changes if work arrives after service instead of before? Why can a huge buffer harm a freshness requirement?

Mini Drill or Application​

Add two slots of zero service, then restore capacity. Predict the backlog and losses. Compare a larger limit with added service capacity, documenting which problem each addresses. Define a separate retry/deadline experiment without silently changing the base contract.

Read This Only If Stuck​

Fundamentals of Software Architecture — operational characteristics. Google SRE: Handling Overload provides production context for overload responses and retry decisions; this finite simulator does not certify those mechanisms.