Managing queues: the invisible inventory

Product development inventory is information, so it is invisible both physically and on the balance sheet.

A factory sees its work-in-process as piles on the floor and dollars on the books; a product team's queue is specs waiting for review, branches waiting for QA, and tickets in a backlog. Because nobody trips over it, nobody manages it. Make it visible: count the items waiting at every hand-off.

Queues are the root cause of most of the economic waste in product development.

Every item waiting in a queue adds cycle time, and with it more risk (the market and technology drift while it waits), more variability, more overhead to track and re-plan it, lower quality (feedback arrives late), and less motivation (no urgency for work that will just wait downstream).

Queue size grows explosively, not linearly, as capacity utilization approaches 100%.

For an M/M/1 queue the number of items in the system is ρ/(1−ρ): going from 80% to 90% utilization more than doubles it (4 → 9), and 95% doubles it again (19). A team planned to 100% loading has, in theory, an infinite queue.

Most of a queue's damage comes from rare high-queue states, not from typical ones.

Queue size follows a geometric distribution: long queues are improbable, but each one delays many items at once, so the low-probability tail carries a large share of the total delay. Watch for, and react to, those states rather than to the average.

Variability in arrivals and in service times increases queue size roughly linearly.

Kingman's approximation multiplies the M/M/1 queue by (Ca² + Cs²)/2: halving the variability of task durations (e.g. M/M/1 → M/D/1) halves the queue. Useful, but utilization is the bigger lever because it acts exponentially.

At high utilization, small changes in load produce large swings in queue size and cycle time.

The slope of the queueing curve, dLq/dρ, rises steeply near 100%. At 95% loading a 1-point change in load moves the queue by several items; at 70% it barely moves. Highly loaded processes amplify the variability fed into them.

Serve pooled demand with reliable, high-capacity servers.

One shared queue feeding several engineers beats a separate queue per engineer, because no one idles while another's line grows and one long job cannot block everyone behind it. A single fast server is even faster, but becomes a single point of failure.

Adjacent queues see either the arrival variability or the service variability of the upstream process, depending on how loaded it is.

A lightly loaded upstream stage passes its arrival pattern straight through; a heavily loaded one releases work at the rhythm of its own service times. Look at the whole chain: smoothing a high-variability upstream step can shrink the queue at a downstream bottleneck.

The right queue size is an economic trade-off between the cost of capacity and the cost of delay.

More capacity costs money but cuts queues; less capacity saves money but delays everything. Add the two costs and pick the utilization at the bottom of the U-curve. The higher the cost of delay, the lower the economic utilization.

The cost of a queue depends on the order in which its jobs are served.

When jobs differ in cost of delay and duration, first-in-first-out is rarely optimal; serving the highest cost of delay per unit of capacity first (WSJF) minimizes total delay cost. Sequencing matters most when queues are large; making queues small is better still.

Use cumulative flow diagrams to monitor queues.

Plot cumulative arrivals and cumulative departures over time: the vertical gap is the current queue, the horizontal gap is the time an item spends in it, and the slopes are demand and throughput. Emerging queues show up as a widening gap long before deadlines slip.

Average wait time equals average queue size divided by average processing rate.

Little's Law (W = L / λ) holds for any stable queue regardless of distributions. If a test team finishes 10 builds a week and 40 are waiting, a new build waits about four weeks; you can forecast cycle time from a queue count.

Don't control capacity utilization; control queue size.

Utilization is hard to measure and, near 100%, tiny errors in it mean huge differences in queues. Queue size is easy to count, and it is what actually causes delay, so set limits on it and act when they are exceeded.

Don't control cycle time; control queue size.

Cycle time is a lagging indicator: you only learn it when items finish, long after the queue built up. Queue size is a leading indicator that moves the moment arrivals outpace capacity, so it lets you intervene earlier.

Over time, queues randomly drift into seriously out-of-control states and can stay there for long periods.

A queue is the running sum of random differences between arrivals and departures, a random walk whose spread grows with the square root of time. Even with balanced average load, some paths wander far from zero and take a long time to come back.

Randomness won't reliably fix a queue that randomness created; intervene quickly when it gets large.

Because a random walk has no memory, a large queue is as likely to grow as to shrink. Set a queue limit and, when it is hit, act: add capacity, shed or block low-value demand, or purge work. Fast, small interventions keep queues bounded.