Logical KV savings are not physical memory savings. Interactive figures.

Paged KV cache runtimes allocate and free memory in whole blocks, so the unit of reuse is a fully free block, not a single dead token. Whether token eviction returns memory depends on the method. Hover any point or bar for exact values, click a legend entry to hide a series, and use the controls to switch distributions or axis scales. These are memory-accounting simulations, not measurements from a real GPU or model.

The worst case bound, random scattered eviction

Under random independent eviction a block is freed only when all B of its tokens are evicted, so the freed fraction is qB while the logical saving is only q. This curve is the worst case for scatter, a lower bound on physical reclaim rather than a prediction of any deployed method. Reclaim efficiency, the physical reclaim earned per unit of logical saving, is qB−1 and collapses as the block grows.

Logical saving (dashed) versus the physical block reclaim bound qB. At B=16 a 50 percent token cut bounds reclaim near 0.0015 percent of blocks. Toggle a log axis to see the smaller block sizes.
Reclaim efficiency bound qB−1. Even a large token cut earns almost no reusable blocks at production block sizes until the eviction ratio is very high.

Realistic methods, reclaim is method dependent

Deployed eviction methods are not random. At a matched retained ratio their geometry decides physical reclaim. StreamingLLM keeps contiguous sink tokens and a contiguous recent window, so its evictions are block coherent and it frees almost the whole logical saving with no copying. H2O keeps scattered heavy hitters, so survivors touch almost every block and it frees nearly nothing at eviction. SnapKV keeps a clustered recent selection and lands in between. The random independent model is shown as the worst case bound only, not as a real method.

Physical reclaim at eviction with zero copy, mean over 600 modelled requests with standard deviation bars. Window plus sink is high, scattered heavy hitter is near zero, clustered recent sits between. The dashed line is the logical saving.
The same reclaim across block sizes 4 to 64. Window plus sink stays high at every block size, scattered heavy hitter falls toward zero as blocks grow. Click a legend entry to isolate a method.
Reclaim as the retained ratio varies. Even when only 20 percent of tokens are kept, scattered heavy hitter frees far less than window plus sink. Bands are one standard deviation.
The complementary view, compaction copy cost to close the gap. Survivor tokens moved to reach the minimal footprint, as a fraction of tokens. Scattered heavy hitter holds reclaim that only compaction can unlock, and the copy bill is the price. Hover for the reclaim per move return.

A controlled comparison, eviction strategy decides reclaim

Four strategies remove the same number of tokens on the same synthetic importance, so logical saving is held fixed and only the physical consequences differ. Lowest-importance eviction minimizes quality loss yet frees no blocks when importance is scattered. Block-wise eviction frees blocks at higher quality loss. Reclaimability-aware eviction sits between them. Values are means over 30 seeds with standard deviation bars.

Physical reclaim by strategy at q=0.5. The dashed line is the logical saving. Random and lowest-importance token eviction sit near zero under scattered importance.
Quality versus reclaim tradeoff, with standard deviation bars. Lowest-importance is low-loss but low-reclaim, block-wise is high-reclaim but higher-loss, reclaimability-aware occupies the favorable interior.
Internal fragmentation of surviving blocks. Scattered strategies strand about half of each live block. Block-wise eviction leaves near zero.
Physical reclaim as the eviction ratio grows, lognormal importance. Block-wise tracks the ideal, reclaimability-aware trails it, scattered strategies stay flat until very high ratios.

Importance geometry decides reclaim

The scattered result is not a law, it is a property of the importance geometry. As a spatial correlation knob rho rises from 0 to 1 the importance proxy gains a smooth field with a recency and a sink bias, so the lowest value tokens fall into contiguous valleys. Lowest-importance eviction then starts to free whole blocks and the gap to block-wise eviction shrinks. Values are means over 30 seeds with standard deviation bands.

Physical reclaim versus the spatial correlation knob rho. At rho 0 importance is scattered and lowest-importance frees nothing. By rho 1 survivors cluster and the gap to block-wise nearly closes.

Compaction cost and an honest return

Compaction moves survivor entries into fewer blocks and frees the emptied ones. We separate blocks that are free at eviction with zero copy from blocks freed only by moving survivors, and we report the delta based return, reclaimable slots unlocked per survivor token moved. Values are means over 30 seeds.

Where free blocks come from. Scattered patterns get free blocks only through copy heavy compaction. Clustered and block-coherent holes are already free at eviction.
Delta based return, reclaimable slots freed by compaction per survivor token moved. It is bounded and modest, near two to seven. Block-coherent holes are already free, so they need no copy and appear in the next chart at zero.
Survivor tokens moved to reach the minimal footprint. This move count is a bandwidth only lower bound on cost. It leaves out metadata updates, kernel disruption, synchronization, and rotary re-embedding of moved KV.

New request admission

A full pool of 512 blocks is evicted by 50 percent and new requests each need 8 free blocks. The logical saving is identical across strategies, yet admission is not, because admission is gated by physically free blocks. Values are means over 30 seeds with standard deviation bars.

New requests admitted after an identical 50 percent logical eviction. The dashed line is what logical saving alone would predict. Random and lowest-importance admit zero.
Admission as the eviction ratio grows. Block-wise and random-plus-compaction overlap at the ceiling. Use the legend to separate overlapping lines.

Reproduce with python kv_reclaimability_blog_experiments.py, which writes the static figures, results_summary.csv, and the interactive_data.json that drives this page. Fixed base seed, 30 seed replicas, numpy, pandas, matplotlib only. Importance scores are synthetic proxies for token value, not attention weights from a real model, and the realistic method curves reproduce retention geometry only, not accuracy.