All Blog Posts

2026

The Closing of the Recipe

Part 5. When every model shares one skeleton, the moat becomes the thing nobody publishes.

The Death of Chinchilla

Part 4. Why the most cited recipe in pretraining optimizes the wrong thing, and why emergence is mostly a measurement artifact.

Logical KV Savings Are Not Physical Memory Savings

Token-level KV cache eviction can report large logical savings while returning little usable GPU memory to a paged allocator. How much it actually returns is...

Why a Busy GPU Is Not a Useful GPU in LLM Serving

A reproducible mechanism study of schedulability, KV-cache lifecycle, and useful tokens per dollar in LLM serving, with a controlled scheduler ablation and h...

Pretraining Is a Fab Problem

Part 3. Why the frontier is gated by memory bandwidth, packaging capacity, and depreciation, and barely at all by ideas about intelligence.

The Hardware Lottery

Part 2. Why every frontier model has converged on the same skeleton, and why the silicon, not the science, is what chose it.

Data Is the Whole Game

Part 1. Why the corpus, not the architecture, is the moat, and why the two loudest panics about running out of data are both told wrong.

The Fossil Fuel of Intelligence

A field guide to pretraining for engineers, and why most of what you have read about it is already obsolete.

The Case for System-Closed Reliability in the HBM Supply Chain

Thoughts on a possible path for scaling HBM bandwidth through system-level reliability