Part A

How little of the web survives the pipeline

A pretraining corpus does not come from the web. It comes from what is left of the web after a long chain of filters. Each stage below removes more than the last. Hover a stage to see what it takes out.

6.8% of the raw crawl reaches training. The rest is thrown away, on purpose.

Fractions are cumulative shares of the raw crawl and are illustrative, chosen to match the shape reported for pipelines like RefinedWeb and FineWeb. The lesson is the order of magnitude, not the decimals.

Part B

The mixture is a set of bets on capability

What survives still has to be blended. Drag the sliders to change the training mixture. They always renormalize to 100 percent. Watch the capability profile move, and notice that code and math lift reasoning and coding together, because reasoning transfers across domains.

Capability scores are a toy linear model, each data category contributes a fixed weight to each capability. Numbers are illustrative and exist only to show direction of tradeoff, not to predict any real benchmark.