Next token prediction, and what attention looks at

The model reads left to right and predicts the next token from everything before it. Attention is how each position pulls information from earlier tokens. Step through the sentence, and click any token to inspect where its attention goes.

Distributions and attention weights here are illustrative, chosen to look plausible. They are not read from a real model. The shape of the mechanism is the point.