A free interactive primer
World Models101
The phrase means at least five different things, and the people using it rarely say which. Start with the map, then the machinery.
9 chapters · 2h 8m · no signup
Five things people mean
Start here if you arrived confused. It is the most common reason to.
Ordered by what they predict
More concrete ← what gets predicted → more abstract
Not a system you run
Five things the phrase is used to mean. Four of them are systems you can run; the fifth is a claim about what is inside one. Pick any of them.
The premise
Four labs will tell you they build world models and mean four incompatible things. One generates video you can steer. One produces 3-D geometry a robot can be trained inside. One learns a compact simulator it can plan against. One predicts embeddings and throws the prediction away.
A fifth group is not describing a system at all. When people argue about whether a language model “has a world model”, they are making a claim about structure found inside a network trained for something else, answered with interpretability evidence, not with anything you could run. Two people can agree on every fact and still disagree, because one is asking whether a system can simulate and the other is asking whether a network contains something.
Underneath the naming mess there is a real and old subject. It runs from Kalman filters through Schmidhuber’s 1990 papers to Dreamer, JEPA, Genie, and Marble. The literature is enormous and almost entirely written for people who have already read it.
So: nine chapters, built around things you can poke at. No prerequisites past comfort with a gradient and a bit of linear algebra. Chapter 1 is the field guide; everything after it is the machinery, with the definition in use stated wherever it matters.
Contents
9 chapters
- 01
What People Mean
The phrase now covers at least five different things, and the people using it rarely say which. A field guide to telling them apart before you read another paper about them.
The map: five definitions of the term, and the question that tells them apart.
13 minRead → - 02
The Idea
The oldest of the five definitions, and the one that gave the term its technical meaning: a learned simulator you can roll forward and plan against.
Two cars, one corner: a reflex controller and a predictive one, side by side.
12 minRead → - 03
Prediction as Learning
Guess the next thing, check, adjust. Why that loop is enough to extract structure from raw experience, and why compression and prediction turn out to be the same problem.
Train a next-state predictor live and watch its loss surface find the physics.
14 minDrafting - 04
Latent Space
You cannot predict pixels and you should not try. Encoders, bottlenecks, and the move from what the world looks like to what state it is in.
Drag a point through a 2-D latent space and watch the world it decodes to.
15 minSoon - 05
Dynamics
The transition model: given where we are and what we do, where do we end up? RNNs, state-space models, transformers, and the compounding error that eventually eats them all.
Roll out an imagined trajectory against the real one and watch them peel apart.
16 minSoon - 06
Learning in a Dream
Once you have a simulator, you can practise inside it. Policy learning in imagination, why it is so sample-efficient, and how agents learn to exploit the bugs in their own dreams.
An agent trained only in the dream, dropped into the real environment.
15 minSoon - 07
The Case Against Generation
Predicting every pixel means spending capacity on the position of every leaf. The argument for predicting in representation space, and what JEPA actually changes.
The same ambiguous future scored by pixel loss and by representation loss.
14 minSoon - 08
Video as World Simulator
Sora, Genie, GameNGen, Cosmos. What happens when you scale generative video until it becomes controllable, and what 'emergent physics' does and does not mean.
Action-conditioned rollout: hold a key, steer a generated world.
16 minSoon - 09
What's Broken
Long-horizon drift, object permanence, counterfactuals, and the fact that nobody agrees how to evaluate any of this. The honest state of the field.
A drift meter: watch a state-of-the-art rollout lose the plot over 1,000 steps.
13 minSoon
Who this is for
The engineer
You ship models but the RL-adjacent literature has always felt like a different country. You want the map, not the tour.
The researcher next door
You work in another subfield and need to know what people mean when they say a video model 'understands physics'.
The student
You have the maths and the time and want a path through the papers that is ordered rather than alphabetical.
The sceptic
You suspect a lot of this is overclaimed. Chapter 9 is for you, and you are probably partly right.
A note when the next chapter lands
One email per chapter, which works out to roughly one a month. No newsletter, no digest. Unsubscribe link in every send, and the list is never shared.