1. 1 · Learning

    The model watches an agent move through two rooms, as images only. Each dot is one place, positioned by the 192 numbers the model uses to describe that image and seen here flat, from outside. Colour shows where the place really is (key, bottom left). The cloud spreads because training forbids the descriptions from collapsing onto a single point.

  2. 2 · Unrolling

    Flat, the cloud looks tangled, because the descriptions lie on a curved surface. Here that surface is laid out by measuring distances along it. It turns out to be two rooms joined by a door, a layout the model was never told.

  3. 3 · Imagining

    Given a picture of the goal (gold ring), the planner samples 300 action sequences and the model imagines where each one leads, without touching the real rooms. A hundred are drawn: faint lines are imagined futures, gold ones those that end closest to the goal. Each round samples again around the best.

  4. 4 · Walking

    The agent carries out the plan in the real rooms (bottom left), then plans once more from where it landed. The white line is its real path, placed on the map by describing what it actually saw at each step.