Memory for long-horizon video generation must satisfy two competing requirements: it must remain bounded so generation can continue indefinitely, while still recovering what was observed at a place when the camera returns, regardless of how long ago it was seen. Reconstruction-based memories provide spatial recall by caching and reprojecting explicit geometry, but require growing storage and depend on estimated 3D structure. Recurrent memories keep a constant-size state, but retention is governed by temporal recurrence, causing old observations to fade with time.
We introduce Smart, a slot memory addressed by region tokens, which combines bounded storage with spatially addressed recall. Smart stores content in a constant-size slot recurrence whose writes are addressed by the observing camera and whose reads are queried by the rendering camera, without constructing an explicit 3D representation or maintaining a growing cache. Consequently, recall is conditioned on inter-view geometry rather than on recency. Each read additionally provides a confidence estimate that conditions a chunk-causal diffusion transformer, allowing it to selectively trust retrieved content while relying on its generative prior elsewhere.
On long trajectories in Memory Maze and Minecraft, Smart maintains high-fidelity recall far beyond its training horizon while using constant memory and per-step cost.
Smart has two components. A recurrent slot memory stores content under pose-derived keys and, when queried with a target camera, returns a coarse draft of that view together with a confidence estimate in latent space. A chunk-causal diffusion model sharpens the draft into video, trusting each cell in proportion to the memory's confidence. Generated chunks are written back into the memory, so rollouts of arbitrary length run at a constant memory size.
The agent walks forward, turns around and walks back along the same path. Up to the red line, the ground-truth walk is written into memory. Afterwards, each method generates the way back, so every frame revisits a place seen before. Smart keeps the maze's walls and colours even after a 1,601-frame walk, while DFoT paints walls in the wrong colours and Causal (Full) blurs.
Forward walk of 401 frames
Forward walk of 801 frames
Forward walk of 1,601 frames
Up to the red line, the ground-truth context is written into memory. From that point on, each method generates the rest of the trajectory from the agent's actions. The agent keeps coming back to places from the context. Smart brings back their landmarks (trees, terraced cliffs, a sand path along a river), while MilliVid and FramePack drift into scenery that was never there.
Write. (a) Euclidean distance between the positions of two forward frames and (b) the probability that their region tokens land in the same slot. (c) The episode's trajectory from its start to its end position. Read. (d) Mean read mass per slot against the distance from the query to the content the slot holds, where that content's position is the decayed average of everything written to the slot. (e, f) Attribution of each reverse read to the forward frames in memory; the white trace marks the pose-nearest forward frame.
Smart addresses its slot memory with region tokens, which decouples forgetting from elapsed time: a region is written into the slot selected by its camera pose, so a slot only decays when it is written again, when the camera returns to its place. On 401-frame Memory Maze episodes, where the forward leg is written and the reverse leg is read back from the state, frames that are close in position (a) are likely to land in the same slot (b). Reading is a softmax attention over the camera pose encodings, which yields a convex combination of write tokens, so every read is attributable to the frames it came from. Slots holding the query's place draw 1.7× the uniform read mass (d). Not all heads behave alike: some localize, retrieving the frames nearest the query (e), while others keep a global attention pattern (f).
A Minecraft rollout on the left, and Smart's memory on the right: every square is one slot, grouped by layer (L0–L7, columns) and head (h0–h5, rows), with 16 slots per head. While the context is recorded, each frame's region tokens are routed to slots, which fill up as the camera sees new places; slots written by the current frame light up in orange. Once generation starts, the generated frames are written back into memory. The counter under the rollout tracks how many of the 768 slots have been written so far.
@article{cetin2026smart,
author = {Cetin, Eren and Erbach, Julius and Bozic, Vukasin and Go, Hyojun and Zhao, Zixiang and Kalischek, Nikolai and Tombari, Federico and Schindler, Konrad and Narnhofer, Dominik},
title = {Long-horizon Video Generation with Slot Memory Addressed By Region Tokens},
journal = {arXiv preprint arXiv:2610.19257},
year = {2026},
}Special thanks to the authors of MilliVid for their generous contribution: their Minecraft dataset generation pipeline, and their highly descriptive checkpoints for causal rollout generation, FramePack and MilliVid, made our Minecraft experiments and comparisons possible.
We also thank the authors of DFoT and Lyra 2.0 for their open-source contributions: our Memory Maze baselines, DFoT and GeoCache, are trained from their official codebases.