Abstract
This reading record examines latent-space video prediction as a compromise between pixel-level fidelity and long-horizon consistency.
Problem
Directly predicting high-resolution video is expensive and often unstable over long rollouts. A compact latent state may preserve the variables needed for prediction while reducing the burden on the dynamics model.
Method
The central design pattern is to encode observations into a structured latent state, evolve that state through time, and decode only when visual output is required.
Limitations
Latent compression can hide errors that become visible only when the model is asked to plan or generate many steps ahead. Evaluation must therefore include both reconstruction quality and rollout behavior.