PAN 2 world model released for action-conditioned video simulation
IFM posted a PAN 2 model card on Hugging Face describing a released world-model checkpoint that simulates how actions change a scene. According to the model card, it accepts an optional image and text, generates videos up to 20 seconds at 704x1248 and 24 fps, and uses a 29B-parameter MMDiT perceptual backbone alongside a Qwen3.5-9B text encoder and Wan 2.2 VAE decoder. These capabilities and training details are vendor claims reported through a single r/LocalLLaMA post.
PAN 2 is presented as a tool for visually comparing possible action outcomes before an intelligent agent selects a plan, making it notable as a released world-model checkpoint rather than a conventional video generator.
| Overview PAN 2 is a general-purpose world model: it simulates how the world will change in response to actions. Given a current observation (an optional image plus text) and actions described in text, it simulates the outcome as a video of up to 20 seconds. This supports simulative reasoning: an agent can imagine the outcomes of its candidate actions before choosing among them, by simulating each plan with PAN 2 and acting on the one whose simulated outcome is best. PAN 2 follows the Generative Latent Prediction (GLP) architecture formalized in Critique of World Models and couples conceptual reasoning by a large language model with perceptual reasoning by a latent diffusion model. The language model, the conceptual reasoning backbone, reasons over the observation and the requested actions and writes the next world state as a structured, time-grounded description of events, timed actions bound to actors, and camera motion. The frozen text encoder maps the structured description to the conditioning features used by the diffusion model. The 29B-parameter MMDiT, the perceptual reasoning backbone, trained from scratch on hundreds of millions of videos annotated in the same format, predicts the perceptual detail that the description leaves open: the video latents that realize the state, conditioned on it and on the VAE latent of the starting image. The Wan 2.2 VAE decoder renders the predicted latents as the next observation, so every prediction can be seen and checked against real footage. Checkpoints PAN 2's latent diffusion model is released as a single checkpoint that serves both tasks. The frozen text encoder (Qwen3.5-9B) and VAE (Wan 2.2) are downloaded from their own repositories. submitted by /u/pmttyji [link] [comments] |
#PAN 2#IFM#Qwen3.5-9B#Wan 2.2 VAE#MMDiT#Generative Latent Prediction