PAN 2 发布可模拟行动后果的视频世界模型
AI 总结
IFM 在 Hugging Face 发布 PAN 2 模型卡,称其检查点可模拟行动如何改变场景。根据模型卡,该模型接收可选图像和文本,可生成最长20秒、704x1248、24 fps的视频,并采用29B参数 MMDiT 感知骨干、Qwen3.5-9B文本编码器和 Wan 2.2 VAE 解码器。相关能力与训练信息均为厂商自述,目前仅由一则 r/LocalLLaMA 帖子报道。
深度见解
PAN 2 被定位为在智能体选择计划前可视化比较候选行动结果的世界模型,因此其检查点发布本身具有较高关注度。
| Overview PAN 2 is a general-purpose world model: it simulates how the world will change in response to actions. Given a current observation (an optional image plus text) and actions described in text, it simulates the outcome as a video of up to 20 seconds. This supports simulative reasoning: an agent can imagine the outcomes of its candidate actions before choosing among them, by simulating each plan with PAN 2 and acting on the one whose simulated outcome is best. PAN 2 follows the Generative Latent Prediction (GLP) architecture formalized in Critique of World Models and couples conceptual reasoning by a large language model with perceptual reasoning by a latent diffusion model. The language model, the conceptual reasoning backbone, reasons over the observation and the requested actions and writes the next world state as a structured, time-grounded description of events, timed actions bound to actors, and camera motion. The frozen text encoder maps the structured description to the conditioning features used by the diffusion model. The 29B-parameter MMDiT, the perceptual reasoning backbone, trained from scratch on hundreds of millions of videos annotated in the same format, predicts the perceptual detail that the description leaves open: the video latents that realize the state, conditioned on it and on the VAE latent of the starting image. The Wan 2.2 VAE decoder renders the predicted latents as the next observation, so every prediction can be seen and checked against real footage. Checkpoints PAN 2's latent diffusion model is released as a single checkpoint that serves both tasks. The frozen text encoder (Qwen3.5-9B) and VAE (Wan 2.2) are downloaded from their own repositories. submitted by /u/pmttyji [link] [comments] |
#PAN 2#IFM#Qwen3.5-9B#Wan 2.2 VAE#MMDiT#Generative Latent Prediction