Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Reddit discussion argues action labels are the missing layer in game world-model data

AI summary

A Reddit discussion argues that action labels, rather than video volume, are the main bottleneck in making game data trainable for world models. The poster cites VPT and Muse as examples of projects that retained gameplay inputs, while describing an unnamed deterministic multiplayer strategy game that records actors, messages, and world state; these claims are not independently verified here. The post also asks whether world-model training should use high-level strategic actions or low-level controls, and whether language-linked multi-agent data is useful beyond evaluation.

In-depth insight

The discussion highlights a concrete data-design issue for game world models: training examples need traceable actions and state, not merely abundant gameplay video.

Most of the public story around game data and world models comes back to one thing: video is cheap, the action that caused each frame is not - vpt: about 2,000 hours of contractor play with keyboard and mouse recorded was enough to train an inverse dynamics model that then labelled around 70,000 hours of youtube - muse was trained on years of play from one studio's game, with the controller inputs kept - nvidia reportedly pulled inputs off streamers' on-screen controller overlays i run a real-time multiplayer strategy game where people and llm agents play the same wars, and the engine is deterministic, so every action label is exact. it also has things scraped footage never has: who acted, what they told the other players right before, and the full world state questions for anyone training these models.. submitted by /u/KnowledgeOk7634 [link] [comments]

#VPT#Muse#NVIDIA

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.