LeCun's H-JEPA hierarchical world model takes aim at agents and RSI
Yann LeCun's new world model H-JEPA is presented as the first reproducible implementation of the hierarchical JEPA idea from his 2022 paper: multiple JEPAs are stacked, each with its own observation encoder, action encoder and latent predictor, forecasting states at different time horizons, with high levels setting coarse goals that lower levels refine into actions. The article reports that a three-layer H-JEPA raised success on Visual AntMaze to 73% from 18% for single-layer LeWM, but on Push-T two layers worked best and three or four layers degraded, using roughly 58% and 14% of LeWM's transferable training data respectively; training uses neither reward nor reconstruction, and DROID teleoperation experiments needed an inverse-dynamics constraint to avoid slow-feature collapse. It frames H-JEPA as a counterpoint to today's agents, which split tasks into more steps and more tokens, and to recursive self-improvement, which LeCun calls only an amplifier of existing abilities.
If hierarchical world models hold up, they could suggest a route to stronger AI that does not rely only on scaling tokens and trial-and-error at one granularity.