TinyWorldFormer在树莓派Pico 2 W运行生成游戏
开发者在 Raspberry Pi Pico 2 W 上同时运行了 TinyWorldFormer 的两个小型神经网络:世界模型生成下一帧和奖励,行动网络则根据生成画面选择动作。世界模型采用循环式、int8 量化的 transformer,行动网络还能在生成环境中进行流式强化学习,两者均运行于 RP2350 的第一核心。项目还提供五种小游戏、量化细节、基准数据、源代码和复现步骤。
该项目展示了世界模型生成与在线控制可在低成本微控制器上协同运行,并公开了实现细节和复现方法。
Using 2 small neural networks to generate and play the game at the same time on a Raspberry Pi Pico 2 W. The first one creates the game: given the current frame and the action, it predicts the next frame and the reward. The second one plays that game: it uses the generated frame and picks an action.
Every frame above was sampled by the world model network, and the actions were taken by the actor network.
Why
I got interested in looped transformers, where one transformer block is applied several times with the same weights instead of stacking different layers. Reusing weights is cheap on memory, so I wanted to see if a looped world model could be small enough to run on a microcontroller. I picked the Pico 2 W because it is cheap and has an FPU, and small games because I could write the real versions directly into numpy and check every generated frame against them. Later I added a streaming RL agent so the board could also learn to play inside its own dream.
部分来源内容未展示,可前往原文继续阅读。