TinyStories模型登陆2006款iPod
AI 总结
Reddit用户r0tlier0称,其开发的Rockbox插件可在iPod Video 5.5G上运行Karpathy的llama2.c TinyStories模型。设备配备80 MHz ARM7TDMI处理器、64 MB内存且没有FPU,因此开发者将llama2.c的int8推理代码改为纯整数运算,并用查找表处理指数和softmax计算。该1500万参数模型占17 MB,可从iPod硬盘载入音乐缓冲区,但推理期间无法播放音乐。
深度见解
这一项目展示了在没有浮点运算单元的2006年便携设备上,以纯整数边缘AI路径运行小型语言模型的技术尝试。
| hardware: PortalPlayer PP5022, ARM7TDMI @ 80 MHz, 64 MB ram wrote a fun rockbox plugin that runs Karpathy's llama2.c TinyStories models on an ipod video 5.5G. since there's no FPU, I ported llama2.c's int8 runq.c to use only integer math. it uses lookup tables -> exp/softmax + an integer square root for RMSNorm the 15M model is 17 MB, so it loads directly off the hard drive into the music buffer. Unfortunately you can't listen to music while it works lol want to eventually run this on both cores, and set up a more ipod themed model that does local music recommendation submitted by /u/r0tlier0 [link] [comments] |
#llama2.c#TinyStories#Rockbox#iPod Video 5.5G#PortalPlayer PP5022