llama2.c TinyStories Runs on a 2006 iPod
Reddit user r0tlier0 reported building a Rockbox plugin that runs Karpathy's llama2.c TinyStories models on an iPod Video 5.5G. The device has an 80 MHz ARM7TDMI processor, 64 MB of RAM and no FPU, so the developer ported llama2.c's int8 inference code to integer-only mathematics with lookup tables for exponential and softmax operations. The 15M-parameter model is 17 MB and loads from the iPod's hard drive into its music buffer, although music cannot play while inference is underway.
The report shows a technically substantive attempt to run a small language model through an integer-only edge-AI path on a 2006 portable device without a floating-point unit.
| hardware: PortalPlayer PP5022, ARM7TDMI @ 80 MHz, 64 MB ram wrote a fun rockbox plugin that runs Karpathy's llama2.c TinyStories models on an ipod video 5.5G. since there's no FPU, I ported llama2.c's int8 runq.c to use only integer math. it uses lookup tables -> exp/softmax + an integer square root for RMSNorm the 15M model is 17 MB, so it loads directly off the hard drive into the music buffer. Unfortunately you can't listen to music while it works lol want to eventually run this on both cores, and set up a more ipod themed model that does local music recommendation submitted by /u/r0tlier0 [link] [comments] |
#llama2.c#TinyStories#Rockbox#iPod Video 5.5G#PortalPlayer PP5022