TinyStories
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 3
- 4-bit GPT training speeds up on a single Nvidia B200
VizuaraAI's author reports training GPT models end to end on one Nvidia B200 with an NVFP4 forward pass, an FP8 backward pass using stochastic rounding, and high-precision master weights. For the 840M-parameter model, 2,000 steps took about 4.3 minutes versus 5.5 minutes in BF16, while final validation losses were 1.691 and 1.688 respectively. Reported per-step speedups ranged from 18% for the 219M model to 38% for the 3.3B model, although the largest run covered only 1,000 steps.
r/LocalLLaMA · 🔥 0 - llama2.c TinyStories Runs on a 2006 iPod
Reddit user r0tlier0 reported building a Rockbox plugin that runs Karpathy's llama2.c TinyStories models on an iPod Video 5.5G. The device has an 80 MHz ARM7TDMI processor, 64 MB of RAM and no FPU, so the developer ported llama2.c's int8 inference code to integer-only mathematics with lookup tables for exponential and softmax operations. The 15M-parameter model is 17 MB and loads from the iPod's hard drive into its music buffer, although music cannot play while inference is underway.
r/LocalLLaMA · 🔥 0 - Creator announces NanoStories dataset with 50 million generated stories
u/Tall_Abrocoma_3533 announced NanoStories, a TinyStories-style dataset that the creator says contains 50 million stories, about 25 times the size of TinyStories. According to the creator, the stories were produced with a rule-based algorithm rather than a language model. The creator reported that generation took just under 10 minutes on a phone CPU, with Parquet conversion and upload to Hugging Face taking a few more minutes. The count, size comparison and processing times have not been independently verified.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my TinyStories experienceAsk about TinyStories
Nobody has shared their experience with TinyStories yet.