Creator announces NanoStories dataset with 50 million generated stories
AI summary
u/Tall_Abrocoma_3533 announced NanoStories, a TinyStories-style dataset that the creator says contains 50 million stories, about 25 times the size of TinyStories. According to the creator, the stories were produced with a rule-based algorithm rather than a language model. The creator reported that generation took just under 10 minutes on a phone CPU, with Parquet conversion and upload to Hugging Face taking a few more minutes. The count, size comparison and processing times have not been independently verified.
| It is a Tinystories style dataset, that contains 50M stories, about 25x that of Tinystories, it was generated using a rule based algorithm, instead of a language model. It took just under 10 Minutes to generate and a few minutes to convert to parquet and upload to huggingface on a phone CPU. Any questions, feedback, or criticism is welcome! submitted by /u/Tall_Abrocoma_3533 [link] [comments] |