Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Claude-Generated DOS Language Model Fits on a Floppy Disk

AI summary

A Reddit developer reports that Fable 5.2 wrote the transformer for a DOS language model, while Opus 5.5 generated roughly 3,500 facts along with small talk and math for training. The model reportedly has 367,000 parameters, uses 8-bit integer weights, and totals about 435 KB with its program, allowing it to run in DOS memory and fit on a floppy disk. The developer says it handles familiar words and facts reasonably well but produces nonsense on complex or lengthy prompts.

In-depth insight

The report presents an unusual demonstration of Claude models helping build and train a very small language model that can run on an Intel Pentium under DOS, though no independent benchmark results are supplied.

I let Fable 5.2 write its own transformer. Opus 5.5 generated the training data: about 3500 facts (and small talk + math), each asked and answered in different ways. It has 367K parameters and runs on an Intel Pentium. The weights are stored as 8bit integers with one scale per row, so the program and weights together come to about 435 KB and fit in DOS memory and on a floppy. It reads and writes one letter at a time. It does pretty well on words and facts it has seen, but if I ask something complex or too long, it spits out utter nonsense. Edit: https://github.com/WollyMonjac/Tiny-2.1 submitted by /u/wollymonjac [link] [comments]

#Tiny-2.1#Claude#Fable 5.2#Opus 5.5#Anthropic

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.