modded-nanoGPT

Headlines · 1

1. Nov 3
   Karpathy's nanochat offers a low-cost LLM training harness

   Andrej Karpathy's MIT-licensed nanochat repository provides a minimal, hackable experimental harness for training language models across tokenization, pretraining, fine-tuning, evaluation, and inference. It is designed to run on a single GPU node, with model size selected through a depth setting while other hyperparameters are calculated automatically. Karpathy says a GPT-2-capability model can be trained for about $48 in roughly two hours on an 8xH100 node, or for closer to $15 on a spot instance; the report does not independently verify these figures.

   GitHub Trending · Python · 🔥 0

Experience and discussion from the community

Share my modded-nanoGPT experienceAsk about modded-nanoGPT