nanochat

Headlines · 3

1. 3d
   DoubleAI Introduces Mechanistic Intervention for Model Self-Modification

   新智元 reported that DoubleAI introduced “mechanistic intervention,” a method that places programming agents into model training and inference so they can generate and insert Python code layers. DoubleAI’s reported experiments include a 20-line layer that enabled gradient descent during one forward pass, while DeepSeek-V4.1-Flash reportedly used a self-written probe-and-resampling program to score 71.0% on AIME and HMMT 2026, compared with 63.9% for standard long-chain reasoning and 65.4% for length-only resampling. The supplied evidence comes from one secondary account and includes no independent replication or implementation audit.

   新智元 · 🔥 0
2. Nov 3
   Karpathy's nanochat offers a low-cost LLM training harness

   Andrej Karpathy's MIT-licensed nanochat repository provides a minimal, hackable experimental harness for training language models across tokenization, pretraining, fine-tuning, evaluation, and inference. It is designed to run on a single GPU node, with model size selected through a depth setting while other hyperparameters are calculated automatically. Karpathy says a GPT-2-capability model can be trained for about $48 in roughly two hours on an 8xH100 node, or for closer to $15 on a spot instance; the report does not independently verify these figures.

   GitHub Trending · Python · 🔥 0
3. Oct 22
   NVIDIA adds official eight-node DGX Spark clusters, reaching 1 TB unified memory

   At GTC Berlin, NVIDIA announced official support for eight-node DGX Spark clusters, which scale unified memory from 128 GB on a single node to 512 GB on four nodes and 1 TB on eight, per the company's technical blog. NVIDIA says the capacity lets the full 753B GLM 5.3 checkpoint from Z.ai run across the 8x 128 GB cluster, and it measured roughly 80 tokens per second generating with GLM 5.3 Flash. Vendor benchmarks also show dense Qwen3.8-27B decode rising from about 12 TPS on one node to 52 TPS on eight, and nanochat pre-training delivering a 7.79x speedup at 97.4% scaling efficiency; NVIDIA says eight-node support in NVIDIA Sync, whose Cluster Assistant handles setup and device management, is coming by the end of the year.

   NVIDIA Technical Blog · 🔥 0

Experience and discussion from the community

Share my nanochat experienceAsk about nanochat