Forth Interpreter Embedded in Nemotron 3.5 Lightning Weights
Reddit author /u/Mr_Finious reports adding an untrained 27-layer Forth interpreter network alongside an unchanged NVIDIA Nemotron 3.5 Lightning model. The interpreter executes each Forth word as the model generates it, with all 765,734 tested machine tokens matching a Python reference. LoRA fine-tuning raised the author's score on a 600-problem held-out test from 79.1% to 92.1%, although the model trails Python on long integers and is not intended for production.
The self-reported experiment provides a concrete demonstration of embedding a working machine in model weights, while also showing that reliability depends on fine-tuning and incurs a full forward pass per machine step.
I learned RPN on an HP calculator and later spent more time with Forth than was strictly useful. When I saw Qwenputer compile a machine into a language model's weights, I wanted to know two things. Would a stack language fit that idea well? And could a model with only 3B active parameters drive a machine like that?
The setup: NVIDIA Nemotron 3.5 Lightning (30B, 3B active) is left unchanged. Next to it, in the same model, is a second network of 27 layers. A Python script calculated its weights; none of them are trained. It is a Forth interpreter. The model writes a Forth block as ordinary text. The interpreter runs each word while the model is still writing, one step per forward pass. When the block closes, it writes the output as the model's next tokens. No tool call occurs, and generation does not stop.
RPN suits this well. A Forth word ends at a space, so the machine can run it as soon as the next word starts, without parsing the whole program. Operands come before the operator, so when the model writes +, both numbers are already on the stack. That is the order a left-to-right model writes in anyway. The stack itself lives in Mamba-2 state. One entry is one row, 20 digits wide, and popping a number means adding its exact negative.
#NVIDIA#Nemotron 3.5 Lightning#Forth#Qwenputer#Qwen#DeepMind#LoRA#vLLM#NeMo AutoModel