Developer runs a tiny autoregressive language model on a 1975 MOS 6502-powered BBC Micro with 32KB RAM

The author trained a tiny Mamba-based autoregressive language model and built an inference engine for the 8-bit MOS 6502 processor used in machines like the BBC Micro and Apple II. The final system fits into about 25KB of usable memory, with 9KB for inference code and 13KB for model weights, and can generate simple story-like text directly on a vintage BBC Micro.

The project shows how modern ML techniques can be aggressively compressed for extreme edge computing, even on hardware without native multiplication or modern numeric types. While not practical for mainstream AI workloads, it highlights a growing interest in minimal models, efficient inference, and understanding AI systems by pushing them onto highly constrained devices.