Developer releases TurboFieldfare, an open-source Mac inference engine streaming Gemma 4 26B from SSD in about 2GB RAM

TurboFieldfare is a Swift and Metal inference engine that runs 4-bit Gemma 4 26B-A4B-IT on Apple Silicon Macs using about 2GB of RAM. It keeps shared model components and KV cache in memory while streaming routed experts from SSD, reaching 5–6 tokens per second on an 8GB M2 MacBook Air and up to 35 tokens per second on an M5 MacBook Pro.

The project shows how on-device AI can stretch beyond RAM limits by combining model architecture awareness with SSD streaming and caching. If approaches like this mature, larger local models could become practical on mainstream laptops without requiring high-memory hardware.