Researchers propose BITCOS compression to cut ternary LLM weight storage below the 1.58-bit theoretical baseline

A new paper finds that real ternary LLMs do not use {-1, 0, +1} weights evenly, with zeros making up as much as 51.5% of weights across 29 tested models. The authors introduce BITCOS, a distribution-aware layout using a presence bitmap and compact sign vector, reaching as low as 1.485 bits per weight and improving inference throughput by up to 1.27x on tested CPUs and GPUs.

The work shows that practical model distributions can beat nominal information-theoretic assumptions when compression schemes exploit real sparsity patterns. If adopted in inference stacks, ternary LLM deployments could gain lower memory bandwidth pressure and faster decoding without changing model architecture.