Dharma-AI says domain-specific vocabularies cut Qwen3-4B essay grader inference cost by 77%

Dharma-AI previews VocabX, a vocabulary-specialization method tested on a production Portuguese essay grader built on Qwen3-4B. By weighting token choices by measured serving cost rather than frequency, it reduced the output vocabulary from about 152,000 to 65,000 rows and cut measured inference cost by 77% without changing the base model, data, or training recipe.

For teams serving small, domain-specific models, the reported 7.4x higher cost of generated tokens suggests tokenizer design can be a deployment lever, not just a preprocessing detail. The trade-off is narrower vocabulary coverage for lower decode cost; the limitation is that these results are from one grader, and the paper is still under review, so savings may vary by hardware, batch size, context length, and task.