AI code reviewer cuts inference costs 68%, from $2.43 to $0.77 per review, after prompt-cache, call-count, and model changes

The company says its AI code reviewer’s average cost rose from $1.19 in April to about $2.46 by late August, largely after model changes and token growth, before optimization work cut the average to $0.77. Key savings came from sharing a stable cached prompt prefix across repositories, reducing unnecessary LLM calls, moving reference material into prompts, and experimenting with models and effort levels; median review time fell from 4m 09s to 1m 32s.

For teams building agentic code-review tools, the reported cache-write/read split shows why token instrumentation by call can drive concrete savings: fixing prompt caching alone helped reduce cost from $2.47 to $2.02 per review. The trade-off is less prompt flexibility, since they removed repository-specific content before the shared prefix and disabled changing tool descriptions; the unanswered question is how these optimizations affect review quality outside this specific workload.