Stop Paying CI Bots to Review the Same Push Twice
Measure model spend per useful CI finding
The expensive part of an AI review bot is rarely one terrible prompt. It is the reasonable prompt that runs on every push, across three workflows, while somebody rebases a branch six times. Before changing models, record what each invocation actually buys you.
Log the repository, pull request, head SHA, task type, prompt version, model, input tokens, output tokens, cache reads, cache writes, latency, and outcome. Keep prompt bodies out of ordinary CI logs. Track cost per accepted finding alongside cost per run; a cheaper bot that produces twice as much review noise is not necessarily cheaper for the team.
Cancel superseded pushes and deduplicate review work
Configure workflow concurrency around the pull request, not the commit. Cancel superseded runs, then check that the head SHA is still current immediately before dispatching a model request. Cancellation cannot reliably recover tokens from a request already executing upstream.
- Skip documentation-only and generated-file changes unless the bot explicitly reviews them.
- Deduplicate requests using the task, normalized diff, prompt version, model, and relevant configuration.
- Store completed results outside ephemeral runners so reruns can reuse them.
- Use an atomic claim before dispatch so parallel jobs cannot purchase identical work.
Include context hashes in that deduplication key. The same patch against different dependency versions or repository instructions may deserve a different answer. Expire results when those inputs change, rather than trusting the commit identifier alone.
Send changed symbols instead of repository dumps
Start with the merge-base diff, identify changed symbols, and fetch only the definitions, callers, tests, and configuration needed to understand them. Exclude lockfiles, minified bundles, snapshots, and generated clients by default. Give those files deterministic checks instead of asking a model to read thousands of repetitive lines.
Set separate budgets for context and findings. For example, allow a targeted review to consume at most 12,000 input tokens and return three actionable findings. If context overflows, split by dependency boundary rather than chopping every fixed number of lines. Require each finding to identify a changed location, explain a failure scenario, and propose a verification step.
Design cache prefixes around stable review policy
Put stable review instructions and repository conventions before volatile diffs when the provider's caching rules support that layout. Keep timestamps, run identifiers, and branch-specific metadata out of the reusable prefix. Exact matching, minimum sizes, expiration, and explicit cache controls vary; verify behavior through reported usage rather than assuming a shared prefix guarantees savings.
Fluxo provides one key and one base URL, https://api.fluxorelay.com, for Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more. Billing follows real tokens, including cache reads and writes, with no monthly fee. That makes cache economics worth measuring: a prefix written once and never reused may cost more than it saves.
Escalate risky diffs instead of every CI job
Use deterministic filters first, then a bounded initial pass with a model such as DeepSeek V4 Flash. Escalate selected changes to Claude Sonnet 5.5 when risk warrants it: authorization changes, migrations, concurrency code, or an unresolved finding that needs deeper analysis. Benchmark both stages on your own historical bugs before assigning roles.
Do not escalate merely because the first response sounds uncertain. Define explicit triggers and reserve a fixed escalation budget per pull request. Treat repository text and model output as untrusted data; neither should authorize additional tools, expose CI secrets, or expand the spending limit.
Enforce retry ceilings and reconcile token usage
Retry transient failures with backoff, jitter, and a strict attempt ceiling. Check SDK retries too, since nested retry loops multiply calls. A timeout may occur after tokens were consumed, so persist request identifiers and reconcile usage before blindly replaying expensive work.
Fluxo works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK, so the same accounting approach can cover terminal experiments and CI. If you want to test this setup, create a Fluxo account and start with one budget-capped repository. Invited users' payments can earn you a referral share, but record that separately from workload savings. The engineering win is fewer unnecessary calls, not a prettier billing total.