Skip to content

devlog/token-math/sending-the-whole-file-to-the-model-burns-tokens-1102.md

Token mathEN

Sending the Whole File to the Model Burns Tokens

Pasting a whole file when three hunks would do

The default reflex in every editor and terminal is to paste the entire module the moment the model asks for context. Nine hundred lines go in, eight hundred of them unchanged since the branch started. The model reads every byte anyway, and on the next turn you pay for the same untouched imports and boilerplate all over again.

A diff describes the same change in a fraction of the tokens. Three hunks with a few lines of context around each one communicate what actually happened far better than a wall of unchanged code, because the boundaries of the edit become visible. Keep the full file for the opening turn of a task, then switch to diffs for every revision that follows.

  • Send the diff with a small context window whenever the model only needs to reason about the edit itself.
  • Reserve full file dumps for the first turn and for moments when the model explicitly reports missing context.
  • Strip lockfiles, generated bundles, and snapshot fixtures before the request ever leaves your terminal.

This matters more when you meter usage honestly. With Fluxo you pay per real token, including cache reads and cache writes, with no monthly fee, so every redundant line has a price you can actually see. The one key and the one base URL at https://api.fluxorelay.com sit in front of Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more, so the same diff habit pays off across every model you switch to.

Diffs without context lines that leave the model guessing

The second mistake is sending a patch that is technically valid and practically unreadable. A hunk with no surrounding lines forces the model to infer what the function looked like before the edit, and inference is exactly where hallucinated code comes from. It will happily rewrite a line it never actually saw.

Context lines are the contract between your working tree and the model's understanding of it. They anchor the hunk inside a recognizable function, name the local variables that matter, and show the indentation style the reply must match. Without them the model is editing a fragment in a vacuum, and its suggestions drift away from your codebase.

  • Include at least three lines of context above and below every hunk so the edit sits inside a recognizable function.
  • Keep the file path and the hunk headers exactly as the diff tool wrote them.
  • Watch for tabs, trailing whitespace, and line-ending drift that make an unchanged line look modified.
  • Prefer a plain unified diff over a colored terminal excerpt or a hand-copied snippet.

Here is the practical test: if you cannot tell which function a hunk belongs to without opening the file yourself, neither can the model. Regenerate the patch with more context and send that instead of apologizing in prose. Most editors can produce this from the command line with a single git invocation, and the extra six lines per hunk cost far less than one wasted correction round.

Diffs generated against a stale base revision

A diff is only meaningful relative to the state the model assumes it is patching. The classic failure is generating the patch, continuing to edit the file for another twenty minutes, and then sending the patch anyway. Line numbers no longer match, hunks overlap each other, and the model applies a change to the wrong function or reports a conflict that is really your own drift.

Regenerate diffs immediately before the request, and make the base explicit. A commit hash or a revision identifier costs a handful of tokens and saves an entire debugging loop. When a reply references a function that does not exist in your tree, the base is almost always the problem rather than the model.

  • Regenerate the diff right before sending it instead of reusing one captured earlier in the session.
  • Include the revision identifier or commit hash so the model knows exactly which base it is reasoning about.
  • Rebase or stash local experiments before diffing so hidden edits do not appear as unrelated noise.

Re-sending identical hunks and poisoning the cache

Agent loops love to restate themselves. On every turn the conversation is replayed, so a diff pasted at the start of a long run is paid for again and again. Worse, small edits to an earlier message shift the prefix, which can invalidate a cache that would otherwise have turned those repeated reads into the cheapest tokens on the bill.

Keep the transcript append-only. Never rewrite an earlier message to fix a typo, because that changes the prefix and converts cache reads into cache writes. Put stable project context first and the volatile patch last, so each new hunk extends the cached region instead of displacing it.

  • Append new hunks at the end of the conversation instead of rewriting the original request.
  • Correct mistakes in a fresh message so the earlier prefix stays byte-identical.
  • Batch related edits into one diff rather than sending five tiny patches across five turns.

Because Fluxo bills per real token including cache reads and cache writes, with no monthly fee, a stable prefix shows up as a visible discount rather than a hidden optimization. The habit pays the same way whether you are driving Claude Code, Codex, Gemini CLI, or any OpenAI SDK against the same endpoint.

Counting cache reads and cache writes instead of guessing

Most rough estimates only count the prompt and the completion. Real usage also includes the cache write on the first pass, the cache read on every pass after that, and the tool output your agent keeps appending to the transcript. If you only watch completion length you will wrongly conclude that diffs did not help.

Measure it properly: run one task with full files and the same task with diffs, then compare prompt tokens, cache reads, and wall-clock time. The gap is usually large enough to change how you structure every subsequent session, and it gets larger as the repository grows.

  • Compare prompt tokens and cache reads across two runs of the same task before drawing conclusions.
  • Track tool output separately because agent transcripts often grow faster than your source files.
  • Use a smaller model for diff review and keep a larger one for architecture decisions.

Matching model to task is itself a token saving. Claude Haiku 4.5 or DeepSeek V4 Flash can review a three-hunk patch quickly and cheaply, while Claude Opus 5.5 or GPT-6.1 Sol earns its cost on sprawling refactors where the full file is genuinely necessary. Keep the fast model on the tight diff loop and escalate only when the change stops being local.

If you want to try this in your own workflow, create a Fluxo account and point your existing tooling at https://api.fluxorelay.com with one key. If anything is unclear after you sign in, leave a note on the contact page and we will pick it up from there. And when people you invite start paying, you earn a referral share of what they spend.

Next in the buffer