Wrong Token Estimates: Mistakes Wrecking Your LLM Budget

Counting Characters Instead of Tokens in Your Estimates

The most common first mistake is estimating tokens by dividing character count by four. That ratio is a rough average for ordinary English prose on one specific tokenizer, and it drifts the moment your payload is code, JSON, log output, emoji, or text outside the Latin alphabet. A minified JavaScript bundle and a tidy Markdown file can differ by twenty percent in real tokens while having similar character counts, so treat the ratio as a sanity check, never as a billing forecast.

When an exact number matters, run the request body through the tokenizer for the model family you are actually calling, and run it again whenever you switch models. Two models that look similar on a benchmark can split the same paragraph into different token sequences, so a budget calibrated on one tokenizer is quietly wrong on another. Measure the full payload you send, including tool schemas and system instructions, because those bytes are billed like everything else.

• Measure the exact payload you send, including tool schemas and system instructions.
• Recount after every prompt refactor instead of trusting last week's number.
• Keep a small fixture of realistic prompts to compare tokenizer output across releases.
• Record the tokenizer name next to every estimate in your logs.

Resending the Entire Conversation on Every Request

Agent loops and chat interfaces have a habit of appending the full transcript to every call. The prompt grows linearly, so a fifty turn debugging session can turn a two thousand token request into a thirty thousand token request while the useful content barely changes. That is money spent re-reading history, and it is also latency spent waiting for the model to chew through it before the first token arrives.

Trim history deliberately. Keep the system prompt and the tool definitions, keep the last few turns verbatim, and compress older work into a short recap that carries decisions and constraints forward. Drop raw command output once it has been distilled into what actually mattered. When you work in Claude Code, Codex, or Gemini CLI, check which context the tool is replaying before you assume the request is small.

Watch for duplicate context that creeps in through templates, because it is invisible in a stack trace and obvious in an invoice. A system prompt copied into every user message, or a large file pasted after it has already been summarized, inflates the bill without adding information. Log prompt length per call so a sudden jump shows up in the same place you look for latency spikes.

Ignoring Cache Reads and Cache Writes in Your Cost Math

Prompt caching changes the shape of your bill rather than the amount of work you imagined. A cache write costs more than a plain input token because the provider is storing that prefix for you, and a cache read costs less because the stored state is being reused. If you only count a single input number, you will overestimate some calls and badly underestimate others.

Structure prompts so stable content comes first and volatile content comes last. System instructions, tool definitions, and reference documents make a good cacheable prefix, while the user question and the latest file diff belong at the tail where they change every turn. Breaking that order forces a fresh cache write on each request, and repeated writes are the most expensive kind of repetition.

• Count cache reads and cache writes as separate line items in your cost estimates.
• Place unchanging instructions before changing context so cache hits stay possible.
• Watch cache hit rates drop when you reorder prompts during a refactor.

This is one reason Fluxo bills per real token, including cache reads and cache writes, with no monthly fee. You can look at a trace and see exactly where the money went, instead of reconstructing it from a plan tier. When your usage is spiky, a quiet weekend costs nothing and a heavy sprint costs exactly what it used.

Switching Model Families Without Recounting Tokens

Changing models is often a one line change in config, and it silently changes your token math. Claude Sonnet 5.5 and Gemini 3.8 Flash tokenize prose differently, so the same prompt and the same reply can land at different counts and different prices per unit. If you compare two models using raw token totals, you are comparing two rulers with different tick marks.

The clean way to compare is to fix the workload first. Run the same set of real prompts against each candidate, record input tokens, output tokens, cache reads, cache writes, latency, and cost in one table, then judge. Cheaper per token does not mean cheaper per task when one model writes twice as much to answer the same question, and truncation or retries can erase any savings.

Having one key and one base URL at https://api.fluxorelay.com makes that experiment trivial, since Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more all sit behind the same interface. The same key works in Claude Code, Codex, Gemini CLI and any OpenAI SDK, so swapping a model is a config edit rather than a porting project.

Reading Usage Fields Too Late or Trusting max_tokens

With streaming responses, usage data often arrives in the final chunk or in a separate field after the text is done. If your code stops reading the stream the moment the text looks complete, or if you collect usage only in a success path, you lose the numbers that tell you what happened on partial or failed calls. Failed calls are not free calls, and they are frequently the ones with the most wasted tokens.

max_tokens is another trap. A request capped too low truncates the answer, and a truncated answer usually triggers a retry that bills the prompt again plus the discarded output. Raise the ceiling where it matters, stop sequences where they help, and treat repeated retries as a budgeting bug rather than background noise.

• Read the usage block on every response path, including errors and cancellations.
• Record input, output, cache read, and cache write tokens separately in your metrics.
• Alert when output tokens hit the max_tokens ceiling on a real request.

Once the numbers are flowing, keep them where you actually look. A tiny CLI that prints the last call's tokens and cost next to latency is worth more than a dashboard you never open, and a weekly rollup catches drift that a single request hides. If you want to try this in your own workflow, creating a Fluxo account takes a minute and gives you the same key and base URL for every model on the shelf, and when people you invite start paying, you earn a referral share of what they spend. If you get stuck, leave a note on the contact page after signing in.