Decoding a Per-Request Bill in Your Terminal

Four Line Items That Make Up One Request Bill

A per-request bill in Fluxo is short enough to read in one glance, yet it hides more than most people expect. The relay reports four numbers beside every call: input tokens, output tokens, cache store tokens, and cache hit tokens. Each number comes from a different moment in the request lifecycle, and each one is priced on its own terms. If you only scan the total, you will never learn which part of your prompt is actually expensive.

Picture the request as a pipeline. Input tokens are what your client hands to the model right now. Output tokens are what the model writes back to you. Cache store tokens are prompt content that the provider persists after this call so a later call can reuse it. Cache hit tokens are that reused content arriving on a later request, usually at a lower rate. All four are billed as real tokens, and there is no monthly fee smoothing anything over.

• Input tokens count the prompt that enters the model during the request you are currently reading.
• Output tokens count everything the model generates, including any tool payloads it emits back to your client.
• Cache store tokens count the portion of the prompt that the provider writes into its cache for later reuse.
• Cache hit tokens count the cached prefix that a later request reuses instead of paying the full input rate.

Read the four lines in that order every time and the bill stops being mysterious. The rest of this article is about the ways that reading goes wrong, and how to fix each mistake before it costs you a debugging afternoon.

Mistaking Cache Store Tokens for Cache Hit Savings

This one shows up in almost every first week with a relay. You see cache store tokens climbing, you conclude the cache is working, and you move on. That is the wrong signal. A store is a write, and a write costs money. The savings only appear when a later request reports cache hit tokens against the same prefix. Rewrite one line near the top of a system prompt and the prefix changes, the old entry stops matching, and you pay for a fresh store on top of a hit you never received.

Several everyday habits make this worse. People prepend a timestamp for logging, they place a randomized request ID at the very front of the payload, or they let tool schemas serialize in a different order between calls. Each of those changes the beginning of the prompt, so each one invalidates the cached prefix no matter how much of the rest of the context is identical.

• Keep the stable part of every prompt at the very top so the cache prefix survives small edits.
• Remove timestamps, request IDs, and randomized values from the beginning of your payload entirely.
• Serialize tool schemas deterministically so a semantically identical payload hashes the same way twice.
• Check the cache hit line before you celebrate any drop in input tokens on a long session.

A quick test costs almost nothing: send the same request twice in a row from your terminal and compare the two bills. The second call should show cache hit tokens with a small or empty cache store line. If the second call repeats the full store, your prefix is unstable, and no amount of rate guessing will rescue the budget.

Mistaking Growing Agent Context for Fresh Input Tokens

Long sessions in Claude Code, Codex, or Gemini CLI accumulate context fast, and that is where the second mistake lives. Developers sum the input line across every turn of a session, compare that sum to the whole conversation size, and conclude the relay is charging for the same tokens again and again. Per-request billing does not work that way. Each bill describes one request as it left your client, so a ten-turn session legitimately shows ten input lines that overlap heavily.

The fix is to stop reading the numbers cumulatively and start reading them comparatively. Look at how the input line grows from one turn to the next rather than adding it up. The delta is roughly the new content you appended. Then look at how much of that turn arrived as cache hits instead of fresh input, because the delta you care about is the part you paid full price for.

• Compare consecutive input lines in one session instead of summing them into a misleading total figure.
• Track the delta between turns to see which appended content is genuinely new and unbudgeted.
• Count tool results as output on the turn that produced them, not as input on the next turn.
• Remember that a retry after a network error produces a real second bill with real token charges.

Output deserves the same care. When a model calls a tool, the tool call is generated text and lands on the output line, then the tool result you inject comes back as input on the following request. Teams that miss this assume the relay invented tokens. The ledger is honest; the mental model was wrong.

Mistaking One Rate Card for Every Model on the Shelf

Fluxo gives you one key and one base URL, https://api.fluxorelay.com, for Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more. That uniformity is exactly what invites the third mistake: assuming uniform pricing. It is a relay, and every model on the shelf carries its own token rates, its own cache behavior, and its own minimum cacheable prefix. A swap that looks free in your config is never free on the invoice.

Take a concrete pair. Claude Sonnet 5.5 and Gemini 3.8 Flash can sit behind the same endpoint and the same request shape, yet their input, output, store, and hit prices differ, and so does how much prompt each one requires before caching engages. If you benchmark one and then silently route production traffic to the other, your per-request costs drift even though your code did not change at all.

• Record the model name beside every bill so cost regressions are attributable to a specific decision.
• Re-read the cache rules whenever you switch models, because the cacheable prefix threshold moves too.
• Benchmark with your real prompt size rather than a tiny test payload that never triggers caching.
• Keep provider-specific response fields out of the prefix so a model swap does not invalidate the cache.

The same discipline applies when you experiment. Keep one model pinned per environment, and treat a model change as a pricing change that needs its own measurement pass before it reaches a shared team config.

Mistaking a Running Total for a Settled Balance

The last mistake happens at reconciliation time. A dashboard total is a running figure, and requests can land with a small delay while retries and streamed completions settle. People match that total against their own counter built from request logs, find a gap, and assume double billing. Usually the gap is a request in flight, a retry they did not log, or a rounding difference between token counts and billed amounts.

Give yourself a quiet window before you audit, then reconcile per request instead of per session. Read one request, confirm its four lines, and move to the next. This is where a per-request bill earns its keep, because a single disputed call is easy to explain and a murky session total is not. When people you invite start paying for their own calls, you also earn a referral share, so clean logs matter for more than debugging.

• Wait for the billing window to settle before comparing your own counters against the relay total.
• Reconcile one request at a time so any discrepancy has exactly one request attached to it.
• Log retries explicitly, because an automatic second attempt produces a second real token charge.
• Compare token counts first and money second, since rounding hides in the currency conversion step.

If you want to see these four lines on your own traffic, create a Fluxo account and send one request from your editor to https://api.fluxorelay.com. If a bill still looks wrong after you have traced the input, output, store, and hit lines yourself, leave a note on the contact page after signing in and include the request ID so the numbers can be followed end to end.