Token mathEN2026-10-26

The Retry Bill Behind a Hanging Agent Terminal

A hanging terminal can hide two paid attempts

Your agent reads a repository, requests a patch, and stalls. You interrupt it and rerun the command. The second attempt succeeds, but the first request may already have consumed tokens. A timeout tells you that your client stopped waiting; it does not prove that upstream generation stopped.

The cost includes repeated context, repeated output, and developer time spent wondering whether anything happened. Here is a terminal walkthrough that makes those attempts visible before you add automatic retries.

Prepare an isolated SDK experiment

Run these commands in Bash, inside a scratch directory. Keep the prompt small: the first measurement should establish behavior without uploading your repository.

python3 -m venv .venv — source .venv/bin/activate

python -m pip install openai httpx — read -r -s -p 'Fluxo API key: ' OPENAI_API_KEY; echo; export OPENAI_API_KEY

export OPENAI_BASE_URL=https://api.fluxorelay.com/v1 — read -r -p 'Exact model ID from your dashboard: ' FLUXO_MODEL; export FLUXO_MODEL

Choose Claude Sonnet 5.5, then paste its exact API identifier when prompted. Display names and API identifiers can differ, so copying the identifier avoids guessing.

Fluxo provides one key and one base URL, https://api.fluxorelay.com, across providers; these OpenAI SDK commands use its /v1 API path. The same service works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK. You can create a Fluxo account if you want to run this experiment yourself.

Measure one attempt with retries disabled

The OpenAI Python SDK normally retries certain failures automatically. Disable that behavior for the baseline, and print both elapsed time and the complete usage object.

python -c 'import os,time,httpx; from openai import OpenAI; c=OpenAI(max_retries=0,timeout=httpx.Timeout(30.0,connect=3.0)); t=time.monotonic(); r=c.chat.completions.create(model=os.environ["FLUXO_MODEL"],messages=[{"role":"user","content":"Explain exponential backoff in exactly three short sentences."}],max_tokens=200); print("elapsed_seconds=",round(time.monotonic()-t,2)); print("usage=",r.usage.model_dump() if r.usage else None); print(r.choices[0].message.content)'

Save the output. Usage fields vary by model and endpoint, so inspect the returned details instead of assuming every provider reports cache activity identically. Fluxo bills per real token, including cache reads and writes, with no monthly fee. That makes repeated context a billing question even when caching helps.

Trigger a timeout without adding hidden retries

This command deliberately sets an impractically short timeout. It should usually fail, although the precise outcome depends on networking and connection timing.

python -c 'import os,httpx; from openai import OpenAI; c=OpenAI(max_retries=0,timeout=httpx.Timeout(0.001)); c.chat.completions.create(model=os.environ["FLUXO_MODEL"],messages=[{"role":"user","content":"Reply with ready."}],max_tokens=20)'

A connection timeout may happen before a request reaches inference. A read timeout may happen after processing begins. Neither exception alone gives you a complete billing record. Compare the attempt time with available usage records before treating the request as free.

Budget retries across the whole agent run

For one controlled retry, replace max_retries=0 in the baseline command with max_retries=1. Retryable failures can now produce two attempts. A successful response's usage object should not be treated as an aggregate invoice for both attempts.

HTTPX timeouts govern network phases and inactivity; they are not a hard deadline for the entire agent task. Backoff and another attempt can extend elapsed time further. Apply these rules in your agent wrapper:

Set one retry owner so nested layers cannot multiply attempts.
Record each attempt with elapsed time, outcome, and available usage.
Use a separate workflow deadline that includes tools and backoff.
Check tool results before repeating any action with external effects.

Count repeated context before raising the timeout

Suppose one attempt processes 12,000 input tokens and generates 1,000 output tokens. Repeating that work once can mean 24,000 input and 2,000 output tokens before cache adjustments. Apply the actual input, output, cache-read, and cache-write rates separately; a single blended rate hides the expensive part.

Fluxo also gives users a referral share when people they invite pay. Keep that credit separate from request-cost measurements. The useful debugging number is the cost and elapsed time of a completed agent task, including every attempt needed to finish it.