Token Budgets from the Terminal with Fluxo
Set a monthly token budget before writing chatbot code
An internal chatbot becomes expensive when nobody can answer a basic question: how many tokens does one useful conversation consume? Start with a budget, then work backward from that number. For example, assume 40 developers, 15 conversations per developer each month, and 8,000 billed tokens per conversation. The estimate is 4,800,000 tokens monthly, before accounting for cache reads, cache writes, retries, and unusually long threads.
Fluxo gives the relay one API key and one base URL, https://api.fluxorelay.com, across Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok, and more. It bills per real token, including cache reads and writes, so the budget should track actual usage rather than only the visible prompt length.
Configure one terminal environment for the internal chatbot
Create a shell environment with the relay credentials and the model you want to test. I usually keep these values outside the repository so a shared chatbot project does not accidentally commit secrets.
export FLUXO_API_KEY="replace-with-your-fluxo-key" export FLUXO_BASE_URL="https://api.fluxorelay.com" export CHATBOT_MODEL="GPT-5.6 Sol"
The same configuration works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK. For this walkthrough, use curl so the accounting is visible from the terminal.
Measure one representative chatbot request
Send a realistic request rather than a tiny health check. Include the system instructions, a short conversation history, and the internal knowledge excerpt your production chatbot normally supplies.
curl "$FLUXO_BASE_URL/v1/chat/completions" \ -H "Authorization: Bearer $FLUXO_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model":"GPT-5.6 Sol","messages":[{"role":"system","content":"Answer internal engineering questions using the supplied context. State uncertainty clearly."},{"role":"user","content":"Context: deployments require an approved change record. Question: Who approves a production rollback?"}],"max_tokens":600}' > response.json
Inspect the usage object returned by the API. The exact fields depend on the response contract, but the important values are prompt tokens, completion tokens, total tokens, and any cached-token counters.
jq '.usage' response.json — Record this result in a small spreadsheet or log file with the request type, model, prompt tokens, completion tokens, cache reads, cache writes, and response quality. Ten representative samples are usually more useful than one elaborate estimate.
Calculate a per-conversation token ceiling
Separate the fixed and variable parts of each request. Fixed tokens include system instructions, tool definitions, and retrieved policy text. Variable tokens include the conversation history and the latest user question. A practical ceiling might be 4,000 input tokens and 600 output tokens for a short support exchange.
For a simple monthly calculation, multiply conversations by the input and output ceilings, then add a retry allowance. Forty developers multiplied by 15 conversations and 4,600 tokens equals 2,760,000 tokens. Adding a 10 percent retry allowance produces a planning ceiling of 3,036,000 tokens.
Keep cache activity in the calculation. Repeated system instructions may become cheaper to process operationally, but Fluxo reports and bills real token activity including cache reads and writes. That visibility lets you compare a compact prompt against a heavily cached prompt using measured data.
Enforce the budget in a repeatable shell test
Run a small batch before changing models or prompts. This command submits the same request five times, saves every response, and makes outlier behavior easier to spot.
rm -f usage.ndjson for run in 1 2 3 4 5; do curl -s "$FLUXO_BASE_URL/v1/chat/completions" -H "Authorization: Bearer $FLUXO_API_KEY" -H "Content-Type: application/json" -d '{"model":"Claude Haiku 4.5","messages":[{"role":"user","content":"Summarize this internal deployment policy in five bullets: rollback approval requires an incident lead and a service owner."}],"max_tokens":300}' | jq -c '{usage: .usage}' >> usage.ndjson; done jq -s 'map(.usage.total_tokens) | add as $total | {runs: length, total_tokens: $total, average_tokens: ($total / length)}' usage.ndjson
Use Claude Haiku 4.5 for this inexpensive baseline, then compare the same test with GPT-5.6 Sol when answer quality requires a different model. Keep the prompt and maximum output unchanged during the comparison, or the result will not tell you which model caused the difference.
Turn measurements into operating limits
Set an application-level warning at 80 percent of the monthly ceiling and a hard stop at 100 percent. At the warning level, shorten retrieved context, summarize old messages, or route low-risk questions to the cheaper tested model. At the hard stop, return a clear internal message and preserve the request ID for investigation.
Create a Fluxo account when you are ready to run these measurements against your own internal chatbot. If you need help, sign in first and leave a note on the contact page.
Review the budget weekly for the first month. Token use changes when developers paste logs, when retrieval returns larger documents, and when retries follow slow responses. A terminal report based on real usage will keep those changes visible before they become a billing surprise.