Measuring System Prompt Cache Hits from Your Terminal

Repeated system prompts are the expensive constant

My coding requests change constantly. The repository rules, architecture notes, and review checklist usually do not. Sending that same context twenty times makes input tokens worth measuring separately from the answer. A prompt cache can turn an eligible, repeated prefix into cheaper cache reads. It does not remove those tokens from the request.

Fluxo gives me one key and one base URL, https://api.fluxorelay.com, across its model catalog. It bills actual tokens, including cache reads and writes, without a monthly fee. That makes the useful question concrete: how much of this request was billed as ordinary input, a cache write, or a cache read?

Prepare a stable prompt and select GPT-6 Astra

These commands use Bash, curl, jq, and Python 3. Run them inside a repository containing AGENTS.md. Use a substantial, representative instruction file: a tiny prompt may never reach the model's caching threshold. If you need access, you can create a Fluxo account to follow along.

read -rsp 'Fluxo API key: ' FLUXO_API_KEY; printf '\n' export FLUXO_API_KEY export OPENAI_BASE_URL='https://api.fluxorelay.com/v1' curl -fsS "$OPENAI_BASE_URL/models" -H "Authorization: Bearer $FLUXO_API_KEY" | jq '.data[] | {id, name}' read -rp 'Paste the exact model ID for GPT-6 Astra: ' MODEL_ID export MODEL_ID cp AGENTS.md system.txt

The models response supplies the identifier; do not assume a display name is also an API ID. These commands use the OpenAI-compatible /v1 route. Cache eligibility, minimum prefix length, retention, and accounting fields depend on the selected model and upstream behavior. Switching to Claude Sonnet 5.5 requires checking its caching requirements separately.

Build two requests with the identical system prefix

Keep shared instructions first and changing task details last. Do not insert timestamps, request IDs, or branch-specific text ahead of the reusable material. The following command creates two payloads whose system messages match exactly while their user questions differ.

python3 -c 'import json, os, pathlib; system = pathlib.Path("system.txt").read_text(); tasks = ["Explain the repository rules in three sentences.", "List three checks to perform before editing this repository."]; [(pathlib.Path(f"request-{i}.json").write_text(json.dumps({"model": os.environ["MODEL_ID"], "messages": [{"role": "system", "content": system}, {"role": "user", "content": task}]}))) for i, task in enumerate(tasks, 1)]'

This uses automatic prefix caching where the selected route supports it. It does not add provider-specific cache controls or guarantee that either request will qualify.

Send both requests and inspect the usage objects

Run the requests sequentially and close together. Save complete responses so you can inspect any additional accounting fields without repeating the experiment.

for n in 1 2; do curl -fsS "$OPENAI_BASE_URL/chat/completions" -H "Authorization: Bearer $FLUXO_API_KEY" -H 'Content-Type: application/json' --data-binary "@request-$n.json" -o "response-$n.json" || break jq '{usage, error}' "response-$n.json" done

In a standard OpenAI-style usage object, inspect usage.prompt_tokens_details.cached_tokens. A positive value on the second request is evidence of a cache read. Total prompt tokens can remain unchanged because cached tokens are still input tokens. Missing cache fields mean this response does not establish a hit; they do not prove caching is unavailable.

If both requests show zero cached tokens, check prompt length, exact prefix identity, model support, and retention rules. Do not add an invented cache flag. Also check Fluxo's usage records for billing details: the response may not expose cache writes separately.

Calculate the break-even point for your repeated prefix

Use the selected model's current rates. Let P be reusable tokens, N the request count, B the ordinary input rate, W the cache-write rate, and R the cache-read rate. With rates per million tokens, ordinary prefix cost is N × P × B / 1,000,000. Assuming one write and subsequent full hits, cached prefix cost is P × [W + (N − 1) × R] / 1,000,000.

For hypothetical rates B = $1, W = $1.25, and R = $0.10, a 4,000-token prefix across ten requests costs $0.04 ordinarily and $0.0086 with caching: 78.5% less for that prefix. Changing user input and output still cost extra. Expiration, partial hits, and additional writes change the result.

I measure this before moving the same workload into Claude Code, Codex, Gemini CLI, or an OpenAI SDK, all supported by Fluxo. Referral shares from invited users' payments are separate from these token calculations. Keep the benchmark grounded in observed usage and the rates actually billed.