Friday Terminal Retro: Stop Paying for AI Mistakes
The expensive AI mistake is rarely the first bad answer. It is the next six attempts: another repository scan, another speculative patch, another test run that fails for the same reason. By Friday, you remember the annoying session but not what it cost.
I treat those sessions like flaky tests. Capture the failure, make it reproducible, and change one thing before paying to repeat it. A weekly retro needs thirty minutes, a small fixture directory, and an honest ledger.
Capture the retry loop before closing your terminal
Keep a local failure log beside your development notes. Record unsuccessful tasks while the details are fresh, not from memory at the end of the week. Never copy credentials, customer records, or unredacted production logs into evaluation fixtures.
- Record the task, model, repository revision, and expected outcome.
- Save the smallest sanitized prompt that still reproduces the mistake.
- Count every attempt, including abandoned branches and manual recovery steps.
- Record billed usage, cache activity, and minutes spent reviewing changes.
- Label the failure as missing context, wrong assumption, unsafe edit, or tool failure.
That last label matters. A missing environment variable does not become a model-quality problem just because an agent encountered it.
Rank failures by total recovery cost
Pick three failures with the highest recovery cost, not three answers that sounded silly. Keep API charges and human time in separate columns. A cheap response that takes twenty minutes to untangle is not automatically a bargain.
For example, a hypothetical task with a $0.06 first attempt and four $0.05 retries costs $0.26 before review. Across one hundred similar tasks, that pattern costs $26. Removing two retries saves $10 at that volume, assuming comparable usage. These are illustrative numbers, not model prices.
Fluxo bills per real token, including cache reads and writes, with no monthly fee. In the ledger, retain those categories separately using whatever usage records your client exposes. Shorter visible output does not prove a cheaper run, and repeated context does not prove a cache hit.
Replay three mistakes against frozen repository fixtures
Create a fixture for each selected failure: a pinned repository revision, a sanitized task, an executable check, and explicit acceptance criteria. For a migration bug, that might mean preserving existing rows, rejecting duplicate identifiers, and passing a rollback test.
Run each candidate from the same clean state. Keep tool permissions, context, and maximum attempts fixed. Compare total billed cost per accepted result, not cost per response. If nothing passes, record a failure rather than quietly excluding it from the comparison.
You could test Claude Haiku 4.5 and Claude Sonnet 5.5 on those fixtures without assuming either will win. Repeat close results; one successful run is weak evidence. The point is to discover which configuration clears your checks reliably enough to justify its spend.
Ship one guardrail that prevents another paid retry
- Add repository instructions identifying the canonical test command and working directory.
- Require inspection of existing interfaces before generating replacement implementations.
- Limit edits to named files until the first check passes.
- Stop after two equivalent failures and request missing evidence instead.
- Route recurring task types using measured acceptance rates and costs.
Choose one change per failure class. Changing the model, prompt, context, and tools simultaneously makes next Friday's results impossible to explain.
Keep routing changes small inside your existing tools
Fluxo provides one key and one base URL, https://api.fluxorelay.com, across Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok, and more. It works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK, so keep your fixtures stable while changing the configured model.
If this fits your workflow, consider creating a Fluxo account for your next weekly replay. If you need help, leave a note on the contact page after signing in. Users also earn a referral share when people they invite pay; track that separately from engineering savings.
End Friday with one committed guardrail and a baseline for next week. The win is not fewer tokens at any cost. It is fewer paid attempts before the code actually works.