Two Models, One Unit Test File, One Terminal
Picking a model with the test file open
When I ask an AI model for unit tests, I do not want the biggest suite. I want the smallest useful diff: tests that catch broken behavior, follow the repository’s conventions, and survive the next refactor. That makes the test file a better unit of model selection than the entire project.
For a concrete comparison, put Claude Haiku 4.5 beside Claude Opus 5.5. Treat the first as the candidate for routine files and the second as the escalation candidate. Those are starting hypotheses, not benchmark results. The winner has to earn its place against your code.
Give both models the same repository slice
Start with something bounded, such as tests for a retry-delay calculator. Provide the implementation, its public contract, package scripts, and one neighboring test file. Include repository instructions about fixtures, mocks, and naming. Do not hand one model the whole repository while giving the other three pasted functions.
My prompt is deliberately restrictive: “Add tests for this module using the existing framework. Do not change production code or dependencies. Assert public behavior, avoid real timers and network calls, and identify any behavior the contract leaves ambiguous.”
Run each attempt in a fresh worktree from the same commit. Keep the context, output budget, and repair allowance identical. One repair turn is reasonable, provided both models receive the same kind of feedback: actual runner output, not hints tailored to their mistakes.
Compare the generated files side by side
This is the comparison sheet I use. It describes what to inspect, not measured results for either model.
- For Claude Haiku 4.5, check whether the routine path needs any manual repairs.
- For Claude Opus 5.5, check whether additional reasoning produces materially stronger assertions.
- For both candidates, reject tests that silently alter the intended public contract.
- For both candidates, record token charges, elapsed time, and manual editing minutes.
For the retry calculator, both files should cover the initial delay, exponential growth, and the configured cap. If jitter is injectable, test its boundary values deterministically. If it is not injectable, do not reward a model for inventing a constructor option just to make its tests easier.
The interesting difference is rarely syntax. It is whether a test distinguishes the correct implementation from a plausible wrong one. A beautifully named test that merely checks for a number contributes less than a plain assertion that catches an off-by-one retry index.
Make green tests prove they can fail
Run the target file, then the relevant package checks. Next, deliberately introduce small defects: remove the cap, shift the retry index, or replace the growth factor with a constant. Run each mutation separately and restore the implementation afterward. Only score mutations that violate the stated contract.
Count which defects each suite catches. Also inspect isolation: does the file leave fake timers enabled, mutate shared fixtures, or pass only when another file runs first? Green output alone cannot answer those questions. This manual check is cheap enough for one file and more informative than comparing generated line counts.
Keep the relay fixed while changing the model
Fluxo makes this experiment less fiddly: one key and one base URL, https://api.fluxorelay.com, provide access to Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more. It works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK, so the comparison can stay inside your existing editor-and-terminal loop.
Record actual charges rather than guessing from response length. Fluxo bills per real token, including cache reads and writes, with no monthly fee. Keep cache conditions comparable between attempts. If you want to run this comparison yourself, consider creating a Fluxo account. Inviting teammates also earns you a referral share when they pay.
The verdict: default small, escalate the difficult file
My verdict is to start with Claude Haiku 4.5 for bounded unit-test files and escalate to Claude Opus 5.5 when the first attempt misses meaningful cases or costs too much repair time. That is a routing policy, not a claim that either model wins every repository. Keep whichever produces contract-faithful tests at the lowest total cost, including your editing time. Repeat the decision for the next file.