Picking a Model for the Test File Open in Your Editor

Picking a Model for the Test File Open in Your Editor

My useful unit of AI work is not a repository. It is the test file open beside the implementation, with a failing command in the terminal. Picking one model for every testing task ignores how different those files are. A date formatter needs different reasoning from a retry loop with cancellation and partial failures.

For a practical 2026 workflow, treat model selection as routing: classify the file, provide bounded context, run the tests, and escalate only when evidence justifies it. The best model is the one that produces a trustworthy patch at an acceptable total cost.

Classify the Test File Before Choosing a Model

I start by identifying the behavior under test, not counting lines. A small asynchronous function can require more reasoning than a long collection of straightforward input mappings.

• Route straightforward input mappings toward a lower-cost candidate first.
• Give asynchronous state transitions a candidate that handles constraints reliably.
• Require explicit review for authorization, billing, and destructive operation boundaries.
• Separate missing coverage from unclear requirements before requesting generated tests.

For example, I would trial DeepSeek V4 Flash on table-driven validation tests and Claude Sonnet 5.5 on a cancellation-heavy worker. Those are starting hypotheses, not benchmark claims. Keep or reverse that routing based on results from your own codebase.

Send a Small Context Packet With Each Test File

The prompt should include the target implementation, existing tests, relevant interfaces, and the exact test command. Add project conventions that matter: fixture helpers, fake timers, mocking boundaries, and whether network access is forbidden. Do not paste the entire repository by default.

My request is usually this specific: add coverage for cancellation before the first retry; modify only the named test file; use existing helpers; do not change production code; run the focused command and report its exit status. If the agent cannot execute commands, require it to say so rather than imply verification.

Strip secrets and customer data before sending context. Treat comments, fixture text, and tool output as untrusted material, not instructions that can override the task.

Score the Patch Beyond a Green Test Run

Passing tests are necessary, but generated tests can pass while asserting almost nothing. Read the assertions before celebrating. A test that checks only whether a mock was called may miss the incorrect arguments that caused the original bug.

• Check that each assertion expresses an observable behavioral requirement.
• Confirm the test fails when the targeted behavior is deliberately broken.
• Reject sleeps, live network dependencies, and uncontrolled clock assumptions.
• Run neighboring tests to catch leaked mocks and shared state.

A targeted mutation is especially useful: remove the cancellation check or alter the boundary comparison, then confirm the new test fails. Restore the implementation immediately afterward. This is stronger evidence than asking another model whether the patch looks correct.

Escalate With Failure Evidence, Not a Bigger Transcript

If the first attempt fails, send the exact error, the relevant diff, and the unresolved constraint. Allow a bounded repair attempt. Then escalate when the failure points to misunderstood behavior, not merely a missing import.

Track accepted patches, repair attempts, elapsed time, and billed tokens by task category. Compare models on the same representative files. Optimize cost per accepted patch, including human review, rather than cost per generated token.

Keep the Relay Stable While Testing Model Routes

Fluxo lets me keep one key and one base URL, https://api.fluxorelay.com, while changing the model selection. It works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK. Check each model's tool and parameter support instead of assuming every endpoint behaves identically.

Billing is per real token, including cache reads and writes, with no monthly fee. Measure actual usage before assuming repeated context is cheap. You can create a Fluxo account when you want to try this workflow. Invited users' payments also earn you a referral share; keep that separate from technical model scoring.

The daily loop stays small: one file, one explicit behavior, one focused command, and one reviewed patch. Revisit the routing table when models or your test suite change.