Claude Code at the Terminal: Debugging Flaky CI
One intermittent timeout, two debugging workflows
The least useful CI notification is the one that turns green when you click Retry. Nothing changed, nobody understands the failure, and the next developer inherits it. For this comparison, consider an illustrative Node test suite: an export test occasionally times out in CI but passes locally. The visible failure is “expected completed, received pending.”
I would investigate it two ways: first, by asking an assistant to interpret the failed test output; second, by giving Claude Code the same output plus access to the repository and terminal. The question is not which answer sounds smarter. It is which workflow produces a falsifiable explanation and a regression test.
Capture the failed run before asking Claude Code
Start with evidence, not a screenshot of the final assertion. Save the complete test output, including stderr, timestamps, test names, worker identifiers, and the command CI actually executed. In Bash, set -o pipefail before running npm test 2>&1 | tee test-output.log, so tee does not hide a failing test command.
Record the commit, runtime version, runner image, dependency lockfile, worker count, and relevant environment settings. Retain the log as a CI artifact even when the test step fails. Redact credentials and customer data before sharing anything with a model; check repository instructions and configuration files for secrets too.
Side by side: interpreting logs and investigating the repository
- With logs alone, ask for ranked hypotheses and supporting evidence.
- With Claude Code, request repository inspection before proposing any patch.
- With logs alone, treat suggested fixes as experiments, not diagnoses.
- With Claude Code, require a reproduction command and an explicit verification plan.
The log-only prompt is deliberately narrow: “Identify the first unexpected event, distinguish observations from hypotheses, and list missing evidence.” That often exposes an earlier database disconnect or a worker restart buried above the timeout. It is fast, cheap, and useful when repository access is inappropriate.
But logs cannot reveal code that was never logged. In this example, “increase the timeout” and “wait for background work” both fit the assertion. Neither is proven. A confident explanation here is still a guess.
For the repository-aware pass, tell Claude Code: “Read test-output.log, inspect this test and its setup, and trace how the export worker starts. Do not edit files yet. Identify a testable ordering assumption.” Keep command approvals enabled, especially around dependency installation, network access, and destructive cleanup.
Turn a worker startup race into a deterministic test
Suppose inspection reveals that setup launches the worker without awaiting its readiness signal. The test submits an export immediately. A slow CI runner makes that ordering visible; a fast laptop usually hides it. That is a hypothesis until an experiment supports it.
Ask Claude Code to insert a controlled startup delay and run the focused test. If the existing test fails reliably, replace the fire-and-forget setup with an awaited readiness handshake. Give that handshake a bounded timeout and useful failure output. Do not substitute an arbitrary sleep: it merely changes the odds.
Then run the regression with the controlled delay, remove diagnostic instrumentation, and repeat the focused test under CI-like parallelism. Run the relevant suite afterward. Fifty clean repetitions provide evidence, not proof; the stronger result is demonstrating that the old setup fails under controlled conditions while the corrected setup passes.
Keep relay costs separate from debugging evidence
Fluxo fits the terminal workflow with one key and one base URL, https://api.fluxorelay.com, for Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok, and more. It works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK; use the documented configuration for each client rather than assuming identical setup.
For example, keep Claude Sonnet 5.5 fixed throughout this comparison so changing models does not become another variable. Fluxo bills per real token, including cache reads and writes, with no monthly fee. Users also earn a referral share when people they invite pay; that is separate from whether a diagnosis holds up. If this setup suits your workflow, consider creating a Fluxo account.
Verdict: logs for triage, Claude Code for verified fixes
Log-only analysis wins the first pass: it is quick and limits access. Claude Code wins the deeper investigation when repository access is appropriate, because it can connect symptoms to implementation and test the proposed cause. Neither earns trust by producing a green rerun. Ship the fix when the failure mechanism is demonstrated, the regression covers it, and the verification commands are recorded.