Terminal Checks for Model Endpoint Latency and Errors
Set up a repeatable endpoint probe
A single slow response is easy to dismiss; a pattern of slow responses or failed requests is not. I keep a small curl probe near the project so I can compare endpoints from the same machine and shell. This example uses Fluxo’s OpenAI-compatible API base URL and GPT-6.1 Sol. You will need bash, curl, and awk.
Export your key without putting it in the script or shell history. Replace the placeholder with your own key, then confirm curl is installed.
export FLUXO_API_KEY='your_fluxo_key' — curl --version — Create a probe that records the HTTP status and total request time for each attempt. The response body is discarded so the log stays small; curl’s output format still gives us the status and duration.
cat > monitor.sh <<'EOF' #!/usr/bin/env bash set -u : "${FLUXO_API_KEY:?Export FLUXO_API_KEY first}" count="${1:-20}" url='https://api.fluxorelay.com/v1/chat/completions' log='probe.tsv' : > "$log" for i in $(seq 1 "$count"); do result=$(curl --silent --show-error --output /dev/null \ --write-out '%{http_code} %{time_total}' \ --connect-timeout 10 --max-time 60 \ "$url" \ -H "Authorization: Bearer $FLUXO_API_KEY" \ -H 'Content-Type: application/json' \ -d '{"model":"GPT-6.1 Sol","messages":[{"role":"user","content":"Reply with OK."}],"max_tokens":8}') rc=$? if [ "$rc" -eq 0 ]; then printf '%s\t%s\n' "$result" "$i" >> "$log" else printf '000\t0\t%s\n' "$i" >> "$log" fi sleep 2 done EOF
Make it executable and run twenty probes, spaced two seconds apart. A failed curl transport is logged as status 000, so it counts as an error instead of disappearing from the results. — chmod +x monitor.sh — ./monitor.sh 20
Calculate latency and HTTP error rates
Each row in probe.tsv contains status, seconds, and attempt number. Use this awk command to calculate the fraction of non-2xx responses and the average successful-request latency. The last field is deliberately ignored in the calculations.
awk 'BEGIN { n=0; errors=0; total=0 } { n++; if ($1 < 200 || $1 >= 300) errors++; else { total+=$2; good++ } } END { if (n) printf "requests=%d errors=%d error_rate=%.1f%% average_success_seconds=%.3f\n", n, errors, 100*errors/n, good ? total/good : 0 }' probe.tsv
For a quick view of individual failures, print only rows with a status outside the success range:
awk '$1 < 200 || $1 >= 300 { print "status=" $1, "seconds=" $2, "attempt=" $3 }' probe.tsv
Run the probe again after a model, network, or application change, and keep the logs with your experiment notes. These are client-observed timings, not a substitute for provider-side traces: DNS, connection setup, your network, and response generation all affect the number. For a useful comparison, keep the prompt, token limit, machine, and network consistent.
Interpret slow responses and failed requests
Separate connection problems from model latency before changing code. A 000 status usually means curl could not complete the request, while a 4xx response points toward request or authentication issues. A 5xx response is a server-side failure worth tracking across repeated attempts. Inspect the response body when diagnosing one failure by temporarily removing --output /dev/null and checking the returned JSON; avoid logging secrets or user prompts.
Latency also needs context. This probe sends a tiny prompt and limits generated tokens, so it is useful for checking baseline responsiveness, not measuring a long production completion. If your application streams responses, measure time to first token separately from total completion time. Keep a small baseline log and compare percentiles as well as averages when traffic or prompts vary.
Keep endpoint checks close to your workflow
Fluxo gives developers one key and one base URL for Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok, and more, so the same terminal habit can cover multiple model endpoints. It works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK. Billing is per real token, including cache reads and writes, with no monthly fee; that makes it practical to test deliberately without a subscription clock. If you want to try this workflow, you can create a Fluxo account and use its key for the probe.
When you need help, sign in and leave a note on the contact page. Keep the probe small, run it regularly, and treat a sudden change in latency or error rate as a signal to investigate rather than an explanation by itself.