Skip to content

devlog/model-bench/picking-models-for-sql-generation-and-query-review-38.md

Picking Models for SQL Generation and Query Review

Pick SQL models with a database fixture

I pick SQL models by running the same task against a small database and checking the result. Fluent explanations are cheap. Correct joins, explicit assumptions, and useful review comments are what make a model worth keeping in my terminal.

For this walkthrough, compare DeepSeek V4 Flash for generation with Claude Sonnet 5.5 for review. Those are starting candidates, not a claim that either wins every workload. Fluxo gives this experiment one key and one base URL, https://api.fluxorelay.com, across providers.

1. Create a disposable PostgreSQL dataset

You need Docker, curl, Python 3, and Bash. Start an isolated database with fictional customers and orders. Keep production credentials out of this experiment.

docker run --name fluxo-sql-lab -e POSTGRES_PASSWORD=lab-only -d postgres:17

until docker exec fluxo-sql-lab pg_isready -U postgres; do sleep 1; done

docker exec fluxo-sql-lab psql -U postgres -v ON_ERROR_STOP=1 -c "CREATE TABLE customers (id integer PRIMARY KEY, email text NOT NULL); CREATE TABLE orders (id integer PRIMARY KEY, customer_id integer REFERENCES customers(id), total_cents integer NOT NULL, status text NOT NULL, created_at timestamptz NOT NULL); INSERT INTO customers VALUES (1,'[email protected]'),(2,'[email protected]'),(3,'[email protected]'); INSERT INTO orders VALUES (1,1,1200,'paid','2026-01-10T12:00:00Z'),(2,1,500,'refunded','2026-01-11T12:00:00Z'),(3,2,900,'paid','2026-02-01T00:00:00Z');"

The boundary order and customer without orders are deliberate. They expose two common mistakes: including February and dropping customers whose January revenue is zero.

2. Configure one relay and two model identifiers

If you want to follow along, create a Fluxo account and generate an API key. Copy the exact API identifiers for the two example models from the model listing; display names are not necessarily request identifiers. These commands prompt for credentials without putting the key in shell history.

read -r -s -p 'Fluxo API key: ' FLUXO_API_KEY; echo; export FLUXO_API_KEY

read -r -p 'DeepSeek V4 Flash API identifier: ' GEN_MODEL; export GEN_MODEL

read -r -p 'Claude Sonnet 5.5 API identifier: ' REVIEW_MODEL; export REVIEW_MODEL

export FLUXO_BASE_URL=https://api.fluxorelay.com

Fluxo also works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK. Here, curl keeps the request visible and makes swapping models a single variable change.

3. Generate SQL from an explicit contract

Give the model types, timezone semantics, and expected output shape. Save the prompt so every candidate receives identical instructions.

printf '%s' 'Write one PostgreSQL SELECT statement. Return SQL only, without fences or commentary. Schema: customers(id integer PRIMARY KEY, email text NOT NULL); orders(id integer PRIMARY KEY, customer_id integer REFERENCES customers(id), total_cents integer NOT NULL, status text NOT NULL, created_at timestamptz NOT NULL). Return every customer id and January 2026 paid revenue as revenue_cents, including zero revenue. Use UTC boundaries and a half-open date interval. Sort by customer id.' > prompt.txt

Define a reusable request function. It saves the full response before extracting the assistant text, so token usage remains available for inspection.

ask_model() { MODEL="$1" python3 -c 'import json,os,pathlib; print(json.dumps({"model":os.environ["MODEL"],"messages":[{"role":"user","content":pathlib.Path("prompt.txt").read_text()}]}))' > request.json; curl --fail-with-body -sS "$FLUXO_BASE_URL/v1/chat/completions" -H "Authorization: Bearer $FLUXO_API_KEY" -H 'Content-Type: application/json' --data-binary @request.json > "$2.json" && python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["choices"][0]["message"]["content"])' "$2.json" > "$2.txt"; } — ask_model "$GEN_MODEL" generation

4. Check results before trusting query review

Inspect the generated statement first. Proceed only if it is the requested SELECT. The disposable fixture limits exposure; a read-only transaction alone is not a sandbox for arbitrary SQL.

cat generation.txt — { printf '%s\n' 'BEGIN READ ONLY;' "SET LOCAL statement_timeout = '3s';"; cat generation.txt; printf '\n%s\n' ';' 'ROLLBACK;'; } | docker exec -i fluxo-sql-lab psql -U postgres -v ON_ERROR_STOP=1

Expected rows are customer 1 with 1200 cents, customer 2 with zero, and customer 3 with zero. A query that misses this fails regardless of its explanation.

Now ask the reviewer to inspect the same contract and generated query.

cp prompt.txt generation-contract.txt

{ printf '%s\n' 'Review the following contract and SQL. Identify correctness bugs, explain outer-join filter placement, and suggest indexes only with workload justification. Do not rewrite unless necessary.'; cat generation-contract.txt generation.txt; } > prompt.txt — ask_model "$REVIEW_MODEL" review — cat review.txt

5. Choose using correctness and measured cost

  • Repeat with representative schemas, recording correct results and manual repair time.
  • Reject reviewers that invent bugs or recommend indexes without evidence.
  • Compare latency and billed usage across repeated, identical requests.

Fluxo bills real tokens, including cache reads and writes, with no monthly fee; inspect those categories when comparing repeated prompts. Invited users also generate a referral share when they pay. Keep model selection grounded in your own results, and remove the fixture when finished. — docker rm -f fluxo-sql-lab

Next in the buffer

Sign up in a minute and get credit to try every model mentioned here. Sign up →