Node.js SSE Streams That Survive Terminal Disconnects
Streaming AI output without orphaning Node.js requests
The happy path is easy: request streaming output, forward chunks, print tokens. The production path starts when someone closes their terminal, a proxy buffers six seconds of output, or a slow client turns your Node.js process into an accidental queue. Treat server-sent events as a transport with explicit ownership, not a loop that happens to call res.write().
Fluxo gives your editor and terminal workflows one key and one base URL, https://api.fluxorelay.com, across multiple model providers. It works with Claude Code, Codex, Gemini CLI, and any OpenAI SDK. For a streaming experiment, pick GPT-6 Astra or Claude Sonnet 5.5, using the exact model identifier exposed by the API rather than guessing it from the display name.
Frame SSE events independently of network chunks
An HTTP chunk is not an SSE event. One read can contain half a UTF-8 character, several events, or an event missing its final delimiter. If you consume raw fetch() output, use a streaming TextDecoder and retain incomplete input between reads. Parse lines incrementally, accept LF and CRLF, join repeated data fields with newlines, and dispatch only when a blank line completes an event.
Prefer the SDK's stream iterator when it already handles this framing. If you expose your own downstream SSE endpoint, serialize structured payloads with JSON.stringify(), then emit data: followed by the JSON and two newlines. Avoid injecting raw model text into SSE fields: embedded newlines can accidentally alter framing. Treat the upstream completion sentinel as protocol metadata, not generated text.
Commit SSE headers only after upstream acceptance
Validate authentication and request parameters before starting the downstream response. Ideally, establish the upstream request before committing SSE headers, so an upstream rejection can still become a meaningful HTTP error. Set Content-Type to text/event-stream; charset=utf-8 and Cache-Control to no-cache, no-transform, then call res.flushHeaders() when ready.
Disable compression and buffering for this route at every layer you control. X-Accel-Buffering: no can help with configured Nginx deployments, but it is not a universal proxy switch. Test through the actual ingress, not just localhost. Use curl -N against a diagnostic stream that emits numbered events at known intervals; if several arrive together, investigate buffering before blaming inference latency.
Connect res.write() backpressure to upstream cancellation
When res.write(frame) returns false, stop pulling from the upstream iterator until drain fires. Otherwise, your application keeps accumulating output for a client that cannot consume it. Also bound each connection's queued bytes and maximum duration: awaiting drain does not guarantee every SDK or intermediate layer stops buffering.
Create an AbortController for each upstream request. Listen for the outgoing response's close event and abort unfinished generation when the client disappears. A drain wait must also settle on close, error, or cancellation; awaiting drain alone can hang forever after a disconnect. Remove listeners and clear timers in a finally block, and make cleanup idempotent.
Separate SSE heartbeats, errors, and replay semantics
- Send comment heartbeats during idle periods, while respecting backpressure and proxy timeouts.
- After headers are committed, send a structured error event only if the connection remains writable.
- Define an explicit completion event so clients distinguish success from truncated connections.
- Disable automatic reconnection unless your application implements durable event replay.
Browser EventSource reconnects automatically, but replaying a generation request is not resuming its output. Event IDs and Last-Event-ID help only when you persist events and can replay them without generating again. For authenticated POST requests, fetch-based streaming usually provides more control than native EventSource.
Measure token usage after the final SSE event
Record upstream acceptance latency, first-event latency, downstream stalls, abort reasons, and final usage separately. Request usage reporting where supported, and do not assume an interrupted stream delivers final accounting. Fluxo bills per real token, including cache reads and writes, with no monthly fee; reconcile usage records rather than estimating charges from visible text.
If you want to test this against your own terminal workflow, create a Fluxo account. Users also earn a referral share when people they invite pay. Keep that separate from your engineering metrics: the useful result here is a stream that stops work promptly, stays bounded, and fails unmistakably.