The Hidden Cost of Orphaned AI Work
That's not a bug. It's a missing owner.
The user closed the browser 30 seconds ago.
Your logs show the response was never delivered. But the LLM stream is still running. The vector search is still scanning. The reranker is still scoring. The tool calls are still executing.
The invoice will arrive tomorrow.
This is not hypothetical. It is the default behaviour of many AI backends today.
app.post("/chat", async (req, res) => { const stream = await openai.chat.completions.create({ model: "gpt-4.1", stream: true, messages: req.body.messages, }); for await (const chunk of stream) { res.write(chunk.choices[0]?.delta?.content ?? ""); } res.end();});
That code looks reasonable.
It even works — until the user refreshes the page, closes the tab, loses signal, or navigates away.
At that moment:
- the HTTP response is dead
- the client no longer exists
- the user no longer cares
But the async work continues anyway.
The LLM keeps generating tokens. The vector...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
