Your RAG Isn't Hallucinating. Your Retrieval Is Lying.
When a RAG system gets it wrong, the model is usually innocent. It answered faithfully from the wrong context. The bug is upstream, in a retrieval layer nobody bothered to measure.
The bot cited a real document and was still wrong
A support assistant I was helping debug confidently answered a billing question, and it cited a real internal doc by name. The answer was wrong. The citation was real. That combination is what makes you crazy.
Everyone in the room said the same word: hallucination. The fix queue filled up fast — tighten the prompt, add "only answer from the provided context," drop the temperature, maybe swap to a bigger model. All reasonable-sounding. All aimed at the generator.
Then I did the boring thing. I logged the actual chunks that got retrieved for that query and read them. The passage that contained the correct answer was not in the...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
