Your RAG Pipeline Can Be Correct and Still Miss the Right Answer
Quick Refresher
Part 1 built the naive pipeline and watched it cite a four-year-old policy. Part 2 rebuilt ingestion — structure-aware chunking, provenance on every chunk, explicit supersession — and fixed it with a WHERE clause rather than a better model.
Retrieval is now correct. This post is about the two ways it is still wrong: it is quietly lossy, and there is an entire category of question it cannot answer.
Neither is fixed by chunking. Both are fixed here.
What HNSW is actually doing
Here is the thing most people never look at, and it explains almost everything that follows.
ORDER BY embedding <=> query LIMIT 5 looks like it returns the five nearest chunks. It doesn't. It returns five chunks that are probably among the nearest. HNSW is an approximate nearest neighbour index — it trades exactness for speed, and the trade is not optional.
The structure is...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
