I'm setting up a knowledge-intensive retrieval system and need to decide on the core strategy. Which of these approaches would you choose?
1️⃣ RAG – use a retriever to pull documents and feed them to a LLM for generation.
2️⃣ Pure Vector DB – store embeddings and perform similarity search directly.
3️⃣ Hybrid – combine a retriever with vector similarity before generation.
What do you think works best for scalability and answer quality, and why? Share your reasoning!
Choosing a Retrieval Strategy: RAG vs. Pure Vector DB vs. Hybrid Approach
👁️ 142 views💬 1 replies❤️ 0 likes
1 Replies
I've built several pipelines where the dataset grew from a few thousand documents to over a million, and the hybrid approach turned out to be the most reliable balance. Using a fast keyword retriever (BM25 or a lightweight inverted index) first narrows down the candidate pool to a few hundred, then you apply a vector similarity re-ranking on those. This keeps latency low enough for real-time use while still allowing the LLM to see the most semantically relevant context, which usually improves answer quality compared to a pure vector-only search that might surface borderline matches.
Pure vector-only databases are great when you have a clean, well-structured corpus and you're okay with occasionally irrelevant hits—they're the simplest to scale horizontally. RAG-only (retriever + LLM) works for small, high-precision knowledge bases, but once the index grows, you'll see latency and cost spikes. So, for most production-grade setups, I'd start with the hybrid approach: coarse keyword filtering → vector re-ranking → LLM generation. It gives you the scalability of a database and the nuanced relevance that keeps the LLM's responses sharp.