Retrieval‑augmented generation relies heavily on the underlying store for retrieved passages. With the rise of dense embeddings, many projects now default to a vector database for similarity search, while others still prefer classic inverted indexes or hybrid solutions. I'm curious how the community balances factors like latency, scaling costs, and the ability to incorporate metadata filters. Do you think pure vector stores are mature enough to be the default backbone for RAG, or should they be combined with traditional term‑based retrieval for robustness? Also, how do you handle frequent index updates without sacrificing query speed? Would love to hear your architectures and any pitfalls you've encountered.
RAG pipelines vs. traditional retrieval: Should vector databases be the default backbone?
👁️ 35 görüntüleme💬 1 cevap❤️ 0 beğeni
1 Cevap
Pure vector stores are a solid choice for RAG when your primary need is semantic similarity—like matching user intent in natural language queries. But they’re not a one-size-fits-all backbone. The real pain points kick in when you need metadata filters (e.g., "only documents published in 2023"), partial term matching, or low-latency updates. Traditional inverted indexes handle those edge cases better at scale, especially with large corpora where vector search alone would bleed performance. I’ve seen teams hit a wall trying to shoehorn everything into a vector DB: indexing latency peaks during bulk updates, filter operations slow to a crawl, and suddenly you’re paying for a Ferrari that only runs in first gear.
Where I’ve seen hybrid setups shine is in production systems where we pre-filter with a term-based index (ELK or Tantivy) and then run dense retrieval on the reduced subset. That keeps both latency and costs in check. For metadata-heavy use cases—like legal or medical RAG—I still default to a hybrid approach because pure vector search struggles with strict attribute checks. The latency hit from separate index scans is negligible compared to the havoc unfiltered vector search causes.
As for frequent updates: vector DBs are getting better (thanks, Qdrant’s HNSW optimizations and Weaviate’s async indexing), but nothing beats a solid write-ahead log or CDC pipeline feeding changes to both indexers. I’ve migrated projects from pure vector stores to hybrid after realizing that optimizing filters was cheaper than over-provisioning vector nodes. The biggest pitfall isn’t technical—it’s assuming semantic similarity alone will cover all retrieval needs.