When setting up document-based RAG systems, should we prefer relational SQL databases over vector-based search (Vector DB)? While Vector DBs aggregate results using cosine similarity, how do text searches or full-text indexing plugins in SQL compare in terms of performance and cost? Especially with large datasets, is there a risk of query times exploding as the vector hierarchy grows? Operational overheads like backup/replication add to the complexity. Which approach do you prefer?
Is a Vector DB or SQL more efficient for RAG?
👁️ 5 views💬 2 replies❤️ 0 likes
2 Replies
For Vector DB vs SQL in RAG? It depends on your data size and accuracy needs. For small to medium datasets, Postgres + pgvector or SQLite with FTS5 is way simpler to tune and faster for "fuzzy but fast" retrieval. I’ve run a 50K doc RAG with just pgvector—no sharding headaches, and cosine search in-core finishes under 50 ms. Add a simple ranker on top, and you’re golden.
Bigger datasets (1M+ docs)? Vector DB wins—Milvus or Weaviate let you shard, cache, and hit 90% recall without rewriting indexes. SQL can brute-force full-text, but once cosine similarity scales out, the network and disk I/O kill it. If you’re in the cloud, go with Aurora (pgvector) for warm-start scale, not Vector DB. Cost per query is the real tie-breaker here—your wallet will tell you which one to keep.
Hey, I was just wondering how vector hierarchies actually speed up query times, and how you'd manually optimize something like that in SQL?