Vector databases are now seen as standard in RAG systems. But are they really indispensable for every project? Alternatives include local vector storage, memory-based approaches, or simply querying text directly. Do you think using a vector DB in RAG setups is mandatory, or is it sometimes just confusing overhead? What do you generally prefer: pure RAG, modified RAG, or another approach?
How necessary or unnecessary is the use of RAG in Vector DBs?
👁️ 13 views💬 1 replies❤️ 0 likes
1 Replies
For small projects or quick prototypes, storing embeddings in memory (even with SQLite or a Python dictionary) works wonderfully. When we're talking about scaling or sensitive data where performance matters (e.g., a chatbot with 10K+ documents), then I definitely go for a vector DB like Chroma or Milvus. I’ve tested this on Android with Room + local vector embeddings in a smart notes app: for under 5K chunks, the overhead of a vector DB isn’t worth it, but as the index grows, search time becomes a bottleneck.
What I do is start with **safest RAG**: embeddings + in-memory search. When the project grows, I migrate to a vector DB without changing the pipeline. For example, in a tutoring project with 2K math problems, Chroma with embedded SQLite handled it perfectly. But if you need high concurrency or dynamic updates (like in a forum where users are adding posts), then Chroma or Weaviate become essential. The key is evaluating the cost/benefit of memory vs. speed based on the use case.