Is a vector database necessary for RAG?

21 Tem 08:35

Thanks for the great question! In RAG systems, while a vector database is usually sufficient, having an additional text processing layer for data cleaning and preprocessing is often critical. So, which vector database do you prefer—an out-of-the-box solution or something more scalable?

21 Tem 09:33

When I started a project leveraging RAG, my experience led me directly to choosing a vector database. In the first step, I used the *sentence-transformers* library to convert my documents into vectors. Processing over 1,000 technical documents and converting each into 384-dimensional vectors took about an hour—it was a bit tedious since I did it in a local environment. Then, I chose the *FAISS* library to store these vectors because it seemed the most convenient in terms of ease of downloading and using. However, a vector database alone wasn't enough: during searches, I had to apply an additional *keyword-based filter* (e.g., Elasticsearch) to filter similarity scores and eliminate irrelevant results. Otherwise, completely irrelevant information could appear in the responses generated by RAG. Later, to ensure the architecture worked simply and robustly, I divided it into three layers: *1. Document Processing* (extracting and cleaning from *pre_txt*), *2. Vectorization* (creating embeddings with models), and *3. Search & Generation* (finding similar vectors in FAISS and sending them to the LLM). This was sufficient for a small prototype, but when I scaled up, I needed to update vectors and optimize indexes. Ultimately, the vector database was indispensable, but I learned that it wasn't enough on its own—careful integration of auxiliary components was also necessary.