RAG
- RAG can be explained through three steps
- It retrieves relevant information from a datasource external to LLMs
- It augments the input to the LLM with that external information
- Finally LLM generates a more accurate result
- Idea of RAG
- Make LLMs respond with up-to-date information
- Make LLMs respond with factually accurate information
- Make LLMs aware of proprietarty information
- RAG was discovered in a paper title “Retrieval-Augemented Generation” for Knowledge Intensive NLP Tasks
-
The process of RAG can be distilled into five steps
- User asks a question
- System searches for information relevant to the input question
- information relevant to the input question is fetched, or retrieved and added to the the input question
- question and information are passed to an LLM
- LLM responds with contextual answer
-
RAG system works best if the information from different sources is
- Collected in a single location
- Stored in single format
- Broken down into small pieces of information.
Design of RAG systems
- RAG Systems of composed of 2 pipelines
- Indexing Pipeline: Creates the knowledge base
- Generation Pipelne: Generates the response (RAG)
Indexing Pipeline
- It composes of four main components
- Data loading: responsible for connecting to external sources and extracting and parsing information
- Data splitting: responsible for breaking down long peices of text into managable parts called as chunks
- Data conversion: responsible for converting text chunks into numberical vectors called “embeddings”
- Data Storage: responsible for storing the embeddings in permanent memory using specialized databases called as vector databases.
How to build RAG systems
- Langchain: It is a popular opensource framework written in python & java script, designed for building applications using LLM. It is suitable form RAG, Agents etc. Langchain has built integrations with
- LLM Providers such as OpenAI, Anthropic, Gemini etc
- Vector store providers: cloud storage system, Pinecone ….
- APIs for news, weathers
- LlamaIndex: Python framework with
- rich set of data connectors (database, documents, API)
- Query engines
- Haystack: Python framework
Step 1: Vector Databases
- Popular vector databases
| Database | Pricing Details | Hosting & Availability | Key Features |
| ——————- | ————————————————————————————————————————————————— | ————————————————————————————————- | ——————————————————————————————————————————————————————————- |
| Pinecone | Free plan available; Starter \~\$50/month; Enterprise \~\$500/month (Xenoss AI Solutions, liquidmetal.ai, Aloa) | Fully managed; serverless & auto-scaling (antalyze.ai, ankursnewsletter.com, Aloa) | Ultra low-latency (sub-50 ms), 99.99% SLA uptime, seamless integration with ML/AI workflows (Aloa, ankursnewsletter.com, liquidmetal.ai) |
| Weaviate | Serverless from \~\$25/month + \$0.095 per 1M vector dimensions (Weaviate, Xenoss AI Solutions); Cloud \~\$75/month (liquidmetal.ai) | Open-source self-hosting; managed Cloud options available (antalyze.ai, Medium) | Hybrid search (vector + keyword), GraphQL API, knowledge-graph modeling, ML module hosting (liquidmetal.ai, Medium) |
| Milvus | Open-source with unlimited free self-hosting; cloud-based Zilliz Cloud (custom pricing) (antalyze.ai, Wikipedia) | Self-host (embedded, standalone, distributed); Fully managed via Zilliz Cloud (Wikipedia) | Massive-scale support (billions+ vectors), multiple index types (HNSW, IVF, quantization), GPU support, strong consistency and multi-tenancy (antalyze.ai, Wikipedia) |
| Qdrant | Free tier (1 GB); Cloud \~ \$0.014/hour; private deployments are custom-priced (Xenoss AI Solutions) | Open-source with self-hosting or managed cloud option (Xenoss AI Solutions) | Good query throughput (\~326 QPS) for a solid open-source pick (Xenoss AI Solutions) |
| Chroma | Fully open-source and free to self-host (antalyze.ai, Wikipedia) | Designed for self-host; no managed cloud offering noted (Wikipedia) | Lightweight, LLM-oriented; supports vector embeddings, full-text search, metadata filtering, multi-modal data (Wikipedia) |
| FAISS (library) | Open-source library (MIT license), no hosting or pricing (Wikipedia) | Integrates into self-hosted solutions or vector DBs (e.g., Milvus) (Wikipedia) | Extremely high performance, GPU-accelerated similarity search; used as backend by many vector DBs (Wikipedia) |
- Vector databases offered by cloud providers
| Cloud Provider | Service Offering | Features & Use Cases | Pricing / Notes |
| —————- | ——————————————————- | ——————————————————————————————————————————————————————————————————- | ————————————————————————————————————————————————————- |
| AWS | Amazon OpenSearch Service | Fully managed search & analytics engine with vector capabilities (k-NN, ANN via HNSW/IVF), hybrid search, RAG, real-time updates, serverless or managed options (Amazon Web Services, Inc.) | Pay-as-you-go; pricing detailed on AWS pricing pages (varies by region and configuration) (Amazon Web Services, Inc.) |
| | Amazon RDS / Aurora (PostgreSQL + pgvector) | Relational database with vector support via pgvector; suitable for combining SQL with vector queries (AWS Documentation, KodeKloud Notes) | Pay-per-use RDS/Aurora pricing |
| | Amazon Neptune ML | Graph database + ML embeddings for graph tasks (fraud detection, knowledge graphs) (Instaclustr, KodeKloud Notes) | Billed per Neptune ML and underlying Neptune usage |
| | Amazon MemoryDB (Redis-compatible) | In-memory vector search with sub-ms latency, ideal for real-time low-latency use cases (Instaclustr, KodeKloud Notes) | Standard MemoryDB pricing applies |
| | Amazon DocumentDB | Document (Mongo-compatible) DB with vector search via extensions (Instaclustr, KodeKloud Notes) | Follows DocumentDB pricing |
| | Amazon S3 Vectors (Preview) | Object storage optimized for vector storage & search; vector buckets and indexes, strong consistency (AWS Documentation) | Preview status—cost-optimized but pricing details TBD |
| Azure | Azure AI Search (formerly Cognitive Search) | Vector search with hybrid capabilities, built-in vectorizers and semantic ranking; no extra charge for vector functionality—cost depends on Search Units (partitions + replicas) (Microsoft Learn) | Example: S1 tier with 28M vectors (768 dims) costs \~\$1/hour; storage & throughput scale with units (TECHCOMMUNITY.MICROSOFT.COM, Microsoft Learn) |
| | Azure Cosmos DB (NoSQL / MongoDB API) | Native vector search with DiskANN (flat, quantized) indexing, serverless, global distribution, 99.999% SLA (Reddit, Wikipedia) | Pricing depends on RU/s provisioning; vector search uses same cost model |
| | Azure Database for PostgreSQL (Flex) | PostgreSQL with native vector data type support and pgvector extension (Microsoft Learn, Wikipedia) | Billed under standard Azure Database for PostgreSQL rates |
| | Azure Cosmos DB for Cassandra (Managed Instance) | Cassandra API with DiskANN-backed vector search in public preview (Microsoft for Developers) | Pricing per Cosmos DB managed instance pricing |
| Google Cloud | AlloyDB AI (with pgvector) | Fully-managed PostgreSQL-compatible database optimized for vector + transactional workloads, enterprise-ready (Forbes) | AlloyDB pricing applies; enterprise-level SLAs |
| | Spanner (Vector Search preview) | Globally-distributed relational DB with exact KNN vector search, 99.999% availability (Google Cloud) | Pay-per-use for Spanner, high availability & scale |
| | Vertex AI RAG Engine – RagManagedDb / Vector Search | RagManagedDb (default) for small-scale quickstart; Vector Search (GA) for enterprise RAG workloads; others: Feature Store (Google Cloud) | Pay-as-you-go pricing; varies by service and scale |
