Usecase: Making ai applictions which are aware of your private data
RAG ( Retrieval-Augmented Generation):
- RAG augments a LLM with external knowledge source(database, documents or API calls)
- Instead of depending on only whats in the model’s parameters, this system retrieves releavent information at query time and fees it into LLM’s context
- How it works:
- User queries ==> converted into embedding
- Embedding search => retrieve relevent chunks from vector database
- Retrieved data => injected into LLM Prompt (Context Window)
- LLM generates the respnse grounded in both its pretraining & the retrieved info.
- Pros:
- Easy to update (adding or removing documents ) – no training required
- keeps responses accurate and upto date
- Avoid hallucinations by grounding answers in real data
- cost-effective compared to fine tuning
- Cons:
- Dependent on retrieval quality (garbage in -> garbage out)
- Limited by context window size
Best Use-cases: - Dynamic or frequently updated knowledge (eg regulations, product catalogs, research paper, defects …)
- Domain Specific Q&A (Customer support bots, internal company knowledge assistants)
- Multi-document reasoning (chat with pdfs, textbooks or code bases)
- Easy to prototype where requirements chnage often
Fine-tuning
- Fine-tuning means adapting the base LLM’s weights on additional training data so it learns domain specific patterns, styles or tasks
-
How it works:
- Collect task-specific dataset (instruction, response, classification, reasoning steps etc)
- Train the base LLM with this dataset
- The models parameters are updated, making the knowledge/patterns native to the model
-
Pros:
- Encodes knowledge directly -> no need to retrieve every time
- Faster inference
- Can captutre domain-specific reasoning styles (medical diagnosis, legal logic)
- Cons:
- Requires high quality curated dataset
- Expensive (compute + data labelling)
- knowledge becomes stale unless retrained
- Risk of overfitting
-
Best Usecases:
- Stable domain knowledge that doesnt change frequently (medical terminology, legal contracts)
- custom tone/style (brand-specific writing style)
- Highy specialize tasks
- When retrieval isn’t feasibel
-
When to use what
| scenario | RAG | Finetuning |
| ——- | —-| ———–|
| Knowledge updates frequently | yes (just update docs) | No (would required frequent retraining) |
| Need deep domain reasoning | no but still possible with good chunking + retrieval | Yes, fine-tuning adapts with reasoning patterns |
| Custom tone/style | Limited | Best Choice |
| Access to large structured Knowledge Base | Works well | Not ideal |
| Latency-sensitive | this adds retrieval overhead | Works well |
| Prototyping | Quick to build | costly/time consuming |
Hybrid Approach
- In practice RAG + Fine-tuning is powerful together
- Fine tune for style, reasoning strucure or formatting rules
- Use RAG for dynamic knowledge injection
Rules
- Use RAG first (As it is cheaper)
- Move to fine tuning only when you need stable reasoning, style or low latency specilization
- Combine both for enterprise grade systems
scenario 1 : Customer Support for ecommerce company
- You need a chatbot that can answer queries like
- where is my order
- whats your refund policy
- Show me latest iphone accessories available
- Brainstorm:
- knowledge type: FAQ, Policies, product catalog, order transcation history -> they change frequently
- Style: Needs to be polite
- Bestfit: RAG
Scenario 2: Internal policy compliance Copilot (Legal/HR)
- Context: An assitant helps employees and managers to understand internal policies, contracts and HR rules. Typical queries
- Am i eligible for paternity leave
- What are security requirements for handling client type
- Can i expense this type of travel
- Brainstorm
- knowledge type: Company polcies, contract, HR manuals -> Often update
- Bestfit: RAG
Scenario 3: Financial Research Agent (equities, filings, news, KPIs)
- Context: We are building an assistant that
- summarizes earnings calls
- Extracts KPIs
- Tracks guidance changes, risks catalysts from news
- Produces analyist style memos with citations
- Bestfit: Hybrid (RAG + Tools + Light finetuning)
Scenario 4: Competitive Examp prep (IIT-JEE/NEET)
- Context: We are building an exam tutor that
- Maps syllabus topics -> textbooks -> solved examples -> past papers
- Answers conceptual doubts stepwise
- Generates practice problems at varying difficulty
- Analyses students errors and prescribes remediation paths
- must align with official syllabus and marking scheme
- What matters:
- Syllabus grounding
- Freshness
- Bestfit:
- RAG = content grouding
- Finetuning:
- exam strategy
- hinting style
- Teach model to output in exam solving schema
