Why would i fine tune
- Finetuning is not for facts its for behavior.
Example 1
- usecase: I have a need to build an application which will help in retrieving the latest information about organization policies.
- Solution: A RAG would be right fit
Example 2
- usecase: I’m building a customer support chatbot, where the model should respond as a trained customer support of my insurance company.
- Solution:
- Behavior => finetuning
- Facts => RAG
Datasets
- Most popular format over here is
jsonl - In most of the cases we finetune the instruct models, so the jsonl will have conversational examples.
-
The structure of each line depends on which model we ar tuning.
-
Generally we finetune opensource models, what if i have to finetune claude, gemini, gpt model
How do i run the opensource models on my hardware
- Explorations:
- hugging face
- C++ Runtime
- llama.cpp
- Easier options to deploy
- ollama
- docker
- Production:
- vLLM
Next Topics
- Use case for finetuning
- Concepts:
- PEFT
- LORA
- QLORA
