Distillation
Model Types
-
For fine tuning we have two types of models which we can use
- pre-trained model: This knows how to predict next token
- instruct models: This model knows how to behave and ensures safety.
-
Prompt:
Give me list of pretrained models and their urls in ollama and hugging face and do the the same for instruct models. Ensure you give me atleast 10 popular models in markdown format
Which models to choose (Pre-trained vs Instruct)
- If you have massive data sets go for pre-trained and if you have limited data sets (less than 50k records) better to go with instruct models.
- Pre-trained models are tuned by large AI labs where as instruct models can be tuned by startups as well.
Fine tuning options
- On a larger note there are two techniques
- Full Fine tuning
- PEFT (Parameter effecient Fine Tuning)
-
Full Fine Tuning: We take an existing model and change all the weights (For mathematical convinence)
-
In PEFT we would use LORA reduce the number of paramters by adding adapter
- We also have Q-LORA
