Typical Model Training
Stage 1: Base model – Become knowledagble
- Here a model is trained on large volumes of data where the result is model can predict next token
Stage 2: Instruct model – Learn how to interact
- Here a methodolgy called supervised finetuning is used to teach model on how to follow instructions.
Stage 3: RLHF – Ensure you answer right questions.
- Here Reinforcement Learning with Human feedback ensure model does not respond to sensitive/unnecessary instructions.
Unsloth for fine tuning
- To tune an existing models behavior
- Datasets
- Hyperparameters
- metrics to evaluate the tuning
- steps
unsloth docs
- Refer Here for unsloth docs
- Fine tuning guide
Hyper-parameters
-
Rank: Generally it is multiple of 8
- Alpha: Scaling factor. Best practice is to have alpha value 2 times rank.
- Learning Rate:
- Batch size
- gradient_accumulation_steps
- epoch
metrics to observe
- learning loss
-
validation loss
-
Prompt:
Express in a tabular form how to respond to different behaviors of learning loss and validation loss during finetuning using unsloth
