Finetuning LLMS
Deciding on Compute
- Free/Cheap: Google Colab: works with models for 7-8B with QLORA
- Faster/GPU Rentals: A10/A100 on RunPod/Colab Pro/kaggle. Or create the equivelent machines on the cloud and run
Approach
Preparing a dataset
- For instruct models we can give instructions in the form or
{ "instruction": "....", "input":"....", "output": "....."}
Configure the training
- choosing Finetuning model
- Hyperparamters
- Start the training (w&b which can publish some metrics)
- Export, test and save/push
Scenario: Customer Support Reply Copilot
- Hyperparmeters: Refer Here
- Recommedned start values
max_seq_length = 2048
learning_rate = 2e-4
lr_scheduler_type = linear
weight_decay = 0.01
#LORA:
r = 16
lora_apha = 32
lora_dropout = 0.05
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj",],
#QLORA
load_in_4bit = True
bnb_4bit_quant_type = "nf4"
- Brand = AmericanAir
- Refer Here for the colab notebook
