Fine-tune 7B models on free-tier Colab hardware using Unsloth 🦥
text
Fine-tune 7B models on free-tier Colab hardware using Unsloth 🦥
new_line
new_line
Unsloth is a framework for fine tuning language models boasting a 0% loss in accuracy  while using no approximation methods. They offer  a trainer for both supervised fine-tuning (SFT) and direct preference optimization (DPO) that can increase speed of fine-tuning by up to 5x.
text
Unsloth is a framework for fine tuning language models boasting a 0% loss in accuracy  while using no approximation methods. They offer  a trainer for both supervised fine-tuning (SFT) and direct preference optimization (DPO) that can increase speed of fine-tuning by up to 5x.
new_line
This is achieved by adding LoRa adapters. This way they only need to train 1 to 10% of the total parameters. You can export a LoRa adapter or merge to 16-bit for a full finetune. The resulting model is prepared for use in vLLM for faster inference.
text
This is achieved by adding LoRa adapters. This way they only need to train 1 to 10% of the total parameters. You can export a LoRa adapter or merge to 16-bit for a full finetune. The resulting model is prepared for use in vLLM for faster inference.
new_line
new_line
Additionally, Huggingface has integrated Unsloth into the documentation for DPO training and reported 18.6% performance gains on T4. 
text
Additionally, Huggingface has integrated Unsloth into the documentation for DPO training and reported 18.6% performance gains on T4. 
new_line
new_line
This sets a new standard for fine-tuning large language models. If you would like to explore this methodology for yourself I have provided a notebook "AutoSloth," where you can fine tune using either SFT or DPO and it will upload to HF with a prefilled Unsloth README 🦥 and a Q8_0 quantization.
text
This sets a new standard for fine-tuning large language models. If you would like to explore this methodology for yourself I have provided a notebook "AutoSloth," where you can fine tune using either SFT or DPO and it will upload to HF with a prefilled Unsloth README 🦥 and a Q8_0 quantization.
new_line
new_line
The SFT example is set up for free tier usage, but the DPO example is set up for an A100. The DPO example can be altered to work on T4 but I wanted to include more than one example. 
text
The SFT example is set up for free tier usage, but the DPO example is set up for an A100. The DPO example can be altered to work on T4 but I wanted to include more than one example. 
new_line
new_line
Colab Stats during training:
text
Colab Stats during training:
new_line
+ Model: unsloth/mistral-7b-bnb-4bit
text
+ Model: unsloth/mistral-7b-bnb-4bit
new_line
+ Dataset: yahma/alpaca-cleaned
text
+ Dataset: yahma/alpaca-cleaned
new_line
+ Batch size: 2
text
+ Batch size: 2
new_line
+ Gradient steps: 4
text
+ Gradient steps: 4
new_line
+ System RAM: 8.5 / 51.0 GB
text
+ System RAM: 8.5 / 51.0 GB
new_line
+ VRAM (T4): 13.6 / 15.0 GB
text
+ VRAM (T4): 13.6 / 15.0 GB
new_line
new_line
Resources:
text
Resources:
new_line
🦥Unsloth: 
text
🦥Unsloth: 
https://github.com/unslothai/unsloth
https://github.com/unslothai/unsloth
link
new_line
🦥AutoSloth: 
text
🦥AutoSloth: 
https://colab.research.google.com/drive/1Zo0sVEb2lqdsUm9dy2PTzGySxdF9CNkc?usp=sharing
https://colab.research.google.com/drive/1Zo0sVEb2lqdsUm9dy2PTzGySxdF9CNkc?usp=sharing
link
new_line
🤗HF-Unsloth-docs: 
text
🤗HF-Unsloth-docs: 
https://huggingface.co/docs/trl/main/en/dpo_trainer#accelerate-dpo-fine-tuning-using-unsloth
https://huggingface.co/docs/trl/main/en/dpo_trainer#accelerate-dpo-fine-tuning-using-unsloth
link
new_line
🤗HF-Unsloth Blog Post: 
text
🤗HF-Unsloth Blog Post: 
https://huggingface.co/blog/unsloth-trl
https://huggingface.co/blog/unsloth-trl
link
Fine-tune 7B models on free-tier Colab hardware using Unsloth 🦥

Unsloth is a framework for fine tuning language models boasting a 0% loss in accuracy  while using no approximation methods. They offer  a trainer for both supervised fine-tuning (SFT) and direct preference optimization (DPO) that can increase speed of fine-tuning by up to 5x.
This is achieved by adding LoRa adapters. This way they only need to train 1 to 10% of the total parameters. You can export a LoRa adapter or merge to 16-bit for a full finetune. The resulting model is prepared for use in vLLM for faster inference.

Additionally, Huggingface has integrated Unsloth into the documentation for DPO training and reported 18.6% performance gains on T4. 

This sets a new standard for fine-tuning large language models. If you would like to explore this methodology for yourself I have provided a notebook "AutoSloth," where you can fine tune using either SFT or DPO and it will upload to HF with a prefilled Unsloth README 🦥 and a Q8_0 quantization.

The SFT example is set up for free tier usage, but the DPO example is set up for an A100. The DPO example can be altered to work on T4 but I wanted to include more than one example. 

Colab Stats during training:
+ Model: unsloth/mistral-7b-bnb-4bit
+ Dataset: yahma/alpaca-cleaned
+ Batch size: 2
+ Gradient steps: 4
+ System RAM: 8.5 / 51.0 GB
+ VRAM (T4): 13.6 / 15.0 GB

Resources:
🦥Unsloth: https://github.com/unslothai/unsloth
🦥AutoSloth: https://colab.research.google.com/drive/1Zo0sVEb2lqdsUm9dy2PTzGySxdF9CNkc?usp=sharing
🤗HF-Unsloth-docs: https://huggingface.co/docs/trl/main/en/dpo_trainer#accelerate-dpo-fine-tuning-using-unsloth
🤗HF-Unsloth Blog Post: https://huggingface.co/blog/unsloth-trl
❤️
alielfilali01 clem sinanyuksel eramax samusenps osanseviero victor vladbogo ybelkada alvarobartt koalazf99 tomaarsen Citaman Dlbk davanstrien kolaposki julien-c Ji-Ha maywell Lovnir Krasi pandora-s boapps felicityworld pcuenq horiguchidotconf
🤝
ybelkada Dlbk julien-c felicityworld horiguchidotconf
626237d9bbcbd1c34f1bb231
https://cdn-avatars.huggingface.co/v1/production/uploads/626237d9bbcbd1c34f1bb231/EJrOjvAL-68qMCYdnvOrq.png
Ali El Filali
alielfilali01
user
6455cc8d679315e4ef16fbec
https://cdn-avatars.huggingface.co/v1/production/uploads/6455cc8d679315e4ef16fbec/M6Cfifn05BUzkCFd2QDIT.png
Tim Dolan
macadeliccc
user
62441d1d9fdefb55a0b7d12c
https://cdn-avatars.huggingface.co/v1/production/uploads/1648631057413-noauth.png
Younes B
ybelkada
user
6439f43a1514b7ee7fb616a1
https://cdn-avatars.huggingface.co/v1/production/uploads/6439f43a1514b7ee7fb616a1/aFhmyAoicv3zcWKYZ27Z_.png
Jeonghwan Park
maywell
user