content.raw: Fine-tune 7B models on free-tier Colab hardware using Unsloth 🦥
content.type: text
content.value: Fine-tune 7B models on free-tier Colab hardware using Unsloth 🦥
content.type: new_line
content.type: new_line
content.raw: Unsloth is a framework for fine tuning language models boasting a 0% loss in accuracy  while using no approximation methods. They offer  a trainer for both supervised fine-tuning (SFT) and direct preference optimization (DPO) that can increase speed of fine-tuning by up to 5x.
content.type: text
content.value: Unsloth is a framework for fine tuning language models boasting a 0% loss in accuracy  while using no approximation methods. They offer  a trainer for both supervised fine-tuning (SFT) and direct preference optimization (DPO) that can increase speed of fine-tuning by up to 5x.
content.type: new_line
content.raw: This is achieved by adding LoRa adapters. This way they only need to train 1 to 10% of the total parameters. You can export a LoRa adapter or merge to 16-bit for a full finetune. The resulting model is prepared for use in vLLM for faster inference.
content.type: text
content.value: This is achieved by adding LoRa adapters. This way they only need to train 1 to 10% of the total parameters. You can export a LoRa adapter or merge to 16-bit for a full finetune. The resulting model is prepared for use in vLLM for faster inference.
content.type: new_line
content.type: new_line
content.raw: Additionally, Huggingface has integrated Unsloth into the documentation for DPO training and reported 18.6% performance gains on T4. 
content.type: text
content.value: Additionally, Huggingface has integrated Unsloth into the documentation for DPO training and reported 18.6% performance gains on T4. 
content.type: new_line
content.type: new_line
content.raw: This sets a new standard for fine-tuning large language models. If you would like to explore this methodology for yourself I have provided a notebook "AutoSloth," where you can fine tune using either SFT or DPO and it will upload to HF with a prefilled Unsloth README 🦥 and a Q8_0 quantization.
content.type: text
content.value: This sets a new standard for fine-tuning large language models. If you would like to explore this methodology for yourself I have provided a notebook "AutoSloth," where you can fine tune using either SFT or DPO and it will upload to HF with a prefilled Unsloth README 🦥 and a Q8_0 quantization.
content.type: new_line
content.type: new_line
content.raw: The SFT example is set up for free tier usage, but the DPO example is set up for an A100. The DPO example can be altered to work on T4 but I wanted to include more than one example. 
content.type: text
content.value: The SFT example is set up for free tier usage, but the DPO example is set up for an A100. The DPO example can be altered to work on T4 but I wanted to include more than one example. 
content.type: new_line
content.type: new_line
content.raw: Colab Stats during training:
content.type: text
content.value: Colab Stats during training:
content.type: new_line
content.raw: + Model: unsloth/mistral-7b-bnb-4bit
content.type: text
content.value: + Model: unsloth/mistral-7b-bnb-4bit
content.type: new_line
content.raw: + Dataset: yahma/alpaca-cleaned
content.type: text
content.value: + Dataset: yahma/alpaca-cleaned
content.type: new_line
content.raw: + Batch size: 2
content.type: text
content.value: + Batch size: 2
content.type: new_line
content.raw: + Gradient steps: 4
content.type: text
content.value: + Gradient steps: 4
content.type: new_line
content.raw: + System RAM: 8.5 / 51.0 GB
content.type: text
content.value: + System RAM: 8.5 / 51.0 GB
content.type: new_line
content.raw: + VRAM (T4): 13.6 / 15.0 GB
content.type: text
content.value: + VRAM (T4): 13.6 / 15.0 GB
content.type: new_line
content.type: new_line
content.raw: Resources:
content.type: text
content.value: Resources:
content.type: new_line
content.raw: 🦥Unsloth: 
content.type: text
content.value: 🦥Unsloth: 
content.href: https://github.com/unslothai/unsloth
content.raw: https://github.com/unslothai/unsloth
content.type: link
content.type: new_line
content.raw: 🦥AutoSloth: 
content.type: text
content.value: 🦥AutoSloth: 
content.href: https://colab.research.google.com/drive/1Zo0sVEb2lqdsUm9dy2PTzGySxdF9CNkc?usp=sharing
content.raw: https://colab.research.google.com/drive/1Zo0sVEb2lqdsUm9dy2PTzGySxdF9CNkc?usp=sharing
content.type: link
content.type: new_line
content.raw: 🤗HF-Unsloth-docs: 
content.type: text
content.value: 🤗HF-Unsloth-docs: 
content.href: https://huggingface.co/docs/trl/main/en/dpo_trainer#accelerate-dpo-fine-tuning-using-unsloth
content.raw: https://huggingface.co/docs/trl/main/en/dpo_trainer#accelerate-dpo-fine-tuning-using-unsloth
content.type: link
content.type: new_line
content.raw: 🤗HF-Unsloth Blog Post: 
content.type: text
content.value: 🤗HF-Unsloth Blog Post: 
content.href: https://huggingface.co/blog/unsloth-trl
content.raw: https://huggingface.co/blog/unsloth-trl
content.type: link
rawContent: Fine-tune 7B models on free-tier Colab hardware using Unsloth 🦥
  
  Unsloth is a framework for fine tuning language models boasting a 0% loss in accuracy  while using no approximation methods. They offer  a trainer for both supervised fine-tuning (SFT) and direct preference optimization (DPO) that can increase speed of fine-tuning by up to 5x.
  This is achieved by adding LoRa adapters. This way they only need to train 1 to 10% of the total parameters. You can export a LoRa adapter or merge to 16-bit for a full finetune. The resulting model is prepared for use in vLLM for faster inference.
  
  Additionally, Huggingface has integrated Unsloth into the documentation for DPO training and reported 18.6% performance gains on T4. 
  
  This sets a new standard for fine-tuning large language models. If you would like to explore this methodology for yourself I have provided a notebook "AutoSloth," where you can fine tune using either SFT or DPO and it will upload to HF with a prefilled Unsloth README 🦥 and a Q8_0 quantization.
  
  The SFT example is set up for free tier usage, but the DPO example is set up for an A100. The DPO example can be altered to work on T4 but I wanted to include more than one example. 
  
  Colab Stats during training:
  + Model: unsloth/mistral-7b-bnb-4bit
  + Dataset: yahma/alpaca-cleaned
  + Batch size: 2
  + Gradient steps: 4
  + System RAM: 8.5 / 51.0 GB
  + VRAM (T4): 13.6 / 15.0 GB
  
  Resources:
  🦥Unsloth: https://github.com/unslothai/unsloth
  🦥AutoSloth: https://colab.research.google.com/drive/1Zo0sVEb2lqdsUm9dy2PTzGySxdF9CNkc?usp=sharing
  🤗HF-Unsloth-docs: https://huggingface.co/docs/trl/main/en/dpo_trainer#accelerate-dpo-fine-tuning-using-unsloth
  🤗HF-Unsloth Blog Post: https://huggingface.co/blog/unsloth-trl
reactions.reaction: ❤️
reactions.users: alielfilali01 clem sinanyuksel eramax samusenps osanseviero victor vladbogo ybelkada alvarobartt koalazf99 tomaarsen Citaman Dlbk davanstrien kolaposki julien-c Ji-Ha maywell Lovnir Krasi pandora-s boapps felicityworld pcuenq horiguchidotconf
reactions.reaction: 🤝
reactions.users: ybelkada Dlbk julien-c felicityworld horiguchidotconf
commentators._id: 6455cc8d679315e4ef16fbec
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6455cc8d679315e4ef16fbec/M6Cfifn05BUzkCFd2QDIT.png
commentators.fullname: Tim Dolan
commentators.name: macadeliccc
commentators.type: user
commentators._id: 626237d9bbcbd1c34f1bb231
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/626237d9bbcbd1c34f1bb231/EJrOjvAL-68qMCYdnvOrq.png
commentators.fullname: Ali El Filali
commentators.name: alielfilali01
commentators.type: user
commentators._id: 62441d1d9fdefb55a0b7d12c
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1648631057413-noauth.png
commentators.fullname: Younes B
commentators.name: ybelkada
commentators.type: user
commentators._id: 6439f43a1514b7ee7fb616a1
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6439f43a1514b7ee7fb616a1/aFhmyAoicv3zcWKYZ27Z_.png
commentators.fullname: Jeonghwan Park
commentators.name: maywell
commentators.type: user