Low Rank Adaptation (LoRA) is a popular method for fine-tuning but why...? 🤔
text
Low Rank Adaptation (LoRA) is a popular method for fine-tuning but why...? 🤔
new_line
new_line
LoRA is the most widely adopted method for Parameter Efficient Fine-Tuning (PEFT) of Large Language Models (LLMs) 🤖. It reduces the number of trainable parameters and memory usage while achieving comparable performance to full fine-tuning 💾.
text
LoRA is the most widely adopted method for Parameter Efficient Fine-Tuning (PEFT) of Large Language Models (LLMs) 🤖. It reduces the number of trainable parameters and memory usage while achieving comparable performance to full fine-tuning 💾.
new_line
new_line
Good folks at 
text
Good folks at 
@predibase
mention
predibase
  have released LoRA Land! 🌍
text
  have released LoRA Land! 🌍
new_line
new_line
A research paper where they measure the quality of LLMs fine-tuned with quantized low-rank adapters across 10 base models and 31 tasks for a total of a whopping 310 models 📊.
text
A research paper where they measure the quality of LLMs fine-tuned with quantized low-rank adapters across 10 base models and 31 tasks for a total of a whopping 310 models 📊.
new_line
new_line
According to the paper, 4-bit LoRA fine-tuned models outperform base models by 34 points 🔝 and GPT-4 by 10 points on average 👏.
text
According to the paper, 4-bit LoRA fine-tuned models outperform base models by 34 points 🔝 and GPT-4 by 10 points on average 👏.
new_line
new_line
Here comes the best part... 🥁
text
Here comes the best part... 🥁
new_line
new_line
They have also evaluated the latency and concurrency capabilities of LoRAX, an open-source Multi-LoRA inference server that facilitates the deployment of multiple LoRA fine-tuned models on a single GPU using the shared base model weights and dynamic adapter loading 🚀.
text
They have also evaluated the latency and concurrency capabilities of LoRAX, an open-source Multi-LoRA inference server that facilitates the deployment of multiple LoRA fine-tuned models on a single GPU using the shared base model weights and dynamic adapter loading 🚀.
new_line
new_line
And did I mention LoRAX is completely open-source! 🎉
text
And did I mention LoRAX is completely open-source! 🎉
new_line
new_line
Paper: 
text
Paper: 
https://arxiv.org/pdf/2405.00732
https://arxiv.org/pdf/2405.00732
link
new_line
new_line
While the claims are lofty and yet to be vetted, this is amazing for serving fine-tuned models without breaking the bank! 💸
text
While the claims are lofty and yet to be vetted, this is amazing for serving fine-tuned models without breaking the bank! 💸
new_line
new_line
PS: 🤯They got LoRAX to host 25 LoRA fine-tuned Mistral-7B LLMs on a single NVIDIA A100 GPU with 80GB memory 💻.
text
PS: 🤯They got LoRAX to host 25 LoRA fine-tuned Mistral-7B LLMs on a single NVIDIA A100 GPU with 80GB memory 💻.
new_line
new_line
LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report (2405.00732)
https://huggingface.co/papers/2405.00732
2405.00732
paper
resource
https://huggingface.co/papers/2405.00732
Low Rank Adaptation (LoRA) is a popular method for fine-tuning but why...? 🤔

LoRA is the most widely adopted method for Parameter Efficient Fine-Tuning (PEFT) of Large Language Models (LLMs) 🤖. It reduces the number of trainable parameters and memory usage while achieving comparable performance to full fine-tuning 💾.

Good folks at @predibase  have released LoRA Land! 🌍

A research paper where they measure the quality of LLMs fine-tuned with quantized low-rank adapters across 10 base models and 31 tasks for a total of a whopping 310 models 📊.

According to the paper, 4-bit LoRA fine-tuned models outperform base models by 34 points 🔝 and GPT-4 by 10 points on average 👏.

Here comes the best part... 🥁

They have also evaluated the latency and concurrency capabilities of LoRAX, an open-source Multi-LoRA inference server that facilitates the deployment of multiple LoRA fine-tuned models on a single GPU using the shared base model weights and dynamic adapter loading 🚀.

And did I mention LoRAX is completely open-source! 🎉

Paper: https://arxiv.org/pdf/2405.00732

While the claims are lofty and yet to be vetted, this is amazing for serving fine-tuned models without breaking the bank! 💸

PS: 🤯They got LoRAX to host 25 LoRA fine-tuned Mistral-7B LLMs on a single NVIDIA A100 GPU with 80GB memory 💻.

https://huggingface.co/papers/2405.00732