content.raw: Low Rank Adaptation (LoRA) is a popular method for fine-tuning but why...? 🤔
content.type: text
content.value: Low Rank Adaptation (LoRA) is a popular method for fine-tuning but why...? 🤔
content.type: new_line
content.type: new_line
content.raw: LoRA is the most widely adopted method for Parameter Efficient Fine-Tuning (PEFT) of Large Language Models (LLMs) 🤖. It reduces the number of trainable parameters and memory usage while achieving comparable performance to full fine-tuning 💾.
content.type: text
content.value: LoRA is the most widely adopted method for Parameter Efficient Fine-Tuning (PEFT) of Large Language Models (LLMs) 🤖. It reduces the number of trainable parameters and memory usage while achieving comparable performance to full fine-tuning 💾.
content.type: new_line
content.type: new_line
content.raw: Good folks at 
content.type: text
content.value: Good folks at 
content.raw: @predibase
content.type: mention
content.user: predibase
content.raw:   have released LoRA Land! 🌍
content.type: text
content.value:   have released LoRA Land! 🌍
content.type: new_line
content.type: new_line
content.raw: A research paper where they measure the quality of LLMs fine-tuned with quantized low-rank adapters across 10 base models and 31 tasks for a total of a whopping 310 models 📊.
content.type: text
content.value: A research paper where they measure the quality of LLMs fine-tuned with quantized low-rank adapters across 10 base models and 31 tasks for a total of a whopping 310 models 📊.
content.type: new_line
content.type: new_line
content.raw: According to the paper, 4-bit LoRA fine-tuned models outperform base models by 34 points 🔝 and GPT-4 by 10 points on average 👏.
content.type: text
content.value: According to the paper, 4-bit LoRA fine-tuned models outperform base models by 34 points 🔝 and GPT-4 by 10 points on average 👏.
content.type: new_line
content.type: new_line
content.raw: Here comes the best part... 🥁
content.type: text
content.value: Here comes the best part... 🥁
content.type: new_line
content.type: new_line
content.raw: They have also evaluated the latency and concurrency capabilities of LoRAX, an open-source Multi-LoRA inference server that facilitates the deployment of multiple LoRA fine-tuned models on a single GPU using the shared base model weights and dynamic adapter loading 🚀.
content.type: text
content.value: They have also evaluated the latency and concurrency capabilities of LoRAX, an open-source Multi-LoRA inference server that facilitates the deployment of multiple LoRA fine-tuned models on a single GPU using the shared base model weights and dynamic adapter loading 🚀.
content.type: new_line
content.type: new_line
content.raw: And did I mention LoRAX is completely open-source! 🎉
content.type: text
content.value: And did I mention LoRAX is completely open-source! 🎉
content.type: new_line
content.type: new_line
content.raw: Paper: 
content.type: text
content.value: Paper: 
content.href: https://arxiv.org/pdf/2405.00732
content.raw: https://arxiv.org/pdf/2405.00732
content.type: link
content.type: new_line
content.type: new_line
content.raw: While the claims are lofty and yet to be vetted, this is amazing for serving fine-tuned models without breaking the bank! 💸
content.type: text
content.value: While the claims are lofty and yet to be vetted, this is amazing for serving fine-tuned models without breaking the bank! 💸
content.type: new_line
content.type: new_line
content.raw: PS: 🤯They got LoRAX to host 25 LoRA fine-tuned Mistral-7B LLMs on a single NVIDIA A100 GPU with 80GB memory 💻.
content.type: text
content.value: PS: 🤯They got LoRAX to host 25 LoRA fine-tuned Mistral-7B LLMs on a single NVIDIA A100 GPU with 80GB memory 💻.
content.type: new_line
content.type: new_line
content.label: LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report (2405.00732)
content.raw: https://huggingface.co/papers/2405.00732
content.id: 2405.00732
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2405.00732
rawContent: Low Rank Adaptation (LoRA) is a popular method for fine-tuning but why...? 🤔
  
  LoRA is the most widely adopted method for Parameter Efficient Fine-Tuning (PEFT) of Large Language Models (LLMs) 🤖. It reduces the number of trainable parameters and memory usage while achieving comparable performance to full fine-tuning 💾.
  
  Good folks at @predibase  have released LoRA Land! 🌍
  
  A research paper where they measure the quality of LLMs fine-tuned with quantized low-rank adapters across 10 base models and 31 tasks for a total of a whopping 310 models 📊.
  
  According to the paper, 4-bit LoRA fine-tuned models outperform base models by 34 points 🔝 and GPT-4 by 10 points on average 👏.
  
  Here comes the best part... 🥁
  
  They have also evaluated the latency and concurrency capabilities of LoRAX, an open-source Multi-LoRA inference server that facilitates the deployment of multiple LoRA fine-tuned models on a single GPU using the shared base model weights and dynamic adapter loading 🚀.
  
  And did I mention LoRAX is completely open-source! 🎉
  
  Paper: https://arxiv.org/pdf/2405.00732
  
  While the claims are lofty and yet to be vetted, this is amazing for serving fine-tuned models without breaking the bank! 💸
  
  PS: 🤯They got LoRAX to host 25 LoRA fine-tuned Mistral-7B LLMs on a single NVIDIA A100 GPU with 80GB memory 💻.
  
  https://huggingface.co/papers/2405.00732