Answers.Body: <p>I have seen researchers use both approaches.
  Typically freezing is useful if you have very few examples (&lt;100 in my cases), Otherwise</p>
  <ol>
  <li>Do you have enough resources to Fine-Tune with an unfrozen model?</li>
  <li>Is performance better with an unfrozen or frozen model?</li>
  </ol>
  <p>Many people have turned to LoRA to avoid this issue.
  <a href="https://github.com/microsoft/LoRA" rel="nofollow noreferrer">https://github.com/microsoft/LoRA</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-19T17:40:29.547Z
Answers.Text: Why would an unfrozen model get "messed up" easier by a dataset with few examples?
Answers.UserDisplayName: multiheadedattention
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-22T18:04:14.847Z
Answers.Text: What defines an "easier" dataset is unclear, but you must have enough samples to represent the true distribution, otherwise, your model will be underfitting. Simply put, small datasets are more difficult to generalize from, therefore unfreezing the weights might lead to degradation in performance.
Answers.UserDisplayName: Thomas K
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-14T16:00:18.890Z
Answers.Text: What is the difference between pretraining a foundational model and fine-tuning with unfrozen weights, then?
Answers.UserDisplayName: illan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-24T17:46:07.663Z
Answers.Text: Those are two different steps. Pre-training generally means a period of unsupervised/semi-supervised learning with large datasets. After pre-training, you won't have a useful model. It then requires supervised fine-tuning with unfrozen weights to be of some use. While the research has not fully proven why pre-training is important, pre-training models on any large related dataset typically yields performance improvements in fine-tuning.
Answers.UserDisplayName: Thomas K
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-18T21:04:23.750Z
Answers.LastActivityDate: 2023-05-18T21:04:23.750Z
Answers.UserDisplayName: Thomas K
Body: <p>I've had it in my head that generally speaking, it's better to freeze layers when fine-tuning an LLM, as per this quote from HuggingFace's <a href="https://huggingface.co/blog/peft" rel="nofollow noreferrer">article</a>:</p>
  <blockquote>
  <p>PEFT approaches only fine-tune a small number of (extra) model
  parameters while freezing most parameters of the pretrained LLMs,
  thereby greatly decreasing the computational and storage costs. This
  also overcomes the issues of catastrophic forgetting, a behaviour
  observed during the full finetuning of LLMs. PEFT approaches have also
  shown to be better than fine-tuning in the low-data regimes and
  generalize better to out-of-domain scenarios. It can be applied to
  various modalities, e.g., image classification and stable diffusion
  dreambooth.</p>
  </blockquote>
  <p>I think what I might be confused by is what is meant by the &quot;(extra)&quot; part. It led me to try fine-tuning a BERT model in PyTorch by freezing all parameters except for the final feed-forward of the transformer responsible for sequence classification:</p>
  <pre><code>for param in model.parameters():
      param.requires_grad = False
  
  for param in model.classifier.parameters():
      param.requires_grad = True
  </code></pre>
  <p>However, this caused my model to get <em>significantly</em> worse evaluation metrics on my test set than before I did this. This lead me to the following conclusions:</p>
  <ul>
  <li><p>My dataset of ~100K datapoints is not of a &quot;low-data regime&quot; and therefore doesn't benefit from PEFT? But doesn't it say this generalizes better to &quot;out-of-domain scenarios&quot;? How do I know the particular seq classification I'm doing with BERT is out-of-domain? Because it isn't specifically a next-sequence prediction task?</p>
  </li>
  <li><p>Is this the cost of misinterpreting the &quot;(extra)&quot; model parameters part? I'm fine-tuning a small number of extant model parameters here, not extra.</p>
  </li>
  </ul>
  <p>I'm just confused here. The quote I've showed here makes me believe my PEFT model should've outperformed a regular fine-tuning.</p>
  
Tags: <transformer><performance><fine-tuning><catastrophic-forgetting>
Title: Should I be layer freezing when fine-tuning an LLM?
UserDisplayName: multiheadedattention
ThreadText: # Should I be layer freezing when fine-tuning an LLM?
  
  (Asked by: username_0 on 2023-05-18)
  
  I've had it in my head that generally speaking, it's better to freeze layers when fine-tuning an LLM, as per this quote from HuggingFace's [article](https://huggingface.co/blog/peft):
  
  > PEFT approaches only fine-tune a small number of (extra) model
  > parameters while freezing most parameters of the pretrained LLMs,
  > thereby greatly decreasing the computational and storage costs. This
  > also overcomes the issues of catastrophic forgetting, a behaviour
  > observed during the full finetuning of LLMs. PEFT approaches have also
  > shown to be better than fine-tuning in the low-data regimes and
  > generalize better to out-of-domain scenarios. It can be applied to
  > various modalities, e.g., image classification and stable diffusion
  > dreambooth.
  
  I think what I might be confused by is what is meant by the "(extra)" part. It led me to try fine-tuning a BERT model in PyTorch by freezing all parameters except for the final feed-forward of the transformer responsible for sequence classification:
  
  ```
  for param in model.parameters():
      param.requires_grad = False
  
  for param in model.classifier.parameters():
      param.requires_grad = True
  
  ```
  
  However, this caused my model to get *significantly* worse evaluation metrics on my test set than before I did this. This lead me to the following conclusions:
  
  * My dataset of ~100K datapoints is not of a "low-data regime" and therefore doesn't benefit from PEFT? But doesn't it say this generalizes better to "out-of-domain scenarios"? How do I know the particular seq classification I'm doing with BERT is out-of-domain? Because it isn't specifically a next-sequence prediction task?
  * Is this the cost of misinterpreting the "(extra)" model parameters part? I'm fine-tuning a small number of extant model parameters here, not extra.
  
  I'm just confused here. The quote I've showed here makes me believe my PEFT model should've outperformed a regular fine-tuning.
  
  **Tags:** transformer, performance, fine-tuning, catastrophic-forgetting
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-05-18. Score: 0
  
  I have seen researchers use both approaches.
  Typically freezing is useful if you have very few examples (<100 in my cases), Otherwise
  
  1. Do you have enough resources to Fine-Tune with an unfrozen model?
  2. Is performance better with an unfrozen or frozen model?
  
  Many people have turned to LoRA to avoid this issue.
  <https://github.com/microsoft/LoRA>
  
  > **Comments:**
  > **username_0**: Why would an unfrozen model get "messed up" easier by a dataset with few examples?  
  > **username_1**: What defines an "easier" dataset is unclear, but you must have enough samples to represent the true distribution, otherwise, your model will be underfitting. Simply put, small datasets are more difficult to generalize from, therefore unfreezing the weights might lead to degradation in performance.  
  > **username_2**: What is the difference between pretraining a foundational model and fine-tuning with unfrozen weights, then?  
  > **username_1**: Those are two different steps. Pre-training generally means a period of unsupervised/semi-supervised learning with large datasets. After pre-training, you won't have a useful model. It then requires supervised fine-tuning with unfrozen weights to be of some use. While the research has not fully proven why pre-training is important, pre-training models on any large related dataset typically yields performance improvements in fine-tuning.  
  
  ---
  
  