Answers.Body: <p>There are considerable free and excellent resources out there (in the wild).</p>
  <p>You can check The Stanford Natural Language Processing Group teaching <a href="https://nlp.stanford.edu/teaching/" rel="nofollow noreferrer">page</a>; you can easily follow their YouTube courses on NLP (natural language processing) and NLU (natural language understanding).</p>
  <p>Moreover, you can read scientific papers on Transformers, the bread and butter of HuggingFace's models; a quick Google search should do the work.</p>
  <p>Finally, you must know Pytorch since (if I recall correctly) HuggingFace's implementations use it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-08T09:41:05.940Z
Answers.LastActivityDate: 2023-02-08T09:41:05.940Z
Answers.UserDisplayName: Eduard
Body: <p>I want to know what things I should be learning before trying to fine-tune or for that matter working with a large language model.</p>
  <p>In my case, I am trying to fine-tune bloom (<a href="https://huggingface.co/blog/bloom" rel="nofollow noreferrer">https://huggingface.co/blog/bloom</a>).</p>
  <p>But considering my inadequate knowledge of this, I want to know what things I should cover first before taking on something so big.</p>
  <p>Any suggestions are appreciated.</p>
  
Tags: <natural-language-processing><language-model><fine-tuning>
Title: What background should I have before starting to fine tune a Large Language Model?
UserDisplayName: kelly
ThreadText: # What background should I have before starting to fine tune a Large Language Model?
  
  (Asked by: username_0 on 2023-02-08)
  
  I want to know what things I should be learning before trying to fine-tune or for that matter working with a large language model.
  
  In my case, I am trying to fine-tune bloom (<https://huggingface.co/blog/bloom>).
  
  But considering my inadequate knowledge of this, I want to know what things I should cover first before taking on something so big.
  
  Any suggestions are appreciated.
  
  **Tags:** natural-language-processing, language-model, fine-tuning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-02-08. Score: 3 | Accepted answer
  
  There are considerable free and excellent resources out there (in the wild).
  
  You can check The Stanford Natural Language Processing Group teaching [page](https://nlp.stanford.edu/teaching/); you can easily follow their YouTube courses on NLP (natural language processing) and NLU (natural language understanding).
  
  Moreover, you can read scientific papers on Transformers, the bread and butter of HuggingFace's models; a quick Google search should do the work.
  
  Finally, you must know Pytorch since (if I recall correctly) HuggingFace's implementations use it.
  
  
  ---
  
  