Answers.Body: <p><a href="https://openai.com/research/better-language-models" rel="nofollow noreferrer">These models are trained with unsupervised learning</a>, but it's probably more intuitive to describe it as self-supervised learning.</p>
  <p>Your confusion makes sense: the next-token prediction objective, which trains models to predict the next token given the previous token, is basically just doing supervised learning with the inputs being the prefix of the text and the labels being the next token. But, here, the labels are created by us, from the same corpus that the input is from! So the model is supervised by the data itself: &quot;self-supervised&quot; learning.</p>
  <p>Also, to be clear, self-supervised learning <em>is</em> unsupervised*, you don't really have a separate set of labels---we're making them ourselves.</p>
  <p><sub>* I realize that these definitions can be controversial/ambiguous. I'm basing my usage on these: <a href="https://www.cs.rice.edu/%7Eas143/COMP642_Spring22/Scribes/Mar-24" rel="nofollow noreferrer">1</a>, <a href="https://proceedings.neurips.cc/paper/2020/file/70feb62b69f16e0238f741fab228fec2-Paper.pdf" rel="nofollow noreferrer">2</a>, <a href="http://multicomp.cs.cmu.edu/research/self-supervised-learning/" rel="nofollow noreferrer">3</a>, <a href="https://jmlr.csail.mit.edu/papers/volume23/21-1155/21-1155.pdf" rel="nofollow noreferrer">4</a>. You could alternatively say that self-supervised learning is a separate category altogether (e.g., <a href="https://web.eecs.umich.edu/%7Ejustincj/slides/eecs498/WI2022/598_WI2022_lecture22.pdf" rel="nofollow noreferrer">these slides</a>). But, all things considered, you probably don't need to worry too much about definitions as long as you understand the underlying concepts---people will know what you're talking about either way.</sub></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-28T04:22:50.303Z
Answers.LastActivityDate: 2024-02-28T19:58:50.230Z
Answers.LastEditDate: 2024-02-28T19:58:50.230Z
Answers.UserDisplayName: Alexander Wan
Answers.Body: <p>since these labels are derived from the input data itself the <strong>pre-training phase</strong> is considered <strong>self-supervised</strong> learning, <s>which is a subset of supervised learning</s>, as the name would suggest. <s>Calling this process unsupervised is an error</s>, but many people still call it unsupervised. Terminology in every field is commonly misused because people make mistakes and have different levels of understanding of the concepts. For example people often misuse <em>link function</em> and the <em>inverse link</em> across different scientific papers.</p>
  <p>so you are correct, self-supervised learning <s>is a form of supervised learning</s> because we have a target value; the one thing that defines supervised learning</p>
  <p>edit: ok <a href="https://www.ibm.com/topics/self-supervised-learning#:%7E:text=Though%20self%2Dsupervised%20learning%20is,performance%20against%20a%20ground%20truth." rel="nofollow noreferrer">according to IBM</a> self-supervised is a form of unsupervised learning. So maybe the answer isn't so clear. my thinking keeps coming back to autoregressive models, e.g. like ARIMA, which are considered <em>supervised</em>.</p>
  <p>image the following scenario. you have stock prices Y and you use previous Y data to predict the next time step. If you did this using ARIMA I think most people would define this as a supervised task. the target value represents a perfectly accurate ground truth (e.g. the stock market price is known for a fact without any doubt). This is the case despite the target not being defined using <em>explicit external labeling</em>.</p>
  <p>now if we use an LSTM or GPT, we call this self-supervised, and suddenly its an unsupervised model? the only thing we changed is the model type which shouldn't factor in to if a task is supervised/unsupervised.</p>
  <p>the below link clarifies this more and also ellaborates the confusion around this term.</p>
  <p><a href="https://www.ibm.com/topics/self-supervised-learning#:%7E:text=Though%20self%2Dsupervised%20learning%20is,performance%20against%20a%20ground%20truth." rel="nofollow noreferrer">source</a></p>
  <blockquote>
  <p>Though self-supervised learning is a technically a subset of
  unsupervised learning (as it doesn’t require labeled datasets), it’s
  closely related to supervised learning in that it optimizes
  performance against a ground truth.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-28T08:14:00.230Z
Answers.LastActivityDate: 2024-02-28T20:05:40.187Z
Answers.LastEditDate: 2024-02-28T20:05:40.187Z
Answers.UserDisplayName: Phillip Maire
Body: <p>I'm learning about GenAI, such as GPT (Generative Pretrained Transformer), and I'm particularly interested in understanding the training techniques used for these models.</p>
  <p>Deep learning, generally, can involve training with supervised learning with labeled datasets which makes sense.  But I also encounter references to unsupervised learning: specifically, predicting the next word in a sequence. The concept of unsupervised training with next-word prediction is unclear to me. To me, it feels like GPT is trained with supervised learning with a labeled dataset: is this correct?</p>
  
Tags: <machine-learning><natural-language-processing><unsupervised-learning><gpt><language-model>
Title: What technique is used for training Large Language Models like GPT?
UserDisplayName: Exploring
ThreadText: # What technique is used for training Large Language Models like GPT?
  
  (Asked by: username_0 on 2024-02-28)
  
  I'm learning about GenAI, such as GPT (Generative Pretrained Transformer), and I'm particularly interested in understanding the training techniques used for these models.
  
  Deep learning, generally, can involve training with supervised learning with labeled datasets which makes sense. But I also encounter references to unsupervised learning: specifically, predicting the next word in a sequence. The concept of unsupervised training with next-word prediction is unclear to me. To me, it feels like GPT is trained with supervised learning with a labeled dataset: is this correct?
  
  **Tags:** machine-learning, natural-language-processing, unsupervised-learning, gpt, language-model
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2024-02-28. Score: 3
  
  [These models are trained with unsupervised learning](https://openai.com/research/better-language-models), but it's probably more intuitive to describe it as self-supervised learning.
  
  Your confusion makes sense: the next-token prediction objective, which trains models to predict the next token given the previous token, is basically just doing supervised learning with the inputs being the prefix of the text and the labels being the next token. But, here, the labels are created by us, from the same corpus that the input is from! So the model is supervised by the data itself: "self-supervised" learning.
  
  Also, to be clear, self-supervised learning *is* unsupervised\*, you don't really have a separate set of labels---we're making them ourselves.
  
  \* I realize that these definitions can be controversial/ambiguous. I'm basing my usage on these: [1](https://www.cs.rice.edu/%7Eas143/COMP642_Spring22/Scribes/Mar-24), [2](https://proceedings.neurips.cc/paper/2020/file/70feb62b69f16e0238f741fab228fec2-Paper.pdf), [3](http://multicomp.cs.cmu.edu/research/self-supervised-learning/), [4](https://jmlr.csail.mit.edu/papers/volume23/21-1155/21-1155.pdf). You could alternatively say that self-supervised learning is a separate category altogether (e.g., [these slides](https://web.eecs.umich.edu/%7Ejustincj/slides/eecs498/WI2022/598_WI2022_lecture22.pdf)). But, all things considered, you probably don't need to worry too much about definitions as long as you understand the underlying concepts---people will know what you're talking about either way.
  
  
  ---
  
  ### Answer by username_2 on 2024-02-28. Score: 2
  
  since these labels are derived from the input data itself the **pre-training phase** is considered **self-supervised** learning, ~~which is a subset of supervised learning~~, as the name would suggest. ~~Calling this process unsupervised is an error~~, but many people still call it unsupervised. Terminology in every field is commonly misused because people make mistakes and have different levels of understanding of the concepts. For example people often misuse *link function* and the *inverse link* across different scientific papers.
  
  so you are correct, self-supervised learning ~~is a form of supervised learning~~ because we have a target value; the one thing that defines supervised learning
  
  edit: ok [according to IBM](https://www.ibm.com/topics/self-supervised-learning#:%7E:text=Though%20self%2Dsupervised%20learning%20is,performance%20against%20a%20ground%20truth.) self-supervised is a form of unsupervised learning. So maybe the answer isn't so clear. my thinking keeps coming back to autoregressive models, e.g. like ARIMA, which are considered *supervised*.
  
  image the following scenario. you have stock prices Y and you use previous Y data to predict the next time step. If you did this using ARIMA I think most people would define this as a supervised task. the target value represents a perfectly accurate ground truth (e.g. the stock market price is known for a fact without any doubt). This is the case despite the target not being defined using *explicit external labeling*.
  
  now if we use an LSTM or GPT, we call this self-supervised, and suddenly its an unsupervised model? the only thing we changed is the model type which shouldn't factor in to if a task is supervised/unsupervised.
  
  the below link clarifies this more and also ellaborates the confusion around this term.
  
  [source](https://www.ibm.com/topics/self-supervised-learning#:%7E:text=Though%20self%2Dsupervised%20learning%20is,performance%20against%20a%20ground%20truth.)
  
  > Though self-supervised learning is a technically a subset of
  > unsupervised learning (as it doesn’t require labeled datasets), it’s
  > closely related to supervised learning in that it optimizes
  > performance against a ground truth.
  
  
  ---
  
  