Answers.Body: <p>The key ingredient is called Reinforcement Learning from Human Feedback (RLHF), that is having humans rate the model answers and use the feedback to guide the model training.</p>
  <p><a href="https://i.sstatic.net/LhJur.png" rel="noreferrer"><img src="https://i.sstatic.net/LhJur.png" alt="enter image description here" /></a></p>
  <p>The <a href="https://openai.com/blog/chatgpt/" rel="noreferrer">official blog</a> explains this fairly well.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-02T19:56:45.610Z
Answers.LastActivityDate: 2023-01-02T19:56:45.610Z
Answers.UserDisplayName: Rexcirus
Body: <p>I know that large language models like GPT-3 are trained simply to continue pieces of text that have been scraped from the web. But how was ChatGPT trained, which, while also having a good understanding of language, is not directly a language model, but a chatbot? Do we know anything about that? I presume that a lot of conversations was needed in order to train it. Did they simply scrape those conversations from the web, and where did they find such conversations in that case?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-29T10:05:59.517Z
Comments.Text: Can you please focus on one model at a time (either LaMDA or ChatGPT), and ask about the other model in a different post? The answers may change from one model to the other, so it's a good idea to focus, otherwise, you may get different answers, for different ways of training the chatbots, which may be what you're looking for, though.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-02T01:30:05.193Z
Comments.Text: @nbro While I would really be interested in knowing about all promising approaches, I chose to focus on ChatGPT since that's the only chatbot that I've personally tried (in recent times at least).
Comments.UserDisplayName: HelloGoodbye
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-02T03:56:10.093Z
Comments.Text: This article describes it well: ["ChatGPT: How Does It Work Internally?"](https://pub.towardsai.net/chatgpt-how-does-it-work-internally-e0b3e23601a1)
Comments.UserDisplayName: Snehal Patel
Tags: <natural-language-processing><chat-bots><training-datasets><language-model><chatgpt>
Title: How was ChatGPT trained?
UserDisplayName: HelloGoodbye
ThreadText: # How was ChatGPT trained?
  
  (Asked by: username_0 on 2022-12-29)
  
  I know that large language models like GPT-3 are trained simply to continue pieces of text that have been scraped from the web. But how was ChatGPT trained, which, while also having a good understanding of language, is not directly a language model, but a chatbot? Do we know anything about that? I presume that a lot of conversations was needed in order to train it. Did they simply scrape those conversations from the web, and where did they find such conversations in that case?
  
  **Tags:** natural-language-processing, chat-bots, training-datasets, language-model, chatgpt
  
  > **Comments:**
  > **username_1**: Can you please focus on one model at a time (either LaMDA or ChatGPT), and ask about the other model in a different post? The answers may change from one model to the other, so it's a good idea to focus, otherwise, you may get different answers, for different ways of training the chatbots, which may be what you're looking for, though.  
  > **username_0**: @username_1 While I would really be interested in knowing about all promising approaches, I chose to focus on ChatGPT since that's the only chatbot that I've personally tried (in recent times at least).  
  > **username_2**: This article describes it well: ["ChatGPT: How Does It Work Internally?"](https://pub.towardsai.net/chatgpt-how-does-it-work-internally-e0b3e23601a1)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2023-01-02. Score: 6 | Accepted answer
  
  The key ingredient is called Reinforcement Learning from Human Feedback (RLHF), that is having humans rate the model answers and use the feedback to guide the model training.
  
  [![enter image description here](https://i.sstatic.net/LhJur.png)](https://i.sstatic.net/LhJur.png)
  
  The [official blog](https://openai.com/blog/chatgpt/) explains this fairly well.
  
  
  ---
  
  