Answers.Body: <p>Step 2 is (mainly) useful to tune the style, alignement and general feeling of the ChatGPT answers, not the specifics of what is answered.</p>
  <p>The actual &quot;knowledge&quot; comes simply from ingesting a large corpora of text as training dataset and using unsupervised learning technques on it. This knowledge has been encapsulated in the GPT3.5 on which ChatGPT is built on. To be super clear: ChatGPT is a specialised version of GPT3.5.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-13T16:31:45.130Z
Answers.LastActivityDate: 2023-03-13T16:31:45.130Z
Answers.UserDisplayName: Rexcirus
Body: <p>According to OpenAI, ChatGPT is trained in a 3-step process.
  <a href="https://i.sstatic.net/RBf8H.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/RBf8H.png" alt="ChatGPT_diagram" /></a>
  Are the steps where human AI trainers are involved, i.e. training the initial policy and providing the A&gt;B&gt;C&gt;D grading as training sets for the reward model, the ONLY place where actual knowledge is entered? Are there no other learning steps where human trainers are NOT involved and the model learns from high-quality sources like authoritative texts? The sampled prompts in step 2 are supposed to cover EVERYTHING on the internet, so what happens when the trainers do not know anything about the topic? Did OpenAI invite domain experts to grade specific prompts-answer pairs?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-13T11:15:46.000Z
Comments.Text: This is impossible to answer as there is no more public information on how ChatGPT was trained.
Comments.UserDisplayName: Dr. Snoopy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-13T15:51:59.883Z
Comments.Text: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.
Comments.UserDisplayName: Community
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-13T22:08:12.813Z
Comments.Text: I think their prompt dataset contains half the Internet.
Comments.UserDisplayName: Criticize SE actions means ban
Tags: <machine-learning><natural-language-processing><training><chatgpt>
Title: How is ChatGPT trained?
UserDisplayName: Meatball Princess
ThreadText: # How is ChatGPT trained?
  
  (Asked by: username_0 on 2023-03-13)
  
  According to OpenAI, ChatGPT is trained in a 3-step process.
  [![ChatGPT_diagram](https://i.sstatic.net/RBf8H.png)](https://i.sstatic.net/RBf8H.png)
  Are the steps where human AI trainers are involved, i.e. training the initial policy and providing the A>B>C>D grading as training sets for the reward model, the ONLY place where actual knowledge is entered? Are there no other learning steps where human trainers are NOT involved and the model learns from high-quality sources like authoritative texts? The sampled prompts in step 2 are supposed to cover EVERYTHING on the internet, so what happens when the trainers do not know anything about the topic? Did OpenAI invite domain experts to grade specific prompts-answer pairs?
  
  **Tags:** machine-learning, natural-language-processing, training, chatgpt
  
  > **Comments:**
  > **username_1**: This is impossible to answer as there is no more public information on how ChatGPT was trained.  
  > **username_2**: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.  
  > **username_3**: I think their prompt dataset contains half the Internet.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_4 on 2023-03-13. Score: 1
  
  Step 2 is (mainly) useful to tune the style, alignement and general feeling of the ChatGPT answers, not the specifics of what is answered.
  
  The actual "knowledge" comes simply from ingesting a large corpora of text as training dataset and using unsupervised learning technques on it. This knowledge has been encapsulated in the GPT3.5 on which ChatGPT is built on. To be super clear: ChatGPT is a specialised version of GPT3.5.
  
  
  ---
  
  