Answers.Body: <p>ChatGPT derives also from <a href="https://openai.com/research/instruction-following" rel="nofollow noreferrer">InstructGPT</a> that is subject to a reinforcement learning training with human feedback (called <a href="https://openai.com/research/learning-from-human-preferences" rel="nofollow noreferrer">RLHF</a>) in order to enble it for instruction following.</p>
  <p>Basically, to get a chatGPT you take a GPT model, pre-train it to predict the next token in a sentence (so that it learns the structure of the language), and then RLHF is applied to enable it to follow instructions specified in a prompt. Otherwise it will give you just random (but plausible) complexions of sentences.</p>
  <p>The power of chatGPT is the combination of a very large model (billions of parameters), a huge dataset, and a large collection of human annotated prompts and results for the instruction follwing thing.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-09T20:08:08.873Z
Answers.LastActivityDate: 2023-05-09T20:08:08.873Z
Answers.UserDisplayName: Luca Anzalone
Body: <p>Reading Stephen Wolfram's <a href="https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/" rel="nofollow noreferrer">explanation</a> of ChatGPT, it sounds as if first you train a very powerful &quot;autocomplete&quot; function that doesn't know anything specifically about chatbots, and then you create a corpus of chatbot-dialogs on top of that model to show it how to be a chatbot specifically. I'm curious if anyone can explain in a bit more detail how this process works technically, ie taking an already-trained model and &quot;specializing&quot; it with a second training corpus. How are the two neural networks related to one another?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-05-09T20:09:55.527Z
Comments.Text: ChatGPT is a decoder-only transformer model. For a full on (hopefully not too technical explanation), please [read this extensive post](https://ai.stackexchange.com/questions/40179/how-does-the-decoder-only-transformer-architecture-work). It also goes into how the transformer is trained (in two stages).
Comments.UserDisplayName: Robin van Hoorn
Tags: <neural-networks>
Title: How are the parts of GPT connected?
UserDisplayName: ak0000
ThreadText: # How are the parts of GPT connected?
  
  (Asked by: username_0 on 2023-05-08)
  
  Reading Stephen Wolfram's [explanation](https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/) of ChatGPT, it sounds as if first you train a very powerful "autocomplete" function that doesn't know anything specifically about chatbots, and then you create a corpus of chatbot-dialogs on top of that model to show it how to be a chatbot specifically. I'm curious if anyone can explain in a bit more detail how this process works technically, ie taking an already-trained model and "specializing" it with a second training corpus. How are the two neural networks related to one another?
  
  **Tags:** neural-networks
  > **Comments:**
  > **username_1**: ChatGPT is a decoder-only transformer model. For a full on (hopefully not too technical explanation), please [read this extensive post](https://ai.stackexchange.com/questions/40179/how-does-the-decoder-only-transformer-architecture-work). It also goes into how the transformer is trained (in two stages).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-05-09. Score: 1
  
  ChatGPT derives also from [InstructGPT](https://openai.com/research/instruction-following) that is subject to a reinforcement learning training with human feedback (called [RLHF](https://openai.com/research/learning-from-human-preferences)) in order to enble it for instruction following.
  
  Basically, to get a chatGPT you take a GPT model, pre-train it to predict the next token in a sentence (so that it learns the structure of the language), and then RLHF is applied to enable it to follow instructions specified in a prompt. Otherwise it will give you just random (but plausible) complexions of sentences.
  
  The power of chatGPT is the combination of a very large model (billions of parameters), a huge dataset, and a large collection of human annotated prompts and results for the instruction follwing thing.
  
  
  ---
  
  