Answers.Body: <p>ChatGPT has not been trained from scratch. ChatGPT is a fine-tuned version of a model from the GPT-3.5 series. <a href="https://openai.com/blog/chatgpt/" rel="noreferrer">OpenAI writes</a></p>
  <blockquote>
  <p>ChatGPT is <strong>fine-tuned from a model in the GPT-3.5 series</strong>, which finished training in early 2022. You can learn more about the 3.5 series <a href="https://platform.openai.com/docs/model-index-for-researchers" rel="noreferrer">here</a>.</p>
  </blockquote>
  <p>Which models are in the GPT-3.5 series? You can read more about that in the <a href="https://platform.openai.com/docs/model-index-for-researchers" rel="noreferrer">linked blog post</a>.</p>
  <blockquote>
  <p>GPT-3.5 series is a series of models that was trained on a blend of text and code from before Q4 2021. The following models are in the GPT-3.5 series:</p>
  <ul>
  <li><code>code-davinci-002</code> is a base model, so good for pure code-completion tasks</li>
  <li><code>text-davinci-002</code> is an InstructGPT model based on <code>code-davinci-002</code></li>
  <li><code>text-davinci-003</code> is an improvement on <code>text-davinci-002</code></li>
  </ul>
  </blockquote>
  <p>So, ChatGPT must be a fine-tuned version of one of these 3 models, assuming the information in their site is accurate and up-to-date.</p>
  <p>Now, according to <a href="https://platform.openai.com/docs/models/codex" rel="noreferrer">this blog post</a></p>
  <blockquote>
  <p>The <strong>Codex models are descendants of our GPT-3 models</strong> that can understand and generate code. Their training data contains both natural language and billions of lines of public code from GitHub</p>
  </blockquote>
  <p><code>code-davinci-002</code> is a codex model. So, ChatGPT <em>might</em> be a descendent of GPT-3. I don't know what &quot;descendent&quot; exactly means here. Does it mean just fine-tuned or maybe a modified version?</p>
  <p><a href="https://platform.openai.com/docs/models/gpt-3" rel="noreferrer">Here</a> they write that <code>text-davinci-003</code> is the most capable GPT-3 model. Based on the information above, <code>text-davinci-002</code> is an InstructGPT model based on <code>code-davinci-002</code>.</p>
  <p><a href="https://openai.com/blog/instruction-following/" rel="noreferrer">Here</a> they write</p>
  <blockquote>
  <p>We then use this data to <strong>fine-tune GPT-3</strong>.</p>
  <p>The resulting InstructGPT models are much better at following instructions than GPT-3</p>
  </blockquote>
  <p>So, InstructGPT models are fine-tuned GPT-3 models. That most likely implies that <code>text-davinci-002</code> is a GPT-3 model and the only thing that changes is how it was trained. However, they also write</p>
  <blockquote>
  <p>Our labelers prefer outputs from our 1.3B InstructGPT model over outputs from a 175B GPT-3 model, despite having more than 100x fewer parameters.</p>
  </blockquote>
  <p>So, there isn't just one GPT3 model. However, <a href="https://arxiv.org/pdf/2005.14165.pdf" rel="noreferrer">the original GPT3 model</a> had 175 billion parameters.</p>
  <p>I still need to read the InstructGPT and GPT3 papers. Once I've done that, I may have more useful/concrete info, then I will update this answer.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-04T17:52:20.557Z
Answers.Text: Thanks for you answer. I went through more or less the same links (that I've included in my question) and I also checked the InstructGPT paper. The different statements from the blog posts are really confusing especially when it comes to different number of parameters... Maybe they use the last layers from GPT-3 to fine tune the new models with fewer parameters.. but in this case they need to run the input through GPT-3 everytime and I'm not sure this is a practical way to do... Anyway I'm waiting for your update. ^^
Answers.UserDisplayName: iMad
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-04T19:35:08.227Z
Answers.Text: I don't know when I will start reading them. I'll start a project at work that involves these OpenAI models, so I will most likely read these papers (at least to some extent), but I don't know when yet.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-04T17:16:30.463Z
Answers.LastActivityDate: 2023-02-04T17:16:30.463Z
Answers.UserDisplayName: nbro
Answers.Body: <p>I'll complement <a href="https://ai.stackexchange.com/users/2444/nbro">nbro</a>'s  answer with this great visual summary by <a href="https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1" rel="noreferrer">Yao Fu &lt;yao.fu@ed.ac.uk&gt;</a>:</p>
  <p><a href="https://i.sstatic.net/Yintu.png" rel="noreferrer"><img src="https://i.sstatic.net/Yintu.png" alt="enter image description here" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-13T08:19:26.033Z
Answers.Text: This is exaclty what I was looking for. Thanks !
Answers.UserDisplayName: iMad
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-11T03:12:12.400Z
Answers.LastActivityDate: 2023-02-11T03:12:12.400Z
Answers.UserDisplayName: Franck Dernoncourt
Body: <p>In the official <a href="https://openai.com/blog/chatgpt/" rel="noreferrer">blog post about ChatGPT</a> from OpenAI, there is this paragraph explaining how ChatGPT model was trained:</p>
  <blockquote>
  <p>We trained this model using Reinforcement Learning from Human Feedback
  (RLHF), <strong>using the same methods as InstructGPT</strong>, but with slight
  differences in the data collection setup. We trained an initial model
  using supervised fine-tuning: human AI trainers provided conversations
  in which they played both sides—the user and an AI assistant. We gave
  the trainers access to model-written suggestions to help them compose
  their responses. We mixed this new dialogue dataset with the
  InstructGPT dataset, which we transformed into a dialogue format.</p>
  </blockquote>
  <p>Especially this part:</p>
  <blockquote>
  <p>We trained an initial model using supervised fine-tuning</p>
  </blockquote>
  <p>My question is about the said <em>initial model</em>, is it some new model that has been trained from scratch or is it a GPT-3 model that has been fine tuned for specific tasks resulting in <a href="https://platform.openai.com/docs/model-index-for-researchers" rel="noreferrer">GPT-3.5 series</a> ?</p>
  <p>On the other hand, from the <a href="https://openai.com/blog/instruction-following/" rel="noreferrer">InstructGPT blog post</a>, it is clearly stated that:</p>
  <blockquote>
  <p>To make our models safer, more helpful, and more aligned, we use an
  existing technique called reinforcement learning from human feedback
  (RLHF). On prompts submitted by our customers to the API,our labelers
  provide demonstrations of the desired model behavior, and rank several
  outputs from our models. <strong>We then use this data to fine-tune GPT-3</strong>.</p>
  </blockquote>
  <p>So does this mean that GPT-3.5 series models (and consequently ChatGPT) are fine-tuned from GPT-3 base model ?</p>
  
Tags: <open-ai><fine-tuning><chatgpt><gpt-3>
Title: Are GPT-3.5 series models based on GPT-3?
UserDisplayName: iMad
ThreadText: # Are GPT-3.5 series models based on GPT-3?
  
  (Asked by: username_0 on 2023-02-02)
  
  In the official [blog post about ChatGPT](https://openai.com/blog/chatgpt/) from OpenAI, there is this paragraph explaining how ChatGPT model was trained:
  
  > We trained this model using Reinforcement Learning from Human Feedback
  > (RLHF), **using the same methods as InstructGPT**, but with slight
  > differences in the data collection setup. We trained an initial model
  > using supervised fine-tuning: human AI trainers provided conversations
  > in which they played both sides—the user and an AI assistant. We gave
  > the trainers access to model-written suggestions to help them compose
  > their responses. We mixed this new dialogue dataset with the
  > InstructGPT dataset, which we transformed into a dialogue format.
  
  Especially this part:
  
  > We trained an initial model using supervised fine-tuning
  
  My question is about the said *initial model*, is it some new model that has been trained from scratch or is it a GPT-3 model that has been fine tuned for specific tasks resulting in [GPT-3.5 series](https://platform.openai.com/docs/model-index-for-researchers) ?
  
  On the other hand, from the [InstructGPT blog post](https://openai.com/blog/instruction-following/), it is clearly stated that:
  
  > To make our models safer, more helpful, and more aligned, we use an
  > existing technique called reinforcement learning from human feedback
  > (RLHF). On prompts submitted by our customers to the API,our labelers
  > provide demonstrations of the desired model behavior, and rank several
  > outputs from our models. **We then use this data to fine-tune GPT-3**.
  
  So does this mean that GPT-3.5 series models (and consequently ChatGPT) are fine-tuned from GPT-3 base model ?
  
  **Tags:** open-ai, fine-tuning, chatgpt, gpt-3
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2023-02-11. Score: 13 | Accepted answer
  
  I'll complement [nbro](https://ai.stackexchange.com/users/2444/nbro)'s answer with this great visual summary by [Yao Fu](https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1) :
  
  [![enter image description here](https://i.sstatic.net/Yintu.png)](https://i.sstatic.net/Yintu.png)
  
  > **Comments:**
  > **username_0**: This is exaclty what I was looking for. Thanks !  
  
  ---
  
  ### Answer by username_1 on 2023-02-04. Score: 7
  
  ChatGPT has not been trained from scratch. ChatGPT is a fine-tuned version of a model from the GPT-3.5 series. [OpenAI writes](https://openai.com/blog/chatgpt/)
  
  > ChatGPT is **fine-tuned from a model in the GPT-3.5 series**, which finished training in early 2022. You can learn more about the 3.5 series [here](https://platform.openai.com/docs/model-index-for-researchers).
  
  Which models are in the GPT-3.5 series? You can read more about that in the [linked blog post](https://platform.openai.com/docs/model-index-for-researchers).
  
  > GPT-3.5 series is a series of models that was trained on a blend of text and code from before Q4 2021. The following models are in the GPT-3.5 series:
  >
  > * `code-davinci-002` is a base model, so good for pure code-completion tasks
  > * `text-davinci-002` is an InstructGPT model based on `code-davinci-002`
  > * `text-davinci-003` is an improvement on `text-davinci-002`
  
  So, ChatGPT must be a fine-tuned version of one of these 3 models, assuming the information in their site is accurate and up-to-date.
  
  Now, according to [this blog post](https://platform.openai.com/docs/models/codex)
  
  > The **Codex models are descendants of our GPT-3 models** that can understand and generate code. Their training data contains both natural language and billions of lines of public code from GitHub
  
  `code-davinci-002` is a codex model. So, ChatGPT *might* be a descendent of GPT-3. I don't know what "descendent" exactly means here. Does it mean just fine-tuned or maybe a modified version?
  
  [Here](https://platform.openai.com/docs/models/gpt-3) they write that `text-davinci-003` is the most capable GPT-3 model. Based on the information above, `text-davinci-002` is an InstructGPT model based on `code-davinci-002`.
  
  [Here](https://openai.com/blog/instruction-following/) they write
  
  > We then use this data to **fine-tune GPT-3**.
  >
  > The resulting InstructGPT models are much better at following instructions than GPT-3
  
  So, InstructGPT models are fine-tuned GPT-3 models. That most likely implies that `text-davinci-002` is a GPT-3 model and the only thing that changes is how it was trained. However, they also write
  
  > Our labelers prefer outputs from our 1.3B InstructGPT model over outputs from a 175B GPT-3 model, despite having more than 100x fewer parameters.
  
  So, there isn't just one GPT3 model. However, [the original GPT3 model](https://arxiv.org/pdf/2005.14165.pdf) had 175 billion parameters.
  
  I still need to read the InstructGPT and GPT3 papers. Once I've done that, I may have more useful/concrete info, then I will update this answer.
  
  > **Comments:**
  > **username_0**: Thanks for you answer. I went through more or less the same links (that I've included in my question) and I also checked the InstructGPT paper. The different statements from the blog posts are really confusing especially when it comes to different number of parameters... Maybe they use the last layers from GPT-3 to fine tune the new models with fewer parameters.. but in this case they need to run the input through GPT-3 everytime and I'm not sure this is a practical way to do... Anyway I'm waiting for your update. ^^  
  > **username_1**: I don't know when I will start reading them. I'll start a project at work that involves these OpenAI models, so I will most likely read these papers (at least to some extent), but I don't know when yet.  
  
  ---
  
  