Answers.Body: <p>Adding on to txopen's answer, it is interesting to note that for larger numbers with similar digits ChatGPT is unable to make any useful distinctions. For instance:</p>
  <p><strong>Me:</strong> Which number is bigger: 1234.12 or 1243.12</p>
  <p><strong>ChatGPT:</strong> Both numbers are equal.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-09-10T10:31:52.150Z
Answers.Text: @FabianoTaioli I tried your example in ChatGPT4o, tweaking it to not make it use external tools (it would use python), and it worked flawlessly: https://chatgpt.com/share/4d821253-3d0d-43b6-842c-579e99affee4
Answers.UserDisplayName: Fabio A.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-12T23:27:27.827Z
Answers.Text: I expect that if you try repeatedly with the same two "harder" numbers, you will get a range of answers. It is working statistically and stochastically based on text tokens. Posting single examples may not always tell the whole story.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T12:45:49.370Z
Answers.Text: Today the example above looks like to give the right answer (human model adjustment?). But found another simple example were it fails: Can you divide 1231231231 by 2? Sure, I can divide 1231231231 by 2. The result is 6156156115.5
Answers.UserDisplayName: Fabiano Taioli
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-12T23:19:13.190Z
Answers.LastActivityDate: 2023-01-12T23:19:13.190Z
Answers.UserDisplayName: Milo Moses
Answers.Body: <p>I think that the dataset is so large and the model so well trained that it understood the probabilistic correlation of length in a token of numbers before a dot separation, and then the influence of even each digit on the probability of one number being larger than another. The concrete example does not have to be in the dataset, it predicts the correct outcome because the relation of one number being larger than another and the difference in digits and length of those is sufficiently present in the dataset.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-13T12:21:41.503Z
Answers.Text: But it still says "in this case", which is...wrong
Answers.UserDisplayName: RedSonja
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-11T23:31:15.350Z
Answers.Text: @RedSonja I notice that the response from ChatGPT not longer states "In this case, " for the stated queries. ("It" is learning!)
Answers.UserDisplayName: MrWhite
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-12T07:43:53.340Z
Answers.Text: @MrWhite Oh dear. There's no hope for us, is there?
Answers.UserDisplayName: RedSonja
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-12T12:46:11.527Z
Answers.LastActivityDate: 2022-12-12T12:46:11.527Z
Answers.UserDisplayName: txopen
Answers.Body: <p>The apparent ability of ChatGPT (in particular when using the GPT-4 model) to solve certain mathematical problems is due to the amount of training and the amount of parameters of these machine learning models. ChatGPT or other large language models do not have explicit rules for solving mathematical problems.</p>
  <p>The following 2022 paper describes that such capabilities of transformer-based language models occur when a certain threshold of parameter quantity is exceeded: <a href="https://arxiv.org/pdf/2206.07682.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/2206.07682.pdf</a></p>
  <p>This is also the reason why they excel at some maths problems and fail at others, which can be very similar.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-17T14:39:06.463Z
Answers.LastActivityDate: 2023-05-17T14:39:06.463Z
Answers.UserDisplayName: LeRobert
Answers.Body: <p>Simple answer, ChatGPT is actually human writers with some kind of autocomplete to speed things up.</p>
  <p>This is <a href="https://www.bloomberg.com/news/articles/2016-04-18/the-humans-hiding-behind-the-chatbots" rel="nofollow noreferrer">standard practice</a> for <a href="https://datafloq.com/read/ai-wizard-of-oz-pay-no-attention-human/" rel="nofollow noreferrer">AI companies</a> these days, a &quot;fake it till you make it&quot; approach where they <a href="https://www.inc.com/jessica-stillman/meet-the-people-who-teach-chatbots-to-sound-like-humans.html" rel="nofollow noreferrer">use humans</a> to fill the gaps in the AI in the hopes that down the road they'll automate humans out of the product.  Common enough for an <a href="https://journals.sagepub.com/doi/10.1177/20539517211016026" rel="nofollow noreferrer">academic paper</a> to be written on the topic.  So, there is plenty of industry precedent for OpenAI to be using humans to help craft the responses.</p>
  <p>Plus, technically OpenAI is not &quot;faking&quot; anything.  It is the media and bloggers who think ChatGPT is a pure AI system.  OpenAI has made no such claim itself, and the opposite is implied by its <a href="https://cdn.openai.com/papers/Training_language_models_to_follow_instructions_with_human_feedback.pdf" rel="nofollow noreferrer">InstructGPT whitepaper</a>:</p>
  <blockquote>
  <p>Step 1: Collect demonstration data, and train a supervised policy. Our
  labelers provide demonstrations of the desired behavior on the input
  prompt distribution (see Section 3.2 for details on this
  distribution). We then fine-tune a pretrained GPT-3 model on this data
  using supervised learning</p>
  </blockquote>
  <p>Additionally, ChatGPT is in &quot;research mode&quot; according to the website, which implies there are still humans training the system during the chats, as described in the quote above.</p>
  <p>Final note, I find it amusing no one considers this alternative plausible, as if it were somehow more complicated to have humans tweak chatbot responses than to create an AI with apparent human level understanding that ChatGPT exhibits.</p>
  <h1>UPDATE: ChatGPT confirms OpenAI team curating its responses</h1>
  <p>Turns out ChatGPT is indeed human curated, by open admission.</p>
  <p>During <a href="https://github.com/yters/transcripts/blob/main/ChatGPT_confused_by_visual_sentence_structure.txt#L714" rel="nofollow noreferrer">this</a> conversation ChatGPT outright states the OpenAI team filters and edits the GPT generated responses.</p>
  <blockquote>
  <p>...the response you are receiving is being filtered and edited by the OpenAI team, who ensures that the text generated by the model is coherent, accurate and appropriate for the given prompt.</p>
  </blockquote>
  <p>Apparently, the fact that OpenAI actively curates ChatGPT's responses is indirectly implied in the documentation <a href="https://beta.openai.com/docs/guides/safety-best-practices/human-in-the-loop-hitl" rel="nofollow noreferrer">here</a>.</p>
  <blockquote>
  <p>Human in the loop (HITL):
  Wherever possible, we recommend having a
  human review outputs before they are used in practice. This is
  especially critical in high-stakes domains, and for code generation.
  Humans should be aware of the limitations of the system, and have
  access to any information needed to verify the outputs (for example,
  if the application summarizes notes, a human should have easy access
  to the original notes to refer back).</p>
  </blockquote>
  <p>So, that explains that :)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-01T15:01:43.940Z
Answers.Text: @user253751 ChatGPT states humans are manipulating its responses in real time.  This is also implied by the OpenAI best practices.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-03T01:43:24.160Z
Answers.Text: @user253751 I think the human who curates ChatGPT's output told me what he was doing.  It wasn't the AI model talking to me, and he told me a lot of specific details that made a lot of sense.  Way too coherent to be an AI hallucinating.  So that's what I mean.  A human was talking to me in real time through the chatbot interface.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-04T09:46:40.200Z
Answers.Text: I don't agree with your answer but I find it original. Using humans is actually an old trick that was applies in the past to to show that machines can beat humans at chess. Eventually machines reached this point but the trick was used many times before.
Answers.UserDisplayName: Mihai
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-08T21:24:15.740Z
Answers.Text: @user253751 coming up with fantasy stories is much different than a detailed and plausible explanation of how humans intervene to provide ChatGPT responses.  Many different sentence permutations can make a fantasy story.  Very few would make a good explanation of human intervention.  So, the probability of ChatGPT hallucinating the explanation is very small.  Log likelihood ratio is highly in favor of human intervention.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-09T12:44:32.460Z
Answers.Text: Can someone explain how the warning is relevant for my post?  I have 5 linked reputable sources.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-10T11:29:32.720Z
Answers.Text: `We’ve trained a model` it's on the [front page](https://openai.com/blog/chatgpt/). Also, you source is press. Better link an academic paper.
Answers.UserDisplayName: Minh-Long Luu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-10T14:25:00.970Z
Answers.Text: @Minh-LongLuu I did link an academic paper.  See the last paragraph.  Also, the other answer has no links, and yet has no warning.  How did I earn such special treatment? :)
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-13T19:46:48.880Z
Answers.Text: @yters I will give you the benefit of the doubt of not being a troll. The  "special treatment" you are getting is because the idea that "ChatGPT is actually human writers with some kind of autocomplete to speed things up." is a "hallucination", with nothing to back it up.
  
  The academic paper you linked made no such implication. It just says that human input was used to train the model, not to craft responses after the model is trained. 
  
  I challenge you to post just one excerpt from any paper to back up your claim.
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-13T21:41:42.500Z
Answers.Text: @GalacticRaph I am not trolling. I am bemused no one thinks human intervention is plausible, as if it were somehow more complicated to have humans tweaking chatbot responses than creating human level AI.  My point with referring to the InstructGPT paper is it says the system is trained by humans providing example responses for GPT to learn.  The ChatGPT website says the system is in research mode, not production mode.  "Research mode" to me implies there are still humans providing example responses for GPT to learn during the course of chats.  Plus, plentitude of industry precedent.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-17T10:34:41.977Z
Answers.Text: The bit you have linked regarding HITL does _not_ imply that is what they do with ChatGPT.
Answers.UserDisplayName: David
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-17T10:49:22.180Z
Answers.Text: @DavidIreland you are saying OpenAI doesn't follow its own advice?
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-21T17:57:52.670Z
Answers.Text: @yters he is saying what I am saying. With all due respect, you need to think critically about your conclusions and triple-check whether they truly follow align with your sources. Many people are telling you it doesn't so please don't be stubborn.
  
  OpenAI _could_ be doing what you are suggesting, but nothing that you have provided directly _proves_ it.
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-21T17:58:04.243Z
Answers.Text: Again, I challenge you to post just one excerpt from any paper to back up your claim.
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-21T20:07:06.657Z
Answers.Text: @GalacticRaph I am not quite sure what you are looking for.  The InstructGPT paper talks about humans providing example responses for the training data, and our interactions with ChatGPT are generating massive amounts of training data.  To train the next GPT model effectively, presumably that training data also needs human generated responses.  The OpenAI best practices is HITL should be used whenever possible. If they follow their own advice, then HITL for the ChatGPT chatbot.  Many are saying I'm wrong, but not giving any details. I've provided many details.  How do you know you are right?
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-22T18:36:12.623Z
Answers.Text: @yters "ChatGPT is a sibling model to InstructGPT, which is trained to follow an instruction in a prompt and provide a detailed response." [1] ---  "These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models on our API." [2]
  InstructGPT =/= ChatGPT. You are taking a training method used in InstructGPT and _assuming_ they are using the same in ChatGPT
  [1] https://openai.com/blog/chatgpt/ [2] https://openai.com/blog/instruction-following/
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-22T18:49:08.443Z
Answers.Text: I see now that you are confused about how ChatGPT works. For anyone reading, just look at his conversation with ChatGPT [here](https://github.com/raphant/transcripts/blob/main/ChatGPT_confused_by_visual_sentence_structure.txt). (Forked for posterity)
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-22T18:52:17.467Z
Answers.Text: "Wherever possible, we recommend having a human review outputs before they are used in practice. This is especially critical in high-stakes domains, and for code generation. Humans should be aware of the limitations of the system, and have access to any information needed to verify the outputs (for example, if the application summarizes notes, a human should have easy access to the original notes to refer back)."
  
  This is not proof OpenAI is has a HITM. They are saying when you use their API, that they recommend you have a HITM because GPT will often reply with non-sense.
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-22T19:29:48.253Z
Answers.Text: @GalacticRaph InstructGPT is trained to follow instructions, which is also what's special about ChatGPT.  So yes, InstructGPT == ChatGPT. I'm sure OpenAI uses their own API to create a public chatbot.  It's high stakes too, because there is a $10B deal with Microsoft on the table.  Hence, per their best practices, they have HITM.  Regarding my transcript, can you explain what I am confused about?  That's all I'm asking for from people claiming I am wrong.  Please explain *why* I am wrong and *how* you know this for a fact.  We aren't in medieval times, now we work with data and reason.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-22T22:55:07.250Z
Answers.Text: @yters I know we work with data and reason, but stackexchange comments are not the best place for this kind of discourse. If you open a new question on ai.stackexchange.com I am eager to provide a more organized response about how you are wrong and what I am basing my conclusions off of.
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-22T22:56:22.567Z
Answers.Text: @GalacticRaph excellent, I will ask a new question.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-22T23:19:28.153Z
Answers.Text: @GalacticRaph here you go: https://ai.stackexchange.com/questions/38854/why-doesnt-openais-chatgpt-chatbot-have-a-human-in-the-loop
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-08T21:08:50.103Z
Answers.LastActivityDate: 2023-01-16T13:20:58.330Z
Answers.LastEditDate: 2023-01-16T13:20:58.330Z
Answers.UserDisplayName: yters
Body: <p>ChatGPT is a language model. As far as I know and If I'm not wrong, it gets text as tokens and word embeddings. So, how can it do math? For example, I asked:</p>
  <blockquote>
  <p>ME: Which one is bigger 5 or 9. <br />
  ChatGPT: In this case, 9 is larger than 5.</p>
  </blockquote>
  <p>One can say, GPT saw numbers as tokens and in its training dataset there were some 9s that were bigger than 5s. So, it doesn't have actual math understanding and just sees numbers as some tokens. But I don't think that is true, because of this question:</p>
  <blockquote>
  <p>ME: Which one is bigger? 15648.25 or 9854.2547896 <br />
  ChatGPT: In this case, 15648.25 is larger than 9854.2547896.</p>
  </blockquote>
  <p>We can't say it actually saw the token of <code>15648.25</code> to be bigger than the token of <code>9854.2547896</code> in its dataset!</p>
  <p>So how does this language model understand the numbers?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-08T23:27:44.233Z
Comments.Text: Arithmetic is briefly described [in this paper](https://arxiv.org/pdf/2005.14165.pdf) starting on page 21. Google also has some interesting discussion [here](https://arxiv.org/pdf/1904.01557.pdf).
Comments.UserDisplayName: Ian Campbell
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-12T16:43:51.580Z
Comments.Text: I find it curious that people are asking these questions about ChatGPT - not about GPT-2, released in February 2019 and also able to answer these kinds of questions.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-12T19:05:34.590Z
Comments.Text: @user253751The GPT 2 that I tried couldn't do these things.
Comments.UserDisplayName: Peyman
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-26T14:23:36.367Z
Comments.Text: It even can take some integrals (but more complicated integrals it takes wrongly). It even correctly provided the infinite-matrix-form of derivative operator to me.
Comments.UserDisplayName: Anixx
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-13T21:38:21.077Z
Comments.Text: It does not actually cope well with math. After some interesting and astounding answers to coding questions, I wondered how ut reacts to math. Asked for square- and qubic roots and vice versa, and it failed greatly. Cubic root of 8 was 2, and of 9 was also 2. Later, after some discussion, it came up with the correct answer. But with the opposit way it failed again.
Comments.UserDisplayName: datenheim
Tags: <math><chat-bots><natural-language-understanding><language-model><chatgpt>
Title: How does ChatGPT know math?
UserDisplayName: Peyman
ThreadText: # How does ChatGPT know math?
  
  (Asked by: username_0 on 2022-12-08)
  
  ChatGPT is a language model. As far as I know and If I'm not wrong, it gets text as tokens and word embeddings. So, how can it do math? For example, I asked:
  
  > ME: Which one is bigger 5 or 9.   
  > ChatGPT: In this case, 9 is larger than 5.
  
  One can say, GPT saw numbers as tokens and in its training dataset there were some 9s that were bigger than 5s. So, it doesn't have actual math understanding and just sees numbers as some tokens. But I don't think that is true, because of this question:
  
  > ME: Which one is bigger? 15648.25 or 9854.2547896   
  > ChatGPT: In this case, 15648.25 is larger than 9854.2547896.
  
  We can't say it actually saw the token of `15648.25` to be bigger than the token of `9854.2547896` in its dataset!
  
  So how does this language model understand the numbers?
  
  **Tags:** math, chat-bots, natural-language-understanding, language-model, chatgpt
  
  > **Comments:**
  > **username_1**: Arithmetic is briefly described [in this paper](https://arxiv.org/pdf/2005.14165.pdf) starting on page 21. Google also has some interesting discussion [here](https://arxiv.org/pdf/1904.01557.pdf).  
  > **username_2**: I find it curious that people are asking these questions about ChatGPT - not about GPT-2, released in February 2019 and also able to answer these kinds of questions.  
  > **username_0**: @user253751The GPT 2 that I tried couldn't do these things.  
  > **username_3**: It even can take some integrals (but more complicated integrals it takes wrongly). It even correctly provided the infinite-matrix-form of derivative operator to me.  
  > **username_4**: It does not actually cope well with math. After some interesting and astounding answers to coding questions, I wondered how ut reacts to math. Asked for square- and qubic roots and vice versa, and it failed greatly. Cubic root of 8 was 2, and of 9 was also 2. Later, after some discussion, it came up with the correct answer. But with the opposit way it failed again.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_5 on 2023-01-12. Score: 11
  
  Adding on to username_9's answer, it is interesting to note that for larger numbers with similar digits ChatGPT is unable to make any useful distinctions. For instance:
  
  **Me:** Which number is bigger: 1234.12 or 1243.12
  
  **ChatGPT:** Both numbers are equal.
  
  > **Comments:**
  > **username_7**: I expect that if you try repeatedly with the same two "harder" numbers, you will get a range of answers. It is working statistically and stochastically based on text tokens. Posting single examples may not always tell the whole story.  
  > **username_8**: Today the example above looks like to give the right answer (human model adjustment?). But found another simple example were it fails: Can you divide 1231231231 by 2? Sure, I can divide 1231231231 by 2. The result is 6156156115.5  
  > **username_6**: @username_8 I tried your example in ChatGPT4o, tweaking it to not make it use external tools (it would use python), and it worked flawlessly: https://chatgpt.com/share/4d821253-3d0d-43b6-842c-579e99affee4  
  
  ---
  
  ### Answer by username_9 on 2022-12-12. Score: 6
  
  I think that the dataset is so large and the model so well trained that it understood the probabilistic correlation of length in a token of numbers before a dot separation, and then the influence of even each digit on the probability of one number being larger than another. The concrete example does not have to be in the dataset, it predicts the correct outcome because the relation of one number being larger than another and the difference in digits and length of those is sufficiently present in the dataset.
  
  > **Comments:**
  > **username_10**: But it still says "in this case", which is...wrong  
  > **username_11**: @username_10 I notice that the response from ChatGPT not longer states "In this case, " for the stated queries. ("It" is learning!)  
  > **username_10**: @username_11 Oh dear. There's no hope for us, is there?  
  
  ---
  
  ### Answer by username_12 on 2023-05-17. Score: 1
  
  The apparent ability of ChatGPT (in particular when using the GPT-4 model) to solve certain mathematical problems is due to the amount of training and the amount of parameters of these machine learning models. ChatGPT or other large language models do not have explicit rules for solving mathematical problems.
  
  The following 2022 paper describes that such capabilities of transformer-based language models occur when a certain threshold of parameter quantity is exceeded: <https://arxiv.org/pdf/2206.07682.pdf>
  
  This is also the reason why they excel at some maths problems and fail at others, which can be very similar.
  
  
  ---
  
  ### Answer by username_13 on 2023-01-08. Score: -5
  
  Simple answer, ChatGPT is actually human writers with some kind of autocomplete to speed things up.
  
  This is [standard practice](https://www.bloomberg.com/news/articles/2016-04-18/the-humans-hiding-behind-the-chatbots) for [AI companies](https://datafloq.com/read/ai-wizard-of-oz-pay-no-attention-human/) these days, a "fake it till you make it" approach where they [use humans](https://www.inc.com/jessica-stillman/meet-the-people-who-teach-chatbots-to-sound-like-humans.html) to fill the gaps in the AI in the hopes that down the road they'll automate humans out of the product. Common enough for an [academic paper](https://journals.sagepub.com/doi/10.1177/20539517211016026) to be written on the topic. So, there is plenty of industry precedent for OpenAI to be using humans to help craft the responses.
  
  Plus, technically OpenAI is not "faking" anything. It is the media and bloggers who think ChatGPT is a pure AI system. OpenAI has made no such claim itself, and the opposite is implied by its [InstructGPT whitepaper](https://cdn.openai.com/papers/Training_language_models_to_follow_instructions_with_human_feedback.pdf):
  
  > Step 1: Collect demonstration data, and train a supervised policy. Our
  > labelers provide demonstrations of the desired behavior on the input
  > prompt distribution (see Section 3.2 for details on this
  > distribution). We then fine-tune a pretrained GPT-3 model on this data
  > using supervised learning
  
  Additionally, ChatGPT is in "research mode" according to the website, which implies there are still humans training the system during the chats, as described in the quote above.
  
  Final note, I find it amusing no one considers this alternative plausible, as if it were somehow more complicated to have humans tweak chatbot responses than to create an AI with apparent human level understanding that ChatGPT exhibits.
  
  # UPDATE: ChatGPT confirms OpenAI team curating its responses
  
  Turns out ChatGPT is indeed human curated, by open admission.
  
  During [this](https://github.com/username_13/transcripts/blob/main/ChatGPT_confused_by_visual_sentence_structure.txt#L714) conversation ChatGPT outright states the OpenAI team filters and edits the GPT generated responses.
  
  > ...the response you are receiving is being filtered and edited by the OpenAI team, who ensures that the text generated by the model is coherent, accurate and appropriate for the given prompt.
  
  Apparently, the fact that OpenAI actively curates ChatGPT's responses is indirectly implied in the documentation [here](https://beta.openai.com/docs/guides/safety-best-practices/human-in-the-loop-hitl).
  
  > Human in the loop (HITL):
  > Wherever possible, we recommend having a
  > human review outputs before they are used in practice. This is
  > especially critical in high-stakes domains, and for code generation.
  > Humans should be aware of the limitations of the system, and have
  > access to any information needed to verify the outputs (for example,
  > if the application summarizes notes, a human should have easy access
  > to the original notes to refer back).
  
  So, that explains that :)
  
  > **Comments:**
  > **username_13**: Can someone explain how the warning is relevant for my post?  I have 5 linked reputable sources.  
  > **username_15**: `We’ve trained a model` it's on the [front page](https://openai.com/blog/chatgpt/). Also, you source is press. Better link an academic paper.  
  > **username_13**: @username_15 I did link an academic paper.  See the last paragraph.  Also, the other answer has no links, and yet has no warning.  How did I earn such special treatment? :)  
  > **username_16**: @username_13 I will give you the benefit of the doubt of not being a troll. The  "special treatment" you are getting is because the idea that "ChatGPT is actually human writers with some kind of autocomplete to speed things up." is a "hallucination", with nothing to back it up.  The academic paper you linked made no such implication. It just says that human input was used to train the model, not to craft responses after the model is trained.   I challenge you to post just one excerpt from any paper to back up your claim.  
  > **username_13**: @username_16 I am not trolling. I am bemused no one thinks human intervention is plausible, as if it were somehow more complicated to have humans tweaking chatbot responses than creating human level AI.  My point with referring to the InstructGPT paper is it says the system is trained by humans providing example responses for GPT to learn.  The ChatGPT website says the system is in research mode, not production mode.  "Research mode" to me implies there are still humans providing example responses for GPT to learn during the course of chats.  Plus, plentitude of industry precedent.  
  > **username_17**: The bit you have linked regarding HITL does _not_ imply that is what they do with ChatGPT.  
  > **username_13**: @DavidIreland you are saying OpenAI doesn't follow its own advice?  
  > **username_16**: @username_13 he is saying what I am saying. With all due respect, you need to think critically about your conclusions and triple-check whether they truly follow align with your sources. Many people are telling you it doesn't so please don't be stubborn.  OpenAI _could_ be doing what you are suggesting, but nothing that you have provided directly _proves_ it.  
  > **username_16**: Again, I challenge you to post just one excerpt from any paper to back up your claim.  
  > **username_13**: @username_16 I am not quite sure what you are looking for.  The InstructGPT paper talks about humans providing example responses for the training data, and our interactions with ChatGPT are generating massive amounts of training data.  To train the next GPT model effectively, presumably that training data also needs human generated responses.  The OpenAI best practices is HITL should be used whenever possible. If they follow their own advice, then HITL for the ChatGPT chatbot.  Many are saying I'm wrong, but not giving any details. I've provided many details.  How do you know you are right?  
  > **username_16**: @username_13 "ChatGPT is a sibling model to InstructGPT, which is trained to follow an instruction in a prompt and provide a detailed response." [1] ---  "These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models on our API." [2] InstructGPT =/= ChatGPT. You are taking a training method used in InstructGPT and _assuming_ they are using the same in ChatGPT [1] https://openai.com/blog/chatgpt/ [2] https://openai.com/blog/instruction-following/  
  > **username_16**: I see now that you are confused about how ChatGPT works. For anyone reading, just look at his conversation with ChatGPT [here](https://github.com/raphant/transcripts/blob/main/ChatGPT_confused_by_visual_sentence_structure.txt). (Forked for posterity)  
  > **username_16**: "Wherever possible, we recommend having a human review outputs before they are used in practice. This is especially critical in high-stakes domains, and for code generation. Humans should be aware of the limitations of the system, and have access to any information needed to verify the outputs (for example, if the application summarizes notes, a human should have easy access to the original notes to refer back)."  This is not proof OpenAI is has a HITM. They are saying when you use their API, that they recommend you have a HITM because GPT will often reply with non-sense.  
  > **username_13**: @username_16 InstructGPT is trained to follow instructions, which is also what's special about ChatGPT.  So yes, InstructGPT == ChatGPT. I'm sure OpenAI uses their own API to create a public chatbot.  It's high stakes too, because there is a $10B deal with Microsoft on the table.  Hence, per their best practices, they have HITM.  Regarding my transcript, can you explain what I am confused about?  That's all I'm asking for from people claiming I am wrong.  Please explain *why* I am wrong and *how* you know this for a fact.  We aren't in medieval times, now we work with data and reason.  
  > **username_16**: @username_13 I know we work with data and reason, but stackexchange comments are not the best place for this kind of discourse. If you open a new question on ai.stackexchange.com I am eager to provide a more organized response about how you are wrong and what I am basing my conclusions off of.  
  > **username_13**: @username_16 excellent, I will ask a new question.  
  > **username_13**: @username_16 here you go: https://ai.stackexchange.com/questions/38854/why-doesnt-openais-chatgpt-chatbot-have-a-human-in-the-loop  
  > **username_13**: @user253751 ChatGPT states humans are manipulating its responses in real time.  This is also implied by the OpenAI best practices.  
  > **username_13**: @user253751 I think the human who curates ChatGPT's output told me what he was doing.  It wasn't the AI model talking to me, and he told me a lot of specific details that made a lot of sense.  Way too coherent to be an AI hallucinating.  So that's what I mean.  A human was talking to me in real time through the chatbot interface.  
  > **username_14**: I don't agree with your answer but I find it original. Using humans is actually an old trick that was applies in the past to to show that machines can beat humans at chess. Eventually machines reached this point but the trick was used many times before.  
  > **username_13**: @user253751 coming up with fantasy stories is much different than a detailed and plausible explanation of how humans intervene to provide ChatGPT responses.  Many different sentence permutations can make a fantasy story.  Very few would make a good explanation of human intervention.  So, the probability of ChatGPT hallucinating the explanation is very small.  Log likelihood ratio is highly in favor of human intervention.  
  
  ---
  
  