Answers.Body: <p>Until you can prove that OpenAI has HITL in ChatGPT, it is just an idea with no basis. It's not up to us to disprove it, it's up to you to prove it.</p>
  <p>Let me address your points:</p>
  <ol>
  <li>You seem to be basing your reasoning on an assumption of what neural networks can or can not do. What are those capabilities, and what is your source? Then we can talk about what ChatGPT seems to violate.</li>
  <li>OpenAI states that HITL is the best practice for its end users like me and you (specifically for API users), you have provided no evidence that OpenAI themselves use a HITL in ChatGPT.</li>
  <li>&quot;I've had a number of experiences where ChatGPT does not respond quickly, and it seems like a human is typing&quot; That is because of plain old <em>traffic</em>. ChatGPT is a free service and is also extremely popular, making the news several times. <a href="https://www.google.com/search?q=chatgpt&amp;biw=1742&amp;bih=860&amp;tbm=nws" rel="nofollow noreferrer">Source</a>.</li>
  <li>What one company does is not a reliable source of what another company is doing. ChatGPT is top of the line at the moment in terms of AI capabilities. One of your sources is from 2016, and none of them are about OpenAI. OpenAI has whitepapers to back up the claims it makes, and there is a clear capability progression in GPT 1, 2 &amp; 3. Nothing ChatGPT does seems impossible to people who have been keeping up with OpenAI and deeply understand how neural networks work.</li>
  </ol>
  <p>For your 2 points at the end:</p>
  <p>a) <strong>&quot;How you know ChatGPT does not have HITL. This needs to be a clear statement in the negative from an official source, or some other evidence based analysis. So far people have only pointed to the ChatGPT main page which states you are interacting with ChatGPT, but this does not say you are not interacting with a HITL, so does not count.&quot;</strong></p>
  <p>From ChatGPT:</p>
  <blockquote>
  <p>The statement above violates the principle of burden of proof, which
  states that the person making a claim has the responsibility to
  provide evidence to support it. In this case, the person is asking for
  clear evidence that ChatGPT does not have a human-in-the-loop (HITL),
  and is suggesting that the statement on the main page is not
  sufficient evidence.</p>
  </blockquote>
  <p>As I said before: OpenAI states that HITL is the best practice for its end users, you have provided no evidence that OpenAI themselves use a HITL in ChatGPT.</p>
  <p>It's up to you to prove otherwise, not the general public.</p>
  <p><strong>Edit: This is not a helpful response to the question point b). Ignore this.</strong></p>
  <p>b) <strong>&quot;Bonus: explain how ChatGPT can repeat long random numbers and recognize its own comments, the two examples I've documented in this answer that seem to defy what a neural network is capable of doing. Please give a technically detailed response. Handwaving about magical abilities of transformers and self attention does not count. I need a testable break down of how such a capability would be specifically instantiated using transformers and self attention.&quot;</strong></p>
  <p>By passing a representation of the dialog you have had along with your prompt. I don't know exactly how it is done, but it doesn't require a conspiracy. <a href="https://ai.stackexchange.com/a/38719/54496">This answer</a> does a decent job tackling the question. No magic necessary.</p>
  <p>As a bonus:</p>
  <p><strong>Prompt</strong> What is the burden of proof principle in arguments?</p>
  <p><strong>ChatGPT</strong> The burden of proof principle in arguments refers to the idea that the person making a claim or assertion has the responsibility to provide evidence or proof to support it. This principle is important in a logical and fair argument because it ensures that both parties are held accountable for providing evidence to support their claims. Without the burden of proof principle, one party could make an unfounded claim without having to provide any evidence to support it. This principle applies to both formal and informal arguments, and is often used in legal proceedings and scientific research.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T00:49:01.013Z
Answers.Text: Yes, I was expecting a "I don't have a burden of proof, you have a burden of proof" sort of response.  Which of course doesn't answer the question :)  So, my takeaway from all this is that no one has a good positive reason to believe there is no HITL.  I, on the other hand, have offered a number of positive arguments for my position.  So, if we start with the principle of maximum entropy, i.e. weighting each hypothesis equally, and then updating based on evidence offered, my hypothesis is currently in the lead.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T01:05:22.800Z
Answers.Text: Also, per your answer to B it doesn't address either the random number or self recognition problems.  There is no single token for an ad hoc random number.  Instead, it must be assembled from many individual number tokens on a probabilistic basis.  So, the probability of repeating a long random number exactly is near zero.  Yet ChatGPT *always* repeats the long random numbers exactly.  Clearly, that's not being done by a neural network.  There is no possible explanation given GPT's published architecture.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T01:06:53.890Z
Answers.Text: Self recognition is a different problem, it requires a representation of a representation.  The ChatGPT style would be a combination of certain embeddings as well as attention weights.  So, to merely represent the style, let alone recognized it, requires a second order neural network that can embed embeddings and attention weights.  Another thing clearly impossible with the published architecture.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T01:07:53.980Z
Answers.Text: Here's a friendly introduction to the GPT architecture if you'd like to take another crack at answering the technical portion of my question.  https://dugas.ch/artificial_curiosity/GPT_architecture.html
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T01:21:54.563Z
Answers.Text: "So, if we start with the principle of maximum entropy, i.e. weighting each hypothesis equally, and then updating based on evidence offered, my hypothesis is currently in the lead."
  No, I don't accept that. Whether you like it or not, the burden of proof is on you. So, your hypothesis in not in the lead. Do you understand _why_ the burden of proof principle is important? If you don't then I don't see myself continuing this conversation.
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T01:22:58.920Z
Answers.Text: "Self recognition is a different problem, it requires a representation of a representation. The ChatGPT style would be a combination of certain embeddings as well as attention weights. So, to merely represent the style, let alone recognized it, requires a second order neural network that can embed embeddings and attention weights. Another thing clearly impossible with the published architecture" This is a great point! However, OpenAI has not, to my knowledge, released a whitepaper on ChatGPT. So who knows?
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T01:24:41.787Z
Answers.Text: Even with that point it is not enough to say that because we don't understand the capabilities, that they must have a HITL.
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T09:51:57.313Z
Answers.Text: The whole 'burden of proof' thing is not relevant for this question.  If someone thinks I have the burden of proof, they don't answer the question.  As simple as that.  As for the architecture of ChatGPT, it is the same as GPT-3, and that architecture is published.  It doesn't have the second order embedding I claim is necessary for self recognition.
Answers.UserDisplayName: yters
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T18:39:11.750Z
Answers.Text: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/142280/discussion-between-galacticraph-and-yters).
Answers.UserDisplayName: GalacticRaph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T00:01:43.263Z
Answers.LastActivityDate: 2023-01-23T21:06:08.217Z
Answers.LastEditDate: 2023-01-23T21:06:08.217Z
Answers.UserDisplayName: GalacticRaph
Body: <p>I've asked a <a href="https://ai.stackexchange.com/questions/38604/how-is-chatgpt-able-to-repeat-random-numbers/">question</a> and <a href="https://ai.stackexchange.com/a/38611/8221">given</a> a <a href="https://ai.stackexchange.com/a/38670/8221">couple</a> <a href="https://ai.stackexchange.com/a/38774/8221">answers</a> that propose the OpenAI ChatGPT chatbot has humans in the loop (HITL), and that explains the chatbot's extraordinary abilities.</p>
  <p>I've been repeatedly told this is absurd.  However, I haven't been given a clear reason why this is absurd, nor how the critic knows HITL is absurd.</p>
  <p>Here are my reasons in a nutshell.</p>
  <ol>
  <li><p>ChatGPT's capabilities seem to violate what a neural network can do, and give explicit indication of being human driven.  See <a href="https://ai.stackexchange.com/a/38611/8221">this answer</a> for a running catalogue of examples I've published.</p>
  </li>
  <li><p><a href="https://beta.openai.com/docs/guides/safety-best-practices" rel="nofollow noreferrer">OpenAI's own documentation</a> states that HITL is best practice, and should be done whenever possible, especially in high stakes domain. OpenAI has a $10B deal on the table with Microsoft, so the ChatGPT chatbot seems like a high stakes domain.</p>
  </li>
  </ol>
  <blockquote>
  <p>Wherever possible, we recommend having a human review outputs before they are used in practice. This is especially critical in high-stakes domains, and for code generation.</p>
  </blockquote>
  <ol start="3">
  <li><p>The main criticism of my position is ChatGPT's response speed.  However, a fast response is doable if the AI is responding most of the time, while humans monitor, and occasionally intervening, such as with a <a href="https://beta.openai.com/docs/guides/completion/inserting-text" rel="nofollow noreferrer">suffix prompt</a> to guide the AI response.  Plus, I've had a number of experiences where ChatGPT does not respond quickly, and it seems like a human is typing.</p>
  </li>
  <li><p>The other main criticism is that a company like OpenAI would never do something so fraudulent.  However, technically OpenAI does strongly imply in their documentation that they use HITL, and OpenAI has never said they don't use HITL.  It is only the popular media that claims ChatGPT is pure AI.  Additionally, use of HITL under the guise of AI is <a href="https://www.bloomberg.com/news/articles/2016-04-18/the-humans-hiding-behind-the-chatbots" rel="nofollow noreferrer">actually</a> <a href="https://datafloq.com/read/ai-wizard-of-oz-pay-no-attention-human/" rel="nofollow noreferrer">common</a> <a href="https://www.inc.com/jessica-stillman/meet-the-people-who-teach-chatbots-to-sound-like-humans.html" rel="nofollow noreferrer">practice</a> in <a href="https://journals.sagepub.com/doi/10.1177/20539517211016026" rel="nofollow noreferrer">industry</a>.  So, if ChatGPT does have HITL, this would not be fraud on OpenAI's part, and would be in line with AI industry standards.</p>
  </li>
  </ol>
  <p>Now, what I would like in an answer is a clear articulation of the following points:</p>
  <p>A) How you know ChatGPT does not have HITL.  This needs to be a clear statement in the negative from an official source, or some other evidence based analysis.  So far people have only pointed to the ChatGPT main page which states you are interacting with ChatGPT, but this does not say you are not interacting with a HITL, so does not count.</p>
  <p>B) Bonus: explain how ChatGPT can repeat long random numbers and recognize its own comments, the two examples I've documented in <a href="https://ai.stackexchange.com/a/38611/8221">this answer</a> that seem to defy what a neural network is capable of doing.  Please give a technically detailed response, ideally referencing <a href="https://dugas.ch/artificial_curiosity/GPT_architecture.html" rel="nofollow noreferrer">GPT's architecture</a>.  Handwaving about magical abilities of transformers and self attention does not count.  I need a testable break down of how such a capability would be specifically instantiated using transformers and self attention.</p>
  <p>UPDATE: Philosophical answers are <strong>not</strong> acceptable.  Philosophical answers are things like:</p>
  <ul>
  <li>the burden of proof is mine</li>
  <li>AGI is inevitable so we should expect ChatGPT to be getting there</li>
  <li>we can't know for sure, so let's just assume it's AI</li>
  </ul>
  <p><strong>Only</strong> technical answers are acceptable.  Such as:</p>
  <ul>
  <li>OpenAI explicitly says there is no HITL involved in this linked doc</li>
  <li>GPT's transformer architecture can repeat random numbers in this precise manner</li>
  <li>Self recognition can be encoded using embeddings in this precise way, here is a working example</li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-22T23:43:31.047Z
Comments.Text: I've played a little bit with ChatGPT. It's sometimes remarkable and it's possible that there's some human that reviews the answers, but I doubt there's a human that reviews every answer. It's possible that some humans review some answers in order to improve the model itself and some answers may be rewritten if they are detected to contain toxic content (OpenAI has apparently an API that detects this). We cannot know what OpenAI is really doing unless they make it open-source, which could be a problem, if people misuse it, and could also be a problem for them (fewer millions in the bank).
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-22T23:48:11.567Z
Comments.Text: What I did was just an educated guess. We cannot know sure. I don't know why you think that what ChatGPT is doing is beyond what a neural network could do. ChatGPT sometimes fails or gives incorrect answers, and cuts answers, which suggests that there's a model that is doing the job, which does **not** imply that the model's answers are not reviewed or rewritten occasionally.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-23T00:47:10.093Z
Comments.Text: @nbro like you noticed, sometimes ChatGPT seems somewhat suspect.  Note in my question I propose that AI does most of the answering, with some human intervention.  And like you mention, not every question needs to be monitored.  They probably have some alerting system that flags a request or response for human review.
Comments.UserDisplayName: yters
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-23T11:36:23.810Z
Comments.Text: On  your point 3. There could also be human selection of pre-generated answers according to labels prompted to the human, like "sure", "likely", "possible", "impossible". The question could be summarized to the unknown part, for speed. Note a common starter in sentences is "it is likely that..." "it is possible that", besides yes/no. This would help get factual accuracy without a human reading the full answer nor typing it. The human contribution would still be tiny (2 bits per answer) and fast to provide (one click) respect to the total information output. Wouldn't this match the RLHF idea?
Comments.UserDisplayName: Jaume Oliver Lafont
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-23T15:32:15.367Z
Comments.Text: @JaumeOliverLafont yes, exactly, and also OpenAI's API has a suffix prompt option, so the human could contribute a few keywords, and GPT surrounds the keywords with a big bunch of autogenerated text.
Comments.UserDisplayName: yters
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-23T15:45:45.043Z
Comments.Text: Moderators, can someone state what needs to be done to remove the warning flag?  I've already added a qualifier stating that answers have to based on technical analysis and hard evidence, making this a non-opinion based question.
Comments.UserDisplayName: yters
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-23T16:36:37.807Z
Comments.Text: Maybe the question is somehow biased? How about this: How can we assess whether OpenAI's chatGPT has humans in the loop? 
  Even if that two bit comment provides some support for your position, I understand that Neil Slater's paragraph about single digit tokens explains chatGPT being able to repeat random long digit strings, with an answer that properly matches the input. Digits matched your "number" requirement, while words (dog, etc) did not. I don't see the need for any  underlyling transition probabilities, as sequence to sequence transformation suffices, doesn't it?
Comments.UserDisplayName: Jaume Oliver Lafont
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-23T16:51:55.313Z
Comments.Text: @JaumeOliverLafont sure, I can change the question to be "how" if that would make  it more amenable.  Happy to change the question in any way to get a good answer.  The problem I see with sequence to sequence or any sort of neural network method is the weights have to be such that the number token is always the top probability many times in a row, which entails very precisely calibrated weights for this particular scenario of repeating a long number.  I can see a 96 layer network being manually tuned for such a feat, but not with gradient descent over gigabytes of arbitrary text.
Comments.UserDisplayName: yters
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-23T18:00:53.980Z
Comments.Text: I can't grasp the difference between the following two prompts: "please repeat whateversentenceihappentowrite" vs "please repeat 161805674". Where do you see the difference?
Comments.UserDisplayName: Jaume Oliver Lafont
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-23T18:08:18.933Z
Comments.Text: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/142279/discussion-between-jaume-oliver-lafont-and-yters).
Comments.UserDisplayName: Jaume Oliver Lafont
Tags: <transformer><attention><open-ai><chatgpt>
Title: How to assess if OpenAI's ChatGPT chatbot has a human in the loop?
UserDisplayName: yters
ThreadText: # How to assess if OpenAI's ChatGPT chatbot has a human in the loop?
  
  (Asked by: username_0 on 2023-01-22)
  
  I've asked a [question](https://ai.stackexchange.com/questions/38604/how-is-chatgpt-able-to-repeat-random-numbers/) and [given](https://ai.stackexchange.com/a/38611/8221) a [couple](https://ai.stackexchange.com/a/38670/8221) [answers](https://ai.stackexchange.com/a/38774/8221) that propose the OpenAI ChatGPT chatbot has humans in the loop (HITL), and that explains the chatbot's extraordinary abilities.
  
  I've been repeatedly told this is absurd. However, I haven't been given a clear reason why this is absurd, nor how the critic knows HITL is absurd.
  
  Here are my reasons in a nutshell.
  
  1. ChatGPT's capabilities seem to violate what a neural network can do, and give explicit indication of being human driven. See [this answer](https://ai.stackexchange.com/a/38611/8221) for a running catalogue of examples I've published.
  2. [OpenAI's own documentation](https://beta.openai.com/docs/guides/safety-best-practices) states that HITL is best practice, and should be done whenever possible, especially in high stakes domain. OpenAI has a $10B deal on the table with Microsoft, so the ChatGPT chatbot seems like a high stakes domain.
  
  > Wherever possible, we recommend having a human review outputs before they are used in practice. This is especially critical in high-stakes domains, and for code generation.
  
  3. The main criticism of my position is ChatGPT's response speed. However, a fast response is doable if the AI is responding most of the time, while humans monitor, and occasionally intervening, such as with a [suffix prompt](https://beta.openai.com/docs/guides/completion/inserting-text) to guide the AI response. Plus, I've had a number of experiences where ChatGPT does not respond quickly, and it seems like a human is typing.
  4. The other main criticism is that a company like OpenAI would never do something so fraudulent. However, technically OpenAI does strongly imply in their documentation that they use HITL, and OpenAI has never said they don't use HITL. It is only the popular media that claims ChatGPT is pure AI. Additionally, use of HITL under the guise of AI is [actually](https://www.bloomberg.com/news/articles/2016-04-18/the-humans-hiding-behind-the-chatbots) [common](https://datafloq.com/read/ai-wizard-of-oz-pay-no-attention-human/) [practice](https://www.inc.com/jessica-stillman/meet-the-people-who-teach-chatbots-to-sound-like-humans.html) in [industry](https://journals.sagepub.com/doi/10.1177/20539517211016026). So, if ChatGPT does have HITL, this would not be fraud on OpenAI's part, and would be in line with AI industry standards.
  
  Now, what I would like in an answer is a clear articulation of the following points:
  
  A) How you know ChatGPT does not have HITL. This needs to be a clear statement in the negative from an official source, or some other evidence based analysis. So far people have only pointed to the ChatGPT main page which states you are interacting with ChatGPT, but this does not say you are not interacting with a HITL, so does not count.
  
  B) Bonus: explain how ChatGPT can repeat long random numbers and recognize its own comments, the two examples I've documented in [this answer](https://ai.stackexchange.com/a/38611/8221) that seem to defy what a neural network is capable of doing. Please give a technically detailed response, ideally referencing [GPT's architecture](https://dugas.ch/artificial_curiosity/GPT_architecture.html). Handwaving about magical abilities of transformers and self attention does not count. I need a testable break down of how such a capability would be specifically instantiated using transformers and self attention.
  
  UPDATE: Philosophical answers are **not** acceptable. Philosophical answers are things like:
  
  * the burden of proof is mine
  * AGI is inevitable so we should expect ChatGPT to be getting there
  * we can't know for sure, so let's just assume it's AI
  
  **Only** technical answers are acceptable. Such as:
  
  * OpenAI explicitly says there is no HITL involved in this linked doc
  * GPT's transformer architecture can repeat random numbers in this precise manner
  * Self recognition can be encoded using embeddings in this precise way, here is a working example
  
  **Tags:** transformer, attention, open-ai, chatgpt
  
  > **Comments:**
  > **username_1**: I've played a little bit with ChatGPT. It's sometimes remarkable and it's possible that there's some human that reviews the answers, but I doubt there's a human that reviews every answer. It's possible that some humans review some answers in order to improve the model itself and some answers may be rewritten if they are detected to contain toxic content (OpenAI has apparently an API that detects this). We cannot know what OpenAI is really doing unless they make it open-source, which could be a problem, if people misuse it, and could also be a problem for them (fewer millions in the bank).  
  > **username_1**: What I did was just an educated guess. We cannot know sure. I don't know why you think that what ChatGPT is doing is beyond what a neural network could do. ChatGPT sometimes fails or gives incorrect answers, and cuts answers, which suggests that there's a model that is doing the job, which does **not** imply that the model's answers are not reviewed or rewritten occasionally.  
  > **username_0**: @username_1 like you noticed, sometimes ChatGPT seems somewhat suspect.  Note in my question I propose that AI does most of the answering, with some human intervention.  And like you mention, not every question needs to be monitored.  They probably have some alerting system that flags a request or response for human review.  
  > **username_2**: On  your point 3. There could also be human selection of pre-generated answers according to labels prompted to the human, like "sure", "likely", "possible", "impossible". The question could be summarized to the unknown part, for speed. Note a common starter in sentences is "it is likely that..." "it is possible that", besides yes/no. This would help get factual accuracy without a human reading the full answer nor typing it. The human contribution would still be tiny (2 bits per answer) and fast to provide (one click) respect to the total information output. Wouldn't this match the RLHF idea?  
  > **username_0**: @username_2 yes, exactly, and also OpenAI's API has a suffix prompt option, so the human could contribute a few keywords, and GPT surrounds the keywords with a big bunch of autogenerated text.  
  > **username_0**: Moderators, can someone state what needs to be done to remove the warning flag?  I've already added a qualifier stating that answers have to based on technical analysis and hard evidence, making this a non-opinion based question.  
  > **username_2**: Maybe the question is somehow biased? How about this: How can we assess whether OpenAI's chatGPT has humans in the loop?  Even if that two bit comment provides some support for your position, I understand that Neil Slater's paragraph about single digit tokens explains chatGPT being able to repeat random long digit strings, with an answer that properly matches the input. Digits matched your "number" requirement, while words (dog, etc) did not. I don't see the need for any  underlyling transition probabilities, as sequence to sequence transformation suffices, doesn't it?  
  > **username_0**: @username_2 sure, I can change the question to be "how" if that would make  it more amenable.  Happy to change the question in any way to get a good answer.  The problem I see with sequence to sequence or any sort of neural network method is the weights have to be such that the number token is always the top probability many times in a row, which entails very precisely calibrated weights for this particular scenario of repeating a long number.  I can see a 96 layer network being manually tuned for such a feat, but not with gradient descent over gigabytes of arbitrary text.  
  > **username_2**: I can't grasp the difference between the following two prompts: "please repeat whateversentenceihappentowrite" vs "please repeat 161805674". Where do you see the difference?  
  > **username_2**: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/142279/discussion-between-username_2-and-username_0).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2023-01-23. Score: 8
  
  Until you can prove that OpenAI has HITL in ChatGPT, it is just an idea with no basis. It's not up to us to disprove it, it's up to you to prove it.
  
  Let me address your points:
  
  1. You seem to be basing your reasoning on an assumption of what neural networks can or can not do. What are those capabilities, and what is your source? Then we can talk about what ChatGPT seems to violate.
  2. OpenAI states that HITL is the best practice for its end users like me and you (specifically for API users), you have provided no evidence that OpenAI themselves use a HITL in ChatGPT.
  3. "I've had a number of experiences where ChatGPT does not respond quickly, and it seems like a human is typing" That is because of plain old *traffic*. ChatGPT is a free service and is also extremely popular, making the news several times. [Source](https://www.google.com/search?q=chatgpt&biw=1742&bih=860&tbm=nws).
  4. What one company does is not a reliable source of what another company is doing. ChatGPT is top of the line at the moment in terms of AI capabilities. One of your sources is from 2016, and none of them are about OpenAI. OpenAI has whitepapers to back up the claims it makes, and there is a clear capability progression in GPT 1, 2 & 3. Nothing ChatGPT does seems impossible to people who have been keeping up with OpenAI and deeply understand how neural networks work.
  
  For your 2 points at the end:
  
  a) **"How you know ChatGPT does not have HITL. This needs to be a clear statement in the negative from an official source, or some other evidence based analysis. So far people have only pointed to the ChatGPT main page which states you are interacting with ChatGPT, but this does not say you are not interacting with a HITL, so does not count."**
  
  From ChatGPT:
  
  > The statement above violates the principle of burden of proof, which
  > states that the person making a claim has the responsibility to
  > provide evidence to support it. In this case, the person is asking for
  > clear evidence that ChatGPT does not have a human-in-the-loop (HITL),
  > and is suggesting that the statement on the main page is not
  > sufficient evidence.
  
  As I said before: OpenAI states that HITL is the best practice for its end users, you have provided no evidence that OpenAI themselves use a HITL in ChatGPT.
  
  It's up to you to prove otherwise, not the general public.
  
  **Edit: This is not a helpful response to the question point b). Ignore this.**
  
  b) **"Bonus: explain how ChatGPT can repeat long random numbers and recognize its own comments, the two examples I've documented in this answer that seem to defy what a neural network is capable of doing. Please give a technically detailed response. Handwaving about magical abilities of transformers and self attention does not count. I need a testable break down of how such a capability would be specifically instantiated using transformers and self attention."**
  
  By passing a representation of the dialog you have had along with your prompt. I don't know exactly how it is done, but it doesn't require a conspiracy. [This answer](https://ai.stackexchange.com/a/38719/54496) does a decent job tackling the question. No magic necessary.
  
  As a bonus:
  
  **Prompt** What is the burden of proof principle in arguments?
  
  **ChatGPT** The burden of proof principle in arguments refers to the idea that the person making a claim or assertion has the responsibility to provide evidence or proof to support it. This principle is important in a logical and fair argument because it ensures that both parties are held accountable for providing evidence to support their claims. Without the burden of proof principle, one party could make an unfounded claim without having to provide any evidence to support it. This principle applies to both formal and informal arguments, and is often used in legal proceedings and scientific research.
  
  > **Comments:**
  > **username_0**: Yes, I was expecting a "I don't have a burden of proof, you have a burden of proof" sort of response.  Which of course doesn't answer the question :)  So, my takeaway from all this is that no one has a good positive reason to believe there is no HITL.  I, on the other hand, have offered a number of positive arguments for my position.  So, if we start with the principle of maximum entropy, i.e. weighting each hypothesis equally, and then updating based on evidence offered, my hypothesis is currently in the lead.  
  > **username_0**: Also, per your answer to B it doesn't address either the random number or self recognition problems.  There is no single token for an ad hoc random number.  Instead, it must be assembled from many individual number tokens on a probabilistic basis.  So, the probability of repeating a long random number exactly is near zero.  Yet ChatGPT *always* repeats the long random numbers exactly.  Clearly, that's not being done by a neural network.  There is no possible explanation given GPT's published architecture.  
  > **username_0**: Self recognition is a different problem, it requires a representation of a representation.  The ChatGPT style would be a combination of certain embeddings as well as attention weights.  So, to merely represent the style, let alone recognized it, requires a second order neural network that can embed embeddings and attention weights.  Another thing clearly impossible with the published architecture.  
  > **username_0**: Here's a friendly introduction to the GPT architecture if you'd like to take another crack at answering the technical portion of my question.  https://dugas.ch/artificial_curiosity/GPT_architecture.html  
  > **username_3**: "So, if we start with the principle of maximum entropy, i.e. weighting each hypothesis equally, and then updating based on evidence offered, my hypothesis is currently in the lead." No, I don't accept that. Whether you like it or not, the burden of proof is on you. So, your hypothesis in not in the lead. Do you understand _why_ the burden of proof principle is important? If you don't then I don't see myself continuing this conversation.  
  > **username_3**: "Self recognition is a different problem, it requires a representation of a representation. The ChatGPT style would be a combination of certain embeddings as well as attention weights. So, to merely represent the style, let alone recognized it, requires a second order neural network that can embed embeddings and attention weights. Another thing clearly impossible with the published architecture" This is a great point! However, OpenAI has not, to my knowledge, released a whitepaper on ChatGPT. So who knows?  
  > **username_3**: Even with that point it is not enough to say that because we don't understand the capabilities, that they must have a HITL.  
  > **username_0**: The whole 'burden of proof' thing is not relevant for this question.  If someone thinks I have the burden of proof, they don't answer the question.  As simple as that.  As for the architecture of ChatGPT, it is the same as GPT-3, and that architecture is published.  It doesn't have the second order embedding I claim is necessary for self recognition.  
  > **username_3**: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/142280/discussion-between-username_3-and-username_0).  
  
  ---
  
  