Answers.Body: <p>Incalculable times i caught it literally making things up it does not know. For example an hour ago i asked it if Arnold Schwarzenegger says the line &quot;i don't want to hurt you&quot; in any of his movies. It first correctly returned no, then i wrote try harder and it apologized and said he says it in Batman and Robin 1997. I asked for exact minute it said 1:03, i looked no such line, then it said it's at 1:26 then i looked and there said there is no line there either. Then it apologize again and said there is no such line. So It lied 3 times, totally making it up. And i have personally caught it lying already countless times.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-16T00:18:47.830Z
Answers.Text: Even at technical questions, it really wants to please. [Coprime Matroid](https://chat.openai.com/c/dc92867a-87a3-4339-a4b3-1b84755da230) looks like a formal mathematical proof, but some of the smaller details don't actually make sense. It would likely fool a non-mathematician. What's really impressive though is that even though there is actually no such thing, it invented the name "the coprime matroid", which would have been the perfect name for it had such a thing actually existed, making it look like it really had researched the answer and found various references to it using this name.
Answers.UserDisplayName: Ray Butterworth
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-17T05:01:59.293Z
Answers.Text: I know, i been talking to it about physics and electrical engineering, when it does not know it simply makes it up and pretends it's a fact altho it is 100% bs. Like this guy puts it "it creates authorative bullshit" https://youtu.be/1wzPr4cUoMQ?t=479
Answers.UserDisplayName: Nix
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-15T23:29:10.597Z
Answers.LastActivityDate: 2023-04-17T16:53:54.383Z
Answers.LastEditDate: 2023-04-17T16:53:54.383Z
Answers.UserDisplayName: Nix
Answers.Body: <p>The reason is that ChatGPT is not a data retrieval system.  In other words, it is not consulting a database, nor is it performing searches of the internet, to find answers to questions posed.</p>
  <p>Instead, it is generating novel answers based on the trained parameters within the network.  In a sense, this is far more impressive.</p>
  <p>If you think about it, ChatGPT and large language models generally provide a very interesting method of compressing data.  Of course, compression isn't the point of LLMs, but you can view this as one of the outcomes.  How so?</p>
  <p>During training, the parameters are learned such that the model is able to reasonably predict the next most likely token (word/bigram/trigram/letter/symbol) based on all of the tokens that have come before it in that session, including any base prompt that is included silently.  When you ask it to recite something like <em>The Lovesong of J. Alfred Prufrock,</em> it can do so very accurately... yet, it does not have a copy of Prufrock &quot;memorized&quot; somewhere.  Instead, it is recreating Prufrock generatively.</p>
  <p>This is also why its answers, while incredibly confident, can be wildly incorrect.  It is essentially generating the most likely token that comes next, not reasoning or thinking about what the text means.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-24T11:29:31.587Z
Answers.Text: How can we assess to what extent it compresses or rather memorizes data?
Answers.UserDisplayName: Jaume Oliver Lafont
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-24T11:45:25.637Z
Answers.Text: That's a super question.  Intuitively, we could evaluate the billions of parameters in the model and attempt to recreate data from them directly... However, since they are in an encoded representation (I would think of it as an eigenvector of some high dimensional space), that would be super challenging.  Definitely an interesting thing to study though!
Answers.UserDisplayName: David Hoelzer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-22T11:11:39.143Z
Answers.Text: its also funny that it claims it's going to "double-check" its sources in the future, considering it has no sources and it's not performing any sort of review on what it says
Answers.UserDisplayName: Ivan Viti
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-24T11:23:33.303Z
Answers.LastActivityDate: 2023-02-24T11:23:33.303Z
Answers.UserDisplayName: David Hoelzer
Body: <p>I asked ChatGPT a <a href="https://chat.openai.com/chat/7e5fbd69-e102-44b3-9ca8-7df2c9b265b7" rel="nofollow noreferrer">question</a>, and its response included this statement:</p>
  <blockquote>
  <p>In &quot;The Ruling Class&quot; film, which was released in 1972, Patrick McGoohan played the role of Bishop Lampton.</p>
  </blockquote>
  <p>Unlike many of its vague or weaselly responses, this is a direct statement of fact.
  But it is false and very easy to disprove, so I wondered how it came up with that misinformation?</p>
  <p>I asked it how, and it apologized and admitted it was wrong:</p>
  <blockquote>
  <p>Q: You say &quot;Patrick McGoohan played the role of Bishop Lampton&quot;, when in fact it was Alastair Sim that played that role.  Where did you get this misinformation?</p>
  <p>A: I apologize for the error in my previous response. You are correct that Patrick McGoohan did not appear in &quot;The Ruling Class&quot; film and that Alastair Sim played the role of Bishop Lampton.</p>
  <p>I apologize for any confusion or inconvenience that my previous response may have caused. As an AI language model, I strive to provide accurate information, and I will make sure to double-check my sources more carefully in the future. Thank you for bringing this to my attention.</p>
  </blockquote>
  <p>But it avoided answering the question:</p>
  <blockquote>
  <p>Q: But my question was &quot;Where did you get this misinformation?&quot;.</p>
  <p>A: I apologize for the confusion. As an AI language model, I do not have personal experiences, so <strong>I did not get the misinformation from anywhere</strong>. I must have made a mistake in my response due to an error in processing or a misunderstanding of the question.</p>
  </blockquote>
  <p>It wouldn't have just made it up in order to provide an amazingly good answer,
  so how can it give a false statement of fact and claim that it didn't get the information from anywhere? (I.e. what mechanism does it use to generate facts if they don't originate somewhere else?)</p>
  <p>And worse, it actually lied while apologizing:</p>
  <blockquote>
  <p>I will make sure to double-check my sources more carefully in the future.</p>
  </blockquote>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-24T09:15:42.100Z
Comments.Text: You should not trust anything that ChatGPT says, its a Stochastic Parrot, and you are asking us about mechanisms, when there is no public information about its inner workings.
Comments.UserDisplayName: Dr. Snoopy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-24T20:51:48.043Z
Comments.Text: The current term for this phenomenon is "hallucination" . It is very easy to induce, once you know how. https://medium.com/@cooper.white_86633/ai-hallucinations-the-ethical-burdens-of-using-chatgpt-5e62dd0f75c7
Comments.UserDisplayName: chasly - supports Monica
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-05T16:10:03.997Z
Comments.Text: From what I experimented Chatgpt is not very good with movie.  It seems to check only a few term of the question and return something that is induced from these terms, not the full question.
Comments.UserDisplayName: Ptit Xav
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-16T07:47:01.493Z
Comments.Text: I think the term parrot is very misleading. It is not small, it is not in a cage and it moves as a lot of us move. I prefer the name myrianthrope.
Comments.UserDisplayName: Jaume Oliver Lafont
Tags: <chatgpt><question-answering>
Title: ChatGPT claims that it doesn't get information from anywhere, so can't say why it gave a blatantly false statement. How is that possible?
UserDisplayName: Ray Butterworth
ThreadText: # ChatGPT claims that it doesn't get information from anywhere, so can't say why it gave a blatantly false statement. How is that possible?
  
  (Asked by: username_0 on 2023-02-24)
  
  I asked ChatGPT a [question](https://chat.openai.com/chat/7e5fbd69-e102-44b3-9ca8-7df2c9b265b7), and its response included this statement:
  
  > In "The Ruling Class" film, which was released in 1972, Patrick McGoohan played the role of Bishop Lampton.
  
  Unlike many of its vague or weaselly responses, this is a direct statement of fact.
  But it is false and very easy to disprove, so I wondered how it came up with that misinformation?
  
  I asked it how, and it apologized and admitted it was wrong:
  
  > Q: You say "Patrick McGoohan played the role of Bishop Lampton", when in fact it was Alastair Sim that played that role. Where did you get this misinformation?
  >
  > A: I apologize for the error in my previous response. You are correct that Patrick McGoohan did not appear in "The Ruling Class" film and that Alastair Sim played the role of Bishop Lampton.
  >
  > I apologize for any confusion or inconvenience that my previous response may have caused. As an AI language model, I strive to provide accurate information, and I will make sure to double-check my sources more carefully in the future. Thank you for bringing this to my attention.
  
  But it avoided answering the question:
  
  > Q: But my question was "Where did you get this misinformation?".
  >
  > A: I apologize for the confusion. As an AI language model, I do not have personal experiences, so **I did not get the misinformation from anywhere**. I must have made a mistake in my response due to an error in processing or a misunderstanding of the question.
  
  It wouldn't have just made it up in order to provide an amazingly good answer,
  so how can it give a false statement of fact and claim that it didn't get the information from anywhere? (I.e. what mechanism does it use to generate facts if they don't originate somewhere else?)
  
  And worse, it actually lied while apologizing:
  
  > I will make sure to double-check my sources more carefully in the future.
  
  **Tags:** chatgpt, question-answering
  
  > **Comments:**
  > **username_1**: You should not trust anything that ChatGPT says, its a Stochastic Parrot, and you are asking us about mechanisms, when there is no public information about its inner workings.  
  > **username_2**: The current term for this phenomenon is "hallucination" . It is very easy to induce, once you know how. https://medium.com/@cooper.white_86633/ai-hallucinations-the-ethical-burdens-of-using-chatgpt-5e62dd0f75c7  
  > **username_3**: From what I experimented Chatgpt is not very good with movie.  It seems to check only a few term of the question and return something that is induced from these terms, not the full question.  
  > **username_4**: I think the term parrot is very misleading. It is not small, it is not in a cage and it moves as a lot of us move. I prefer the name myrianthrope.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_6 on 2023-02-24. Score: 7 | Accepted answer
  
  The reason is that ChatGPT is not a data retrieval system. In other words, it is not consulting a database, nor is it performing searches of the internet, to find answers to questions posed.
  
  Instead, it is generating novel answers based on the trained parameters within the network. In a sense, this is far more impressive.
  
  If you think about it, ChatGPT and large language models generally provide a very interesting method of compressing data. Of course, compression isn't the point of LLMs, but you can view this as one of the outcomes. How so?
  
  During training, the parameters are learned such that the model is able to reasonably predict the next most likely token (word/bigram/trigram/letter/symbol) based on all of the tokens that have come before it in that session, including any base prompt that is included silently. When you ask it to recite something like *The Lovesong of J. Alfred Prufrock,* it can do so very accurately... yet, it does not have a copy of Prufrock "memorized" somewhere. Instead, it is recreating Prufrock generatively.
  
  This is also why its answers, while incredibly confident, can be wildly incorrect. It is essentially generating the most likely token that comes next, not reasoning or thinking about what the text means.
  
  > **Comments:**
  > **username_4**: How can we assess to what extent it compresses or rather memorizes data?  
  > **username_6**: That's a super question.  Intuitively, we could evaluate the billions of parameters in the model and attempt to recreate data from them directly... However, since they are in an encoded representation (I would think of it as an eigenvector of some high dimensional space), that would be super challenging.  Definitely an interesting thing to study though!  
  > **username_7**: its also funny that it claims it's going to "double-check" its sources in the future, considering it has no sources and it's not performing any sort of review on what it says  
  
  ---
  
  ### Answer by username_5 on 2023-04-15. Score: 1
  
  Incalculable times i caught it literally making things up it does not know. For example an hour ago i asked it if Arnold Schwarzenegger says the line "i don't want to hurt you" in any of his movies. It first correctly returned no, then i wrote try harder and it apologized and said he says it in Batman and Robin 1997. I asked for exact minute it said 1:03, i looked no such line, then it said it's at 1:26 then i looked and there said there is no line there either. Then it apologize again and said there is no such line. So It lied 3 times, totally making it up. And i have personally caught it lying already countless times.
  
  > **Comments:**
  > **username_0**: Even at technical questions, it really wants to please. [Coprime Matroid](https://chat.openai.com/c/dc92867a-87a3-4339-a4b3-1b84755da230) looks like a formal mathematical proof, but some of the smaller details don't actually make sense. It would likely fool a non-mathematician. What's really impressive though is that even though there is actually no such thing, it invented the name "the coprime matroid", which would have been the perfect name for it had such a thing actually existed, making it look like it really had researched the answer and found various references to it using this name.  
  > **username_5**: I know, i been talking to it about physics and electrical engineering, when it does not know it simply makes it up and pretends it's a fact altho it is 100% bs. Like this guy puts it "it creates authorative bullshit" https://youtu.be/1wzPr4cUoMQ?t=479  
  
  ---
  
  