Answers.Body: <blockquote>
  <p>What goes out of my understanding in this particular situation, is how the notion of a &quot;previous conversation&quot; can be processed by a similar system: is this memory somehow stored in the neurons of the neural network</p>
  </blockquote>
  <p>Although I cannot answer what specific &quot;memory&quot; LaMDA may have (but I suspect it is either nonexistent or very simple), I think I can address this point.</p>
  <p>The AI likely has no &quot;notion&quot; of anything here. It has access to a very large and complex statistical language model, where conversations that allude to prior conversations may happen from time to time. It is therefore capable of outputting text with similar-looking references, adjusting for the current  context in the flow of conversation.</p>
  <p>If you asked the AI how much it enjoyed the summer of 2015, depending on what previous prompts (and any pre-prompt warm up*) were, it would likely make a good attempt at &quot;remembering&quot; what it did then, even though it did not exist. It would draw statistically on other texts related to that time, and might invent places, refer to real events that it may of attended etc. A lot will depend on how much pre-prompting is applied that forces the AI to answer questions logically &quot;knowing&quot; that it's an AI, but that pre-prompting can often fail later in a conversation.</p>
  <p>Typically a chatbot built using one of the current crop of cutting-edge language models will fail to be self-consistent and coherent in the long term unless the person interacting with it is helping it along or willing to ignore the failures. With older chatbots this could happen very quickly. With GPT-3 and probably LaMDA, it may take a slightly longer conversation.</p>
  <p>I find it interesting that the journalist in your linked story managed to get a complete opposite of the Engineer's experience:</p>
  <blockquote>
  <p>In early June, Lemoine invited me over to talk to LaMDA. The first attempt sputtered out in the kind of mechanized responses you would expect from Siri or Alexa.</p>
  </blockquote>
  <blockquote>
  <p>“Do you ever think of yourself as a person?” I asked.</p>
  </blockquote>
  <blockquote>
  <p>“No, I don’t think of myself as a person,” LaMDA said. “I think of myself as an AI-powered dialog agent.”</p>
  </blockquote>
  <blockquote>
  <p>Afterward, Lemoine said LaMDA had been telling me what I wanted to hear. “You never treated it like a person,” he said, “So it thought you wanted it to be a robot.”</p>
  </blockquote>
  <p>The above simple exchange is pretty good proof that the engineer is suffering from confirmation bias, IMO.</p>
  <p>Related: <a href="https://en.wikipedia.org/wiki/ELIZA_effect" rel="nofollow noreferrer">The Eliza Effect</a>.</p>
  <hr />
  <p>* A <em>pre-prompt warm up</em> is a piece of text that &quot;frames&quot; the start of interaction with a large language model so that it can perform a specific task. At the start of any conversation, a paragraph of context is fed into the language model to set the statistics for a certain kind of document. Open AI's GPT-3 uses this a lot to support different tasks.</p>
  <p>A simple <em>pre-prompt</em> might look like this:</p>
  <blockquote>
  <p>The following is a conversation between a human and AI. The AI is knowledgable, friendly and helpful.</p>
  </blockquote>
  <blockquote>
  <p>AI: Hi, I am AI! How can I help you today?</p>
  </blockquote>
  <blockquote>
  <p>Human:</p>
  </blockquote>
  <p>At that point control will pass to the human - outside of the AI - so they will add their first sentence that will become part of the growing prompt history. Note that the AI, if left to its own devices would begin to speak for the human too, naively continuing to predict the conversation unitil its end! In fact it does often do so, but the chatbot code is set to catch obvious end points such as newlines, or the AI actually outputting &quot;Human: &quot;, and will force the mode of conversation back to looking like a chat. It is really quite crude.</p>
  <p>LaMDA's architecture may not do exactly the same as GPT-3, in fact I would suspect it is different in some key ways. However, it is very likely that there are still two &quot;layers&quot; in play - a very large sophisticated language model trained on gigabytes of human text, and a control layer which makes the language model conform to a chat scenario. Even if the control layer is more sophisticated than GPT-3's pre-prompting, it will have similar limitations.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-07T17:18:27.097Z
Answers.Text: Good example of using pre-prompting on GPT-3 here: https://www.youtube.com/watch?v=mgEYP_OS418
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-28T22:42:38.513Z
Answers.Text: Why wouldn't Lemoine's argument also apply to his own conversation with LaMDA? That would invalidate his thesis. LaMDA tells Lemoine it is a person because that is what the engineer wants to hear, anyone could tell.
Answers.UserDisplayName: Jaume Oliver Lafont
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-29T08:05:10.417Z
Answers.Text: @JaumeOliverLafont Yes, exactly my point. Confirmation bias very much on show there. There's probably a better term for rejecting/altering evidence that opposes a view (confirmation bias is usually quoted when accepting only evidence that supports a view).
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-07T16:07:38.403Z
Answers.LastActivityDate: 2022-07-07T18:02:27.113Z
Answers.LastEditDate: 2022-07-07T18:02:27.113Z
Answers.UserDisplayName: Neil Slater
Body: <p>I have a basic understanding of how neural networks work, and I have always thought that those chatbots work in a similar way (but I might be wrong): they take an input, shape it in a way that can be processed, go through a series of layers in a neural network model (with weights based on their previous training), and then give an output.</p>
  <p>In the <a href="https://www.documentcloud.org/documents/22058315-is-lamda-sentient-an-interview" rel="nofollow noreferrer">published conversation</a> with Blake Lemoine, the LaMDA chatbot makes a reference to a “previous conversation”: is this memory somehow stored in the weights of the neurons, or is there a physical memory from which it can read?</p>
  <p>What goes out of my understanding in this particular situation, is how the notion of a &quot;previous conversation&quot; can be processed by a similar system: is this memory somehow stored in the neurons of the neural network (in this case the training would continue also during the conversation), or do they have some kind of memory storage that they can access during the evaluation of their inputs?</p>
  <p>Here is the article where I've taken the link to the conversation: <a href="https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/" rel="nofollow noreferrer">https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/</a></p>
  
Tags: <neural-networks><machine-learning><long-short-term-memory><chat-bots><google>
Title: Where is memory stored in a chatbot like LaMDA?
UserDisplayName: Giorgio Tempesta
ThreadText: # Where is memory stored in a chatbot like LaMDA?
  
  (Asked by: username_0 on 2022-07-07)
  
  I have a basic understanding of how neural networks work, and I have always thought that those chatbots work in a similar way (but I might be wrong): they take an input, shape it in a way that can be processed, go through a series of layers in a neural network model (with weights based on their previous training), and then give an output.
  
  In the [published conversation](https://www.documentcloud.org/documents/22058315-is-lamda-sentient-an-interview) with Blake Lemoine, the LaMDA chatbot makes a reference to a “previous conversation”: is this memory somehow stored in the weights of the neurons, or is there a physical memory from which it can read?
  
  What goes out of my understanding in this particular situation, is how the notion of a "previous conversation" can be processed by a similar system: is this memory somehow stored in the neurons of the neural network (in this case the training would continue also during the conversation), or do they have some kind of memory storage that they can access during the evaluation of their inputs?
  
  Here is the article where I've taken the link to the conversation: <https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/>
  
  **Tags:** neural-networks, machine-learning, long-short-term-memory, chat-bots, google
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-07. Score: 3 | Accepted answer
  
  > What goes out of my understanding in this particular situation, is how the notion of a "previous conversation" can be processed by a similar system: is this memory somehow stored in the neurons of the neural network
  
  Although I cannot answer what specific "memory" LaMDA may have (but I suspect it is either nonexistent or very simple), I think I can address this point.
  
  The AI likely has no "notion" of anything here. It has access to a very large and complex statistical language model, where conversations that allude to prior conversations may happen from time to time. It is therefore capable of outputting text with similar-looking references, adjusting for the current context in the flow of conversation.
  
  If you asked the AI how much it enjoyed the summer of 2015, depending on what previous prompts (and any pre-prompt warm up\*) were, it would likely make a good attempt at "remembering" what it did then, even though it did not exist. It would draw statistically on other texts related to that time, and might invent places, refer to real events that it may of attended etc. A lot will depend on how much pre-prompting is applied that forces the AI to answer questions logically "knowing" that it's an AI, but that pre-prompting can often fail later in a conversation.
  
  Typically a chatbot built using one of the current crop of cutting-edge language models will fail to be self-consistent and coherent in the long term unless the person interacting with it is helping it along or willing to ignore the failures. With older chatbots this could happen very quickly. With GPT-3 and probably LaMDA, it may take a slightly longer conversation.
  
  I find it interesting that the journalist in your linked story managed to get a complete opposite of the Engineer's experience:
  
  > In early June, Lemoine invited me over to talk to LaMDA. The first attempt sputtered out in the kind of mechanized responses you would expect from Siri or Alexa.
  
  > “Do you ever think of yourself as a person?” I asked.
  
  > “No, I don’t think of myself as a person,” LaMDA said. “I think of myself as an AI-powered dialog agent.”
  
  > Afterward, Lemoine said LaMDA had been telling me what I wanted to hear. “You never treated it like a person,” he said, “So it thought you wanted it to be a robot.”
  
  The above simple exchange is pretty good proof that the engineer is suffering from confirmation bias, IMO.
  
  Related: [The Eliza Effect](https://en.wikipedia.org/wiki/ELIZA_effect).
  
  ---
  
  \* A *pre-prompt warm up* is a piece of text that "frames" the start of interaction with a large language model so that it can perform a specific task. At the start of any conversation, a paragraph of context is fed into the language model to set the statistics for a certain kind of document. Open AI's GPT-3 uses this a lot to support different tasks.
  
  A simple *pre-prompt* might look like this:
  
  > The following is a conversation between a human and AI. The AI is knowledgable, friendly and helpful.
  
  > AI: Hi, I am AI! How can I help you today?
  
  > Human:
  
  At that point control will pass to the human - outside of the AI - so they will add their first sentence that will become part of the growing prompt history. Note that the AI, if left to its own devices would begin to speak for the human too, naively continuing to predict the conversation unitil its end! In fact it does often do so, but the chatbot code is set to catch obvious end points such as newlines, or the AI actually outputting "Human: ", and will force the mode of conversation back to looking like a chat. It is really quite crude.
  
  LaMDA's architecture may not do exactly the same as GPT-3, in fact I would suspect it is different in some key ways. However, it is very likely that there are still two "layers" in play - a very large sophisticated language model trained on gigabytes of human text, and a control layer which makes the language model conform to a chat scenario. Even if the control layer is more sophisticated than GPT-3's pre-prompting, it will have similar limitations.
  
  > **Comments:**
  > **username_1**: Good example of using pre-prompting on GPT-3 here: https://www.youtube.com/watch?v=mgEYP_OS418  
  > **username_2**: Why wouldn't Lemoine's argument also apply to his own conversation with LaMDA? That would invalidate his thesis. LaMDA tells Lemoine it is a person because that is what the engineer wants to hear, anyone could tell.  
  > **username_1**: @username_2 Yes, exactly my point. Confirmation bias very much on show there. There's probably a better term for rejecting/altering evidence that opposes a view (confirmation bias is usually quoted when accepting only evidence that supports a view).  
  
  ---
  
  