Answers.Body: <p>First, for almost every question of the form "Can AI be made to X", the most obvious and straightforward answer is something like "We don't know. Probably, but if it hasn't been done yet, we're really not sure."</p>
  
  <p>It's also important to understand that, from a technology standpoint, there isn't one "thing" called "AI".  There are many, many different technologies, which are loosely related (at best) and are generally lumped together under the overall rubric of "Artificial Intelligence".</p>
  
  <p>All of that said, yes, there has been work on adding memory, even long-term memory, to various kinds of "AI".  The most notable recent example is the advent of <a href="https://en.wikipedia.org/wiki/Long_short-term_memory" rel="nofollow noreferrer">LSTM</a> in recurrent neural networks.  </p>
  
  <p>Additionally, some of the work done on "cognitive architectures" has focused on the use of memory.  For more info on that, look up ACT-R and/or SOAR and read some of those papers.  </p>
  
  <p>What isn't clear to me offhand, is whether or not anybody has tried applying any of these techniques to chat-bots in particular.  I wouldn't be surprised if somebody had, but I can't cite any such research off the top of my head.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-01T11:27:25.713Z
Answers.Text: +1. Though LSTMs are 20 years old. The most notable recent example is probably rather Deepminds neural turing machines.
Answers.UserDisplayName: BlindKungFuMaster
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-01T20:58:09.203Z
Answers.Text: Fair enough.  It's "recent" relative to the development of perceptrons / ANN's in general, but I guess most people wouldn't really call something "recent" that is 20 years old.  :-)
Answers.UserDisplayName: mindcrime
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-02-26T09:26:38.610Z
Answers.LastActivityDate: 2017-02-26T09:26:38.610Z
Answers.UserDisplayName: mindcrime
Answers.Body: <p>It could be said that "maintaining a thought" is a basic requirement of computing, and can be represented as a string of binary digits in the context of a <a href="https://en.wikipedia.org/wiki/Turing_machine" rel="nofollow noreferrer">Turing Machine</a>.</p>
  
  <p><em>"Basically, it occurred to me, that when I'm thinking or speaking, there is a constant feedback loop, in which I am formulating which words to use next, which sentences to form, and which concepts to explore, based on my most recent statements and the flow of the dialogue or monologue. I'm not just responding to outside stimulus but also to myself. In other words, I am usually maintaining a train of thought."</em></p>
  
  <p>This sounds an awful lot like a recursive function.</p>
  
  <p>My analysis of the chatbot problem is that it reveals a poor quality reasoning on the part of the bots, as opposed to lack of reasoning. It's not so much a question on the raw ability of an algorithm to maintain a train of thought, because the "train of though" is the function itself, but the quality of the algorithm and, by some measures, the "humanness" of the output.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-01T14:35:50.300Z
Answers.Text: what I mean though, is an ai that actually has a train of thought. That is, an ai that is aware of it's most recent statements, and that will take those statements into consideration, in order to continue a "train of thought". I haven't actually looked at any chatbot code (and wouldn't understand it if I did), but I'm under the impression that chatbots usually only respond to user input (and don't maintain a train of thought).
Answers.UserDisplayName: Sebastian Hahn
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-01T16:27:11.450Z
Answers.Text: @SebastianHahn Awareness of past statements, and previous user inputs, would simply be an additional function.  Implementing such a system is trivial from a coding standpoint.  The difficulty, again, is with the quality of the output, which in this case, ideally passes a "Turing test".
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-02T11:19:08.797Z
Answers.Text: Unless I am misunderstanding something, it would not be trivial. Simply due to the fact that, while bots can learn response patterns, they often lack an understanding of the relationships between the various concepts which their statements and responses describe (and lacking an understanding of those relationships, a bot would not be able to maintain a coherent train of thought).
Answers.UserDisplayName: Sebastian Hahn
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-02T20:30:30.750Z
Answers.Text: @SebastianHahn I strongly concur that applying meaning is exceptionally non-trivial, but the disagreement may be semantic, in the sense of "coherent to whom?" (i,.e. coherence is a spectrum and subjective.)  If we define coherent as "logical and consistent", a response that is meaningless to humans could, and must, still be coherent in an algorithmic sense to functionally produce output of any kind.  Coherent to a human is a different animal, and here is a function of a "Turing" threshold.
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-02-28T20:03:56.887Z
Answers.LastActivityDate: 2017-02-28T20:03:56.887Z
Answers.UserDisplayName: DukeZhou
Body: <p>This mostly refers to human-like or chatbot AI, but could maybe be used in other applications (math or something?). </p>
  
  <p>Basically, it occurred to me, that when I'm thinking or speaking, there is a constant feedback loop, in which I am formulating which words to use next, which sentences to form, and which concepts to explore, based on my most recent statements and the flow of the dialogue or monologue. I'm not just responding to outside stimulus but also to myself. In other words, I am usually maintaining a train of thought.</p>
  
  <p>Can AI be made capable of this? If so, has it been demonstrated? And to what extent? While typing this, I discovered the term "thought vectors", and I think it might be related. </p>
  
  <p>If I read correctly, thought vectors have something to do with allowing AI to store or identify the relationships between different concepts; and if I had to guess, I'd say that if an AI lacks a strong understanding of the relationships between concepts, then it would be impossible for it to maintain a coherent train of thought. Would that be a correct assumption?</p>
  
  <p>(ps. in my limited experience with AI chatbots, they seem to be either completely scripted, or otherwise random and often incoherent, which is what leads me to believe that they do not maintain a train of thought)</p>
  
Tags: <ai-design><chat-bots><thought-vectors>
Title: Can an AI be made to maintain a train of thought?
UserDisplayName: Sebastian Hahn
ThreadText: # Can an AI be made to maintain a train of thought?
  
  (Asked by: username_0 on 2017-02-25)
  
  This mostly refers to human-like or chatbot AI, but could maybe be used in other applications (math or something?).
  
  Basically, it occurred to me, that when I'm thinking or speaking, there is a constant feedback loop, in which I am formulating which words to use next, which sentences to form, and which concepts to explore, based on my most recent statements and the flow of the dialogue or monologue. I'm not just responding to outside stimulus but also to myself. In other words, I am usually maintaining a train of thought.
  
  Can AI be made capable of this? If so, has it been demonstrated? And to what extent? While typing this, I discovered the term "thought vectors", and I think it might be related.
  
  If I read correctly, thought vectors have something to do with allowing AI to store or identify the relationships between different concepts; and if I had to guess, I'd say that if an AI lacks a strong understanding of the relationships between concepts, then it would be impossible for it to maintain a coherent train of thought. Would that be a correct assumption?
  
  (ps. in my limited experience with AI chatbots, they seem to be either completely scripted, or otherwise random and often incoherent, which is what leads me to believe that they do not maintain a train of thought)
  
  **Tags:** ai-design, chat-bots, thought-vectors
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2017-02-26. Score: 1
  
  First, for almost every question of the form "Can AI be made to X", the most obvious and straightforward answer is something like "We don't know. Probably, but if it hasn't been done yet, we're really not sure."
  
  It's also important to understand that, from a technology standpoint, there isn't one "thing" called "AI". There are many, many different technologies, which are loosely related (at best) and are generally lumped together under the overall rubric of "Artificial Intelligence".
  
  All of that said, yes, there has been work on adding memory, even long-term memory, to various kinds of "AI". The most notable recent example is the advent of [LSTM](https://en.wikipedia.org/wiki/Long_short-term_memory) in recurrent neural networks.
  
  Additionally, some of the work done on "cognitive architectures" has focused on the use of memory. For more info on that, look up ACT-R and/or SOAR and read some of those papers.
  
  What isn't clear to me offhand, is whether or not anybody has tried applying any of these techniques to chat-bots in particular. I wouldn't be surprised if somebody had, but I can't cite any such research off the top of my head.
  
  > **Comments:**
  > **username_2**: +1. Though LSTMs are 20 years old. The most notable recent example is probably rather Deepminds neural turing machines.  
  > **username_1**: Fair enough.  It's "recent" relative to the development of perceptrons / ANN's in general, but I guess most people wouldn't really call something "recent" that is 20 years old.  :-)  
  
  ---
  
  ### Answer by username_3 on 2017-02-28. Score: 0
  
  It could be said that "maintaining a thought" is a basic requirement of computing, and can be represented as a string of binary digits in the context of a [Turing Machine](https://en.wikipedia.org/wiki/Turing_machine).
  
  *"Basically, it occurred to me, that when I'm thinking or speaking, there is a constant feedback loop, in which I am formulating which words to use next, which sentences to form, and which concepts to explore, based on my most recent statements and the flow of the dialogue or monologue. I'm not just responding to outside stimulus but also to myself. In other words, I am usually maintaining a train of thought."*
  
  This sounds an awful lot like a recursive function.
  
  My analysis of the chatbot problem is that it reveals a poor quality reasoning on the part of the bots, as opposed to lack of reasoning. It's not so much a question on the raw ability of an algorithm to maintain a train of thought, because the "train of though" is the function itself, but the quality of the algorithm and, by some measures, the "humanness" of the output.
  
  > **Comments:**
  > **username_0**: what I mean though, is an ai that actually has a train of thought. That is, an ai that is aware of it's most recent statements, and that will take those statements into consideration, in order to continue a "train of thought". I haven't actually looked at any chatbot code (and wouldn't understand it if I did), but I'm under the impression that chatbots usually only respond to user input (and don't maintain a train of thought).  
  > **username_3**: @SebastianHahn Awareness of past statements, and previous user inputs, would simply be an additional function.  Implementing such a system is trivial from a coding standpoint.  The difficulty, again, is with the quality of the output, which in this case, ideally passes a "Turing test".  
  > **username_0**: Unless I am misunderstanding something, it would not be trivial. Simply due to the fact that, while bots can learn response patterns, they often lack an understanding of the relationships between the various concepts which their statements and responses describe (and lacking an understanding of those relationships, a bot would not be able to maintain a coherent train of thought).  
  > **username_3**: @SebastianHahn I strongly concur that applying meaning is exceptionally non-trivial, but the disagreement may be semantic, in the sense of "coherent to whom?" (i,.e. coherence is a spectrum and subjective.)  If we define coherent as "logical and consistent", a response that is meaningless to humans could, and must, still be coherent in an algorithmic sense to functionally produce output of any kind.  Coherent to a human is a different animal, and here is a function of a "Turing" threshold.  
  
  ---
  
  