Answers.Body: <blockquote>
    <p>I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for "repeat the character that is 14 characters to the left" and so on</p>
  </blockquote>
  
  <p>Creating a system of rules like your example above is the very opposite of training an RNN to perform this task.</p>
  
  <p>If you would like to train an RNN to answer simple questions you would not need to come up with ingenious rules, but with enough training data of the form</p>
  
  <pre><code>question -&gt; answer
  </code></pre>
  
  <p>Then you could use one of many popular sequence-to-sequence NLP tools to try and learn this behaviour, effectively treating the problem as if it were machine translation between languages.</p>
  
  <hr>
  
  <p>More broadly, yes, question answering has been done before, it is very popular in fact. It is an active subfield of NLP research and many methods have been developed, some of them involving RNN networks.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-28T19:40:41.287Z
Answers.LastActivityDate: 2020-01-28T19:40:41.287Z
Answers.UserDisplayName: Mathias Müller
Body: <p>I would like to train an RNN to follow the sentences:</p>
  
  <p>"Would you like some cheese"? with "Yes, I would like some cheese."</p>
  
  <p>So whenever the template "Would you like some ____?" appears then RNN produces the sequence above. And it should even work on sentences which are new like "Would you like some blumf?"</p>
  
  <p>I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for "repeat the character that is 14 characters to the left" and so on.</p>
  
  <p>Has this been done before or is there a better way?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-28T23:16:13.000Z
Comments.Text: One of the important things to realize about the modern approach to AI/ML is that you *do not provide it rules*, you use the training data to get the network to *work out rules for itself*.
Comments.UserDisplayName: David Hoelzer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-29T00:46:58.177Z
Comments.Text: Except an RNN can't learn the rule "repeat last character." It can only learn 26 separate rules for each character. Which is kind of useless to learn template type things.
Comments.UserDisplayName: zooby
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-29T00:57:43.807Z
Comments.Text: Why limit yourself to letters?  You could certainly train it to look for and predict word sequences.
Comments.UserDisplayName: David Hoelzer
Tags: <natural-language-processing><recurrent-neural-networks><long-short-term-memory>
Title: Training an RNN to answer simple quesitons
UserDisplayName: zooby
ThreadText: # Training an RNN to answer simple quesitons
  
  (Asked by: username_0 on 2020-01-09)
  
  I would like to train an RNN to follow the sentences:
  
  "Would you like some cheese"? with "Yes, I would like some cheese."
  
  So whenever the template "Would you like some \_\_\_\_?" appears then RNN produces the sequence above. And it should even work on sentences which are new like "Would you like some blumf?"
  
  I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for "repeat the character that is 14 characters to the left" and so on.
  
  Has this been done before or is there a better way?
  
  **Tags:** natural-language-processing, recurrent-neural-networks, long-short-term-memory
  
  > **Comments:**
  > **username_1**: One of the important things to realize about the modern approach to AI/ML is that you *do not provide it rules*, you use the training data to get the network to *work out rules for itself*.  
  > **username_0**: Except an RNN can't learn the rule "repeat last character." It can only learn 26 separate rules for each character. Which is kind of useless to learn template type things.  
  > **username_1**: Why limit yourself to letters?  You could certainly train it to look for and predict word sequences.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-01-28. Score: 1
  
  > I have thought of various ways of doing this. Such as, as well as having 26 outputs for letters of the alphabet have about 20 more for "repeat the character that is 14 characters to the left" and so on
  
  Creating a system of rules like your example above is the very opposite of training an RNN to perform this task.
  
  If you would like to train an RNN to answer simple questions you would not need to come up with ingenious rules, but with enough training data of the form
  
  ```
  question -> answer
  
  ```
  
  Then you could use one of many popular sequence-to-sequence NLP tools to try and learn this behaviour, effectively treating the problem as if it were machine translation between languages.
  
  ---
  
  More broadly, yes, question answering has been done before, it is very popular in fact. It is an active subfield of NLP research and many methods have been developed, some of them involving RNN networks.
  
  
  ---
  
  