Answers.Body: <p>Seems like recurrent neural networks (RNN) should work for your use case. An excellent introduction is available at: <a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/" rel="nofollow noreferrer">The Unreasonable Effectiveness of Recurrent Neural Networks</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-26T23:34:34.057Z
Answers.LastActivityDate: 2018-01-26T23:34:34.057Z
Answers.UserDisplayName: Adnan S
Body: <p>I'm relatively new to AI, and I've tried to create one that "speaks". Here's how it works:<br><br>
  1. Get training data e.g 'Jim ran to the shop to buy candy'<br>
  2. The data gets split into overlapping 'chains' of three e.g ['Jim ran to', 'ran to the', 'to the shop', 'the shop to'...]<br>
  3. User enters two words<br>
  4. Looks through the chains to find if the two words have been seen before.<br>
  5. If they have, finds out which word followed it and how many times.<br>
  6. Work out the probability e.g: if 'or' followed the two words 3 times, 'because' followed the two words 1 time and 'but' followed it 1 time it would be 0.6, 0.2 and 0.2<br>
  7. Generate a random decimal<br>
  8. If the random decimal is in the range of the first word (0 - 0.6) pick that one or if it's in the range of the second word (0.6 - 0.8) pick that word or if it's in the range of the third (0.8 - 1) pick that word<br>
  9. Output the word picked<br>
  10. Repeat from 4 but with the new last two words e.g if the last words had been 'to be' and it picked 'or' the new last two words would be 'be or'.</p>
  
  <p>It does work, but it doesn't stick to a particular topic. For example, after training with 800 random Wikipedia articles:</p>
  
  <blockquote>
    <p>In the early 1990s the frequency had a plastic pickguard and separate hardtail bridge with the council hoped that the bullet one replaced with the goal of educating the next orders could revert to the north island or string of islands in a new urban zone close to the west.</p>
  </blockquote>
  
  <p>As you can see the topic changes many times mid-sentence. I thought of increasing the number of words it considered from two to three or four, but I thought it might start simply quoting the articles. If I'm wrong please tell me.</p>
  
  <p>Any help is greatly appreciated. If I haven't explained clearly enough or you have any questions please ask.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-26T20:41:59.920Z
Comments.Text: Welcome to AI!  *(That is very elegant output, imo, nonsensicality aside;)*
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-26T21:10:31.387Z
Comments.Text: How come you pick one at "random"? It seems like that be less accurate than just picking the one with the highest decimal
Comments.UserDisplayName: Emil Wormbs
Tags: <natural-language-processing><probability>
Title: How can I improve this word-prediction AI?
UserDisplayName: user117279
ThreadText: # How can I improve this word-prediction AI?
  
  (Asked by: username_0 on 2018-01-26)
  
  I'm relatively new to AI, and I've tried to create one that "speaks". Here's how it works:  
    
  1. Get training data e.g 'Jim ran to the shop to buy candy'  
  2. The data gets split into overlapping 'chains' of three e.g ['Jim ran to', 'ran to the', 'to the shop', 'the shop to'...]  
  3. User enters two words  
  4. Looks through the chains to find if the two words have been seen before.  
  5. If they have, finds out which word followed it and how many times.  
  6. Work out the probability e.g: if 'or' followed the two words 3 times, 'because' followed the two words 1 time and 'but' followed it 1 time it would be 0.6, 0.2 and 0.2  
  7. Generate a random decimal  
  8. If the random decimal is in the range of the first word (0 - 0.6) pick that one or if it's in the range of the second word (0.6 - 0.8) pick that word or if it's in the range of the third (0.8 - 1) pick that word  
  9. Output the word picked  
  10. Repeat from 4 but with the new last two words e.g if the last words had been 'to be' and it picked 'or' the new last two words would be 'be or'.
  
  It does work, but it doesn't stick to a particular topic. For example, after training with 800 random Wikipedia articles:
  
  > In the early 1990s the frequency had a plastic pickguard and separate hardtail bridge with the council hoped that the bullet one replaced with the goal of educating the next orders could revert to the north island or string of islands in a new urban zone close to the west.
  
  As you can see the topic changes many times mid-sentence. I thought of increasing the number of words it considered from two to three or four, but I thought it might start simply quoting the articles. If I'm wrong please tell me.
  
  Any help is greatly appreciated. If I haven't explained clearly enough or you have any questions please ask.
  
  **Tags:** natural-language-processing, probability
  > **Comments:**
  > **username_1**: Welcome to AI!  *(That is very elegant output, imo, nonsensicality aside;)*  
  > **username_2**: How come you pick one at "random"? It seems like that be less accurate than just picking the one with the highest decimal  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2018-01-26. Score: 1 | Accepted answer
  
  Seems like recurrent neural networks (RNN) should work for your use case. An excellent introduction is available at: [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)
  
  
  ---
  
  