Answers.Body: <p>This is an idea that I used for my model - 
  Try using two RNN (GRU) Networks, one of them to manage current output state and the other to maintain context</p>
  
  <p>Say we are at timestamp <span class="math-container">$t$</span> and the two GRUCells are represented as <span class="math-container">$GRU_c$</span> and <span class="math-container">$GRU_s$</span> for GRU context network and state network. (Your output coming from the state network)</p>
  
  <p>At time stamp <span class="math-container">$t$</span> , the input <span class="math-container">$GRU_s(t) = concat(input, att(all~GRU_c~from ~[0, ~t-1]))$</span> where <span class="math-container">$att$</span> is an attention mechanism to give importance to specific parts of the conversation uptil that point (This is what maintains context)  and input <span class="math-container">$GRU_c(t) = learned~representation~of~GRU_s(t)$</span> , hence updating <span class="math-container">$GRU_c$</span> for that timestamp, which along with the historical information can be used for <span class="math-container">$GRU_s(t+1)$</span>  </p>
  
  <p>Hope this helped!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-26T10:28:55.117Z
Answers.LastActivityDate: 2019-07-26T10:28:55.117Z
Answers.UserDisplayName: ashenoy
Body: <p>I have created a chatbot by Keras based on movie dialog. I used RNN more specifically GRU . My bot can reply well. But the problem is , it can't hold the context . As an example if I say <code>Tell me a joke</code>, the bot will reply something , and then if I say <code>one more</code> , the bot simply doesn't understand that I was asking for another joke and many more similar cases, like if I used a slang against the bot , the bot will reply me with something similar , but if I just say something romantic or good immediately after using slang , the bot will reply to me with something good . I want to keep context or environment . How can I do so . Any lead would be helpful .</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-25T14:49:55.810Z
Comments.Text: Could you add a little more context about your current model? I would guess it is a seq2seq model that "translates" a phrase input by user into a reply phrase. The behaviour you notice is definitely a limitation of using seq2seq like that.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-25T16:57:17.617Z
Comments.Text: Yes u are right , I am using seq2seq . Is there any alternative which can solve my problem ?
Comments.UserDisplayName: Mithun Sarker Shuvro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-25T20:49:45.753Z
Comments.Text: There's lots of ways to make chatbots. You could ask about comparison between a couple of methods (but please don't ask for some full guide or review, it's  too much for a question here). Or your current question is OK in my opinion, if you use [edit] to add some details from your comment, so that people have some clue to what you've got compared to where you want to go next. I'm not a chatbot expert, but I think you are in the right place to ask such a question. I just think it would help you get the best answer if you say a little more about your current model.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-25T20:56:18.330Z
Comments.Text: It may also help if you explain your longer-term goal. Is your chatbot intended for some specific use? Or is the main purpose in building it to teach yourself topics in Natural Language Processing and AI? If the former, you may want to fall back on more robust simple methods than neural networks. If the latter, it may not be too important to you if the bot has flaws provided it exhibits some interesting behaviour such as assigning pronouns correctly between different exchanges at least some of the time
Comments.UserDisplayName: Neil Slater
Tags: <recurrent-neural-networks><keras><chat-bots>
Title: How can I keep context in my chatbot
UserDisplayName: Mithun Sarker Shuvro
ThreadText: # How can I keep context in my chatbot
  
  (Asked by: username_0 on 2019-07-25)
  
  I have created a chatbot by Keras based on movie dialog. I used RNN more specifically GRU . My bot can reply well. But the problem is , it can't hold the context . As an example if I say `Tell me a joke`, the bot will reply something , and then if I say `one more` , the bot simply doesn't understand that I was asking for another joke and many more similar cases, like if I used a slang against the bot , the bot will reply me with something similar , but if I just say something romantic or good immediately after using slang , the bot will reply to me with something good . I want to keep context or environment . How can I do so . Any lead would be helpful .
  
  **Tags:** recurrent-neural-networks, keras, chat-bots
  
  > **Comments:**
  > **username_1**: Could you add a little more context about your current model? I would guess it is a seq2seq model that "translates" a phrase input by user into a reply phrase. The behaviour you notice is definitely a limitation of using seq2seq like that.  
  > **username_0**: Yes u are right , I am using seq2seq . Is there any alternative which can solve my problem ?  
  > **username_1**: There's lots of ways to make chatbots. You could ask about comparison between a couple of methods (but please don't ask for some full guide or review, it's  too much for a question here). Or your current question is OK in my opinion, if you use [edit] to add some details from your comment, so that people have some clue to what you've got compared to where you want to go next. I'm not a chatbot expert, but I think you are in the right place to ask such a question. I just think it would help you get the best answer if you say a little more about your current model.  
  > **username_1**: It may also help if you explain your longer-term goal. Is your chatbot intended for some specific use? Or is the main purpose in building it to teach yourself topics in Natural Language Processing and AI? If the former, you may want to fall back on more robust simple methods than neural networks. If the latter, it may not be too important to you if the bot has flaws provided it exhibits some interesting behaviour such as assigning pronouns correctly between different exchanges at least some of the time  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-07-26. Score: 1
  
  This is an idea that I used for my model -
  Try using two RNN (GRU) Networks, one of them to manage current output state and the other to maintain context
  
  Say we are at timestamp $t$ and the two GRUCells are represented as $GRU\_c$ and $GRU\_s$ for GRU context network and state network. (Your output coming from the state network)
  
  At time stamp $t$ , the input $GRU\_s(t) = concat(input, att(all~GRU\_c~from ~[0, ~t-1]))$ where $att$ is an attention mechanism to give importance to specific parts of the conversation uptil that point (This is what maintains context) and input $GRU\_c(t) = learned~representation~of~GRU\_s(t)$ , hence updating $GRU\_c$ for that timestamp, which along with the historical information can be used for $GRU\_s(t+1)$
  
  Hope this helped!
  
  
  ---
  
  