Answers.Body: <p>You can use LSTM in reinforcement learning, of course. You don't give actions to the agent, it doesn't work like that.</p>
  
  <p>The agent give actions to your MDP and you must return proper reward in order to teach the agent. For example if you implement trading bot, the policy(policy=the agent, which is your LSTM network) will say that at step T it is going to have action 34, which means something to your MDP and you return reward for example -0.03 or +0.05 or whatever depending what that actions is doing at the moment T. </p>
  
  <p>So I get the question like you want to do a supervised learning on a reinforcement learning environment.</p>
  
  <p>You can mimic supervised learning as well, but the idea of reinforcement learning is not that.</p>
  
  <p>Here is how to mimic:</p>
  
  <p>Scenario: you are at step T, lets say you have 3 possible actions -1,0,+1;</p>
  
  <p>In a supervised learning you must give the desired action to the learning process.
  In reinforcement learning you must give reward based on if you are happy or not from the agent's action.</p>
  
  <p>So you must have predefined that for -1 you are not happy and you give reward 0.0, for action 0 you are not happy and you give reward 0.0 and for action +1 you are happy and you give reward +100;</p>
  
  <p>I hope this makes things clear.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-20T18:27:26.077Z
Answers.Text: Here's a question on your penultimate paragraph, I can't understand one thing, lstm gives out 3 actions, and how to me reward to give the neural network, the fourth output parameter or something?
Answers.UserDisplayName: alex-rudenkiy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-21T22:25:05.507Z
Answers.Text: Its not that simple. You need a framework that makes everything for you. Otherwise you will need to reinvent the wheel. First value of the output is called "value" and should be with activation "identity" function. This is how the reinforcement learning works. Sometimes it can be made of many networks, not just one. I use computational graph from dl4j framework. Many reinforcement learning introduce the notion of `value-function` which often denoted as V(s) . The value function represent how good is a state for an agent to be in.
Answers.UserDisplayName: Borislav Markov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-21T22:34:53.180Z
Answers.Text: I see you are interested in keras and python. Take a look at this framework :https://github.com/keras-rl/  and the answer to your question is at this row, look how the author has put first output as V(s;theta) and then comes all the actions as one hot vector: https://github.com/keras-rl/keras-rl/blob/master/rl/agents/dqn.py#L122
Answers.UserDisplayName: Borislav Markov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-18T16:18:49.670Z
Answers.LastActivityDate: 2019-09-18T16:18:49.670Z
Answers.UserDisplayName: Borislav Markov
Body: <p>Please tell me that is the LSTM network for the problem of reinforcement learning, as I explain to her what she will get the reward of a prediction, because the output will contain only actions?</p>
  
  <p>Well, well, let's say at first I can play and upload my actions to training so that she sees which actions are right, that is, which she should strive for, but how to make her learn relatively independently?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-08-13T21:22:04.537Z
Comments.Text: Hi Alex and welcome to this community! We're glad to help. However, it is not clear what you're asking. Can you please write your question clearly in your native language and then use e.g. [https://www.deepl.com/translator](https://www.deepl.com/translator) or Google Translate to translate your question to English, then edit your current post with that translation?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-08-14T19:39:33.727Z
Comments.Text: Also useful to translate, re-translate back, then re-retranslate to determine how much of the content is being translated accurately.
Comments.UserDisplayName: DukeZhou
Tags: <reinforcement-learning><keras><long-short-term-memory>
Title: LSTM in reinforcement learning
UserDisplayName: alex-rudenkiy
ThreadText: # LSTM in reinforcement learning
  
  (Asked by: username_0 on 2019-08-13)
  
  Please tell me that is the LSTM network for the problem of reinforcement learning, as I explain to her what she will get the reward of a prediction, because the output will contain only actions?
  
  Well, well, let's say at first I can play and upload my actions to training so that she sees which actions are right, that is, which she should strive for, but how to make her learn relatively independently?
  
  **Tags:** reinforcement-learning, keras, long-short-term-memory
  > **Comments:**
  > **username_1**: Hi Alex and welcome to this community! We're glad to help. However, it is not clear what you're asking. Can you please write your question clearly in your native language and then use e.g. [https://www.deepl.com/translator](https://www.deepl.com/translator) or Google Translate to translate your question to English, then edit your current post with that translation?  
  > **username_2**: Also useful to translate, re-translate back, then re-retranslate to determine how much of the content is being translated accurately.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-09-18. Score: 1
  
  You can use LSTM in reinforcement learning, of course. You don't give actions to the agent, it doesn't work like that.
  
  The agent give actions to your MDP and you must return proper reward in order to teach the agent. For example if you implement trading bot, the policy(policy=the agent, which is your LSTM network) will say that at step T it is going to have action 34, which means something to your MDP and you return reward for example -0.03 or +0.05 or whatever depending what that actions is doing at the moment T.
  
  So I get the question like you want to do a supervised learning on a reinforcement learning environment.
  
  You can mimic supervised learning as well, but the idea of reinforcement learning is not that.
  
  Here is how to mimic:
  
  Scenario: you are at step T, lets say you have 3 possible actions -1,0,+1;
  
  In a supervised learning you must give the desired action to the learning process.
  In reinforcement learning you must give reward based on if you are happy or not from the agent's action.
  
  So you must have predefined that for -1 you are not happy and you give reward 0.0, for action 0 you are not happy and you give reward 0.0 and for action +1 you are happy and you give reward +100;
  
  I hope this makes things clear.
  
  > **Comments:**
  > **username_0**: Here's a question on your penultimate paragraph, I can't understand one thing, lstm gives out 3 actions, and how to me reward to give the neural network, the fourth output parameter or something?  
  > **username_3**: Its not that simple. You need a framework that makes everything for you. Otherwise you will need to reinvent the wheel. First value of the output is called "value" and should be with activation "identity" function. This is how the reinforcement learning works. Sometimes it can be made of many networks, not just one. I use computational graph from dl4j framework. Many reinforcement learning introduce the notion of `value-function` which often denoted as V(s) . The value function represent how good is a state for an agent to be in.  
  > **username_3**: I see you are interested in keras and python. Take a look at this framework :https://github.com/keras-rl/  and the answer to your question is at this row, look how the author has put first output as V(s;theta) and then comes all the actions as one hot vector: https://github.com/keras-rl/keras-rl/blob/master/rl/agents/dqn.py#L122  
  
  ---
  
  