Body: <p>In policy gradients, is it possible to learn the policy if the chain of actions is selected and performed manually/externally (e.g. by myself or by someone else who I have no influence over)?</p>
  <p>For example, we have four actions, and I choose in the beginning an action 2, and we end up in a given state, then I choose action 4 and we end up in another state, etc. (the actions can follow some logic or not but the question is general; some of the actions will end up with positive rewards).</p>
  <p>Can we learn any meaningful policy network from such a chain of actions?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-13T13:09:55.950Z
Comments.Text: Hello. It may be a good idea to explain why you would like to do this, and how exactly do you plan to choose the actions.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-13T15:45:03.933Z
Comments.Text: It seems that what you're proposing is related to [imitation learning](https://ai.stackexchange.com/q/9595/2444), but maybe I am wrong, as imitation learning is really not reinforcement learning but supervised learning applied to an RL problem. You're suggesting that humans provide/specify the behavioural policy. I don't remember now all the details of policy gradients to answer this question, but, in the case of Q-learning (which is off-policy), you could in principle use any exploratory policy that explores enough the environment, so even a policy decided by a human.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><policy-gradients>
Title: Can we learn a policy network via a sequence of manually determined actions?
UserDisplayName: cerebrou
ThreadText: # Can we learn a policy network via a sequence of manually determined actions?
  
  (Asked by: username_0 on 2021-10-13)
  
  In policy gradients, is it possible to learn the policy if the chain of actions is selected and performed manually/externally (e.g. by myself or by someone else who I have no influence over)?
  
  For example, we have four actions, and I choose in the beginning an action 2, and we end up in a given state, then I choose action 4 and we end up in another state, etc. (the actions can follow some logic or not but the question is general; some of the actions will end up with positive rewards).
  
  Can we learn any meaningful policy network from such a chain of actions?
  
  **Tags:** reinforcement-learning, policy-gradients
  
  > **Comments:**
  > **username_1**: Hello. It may be a good idea to explain why you would like to do this, and how exactly do you plan to choose the actions.  
  > **username_1**: It seems that what you're proposing is related to [imitation learning](https://ai.stackexchange.com/q/9595/2444), but maybe I am wrong, as imitation learning is really not reinforcement learning but supervised learning applied to an RL problem. You're suggesting that humans provide/specify the behavioural policy. I don't remember now all the details of policy gradients to answer this question, but, in the case of Q-learning (which is off-policy), you could in principle use any exploratory policy that explores enough the environment, so even a policy decided by a human.  
  
  ---
  
  ## 0 answers
  
  