Body: <p>Recently, some work has been done planning and learning in Non-Markovian Decision Processes, that is, decision-making with temporally extended rewards. In these settings, a particular reward is received only when a particular temporal logic formula is satisfied (LTL or CTL formula). However, I cannot find any work about <em>learning</em> which rewards correspond to which temporally extended behavior.</p>
  <p>In my searches, I came across k-order MDPs (which are non-Markovian). I did not find RL research done on k-order MDPs.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-07T18:11:39.207Z
Comments.Text: non-Markovian reward functions are quite old concepts. They were first introduced in the theory for control systems. You can take any objective fn. of a control system and apply planning and learning concepts to it.
Comments.UserDisplayName: abunickabhi
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-08T09:36:06.887Z
Comments.Text: By "reward functions", do you mean "value functions"?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-08T19:46:02.590Z
Comments.Text: I don't mean the value function. I am assuming that the environment/society knows the rewards to give, and the agent wants to learn a model of what sequence of states visited results in what rewards.
Comments.UserDisplayName: Gavin Rens
Tags: <reinforcement-learning><reference-request><reward-functions><markov-property>
Title: What research has been done on learning non-Markovian reward functions?
UserDisplayName: Gavin Rens
ThreadText: # What research has been done on learning non-Markovian reward functions?
  
  (Asked by: username_0 on 2019-04-07)
  
  Recently, some work has been done planning and learning in Non-Markovian Decision Processes, that is, decision-making with temporally extended rewards. In these settings, a particular reward is received only when a particular temporal logic formula is satisfied (LTL or CTL formula). However, I cannot find any work about *learning* which rewards correspond to which temporally extended behavior.
  
  In my searches, I came across k-order MDPs (which are non-Markovian). I did not find RL research done on k-order MDPs.
  
  **Tags:** reinforcement-learning, reference-request, reward-functions, markov-property
  
  > **Comments:**
  > **username_1**: non-Markovian reward functions are quite old concepts. They were first introduced in the theory for control systems. You can take any objective fn. of a control system and apply planning and learning concepts to it.  
  > **username_2**: By "reward functions", do you mean "value functions"?  
  > **username_0**: I don't mean the value function. I am assuming that the environment/society knows the rewards to give, and the agent wants to learn a model of what sequence of states visited results in what rewards.  
  
  ---
  
  ## 0 answers
  
  