Body: <p>I've been reading the <a href="http://carlosdiuk.github.io/papers/OORL.pdf" rel="nofollow noreferrer">attached paper</a> - which aims to model entities in the world as objects, including the learning agent itself! </p>
  
  <p>To say the least, the goal is to navigate through what seems like a maze (path-planning problem) - and drop off passengers in desired destinations, while avoiding walls in the map of the world (5x5 grid for now). The objects involved are, a taxi, passengers, walls and a destination. </p>
  
  <p>Now, a particular paragraph says the following:</p>
  
  <blockquote>
    <p>"Whereas in the classical MDP
    model, the effect of encountering walls is felt as a property of specific locations in the grid, the OO-MDP view is that wall interactions are the same regardless of their location. As such, agents’ experience can transfer gracefully throughout the state space."</p>
  </blockquote>
  
  <p>What does this mean? <strong>How are the classical MDP and the object-oriented MDP views different?</strong> </p>
  
  <p>I can't make sense of the above extract, at all. Any help would be appreciated!</p>
  
  <p>P.S. I did not consider posting parts of the extract as separate questions since my problem has more to do with understanding the extract as a whole which inevitably relies on understanding the parts. </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-28T19:08:07.257Z
Comments.Text: Maybe it doesn't define the state in terms of the location but instead the local properties. Thus the mdp only considers the agents past local properties. This kind of makes sense.
Comments.UserDisplayName: FourierFlux
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-28T20:40:59.707Z
Comments.Text: it should be also noted this approach will always result in a finite horizon solution, you can't do global planning with it since it's always operating locally,
Comments.UserDisplayName: FourierFlux
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-29T03:31:52.130Z
Comments.Text: Could you elaborate a bit on what you mentioned in the comments, @FourierFlux? Also, what is meant by "local properties"? How does it help in making sense of the line - "wall interactions are same regardless of location" and "agents' experience can transfer gracefully throughout the state space"?
Comments.UserDisplayName: stoic-santiago
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-29T05:17:07.877Z
Comments.Text: The state space im assuming only consists of what the agent sees and directly interacts with.
Comments.UserDisplayName: FourierFlux
Tags: <reinforcement-learning><comparison><markov-decision-process><papers>
Title: How are the classical MDP and the object-oriented MDP views different?
UserDisplayName: stoic-santiago
ThreadText: # How are the classical MDP and the object-oriented MDP views different?
  
  (Asked by: username_0 on 2020-04-28)
  
  I've been reading the [attached paper](http://carlosdiuk.github.io/papers/OORL.pdf) - which aims to model entities in the world as objects, including the learning agent itself!
  
  To say the least, the goal is to navigate through what seems like a maze (path-planning problem) - and drop off passengers in desired destinations, while avoiding walls in the map of the world (5x5 grid for now). The objects involved are, a taxi, passengers, walls and a destination.
  
  Now, a particular paragraph says the following:
  
  > "Whereas in the classical MDP
  > model, the effect of encountering walls is felt as a property of specific locations in the grid, the OO-MDP view is that wall interactions are the same regardless of their location. As such, agents’ experience can transfer gracefully throughout the state space."
  
  What does this mean? **How are the classical MDP and the object-oriented MDP views different?**
  
  I can't make sense of the above extract, at all. Any help would be appreciated!
  
  P.S. I did not consider posting parts of the extract as separate questions since my problem has more to do with understanding the extract as a whole which inevitably relies on understanding the parts.
  
  **Tags:** reinforcement-learning, comparison, markov-decision-process, papers
  
  > **Comments:**
  > **username_1**: Maybe it doesn't define the state in terms of the location but instead the local properties. Thus the mdp only considers the agents past local properties. This kind of makes sense.  
  > **username_1**: it should be also noted this approach will always result in a finite horizon solution, you can't do global planning with it since it's always operating locally,  
  > **username_0**: Could you elaborate a bit on what you mentioned in the comments, @username_1? Also, what is meant by "local properties"? How does it help in making sense of the line - "wall interactions are same regardless of location" and "agents' experience can transfer gracefully throughout the state space"?  
  > **username_1**: The state space im assuming only consists of what the agent sees and directly interacts with.  
  
  ---
  
  ## 0 answers
  
  