Answers.Body: <p>Since most policies depend solely on actions and states/observations, then if you model the space of this field as the Cartesian Product of your state and action spaces, then the policy learns a surface over this combined space, similar to the way a field is parameterized.</p>
  <p>The policy an agent learns could exhibit the same behavior as the field you describe above (obstacles form an repulsive field, and goal(s) form an attractive field).  However, unlike the field described above, it is not guaranteed that the learned policy will capture this behavior - the policy learned depends on:</p>
  <ol>
  <li>The learning algorithm used</li>
  <li>The approximators (e.g. neural networks) used for learning, and their respective hyperparameters</li>
  <li>The formulation of the reward function</li>
  <li>The number of episodes/total steps the policy/agent is trained over.</li>
  </ol>
  <p>To sum this answer up, I believe you could train the policy in such a way (using the mechanisms above) such that it resembles the field you describe.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-29T09:18:16.767Z
Answers.Text: The field has much closer analogy with a value function than with a policy function. The policy is analagous to a solution found by using the field.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-29T14:10:49.077Z
Answers.Text: So isnt technically a policy a field surface, with the exception of the fact that field is more or less deterministic while policy is field + probability assigned to it every action in the field?
Answers.UserDisplayName: gfdsal
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-29T14:12:33.297Z
Answers.Text: @NeilSlater, if we merely assign probability to the field at each state, will then it be close analogy to the policy as policies are probabilistic transition from states to actions?
Answers.UserDisplayName: gfdsal
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-29T14:49:31.777Z
Answers.Text: @gfdsal: A potential field cannot directly represent probabilities. You might derive them from it, according to the local gradients of the field. Again that is a closer analogy to a value function used as a heuristic, not a policy function. A policy can be derived from a state value function in a similar way as a plan can be derived from an APF.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-29T00:53:40.260Z
Answers.LastActivityDate: 2020-07-29T00:53:40.260Z
Answers.UserDisplayName: Ryan S
Body: <p>If a policy maps states to actions in reinforcement learning, then for a path planning with obstacles, can't we simply use <a href="https://www.youtube.com/watch?v=r9FD7P76zJs" rel="nofollow noreferrer">Artificial Potential Field</a> fields for path planning and model policy mathematically as a field where the obstacles form repulsive field and goal form attractive field?</p>
  <p>So, technically, is a policy simply a field?</p>
  
Tags: <reinforcement-learning><markov-decision-process><policies><path-planning>
Title: Is a policy in reinforcement learning analogous to a field such as APF?
UserDisplayName: gfdsal
ThreadText: # Is a policy in reinforcement learning analogous to a field such as APF?
  
  (Asked by: username_0 on 2020-07-28)
  
  If a policy maps states to actions in reinforcement learning, then for a path planning with obstacles, can't we simply use [Artificial Potential Field](https://www.youtube.com/watch?v=r9FD7P76zJs) fields for path planning and model policy mathematically as a field where the obstacles form repulsive field and goal form attractive field?
  
  So, technically, is a policy simply a field?
  
  **Tags:** reinforcement-learning, markov-decision-process, policies, path-planning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-29. Score: 0 | Accepted answer
  
  Since most policies depend solely on actions and states/observations, then if you model the space of this field as the Cartesian Product of your state and action spaces, then the policy learns a surface over this combined space, similar to the way a field is parameterized.
  
  The policy an agent learns could exhibit the same behavior as the field you describe above (obstacles form an repulsive field, and goal(s) form an attractive field). However, unlike the field described above, it is not guaranteed that the learned policy will capture this behavior - the policy learned depends on:
  
  1. The learning algorithm used
  2. The approximators (e.g. neural networks) used for learning, and their respective hyperparameters
  3. The formulation of the reward function
  4. The number of episodes/total steps the policy/agent is trained over.
  
  To sum this answer up, I believe you could train the policy in such a way (using the mechanisms above) such that it resembles the field you describe.
  
  > **Comments:**
  > **username_2**: The field has much closer analogy with a value function than with a policy function. The policy is analagous to a solution found by using the field.  
  > **username_0**: So isnt technically a policy a field surface, with the exception of the fact that field is more or less deterministic while policy is field + probability assigned to it every action in the field?  
  > **username_0**: @username_2, if we merely assign probability to the field at each state, will then it be close analogy to the policy as policies are probabilistic transition from states to actions?  
  > **username_2**: @username_0: A potential field cannot directly represent probabilities. You might derive them from it, according to the local gradients of the field. Again that is a closer analogy to a value function used as a heuristic, not a policy function. A policy can be derived from a state value function in a similar way as a plan can be derived from an APF.  
  
  ---
  
  