Answers.Body: <p><strong>Policy gradient</strong> methods are popular in reinforcement learning because they are fast and easy to implement. Additionally, policy gradient methods often work well for simple problems. for example, if the <a href="https://en.wikipedia.org/wiki/Hellinger_distance#:%7E:text=The%20Hellinger%20distance%20forms%20a,positive%20probability%2C%20and%20vice%20versa." rel="nofollow noreferrer">Hellinger distance</a> between two measures is small.</p>
  <p><strong>Dual LP</strong> methods may be more accurate if the problem is not too simple. However, they can be more difficult to implement and may require more computational resources. they are preferable if you have more information about the underlying distribution of the data.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-16T21:59:17.210Z
Answers.LastActivityDate: 2022-10-16T21:59:17.210Z
Answers.UserDisplayName: Faizy
Body: <p>Why are policy gradient methods popular in reinforcement learning when there exists a dual LP formulation in terms of occupation measures that can be solved easily?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-13T11:33:15.183Z
Comments.Text: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
Comments.UserDisplayName: Community
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-12-31T12:34:55.540Z
Comments.Text: Can you please clarify what you mean by "dual LP formulation in terms of occupation measures". I know what linear programming is, but what do you mean by "occupation measures" and "dual LP formulation in terms of occupation measures"?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-03T05:17:02.983Z
Comments.Text: Occupation measure refers to an MDP's discounted state-action visitation frequency under a given policy. A detailed explanation of how this can be used for obtaining policies and the corresponding dual formulation, you can refer to Section 6.9 Linear Programming in the book Markov Decision Processes(1994) by Martin L. Puterman.
Comments.UserDisplayName: blackbird_h71
Tags: <reinforcement-learning><policy-gradients><linear-programming>
Title: Why are policy gradients popular in RL when there exists a dual LP formulation in terms of occupation measures that can be solved easily?
UserDisplayName: blackbird_h71
ThreadText: # Why are policy gradients popular in RL when there exists a dual LP formulation in terms of occupation measures that can be solved easily?
  
  (Asked by: username_0 on 2022-10-13)
  
  Why are policy gradient methods popular in reinforcement learning when there exists a dual LP formulation in terms of occupation measures that can be solved easily?
  
  **Tags:** reinforcement-learning, policy-gradients, linear-programming
  
  > **Comments:**
  > **username_1**: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.  
  > **username_2**: Can you please clarify what you mean by "dual LP formulation in terms of occupation measures". I know what linear programming is, but what do you mean by "occupation measures" and "dual LP formulation in terms of occupation measures"?  
  > **username_0**: Occupation measure refers to an MDP's discounted state-action visitation frequency under a given policy. A detailed explanation of how this can be used for obtaining policies and the corresponding dual formulation, you can refer to Section 6.9 Linear Programming in the book Markov Decision Processes(1994) by Martin L. Puterman.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2022-10-16. Score: 0
  
  **Policy gradient** methods are popular in reinforcement learning because they are fast and easy to implement. Additionally, policy gradient methods often work well for simple problems. for example, if the [Hellinger distance](https://en.wikipedia.org/wiki/Hellinger_distance#:%7E:text=The%20Hellinger%20distance%20forms%20a,positive%20probability%2C%20and%20vice%20versa.) between two measures is small.
  
  **Dual LP** methods may be more accurate if the problem is not too simple. However, they can be more difficult to implement and may require more computational resources. they are preferable if you have more information about the underlying distribution of the data.
  
  
  ---
  
  