Answers.Body: <p>The reduction of a Dec-POMDP to a centralized MDP that you mention is discussed at length in the technical report <a href="https://www.fransoliehoek.net/docs/Oliehoek14IASTR.pdf" rel="nofollow noreferrer">&quot;Dec-POMDPs as Non-Observable MDPs&quot;</a> by Oliehoek and Amato.</p>
  <p>Further background that can help understand the method can be found in <a href="https://www.ijcai.org/Proceedings/13/Papers/053.pdf" rel="nofollow noreferrer">&quot;Sufficient plan-time statistics for decentralized POMDPs&quot;</a> in IJCAI'13 also by Oliehoek.
  A related approach dealing with common/individual knowledge and information gathering is <a href="https://proceedings.neurips.cc/paper/2020/hash/9db6faeef387dc789777227a8bed4d52-Abstract.html" rel="nofollow noreferrer">&quot;Multi-agent active perception with prediction rewards&quot;</a> in NeurIPS'20 by Lauri and Oliehoek. (Disclaimer: I am one of the authors).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-22T14:55:32.070Z
Answers.LastActivityDate: 2022-07-22T14:55:32.070Z
Answers.UserDisplayName: mikkola
Body: <p>I want to ask about the intuition/where-to-look/what-to-try if I want to prove that an action value function optimal for a problem is also optimal for another reformulation of that smae problem. For example, a common approach to solve DEC-POMDP is to recast the problem into a common-knowledge-MDP. In this case (and in any similar case) What is the approach to follow in order to prove that the optimal action-value function optimal for the single agent reformulation is also optimal for the multi-agent formulation?</p>
  
Tags: <reinforcement-learning><markov-decision-process><decision-theory>
Title: How to prove that an action-value function optimal for one problem formulation is also optimal for another?
UserDisplayName: Souhaielrmx
ThreadText: # How to prove that an action-value function optimal for one problem formulation is also optimal for another?
  
  (Asked by: username_0 on 2022-07-22)
  
  I want to ask about the intuition/where-to-look/what-to-try if I want to prove that an action value function optimal for a problem is also optimal for another reformulation of that smae problem. For example, a common approach to solve DEC-POMDP is to recast the problem into a common-knowledge-MDP. In this case (and in any similar case) What is the approach to follow in order to prove that the optimal action-value function optimal for the single agent reformulation is also optimal for the multi-agent formulation?
  
  **Tags:** reinforcement-learning, markov-decision-process, decision-theory
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-22. Score: 0
  
  The reduction of a Dec-POMDP to a centralized MDP that you mention is discussed at length in the technical report ["Dec-POMDPs as Non-Observable MDPs"](https://www.fransoliehoek.net/docs/Oliehoek14IASTR.pdf) by Oliehoek and Amato.
  
  Further background that can help understand the method can be found in ["Sufficient plan-time statistics for decentralized POMDPs"](https://www.ijcai.org/Proceedings/13/Papers/053.pdf) in IJCAI'13 also by Oliehoek.
  A related approach dealing with common/individual knowledge and information gathering is ["Multi-agent active perception with prediction rewards"](https://proceedings.neurips.cc/paper/2020/hash/9db6faeef387dc789777227a8bed4d52-Abstract.html) in NeurIPS'20 by Lauri and Oliehoek. (Disclaimer: I am one of the authors).
  
  
  ---
  
  