Body: <p>I understand the goals and purposes of RL in the case of a single agent and the underlying model, i.e. MDPs, for RL problems (or sequential decision making with uncertainty in general).</p>
  <p>My question is (and I know this will/may be subjective) are the indicators for choosing to model some decision making problem as a single agent, treating all other factors/noise as part of the environment (MDP or some variant of it) vs multiple agents (a stochastic/Markov game)?</p>
  <p>In the zero-sum/adversarial or pure cooperation setting where the goals of the agents conflict/assist each other, it is obvious that the multi-agent setting is the way to go. But suppose that there is no pure conflict/coordination.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-04-07T10:33:05.217Z
Comments.Text: Note that "Markov games" are already stochastic, so saying "stochastic Markov game" is a bit redundant. Markov games are sometimes called "stochastic games", so I changed "stochastic Markov game" to "stochastic/Markov game" in your post (because maybe that's what you meant originally anyway).
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-04-07T16:02:58.237Z
Comments.Text: Thank @nbro, I understand that Markov games are stochastic but not all stochastic games are Markovian. I was referring to both cases where the Markov property may or may not apply. I should think there may be tricks for making general stochastic games (approximately) Markovian tho.
Comments.UserDisplayName: David
Tags: <reinforcement-learning><comparison><game-theory><multi-agent-rl>
Title: When to model decision-making problem as single agent vs multi-agent problem?
UserDisplayName: David
ThreadText: # When to model decision-making problem as single agent vs multi-agent problem?
  
  (Asked by: username_0 on 2022-04-05)
  
  I understand the goals and purposes of RL in the case of a single agent and the underlying model, i.e. MDPs, for RL problems (or sequential decision making with uncertainty in general).
  
  My question is (and I know this will/may be subjective) are the indicators for choosing to model some decision making problem as a single agent, treating all other factors/noise as part of the environment (MDP or some variant of it) vs multiple agents (a stochastic/Markov game)?
  
  In the zero-sum/adversarial or pure cooperation setting where the goals of the agents conflict/assist each other, it is obvious that the multi-agent setting is the way to go. But suppose that there is no pure conflict/coordination.
  
  **Tags:** reinforcement-learning, comparison, game-theory, multi-agent-rl
  > **Comments:**
  > **username_1**: Note that "Markov games" are already stochastic, so saying "stochastic Markov game" is a bit redundant. Markov games are sometimes called "stochastic games", so I changed "stochastic Markov game" to "stochastic/Markov game" in your post (because maybe that's what you meant originally anyway).  
  > **username_0**: Thank @nbro, I understand that Markov games are stochastic but not all stochastic games are Markovian. I was referring to both cases where the Markov property may or may not apply. I should think there may be tricks for making general stochastic games (approximately) Markovian tho.  
  
  ---
  
  ## 0 answers
  
  