<p>I understand the goals and purposes of RL in the case of a single agent and the underlying model, i.e. MDPs, for RL problems (or sequential decision making with uncertainty in general).</p>
<p>My question is (and I know this will/may be subjective) are the indicators for choosing to model some decision making problem as a single agent, treating all other factors/noise as part of the environment (MDP or some variant of it) vs multiple agents (a stochastic/Markov game)?</p>
<p>In the zero-sum/adversarial or pure cooperation setting where the goals of the agents conflict/assist each other, it is obvious that the multi-agent setting is the way to go. But suppose that there is no pure conflict/coordination.</p>

CC BY-SA 4.0
2022-04-07T10:33:05.217Z
Note that "Markov games" are already stochastic, so saying "stochastic Markov game" is a bit redundant. Markov games are sometimes called "stochastic games", so I changed "stochastic Markov game" to "stochastic/Markov game" in your post (because maybe that's what you meant originally anyway).
nbro
CC BY-SA 4.0
2022-04-07T16:02:58.237Z
Thank @nbro, I understand that Markov games are stochastic but not all stochastic games are Markovian. I was referring to both cases where the Markov property may or may not apply. I should think there may be tricks for making general stochastic games (approximately) Markovian tho.
David
<reinforcement-learning><comparison><game-theory><multi-agent-rl>
When to model decision-making problem as single agent vs multi-agent problem?
David
# When to model decision-making problem as single agent vs multi-agent problem?

(Asked by: username_0 on 2022-04-05)

I understand the goals and purposes of RL in the case of a single agent and the underlying model, i.e. MDPs, for RL problems (or sequential decision making with uncertainty in general).

My question is (and I know this will/may be subjective) are the indicators for choosing to model some decision making problem as a single agent, treating all other factors/noise as part of the environment (MDP or some variant of it) vs multiple agents (a stochastic/Markov game)?

In the zero-sum/adversarial or pure cooperation setting where the goals of the agents conflict/assist each other, it is obvious that the multi-agent setting is the way to go. But suppose that there is no pure conflict/coordination.

**Tags:** reinforcement-learning, comparison, game-theory, multi-agent-rl

> **Comments:**
> **username_1**: Note that "Markov games" are already stochastic, so saying "stochastic Markov game" is a bit redundant. Markov games are sometimes called "stochastic games", so I changed "stochastic Markov game" to "stochastic/Markov game" in your post (because maybe that's what you meant originally anyway).  
> **username_0**: Thank @username_1, I understand that Markov games are stochastic but not all stochastic games are Markovian. I was referring to both cases where the Markov property may or may not apply. I should think there may be tricks for making general stochastic games (approximately) Markovian tho.  

---

## 0 answers

