Answers.Body: <p>Some of the domains in the <a href="https://ipc2018-probabilistic.bitbucket.io/" rel="nofollow noreferrer">International Probabilistic Planning Competition</a>, such as the <em>Wildlife Preserve</em> benchmark, fit quite well the constraints you have given. Note that the problems are modeled with a high-level declarative language, <a href="http://users.cecs.anu.edu.au/~ssanner/Papers/RDDL_Tutorial.pdf" rel="nofollow noreferrer">RDDL</a>. This means that you can define problems as big or as small as your heart desires with relative ease, since you can parametrize state description in terms of functions describing properties of an arbitrary number of objects.</p>
  
  <p>There's also a <a href="https://github.com/thiagopbueno/rddlgym" rel="nofollow noreferrer">quite useful</a> project that allows to instance OpenAI <code>gym</code> environments from the declarative description of the environment, states and actions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-20T04:38:47.467Z
Answers.Text: Thanks for your answer. I looked at the two papers listed under Wildlife Preserve in the page you linked to, but it seems neither is a multi-state game?
Answers.UserDisplayName: user76284
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-21T06:16:45.243Z
Answers.Text: From the definition of the instance (see [this gist](https://gist.github.com/miquelramirez/67d35b824c01e2919241c774b1dcf737), file `wildlife-preserve_inst_mdp__01.rddl`), and the definition of the state variables (see section `state-fluents` in the file `wildlife-preserve_01_mdp.rddl` posted in the gist above) the smallest instance in the suite has 2^4 x 4 x 4 x 2 = 512 states. 
  
  I recommend you too to play around with the `rddlgym` tool I linked in my answer, will make easier to come to grips with the syntax of RDDL.
Answers.UserDisplayName: Miquel Ramirez
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-19T03:21:43.203Z
Answers.LastActivityDate: 2020-02-19T03:21:43.203Z
Answers.UserDisplayName: Miquel Ramirez
Body: <p><a href="https://en.wikipedia.org/wiki/Stochastic_game" rel="nofollow noreferrer">SGs</a> are a generalization of <a href="https://en.wikipedia.org/wiki/Markov_decision_process" rel="nofollow noreferrer">MDPs</a> to multiple agents. Like this <a href="https://ai.stackexchange.com/questions/14246/benchmarks-for-reinforcement-learning-in-discrete-mdps">previous question</a> on MDPs, are there any interesting examples of zero-sum, discrete SGs—preferably with small state and action spaces? I'm hoping to use such examples as benchmarks, but couldn't find much in the literature. One example I can think of is a <a href="https://en.wikipedia.org/wiki/Pursuit-evasion" rel="nofollow noreferrer">pursuit-evasion game</a> on a graph.</p>
  
Tags: <game-theory><environment><markov-decision-process><benchmarks>
Title: Interesting examples of discrete stochastic games
UserDisplayName: user76284
ThreadText: # Interesting examples of discrete stochastic games
  
  (Asked by: username_0 on 2019-12-23)
  
  [SGs](https://en.wikipedia.org/wiki/Stochastic_game) are a generalization of [MDPs](https://en.wikipedia.org/wiki/Markov_decision_process) to multiple agents. Like this [previous question](https://ai.stackexchange.com/questions/14246/benchmarks-for-reinforcement-learning-in-discrete-mdps) on MDPs, are there any interesting examples of zero-sum, discrete SGs—preferably with small state and action spaces? I'm hoping to use such examples as benchmarks, but couldn't find much in the literature. One example I can think of is a [pursuit-evasion game](https://en.wikipedia.org/wiki/Pursuit-evasion) on a graph.
  
  **Tags:** game-theory, environment, markov-decision-process, benchmarks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-02-19. Score: 1 | Accepted answer
  
  Some of the domains in the [International Probabilistic Planning Competition](https://ipc2018-probabilistic.bitbucket.io/), such as the *Wildlife Preserve* benchmark, fit quite well the constraints you have given. Note that the problems are modeled with a high-level declarative language, [RDDL](http://users.cecs.anu.edu.au/~ssanner/Papers/RDDL_Tutorial.pdf). This means that you can define problems as big or as small as your heart desires with relative ease, since you can parametrize state description in terms of functions describing properties of an arbitrary number of objects.
  
  There's also a [quite useful](https://github.com/thiagopbueno/rddlgym) project that allows to instance OpenAI `gym` environments from the declarative description of the environment, states and actions.
  
  > **Comments:**
  > **username_0**: Thanks for your answer. I looked at the two papers listed under Wildlife Preserve in the page you linked to, but it seems neither is a multi-state game?  
  > **username_1**: From the definition of the instance (see [this gist](https://gist.github.com/username_1/67d35b824c01e2919241c774b1dcf737), file `wildlife-preserve_inst_mdp__01.rddl`), and the definition of the state variables (see section `state-fluents` in the file `wildlife-preserve_01_mdp.rddl` posted in the gist above) the smallest instance in the suite has 2^4 x 4 x 4 x 2 = 512 states.   I recommend you too to play around with the `rddlgym` tool I linked in my answer, will make easier to come to grips with the syntax of RDDL.  
  
  ---
  
  