Answers.Body: <p><a href="https://groups.google.com/g/rl-list/c/k9jNdOk6NZk/m/KQWKfyHsBgAJ" rel="nofollow noreferrer">I've asked this on rl-list</a> and got lots of interesting references to research papers.</p>
  <p>So far the most promising one I've seen is this:</p>
  <p><a href="https://www.mircomusolesi.org/papers/aaai20.pdf" rel="nofollow noreferrer">Nicolas Anastassacos, Stephen Hailes and Mirco Musolesi. Partner Selection for the Emergence of Cooperation in Multi-Agent Systems using Reinforcement Learning. In AAAI 2020. New York City, NY, USA. February 2020.</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-12T17:55:41.477Z
Answers.Text: Maybe you can briefly explain why this paper is useful/promising.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-12T15:32:55.730Z
Answers.LastActivityDate: 2021-01-12T15:32:55.730Z
Answers.UserDisplayName: Ram Rachum
Body: <p>I've been working on research into reproducing social behavior using multi-agent reinforcement learning. My focus has been on <a href="https://github.com/cool-RR/grid_royale" rel="nofollow noreferrer">a GridWorld-style game</a>, but I was thinking that maybe a simpler Prisoner's Dilemma game could be a better approach. I tried to find existing research papers in this direction, but couldn't find any, so I'd like to describe what I'm looking for in case anyone here knows of such research.</p>
  <p><strong>I'm looking for research into scenarios where multiple RL agents are playing Iterated Prisoner's Dilemma with each other, and social behaviors emerge.</strong> Let me specify what I mean by &quot;social behaviors.&quot; Most research I've seen into RL/IPD (<a href="https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0188046" rel="nofollow noreferrer">example</a>) focuses on how to achieve the ideal strategy, and how to get there the fastest, and what common archetypes of strategies emerge. That is all nice and well, but not what I'm interested in.</p>
  <p>An agent executing a Tit-for-Tat strategy is giving positive reinforcement to the other player for &quot;good&quot; behavior, and negative reinforcement for &quot;bad&quot; behavior. That is why it wins. My key point here is that this carrot-and-stick method is done individually rather than in groups. <strong>I want to see it evolve within a group.</strong></p>
  <p>I want to see an entire group of agents evolve to punish and reward other players according to how they behaved with the group. I believe that fascinating group dynamics could be observed in that scenario.</p>
  <p>I programmed such a scenario a decade ago, but by writing an algorithm manually, not using deep RL. I want to do it using deep RL, but first I want to know whether there are existing attempts.</p>
  <p><strong>Does anyone know whether such research exists?</strong></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-06T15:06:13.000Z
Comments.Text: Is that https://arxiv.org/pdf/1803.00162.pdf maybe something you are looking for?
Comments.UserDisplayName: Daniel B.
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-06T15:31:36.863Z
Comments.Text: I've seen this research before, though I've only read the abstract, as I did again now. It seems to be talking about a continuum of choices between Defect and Cooperate, but that's not really what I'm looking for here.
Comments.UserDisplayName: Ram Rachum
Tags: <reinforcement-learning><q-learning><reference-request><research>
Title: Research into social behavior in Prisoner's Dilemma
UserDisplayName: Ram Rachum
ThreadText: # Research into social behavior in Prisoner's Dilemma
  
  (Asked by: username_0 on 2021-01-04)
  
  I've been working on research into reproducing social behavior using multi-agent reinforcement learning. My focus has been on [a GridWorld-style game](https://github.com/cool-RR/grid_royale), but I was thinking that maybe a simpler Prisoner's Dilemma game could be a better approach. I tried to find existing research papers in this direction, but couldn't find any, so I'd like to describe what I'm looking for in case anyone here knows of such research.
  
  **I'm looking for research into scenarios where multiple RL agents are playing Iterated Prisoner's Dilemma with each other, and social behaviors emerge.** Let me specify what I mean by "social behaviors." Most research I've seen into RL/IPD ([example](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0188046)) focuses on how to achieve the ideal strategy, and how to get there the fastest, and what common archetypes of strategies emerge. That is all nice and well, but not what I'm interested in.
  
  An agent executing a Tit-for-Tat strategy is giving positive reinforcement to the other player for "good" behavior, and negative reinforcement for "bad" behavior. That is why it wins. My key point here is that this carrot-and-stick method is done individually rather than in groups. **I want to see it evolve within a group.**
  
  I want to see an entire group of agents evolve to punish and reward other players according to how they behaved with the group. I believe that fascinating group dynamics could be observed in that scenario.
  
  I programmed such a scenario a decade ago, but by writing an algorithm manually, not using deep RL. I want to do it using deep RL, but first I want to know whether there are existing attempts.
  
  **Does anyone know whether such research exists?**
  
  **Tags:** reinforcement-learning, q-learning, reference-request, research
  
  > **Comments:**
  > **username_1**: Is that https://arxiv.org/pdf/1803.00162.pdf maybe something you are looking for?  
  > **username_0**: I've seen this research before, though I've only read the abstract, as I did again now. It seems to be talking about a continuum of choices between Defect and Cooperate, but that's not really what I'm looking for here.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-01-12. Score: 1
  
  [I've asked this on rl-list](https://groups.google.com/g/rl-list/c/k9jNdOk6NZk/m/KQWKfyHsBgAJ) and got lots of interesting references to research papers.
  
  So far the most promising one I've seen is this:
  
  [Nicolas Anastassacos, Stephen Hailes and Mirco Musolesi. Partner Selection for the Emergence of Cooperation in Multi-Agent Systems using Reinforcement Learning. In AAAI 2020. New York City, NY, USA. February 2020.](https://www.mircomusolesi.org/papers/aaai20.pdf)
  
  > **Comments:**
  > **username_2**: Maybe you can briefly explain why this paper is useful/promising.  
  
  ---
  
  