  In this work, we develop a reinforcement learning protocol for a multiagent
coordination task in a discrete state and action space: an iterated prisoner's
dilemma game extended into a team based, winner-take all tournament, which
forces the agents to collude in order to maximize their reward. By disallowing
extra communication channels, the agents are forced to embed their coordination
strategy into their actions in the prisoner's dilemma game.
  We develop a representation of the iterated prisoners dilemma that makes it
amenable to Q-learning. We find that the reinforcement learning strategy is
able to consistently train agents that can win the winner take all iterated
prisoners dilemma tournament.
  By using a game with discrete state and action space, we are able to better
analyze and understand both the dynamics and the communication protocols that
are established between the agents. We find that the agents adapt a number of
interesting behaviors, such as the formation of benevolent dictators, that
minimize inequality of scores. We also find that the agents settle on a
remarkably consistent symbology in their actions, such that agents from
independent trials are able to collude with each other without further
training.

Computer Science and Game Theory