Environment Complexity and Nash Equilibria in a Sequential Social Dilemma
<|reference_start|>Environment Complexity and Nash Equilibria in a Sequential Social Dilemma: Multi-agent reinforcement learning (MARL) methods, while effective in zero-sum or positive-sum games, often yield suboptimal outcomes in general-sum games where cooperation is essential for achieving globally optimal outcomes. Matrix game social dilemmas, which abstract key aspects of general-sum interactions, such as cooperation, risk, and trust, fail to model the temporal and spatial dynamics characteristic of real-world scenarios. In response, our study extends matrix game social dilemmas into more complex, higher-dimensional MARL environments. We adapt a gridworld implementation of the Stag Hunt dilemma to more closely match the decision-space of a one-shot matrix game while also introducing variable environment complexity. Our findings indicate that as complexity increases, MARL agents trained in these environments converge to suboptimal strategies, consistent with the risk-dominant Nash equilibria strategies found in matrix games. Our work highlights the impact of environment complexity on achieving optimal outcomes in higher-dimensional game-theoretic MARL environments.<|reference_end|>
@article{yasir2024environment,
    title={Environment Complexity and Nash Equilibria in a Sequential Social
  Dilemma},
    author={Mustafa Yasir, Andrew Howes, Vasilios Mavroudis, Chris Hicks},
    journal={arXiv preprint arXiv:2408.02148},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.02148},
    primaryClass={cs.GT cs.AI cs.MA}
}