A Meta-Game Evaluation Framework for Deep Multiagent Reinforcement Learning
<|reference_start|>A Meta-Game Evaluation Framework for Deep Multiagent Reinforcement Learning: Evaluating deep multiagent reinforcement learning (MARL) algorithms is complicated by stochasticity in training and sensitivity of agent performance to the behavior of other agents. We propose a meta-game evaluation framework for deep MARL, by framing each MARL algorithm as a meta-strategy, and repeatedly sampling normal-form empirical games over combinations of meta-strategies resulting from different random seeds. Each empirical game captures both self-play and cross-play factors across seeds. These empirical games provide the basis for constructing a sampling distribution, using bootstrapping, over a variety of game analysis statistics. We use this approach to evaluate state-of-the-art deep MARL algorithms on a class of negotiation games. From statistics on individual payoffs, social welfare, and empirical best-response graphs, we uncover strategic relationships among self-play, population-based, model-free, and model-based MARL methods.We also investigate the effect of run-time search as a meta-strategy operator, and find via meta-game analysis that the search version of a meta-strategy generally leads to improved performance.<|reference_end|>
@article{li2024a,
    title={A Meta-Game Evaluation Framework for Deep Multiagent Reinforcement
  Learning},
    author={Zun Li and Michael P. Wellman},
    journal={arXiv preprint arXiv:2405.00243},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.00243},
    primaryClass={cs.MA cs.GT}
}