Putting Data at the Centre of Offline Multi-Agent Reinforcement Learning
Offline multi-agent reinforcement learning (MARL) is an exciting direction of
research that uses static datasets to find optimal control policies for
multi-agent systems. Though the field is by definition data-driven, efforts
have thus far neglected data in their drive to achieve state-of-the-art
results. We first substantiate this claim by surveying the literature, showing
how the majority of works generate their own datasets without consistent
methodology and provide sparse information about the characteristics of these
datasets. We then show why neglecting the nature of the data is problematic,
through salient examples of how tightly algorithmic performance is coupled to
the dataset used, necessitating a common foundation for experiments in the
field. In response, we take a big step towards improving data usage and data
awareness in offline MARL, with three key contributions: (1) a clear guideline
for generating novel datasets; (2) a standardisation of over 80 existing
datasets, hosted in a publicly available repository, using a consistent storage
format and easy-to-use API; and (3) a suite of analysis tools that allow us to
understand these datasets better, aiding further development.
66ec6edce65d94c88ebb0543
Claude Formanek
extracted_confirmed
2024-09-20T12:40:36.989Z
6638b36afbc5d09216e016f2
/avatars/f2832db5bd602b0e495d1326b90f3ecc.svg
Claude Formanek
user
claudef
66ec6edce65d94c88ebb0544
Louise Beyers
extracted_pending
2024-09-19T18:35:09.468Z
6638bbea8bd9205c326e8553
/avatars/4bb695a8f51e6ba17448c748280ebf14.svg
Louise Beyers
user
LBey
66ec6edce65d94c88ebb0545
Callum Rhys Tilbury
extracted_pending
2024-09-19T18:35:09.468Z
66840525502121ba24f85f13
https://cdn-avatars.huggingface.co/v1/production/uploads/66840525502121ba24f85f13/VFfRo27chLAzWCYyGoXJt.jpeg
Callum Rhys Tilbury
user
callumtilbury
66ec6edce65d94c88ebb0546
Jonathan P. Shock
66ec6edce65d94c88ebb0547
Arnu Pretorius
claimed_verified
2024-09-20T07:46:03.083Z
63c03b1fdd793d5a62836bca
https://cdn-avatars.huggingface.co/v1/production/uploads/63c03b1fdd793d5a62836bca/OphKA907XmjJCMB6OmD6x.jpeg
Arnu Pretorius
user
Arnu
Putting Data at the Centre of Offline Multi-Agent Reinforcement Learning
Offline multi-agent reinforcement learning (MARL) is an exciting direction of
research that uses static datasets to find optimal control policies for
multi-agent systems. Though the field is by definition data-driven, efforts
have thus far neglected data in their drive to achieve state-of-the-art
results. We first substantiate this claim by surveying the literature, showing
how the majority of works generate their own datasets without consistent
methodology and provide sparse information about the characteristics of these
datasets. We then show why neglecting the nature of the data is problematic,
through salient examples of how tightly algorithmic performance is coupled to
the dataset used, necessitating a common foundation for experiments in the
field. In response, we take a big step towards improving data usage and data
awareness in offline MARL, with three key contributions: (1) a clear guideline
for generating novel datasets; (2) a standardisation of over 80 existing
datasets, hosted in a publicly available repository, using a consistent storage
format and easy-to-use API; and (3) a suite of analysis tools that allow us to
understand these datasets better, aiding further development.
Research in offline multi-agent reinforcement learning lacks a standardized approach to data generation and usage, which is addressed by providing a guideline, standardization, and analysis tools for datasets.
multi-agent reinforcement learning offline MARL static datasets optimal control policies data-driven data methodology dataset characteristics algorithmic performance dataset usage data awareness dataset generation dataset repository storage format API analysis tools