Co-Optimization of Environment and Policies for Decentralized Multi-Agent Navigation
<|reference_start|>Co-Optimization of Environment and Policies for Decentralized Multi-Agent Navigation: This work views the multi-agent system and its surrounding environment as a co-evolving system, where the behavior of one affects the other. The goal is to take both agent actions and environment configurations as decision variables, and optimize these two components in a coordinated manner to improve some measure of interest. Towards this end, we consider the problem of decentralized multi-agent navigation in cluttered environments. By introducing two sub-objectives of multi-agent navigation and environment optimization, we propose an $\textit{agent-environment co-optimization}$ problem and develop a $\textit{coordinated algorithm}$ that alternates between these sub-objectives to search for an optimal synthesis of agent actions and obstacle configurations in the environment; ultimately, improving the navigation performance. Due to the challenge of explicitly modeling the relation between agents, environment and performance, we leverage policy gradient to formulate a model-free learning mechanism within the coordinated framework. A formal convergence analysis shows that our coordinated algorithm tracks the local minimum trajectory of an associated time-varying non-convex optimization problem. Extensive numerical results corroborate theoretical findings and show the benefits of co-optimization over baselines. Interestingly, the results also indicate that optimized environment configurations are able to offer structural guidance that is key to de-conflicting agents in motion.<|reference_end|>
@article{gao2024co-optimization,
    title={Co-Optimization of Environment and Policies for Decentralized
  Multi-Agent Navigation},
    author={Zhan Gao and Guang Yang and Amanda Prorok},
    journal={arXiv preprint arXiv:2403.14583},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.14583},
    primaryClass={cs.RO cs.LG cs.MA}
}