ActFormer: Scalable Collaborative Perception via Active Queries
<|reference_start|>ActFormer: Scalable Collaborative Perception via Active Queries: Collaborative perception leverages rich visual observations from multiple robots to extend a single robot's perception ability beyond its field of view. Many prior works receive messages broadcast from all collaborators, leading to a scalability challenge when dealing with a large number of robots and sensors. In this work, we aim to address \textit{scalable camera-based collaborative perception} with a Transformer-based architecture. Our key idea is to enable a single robot to intelligently discern the relevance of the collaborators and their associated cameras according to a learned spatial prior. This proactive understanding of the visual features' relevance does not require the transmission of the features themselves, enhancing both communication and computation efficiency. Specifically, we present ActFormer, a Transformer that learns bird's eye view (BEV) representations by using predefined BEV queries to interact with multi-robot multi-camera inputs. Each BEV query can actively select relevant cameras for information aggregation based on pose information, instead of interacting with all cameras indiscriminately. Experiments on the V2X-Sim dataset demonstrate that ActFormer improves the detection performance from 29.89% to 45.15% in terms of AP@0.7 with about 50% fewer queries, showcasing the effectiveness of ActFormer in multi-agent collaborative 3D object detection.<|reference_end|>
@article{huang2024actformer:,
    title={ActFormer: Scalable Collaborative Perception via Active Queries},
    author={Suozhi Huang, Juexiao Zhang, Yiming Li, Chen Feng},
    journal={arXiv preprint arXiv:2403.04968},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.04968},
    primaryClass={cs.CV}
}