Robust Task-Oriented Communication Framework for Real-Time Collaborative Vision Perception
<|reference_start|>Robust Task-Oriented Communication Framework for Real-Time Collaborative Vision Perception: Cooperative perception enhances sensing in multi-robot and vehicular networks by aggregating information from multiple agents, improving perception accuracy and range. However, mobility and non-rigid sensor mounts introduce extrinsic calibration errors, necessitating online calibration, which is complicated by limited overlap in sensing regions. Maintaining fresh information is crucial for timely and accurate sensing. To address calibration errors and ensure both perception accuracy and transmission timeliness, we propose a Robust Task-Oriented Communication framework (R-TOCOM) that optimizes calibration and feature transmission in both deployment and streaming phases. First, we formulate an Age of Perceived Targets (AoPT) minimization problem to capture information freshness. Then, in the deployment phase, we introduce a channel-aware self-calibration technique based on re-identification (Re-ID). This technique adaptively compresses key-point features according to channel capacities, effectively addressing calibration issues via spatial and temporal cross-camera correlations. In the streaming phase, we tackle the trade-off between bandwidth and inference accuracy by integrating an Information Bottleneck (IB)-based encoding method that adjusts video compression rates based on task relevance, thereby reducing communication overhead and latency. To mitigate performance degradation from packet loss, we introduce a priority network that filters corrupted features. Extensive studies demonstrate our framework outperforms five baselines, improving multiple object detection accuracy (MODA) by 25.49% and reducing communication costs by 51.36% under severe channel condition.<|reference_end|>
@article{fang2024r-acp:,
    title={R-ACP: Real-Time Adaptive Collaborative Perception Leveraging Robust
  Task-Oriented Communications},
    author={Zhengru Fang, Jingjing Wang, Yanan Ma, Yihang Tao, Yiqin Deng, Xianhao
  Chen, Yuguang Fang},
    journal={arXiv preprint arXiv:2410.04168},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.04168},
    primaryClass={cs.NI}
}