RepVF: A Unified Vector Fields Representation for Multi-task 3D Perception
<|reference_start|>RepVF: A Unified Vector Fields Representation for Multi-task 3D Perception: Concurrent processing of multiple autonomous driving 3D perception tasks within the same spatiotemporal scene poses a significant challenge, in particular due to the computational inefficiencies and feature competition between tasks when using traditional multi-task learning approaches. This paper addresses these issues by proposing a novel unified representation, RepVF, which harmonizes the representation of various perception tasks such as 3D object detection and 3D lane detection within a single framework. RepVF characterizes the structure of different targets in the scene through a vector field, enabling a single-head, multi-task learning model that significantly reduces computational redundancy and feature competition. Building upon RepVF, we introduce RFTR, a network designed to exploit the inherent connections between different tasks by utilizing a hierarchical structure of queries that implicitly model the relationships both between and within tasks. This approach eliminates the need for task-specific heads and parameters, fundamentally reducing the conflicts inherent in traditional multi-task learning paradigms. We validate our approach by combining labels from the OpenLane dataset with the Waymo Open dataset. Our work presents a significant advancement in the efficiency and effectiveness of multi-task perception in autonomous driving, offering a new perspective on handling multiple 3D perception tasks synchronously and in parallel. The code will be available at: https://github.com/jbji/RepVF<|reference_end|>
@article{li2024repvf:,
    title={RepVF: A Unified Vector Fields Representation for Multi-task 3D
  Perception},
    author={Chunliang Li, Wencheng Han, Junbo Yin, Sanyuan Zhao, Jianbing Shen},
    journal={arXiv preprint arXiv:2407.10876},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.10876},
    primaryClass={cs.CV}
}