Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformer
<|reference_start|>Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformer: The complex traffic environment and various weather conditions make the collection of LiDAR data expensive and challenging. Achieving high-quality and controllable LiDAR data generation is urgently needed, controlling with text is a common practice, but there is little research in this field. To this end, we propose Text2LiDAR, the first efficient, diverse, and text-controllable LiDAR data generation model. Specifically, we design an equirectangular transformer architecture, utilizing the designed equirectangular attention to capture LiDAR features in a manner with data characteristics. Then, we design a control-signal embedding injector to efficiently integrate control signals through the global-to-focused attention mechanism. Additionally, we devise a frequency modulator to assist the model in recovering high-frequency details, ensuring the clarity of the generated point cloud. To foster development in the field and optimize text-controlled generation performance, we construct nuLiDARtext which offers diverse text descriptors for 34,149 LiDAR point clouds from 850 scenes. Experiments on uncontrolled and text-controlled generation in various forms on KITTI-360 and nuScenes datasets demonstrate the superiority of our approach.<|reference_end|>
@article{wu2024text2lidar:,
    title={Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular
  Transformer},
    author={Yang Wu, Kaihua Zhang, Jianjun Qian, Jin Xie, Jian Yang},
    journal={arXiv preprint arXiv:2407.19628},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.19628},
    primaryClass={cs.CV}
}