Positional Prompt Tuning for Efficient 3D Representation Learning
<|reference_start|>Positional Prompt Tuning for Efficient 3D Representation Learning: Point cloud analysis has achieved significant development and is well-performed in multiple downstream tasks like point cloud classification and segmentation, etc. Being conscious of the simplicity of the position encoding structure in Transformer-based architectures, we attach importance to the position encoding as a high-dimensional part and the patch encoder to offer multi-scale information. Together with the sequential Transformer, the whole module with position encoding comprehensively constructs a multi-scale feature abstraction module that considers both the local parts from the patch and the global parts from center points as position encoding. With only a few parameters, the position embedding module fits the setting of PEFT (Parameter-Efficient Fine-Tuning) tasks pretty well. Thus we unfreeze these parameters as a fine-tuning part. At the same time, we review the existing prompt and adapter tuning methods, proposing a fresh way of prompts and synthesizing them with adapters as dynamic adjustments. Our Proposed method of PEFT tasks, namely PPT, with only 1.05% of parameters for training, gets state-of-the-art results in several mainstream datasets, such as 95.01% accuracy in the ScanObjectNN OBJ_BG dataset. Codes will be released at https://github.com/zsc000722/PPT.<|reference_end|>
@article{zhang2024positional,
    title={Positional Prompt Tuning for Efficient 3D Representation Learning},
    author={Shaochen Zhang, Zekun Qi, Runpei Dong, Xiuxiu Bai, Xing Wei},
    journal={arXiv preprint arXiv:2408.11567},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.11567},
    primaryClass={cs.CV}
}