CPT: Consistent Proxy Tuning for Black-box Optimization
<|reference_start|>CPT: Consistent Proxy Tuning for Black-box Optimization: Black-box tuning has attracted recent attention due to that the structure or inner parameters of advanced proprietary models are not accessible. Proxy-tuning provides a test-time output adjustment for tuning black-box language models. It applies the difference of the output logits before and after tuning a smaller white-box "proxy" model to improve the black-box model. However, this technique serves only as a decoding-time algorithm, leading to an inconsistency between training and testing which potentially limits overall performance. To address this problem, we introduce Consistent Proxy Tuning (CPT), a simple yet effective black-box tuning method. Different from Proxy-tuning, CPT additionally exploits the frozen large black-box model and another frozen small white-box model, ensuring consistency between training-stage optimization objective and test-time proxies. This consistency benefits Proxy-tuning and enhances model performance. Note that our method focuses solely on logit-level computation, which makes it model-agnostic and applicable to any task involving logit classification. Extensive experimental results demonstrate the superiority of our CPT in both black-box tuning of Large Language Models (LLMs) and Vision-Language Models (VLMs) across various datasets. The code is available at https://github.com/chunmeifeng/CPT.<|reference_end|>
@article{he2024cpt:,
    title={CPT: Consistent Proxy Tuning for Black-box Optimization},
    author={Yuanyang He, Zitong Huang, Xinxing Xu, Rick Siow Mong Goh, Salman
  Khan, Wangmeng Zuo, Yong Liu, Chun-Mei Feng},
    journal={arXiv preprint arXiv:2407.01155},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.01155},
    primaryClass={cs.LG}
}