SKIP: Skill-Localized Prompt Tuning for Inference Speed Boost-Up
<|reference_start|>SKIP: Skill-Localized Prompt Tuning for Inference Speed Boost-Up: Prompt-tuning methods have shown comparable performance as parameter-efficient fine-tuning (PEFT) methods in various natural language understanding tasks. However, existing prompt tuning methods still utilize the entire model architecture; thus, they fail to accelerate inference speed in the application. In this paper, we propose a novel approach called SKIll-localized Prompt tuning (SKIP), which is extremely efficient in inference time. Our method significantly enhances inference efficiency by investigating and utilizing a skill-localized subnetwork in a language model. Surprisingly, our method improves the inference speed up to 160% while pruning 52% of the parameters. Furthermore, we demonstrate that our method is applicable across various transformer-based architectures, thereby confirming its practicality and scalability.<|reference_end|>
@article{yang2024skeleton:,
    title={Skeleton: A New Framework for Accelerating Language Models via Task
  Neuron Localized Prompt Tuning},
    author={Nakyeong Yang, Jiwon Moon, Junseok Kim, Yunah Jang, and Kyomin Jung},
    journal={arXiv preprint arXiv:2404.11916},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.11916},
    primaryClass={cs.CL cs.AI}
}