6.3.2 Data-Efficient Tuning. Data-efficient tuning refers to the process of updating a limited set of prompt parameters for
downstream tasks, instead of fine-tuning the pretrained LLM. It is typically achieved through prompt tuning, where the
weights of a pretrained model are kept fixed, yet only the added prompt tokens are adjusted. Such approaches enable a more
efficient use of data and often yield enhanced performance, particularly as the scale of the model parameters increases.
TheEfficiencySpectrumofLargeLanguageModels-AnAlgorithmicSurvey