ing the soft prompts [P1],··· , [Pi], [Pi+1],··· , [Pl] into the
model input. Nonetheless, P-tuning differs by concatenating
these prompts to form a template and maps it
to obtain
{h1,··· , hi, e(x), hi+1,··· , hl, e(x)}, in which e represents
pretrained embedding layer. The training goal is to optimize
the continuous prompts {h1,··· , hl}. As the weights of PLMs
are fixed and only a few parameters need to be fine-tuned,
the template can be effectively learned in few-shot learning
scenarios. P-tuning employs a bidirectional long short-term
memory network (LSTM) with a ReLU-activated multilayer
perceptron (MLP) to initialize the embedding of soft prompts
through MLP(LSTM(h1,··· , hi): LSTM(hi,··· , hl)).
head = Attn(XWq, [ ˆPk, XWk], [ ˆPv, XWv]),
Parameter-EfficientFine-TuningMethods