addition to leveraging a ∼ 110M parameter frozen T5 encoder. While ID-PT is thus considerably
more “heavy weight” than prompt tuning, ID-PT is still a very small increment to the frozen model,
adding ∼ 0.3% to the number of parameters and ∼ 1% to inference time.
STANDING ON THE SHOULDERS OF GIANT FROZEN LANGUAGE MODELS