2014). The embeddings are then fed to custom downstream
models. Fine-tuning involves copying the weights from a
pre-trained network and tuning them on the downstream
task. Recent work shows that ﬁne-tuning often enjoys better
Parameter-Efficient Transfer Learning for NLP