  Fine-tuning of self-supervised models is a powerful transfer learning method
in a variety of fields, including speech processing, since it can utilize
generic feature representations obtained from large amounts of unlabeled data.
Fine-tuning, however, requires a new parameter set for each downstream task,
which is parameter inefficient. Adapter architecture is proposed to partially
solve this issue by inserting lightweight learnable modules into a frozen
pre-trained model. However, existing adapter architectures fail to adaptively
leverage low- to high-level features stored in different layers, which is
necessary for solving various kinds of speech processing tasks. Thus, we
propose a new adapter architecture to acquire feature representations more
flexibly for various speech tasks. In experiments, we applied this adapter to
WavLM on four speech tasks. It performed on par or better than naive
fine-tuning, with only 11% of learnable parameters. It also outperformed an
existing adapter architecture.

Multimedia;Sound;Audio and Speech Processing