18

feed-forward network while empirically demonstrating its eﬃcacy in domain adapta-
tion tasks. These adapter modules are strategically positioned after the multi-head
attention and feed-forward layers within the transformer architecture. Variants of such
neural adapters are colloquially termed as either bottleneck adapters or serial adapters;
in the present paper, we employ the term ”serial adapters” to refer speciﬁcally to the
architecture described in [98].
Beyond Efficiency