Baselines. Running the large LM twice means doubling inference time. In order to determine
whether this second pass through the frozen LM leads to performance gains, we compare primarily to
prompt-tuning Lester et al. (2021), as an established single-pass frozen model method. Additionally,
since the Connector introduces a nontrivial number of learned parameters, which could affect
performance even if the second pass through the frozen LM is unhelpful, we ran two additional
single-LM-pass baselines, Connector–LM and LM–Connector: consisting of a Connector that runs
either before or after a single pass through the LM, respectively.
STANDING ON THE SHOULDERS OF GIANT FROZEN LANGUAGE MODELS