parameters. Given that our runtime is dominated by the substantial cost of making two passes through
the LM, we can afford to use somewhat heavier machinery to unlock its full potential.
Our neural recursive LMs approach addresses all of these drawbacks. Speciﬁcally, we trained a
Connector network that connects the output of the ﬁrst pass through the frozen LM with the input
of its second pass (see Figure 5c). The inputs to the Connector network are the output embeddings
after the ﬁrst LM pass (before translation into the token vocabulary space), and the outputs of the
Connector network are injected into the ﬁrst Transformer layer of the second LM pass (bypassing its
token embedding matrix). We implemented the Connector as an autoregressive network composed of
a small number of unidirectional Transformer layers, the same building blocks comprising the frozen
LM. Therefore, by “stitching together” two replicas of the frozen LM via such a Connector network,
STANDING ON THE SHOULDERS OF GIANT FROZEN LANGUAGE MODELS