its potential. Table 4 shows this comparison on the Natural Questions test set. Our Neural recursive
LM improved over prompt tuning by 4.4 points (or, over 20%), and over our textual recursive LM by
2.6 points.
To better understand the differences between our two LM recursion methods, we examined the set of
questions for which the saturated prompt tuning baseline was not able to provide the correct answer
even when 64 answers were sampled from it. Given that this setting gave rise to recall of 44% (see
Table 5), this set was roughly half of the Natural Questions development set. We treat this subset
as questions that were beyond the scope of a single pass through our frozen J1-Large model, and
ask how well both LM recursion methods performed on such questions. Our textual recursive LM
received an EM score of 3.8%, implying that it was rarely able to go beyond extracting a correct
answer from those sampled during the ﬁrst pass through the LM. In contrast, the neural recursive LM
STANDING ON THE SHOULDERS OF GIANT FROZEN LANGUAGE MODELS