Prompt tuning. We prompt tuned our frozen J1-Large-7B and J1-Grande-17B LMs to serve as
readers over the data prepared by the re-ranker. We used batch size 32, and considered learning rates
in {1 · 10−1, 5 · 10−1} for J1-Large and {3 · 10−2, 1 · 10−1} for J1-Grande, reporting the best results
on the development set and measuring test scores for the best development set conﬁguration.

8

Preprint.

Retriever

Passage score Reader
Retriever
Re-ranker
Retriever
Re-ranker

J1-Large-7B DPR
J1-Large-7B DPR
J1-Large-7B Spider+BM25
J1-Large-7B Spider+BM25

Recall @ J1 input Avg. #docs Dev EM

77.2
80.4
81.4
83.2

17
17
17
17

46.6
48.7
49.5
50.8
STANDING ON THE SHOULDERS OF GIANT FROZEN LANGUAGE MODELS