Implementing asynchronous MIPS refreshes We asyn-
chronously refresh the MIPS index by running two jobs in
parallel: a primary trainer job, which performs gradient
updates on the parameters, and a secondary index builder
job, which embeds and indexes the documents. As shown

REALM: Retrieval-Augmented Language Model Pre-Training

Figure 3. REALM pre-training with asynchronous MIPS refreshes.
REALM