The training regimen progresses with the model learning
to reconstruct the masked entities by leveraging contextual
information. This exercise cultivates the model’s comprehen-
sion of the textual data’s structural semantics and facilitates
the alignment of pertinent entities within the structured data.
The overarching optimization goal is to train the language
model to accurately restore the obscured spans, thereby en-
riching its understanding of entity semantics [Ye et al., 2020].
RAG forLargeLanguageModels-ASurvey