datasets emerges as a significant challenge. Previous methods employ high-performing LLMs such
as ChatGPT and GPT-4 (OpenAI, 2023) to generate these question-answer pairs (Li et al., 2023a),
but the cost of utilizing those closed-source models for inference can be a concern. In such situations,
harnessing large-scale domain corpora for continual pre-training represents a promising solution to
acquire domain knowledge.
Retrieval-augmented Prompting. Retrieval augmentation enhances LLMs by integrating external
domain-specific information without modifying the model parameters (Li et al., 2023b; Cui et al.,
2023; Huang et al., 2023). LLMs gain domain context from sources like documents, domain-specific
knowledge graphs, or neural networks with parametric domain knowledge. This enables LLMs to
better answer domain-specific questions and address issues like hallucination. In such cases, seam-
less integration of external knowledge into LLMs is crucial, existing methods typically concatenate
ADAPTINGLARGELANGUAGEMODELSVIA READINGCOMPREHENSION