• We propose a simple recipe which automatically converts large-scale raw corpora into reading
comprehension texts, to effectively learn the domain knowledge while concurrently preserving
prompting performance.

• Our experiments show the effectiveness of our method in consistently improving model perfor-

mance in three different domains: biomedicine, finance and law.

2 PRELIMINARY EXPLORATION ON CONTINUED PRE-TRAINING
ADAPTINGLARGELANGUAGEMODELSVIA READINGCOMPREHENSION