Table 1: Domain-specific task scores of general language model (General LLM) and the language
model that has undergone continued pre-training on the domain-specific raw corpora (DAPT (Gu-
rurangan et al., 2020)). We report the average of task scores within each domain under prompting,
fine-tuning and knowledge probing settings.

Method

Prompting

Fine-tuning

BioMed.

Finance Law BioMed.

Finance Law BioMed.

General LLM
DAPT

44.2
41.7

58.6
57.6

34.2
35.0

64.2
66.5

79.9
80.9

42.0
45.4

Knowledge Prob
Law
45.0
45.6

36.5
36.9
ADAPTINGLARGELANGUAGEMODELSVIA READINGCOMPREHENSION