3

Figure 2: The training curve compared with LLaMA under two settings with diﬀerent
training data. The curve is smoothed with 0.6 Exponential Moving Average. Depending on
the amount of training data, the curves have diﬀerent ending steps, thus showing seesaw
shapes with diﬀerent periods.

USMLE(OOD/ID) MedMCQA(ID) PubMedQA(ID)

Method

Human (pass)
Human (expert)
InstructGPT-175B

ChatGPT
LLaMA-7B
LLaMA-33B
LLaMA-7BFull

PMC-LLaMA-7BFull

LLaMA-7BPEFT

Setting
Manual*

Zero-shot*

50.0
87.0
46.0
57.0
27.1
43.4

Full ﬁne-tuning

PEFT

44.55/35.66
44.70/40.61
29.38/27.34
30.64/28.52

–
90.0
44.0
44.7
24.3
30.3
48.15
50.54
32.37
34.33

60.0
78.0
73.2
63.9
5.2
1.8
73.41
69.53
65.81
68.23

PMC-LLaMA-7BPEFT
Table 1: Comparison between LLaMA-7B and PMC-LLaMA-7B under diﬀerent settings.
ACC in percentages is reported in the table. Note that, the manual and zero-shot results
with * are referred from LMFlow [Diao et al., 2023].
PMC-LLaMA- Further Finetuning LLaMA on Medical Papers