LoRA is set by default provided in the python package PEFT [Mangrulkar et al., 2022]. The
data settings for training and testing remain the same as full ﬁne-tuning.
Data-eﬃcient ﬁne-tuning. In addition to measuring the number of training parameters,
we also conduct experiments on the data side, i.e., data-eﬃcient ﬁne-tuning. Speciﬁcally,
we conduct experiments on USMLE dataset, i.e., simply train and test on its own splits,
resembling an USMLE ID evaluation. However, as the training set in USMLE is much
smaller compared with that of combining PubMedQA and MedMCQA (10k vs. 400k), less
medical knowledge can be learned from the downstream training data, it turns out to be
rather more challenging than the previous OOD test, as will be shown by our experimental
results.
Implementation details. For each downstream task, we ﬁne-tune the network using the
same setting as training on PMC papers, speciﬁcally, we set the max context length as 512
PMC-LLaMA- Further Finetuning LLaMA on Medical Papers