3.1 Common Sense Reasoning
Following results from (Conover et al.), we evalu-
ate on 7 standard common sense reasoning tasks:
ARC easy and challenge (Clark et al., 2018),
BoolQ (Clark et al., 2019), HellaSwag (Zellers
et al., 2019), OpenBookQA (Mihaylov et al.,
2018), and Winogrande (Sakaguchi et al., 2019).
We evaluate several models: GPT-J (Wang and
Komatsuzaki, 2021), Pythia (6B and 12B) (Bi-
derman et al., 2023), Dolly v1 and v2 (Conover
et al.), and GPT4All using lm-eval-harness (Gao
et al., 2021). Similar to results in (Ouyang et al.,
2022), instruction-tuning showed performance re-
gressions over the base model. However, we no-
tice in some tasks that the LoRA instruction fine-
tuned models show some performance improve-
ments.

4 Use Considerations
2023_GPT4All-J_Technical_Report_2