3 Evaluation

We perform a preliminary evaluation of our model
using the human evaluation data from the Self-
Instruct paper (Wang et al., 2022). We report the
ground truth perplexity of our model against what
is, to our knowledge, the best openly available
alpaca-lora model, provided by user chainyo on
huggingface. We find that all models have very
large perplexities on a small number of tasks, and
report perplexities clipped to a maximum of 100.
Models finetuned on this collected dataset ex-
hibit much lower perplexity in the Self-Instruct
evaluation compared to Alpaca. This evaluation is
in no way exhaustive and further evaluation work

Figure 3: Model Perplexities. Lower is better. Our
models achieve stochastically lower ground truth per-
plexities than alpaca-lora.

remains. We welcome the reader to run the model
locally on CPU (see Github for files) and get a
qualitative sense of what it can do.

4 Use Considerations
GPT4All- Training an Assistant-style Chatbot with Large Scale Data Distillation from GPT-3.5-Turbo