To sum up, LLaMA is designed to be a base model for further fine-tuning. Its advantages are

Small size
Performant – thanks to extensive training
Open source

Alpaca model

Alpaca model page
Alpaca Github page

Alpaca is a fine-tuned LLaMA model, meaning that the model architecture is the same, but
the weights are slightly different. It is aimed at resolving the lack of instruction-following
capability of LLaMA models.

It behaves like ChatGPT and can follow conversations and instructions.

The 7B and 13B Alpaca models are available.
Training

It was trained to follow instructions like ChatGPT.

The authors first generate the training data using OpenAI’s GPT-3, then convert them to 52k
instruction-following conversational data using the Self-Instruct pipeline.

https://agi-sphere.com/llama-models/

7/18

02/05/2023, 07:05

A brief history of LLaMA models - AGI Sphere

As a result, Alpaca is fine-tuned to respond to conversations like ChatGPT.
Performance
A brief history of LLaMA models - AGI Sphere