  This paper details the process of developing the first native large
generative language model for the Nordic languages, GPT-SW3. We cover all parts
of the development process, from data collection and processing, training
configuration and instruction finetuning, to evaluation and considerations for
release strategies. We hope that this paper can serve as a guide and reference
for other researchers that undertake the development of large generative models
for smaller languages.

Computation and Language;Artificial Intelligence