README.md

---
language:
- sv
license: llama3
library_name: transformers
tags:
- unsloth
datasets:
- neph1/bellman-7b-finetune
- neph1/codefeedback-swedish
---

# Model Card for Model ID

This version of bellman is finetuned from llama-3-instruct-8b.
It is arguable whether it's better at Swedish, because llama-3 is really good. It's however finetuned for prompt question answering, based on a dataset created from 
Swedish wikipedia, with a lot of Sweden-centric questions.
New from previous versions is questions from a translated code-feedback dataset, as well as a number of stories. It's not great at generating stories, 
but better than previosly.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/653cd3049107029eb004f968/IDGX3d9lGe6yx-yHjsrav.png)

## Model Details

Training run on 240606:

Step 	Training Loss 	Validation Loss<br>
 25 	1.506400 	1.164538<br>
 50 	1.128500 	1.059316<br>
 75 	1.095100 	1.040511<br>
100 	1.068700 	1.031033<br>
125 	1.061300 	1.024377<br>
150 	1.035700 	1.017490<br>
175 	1.061200 	1.012095<br>
200 	1.031600 	1.007867<br>
225 	1.031900 	1.002652<br>
250 	0.958300 	1.003817<br>
275 	0.967900 	1.000483<br>
300 	0.950000 	0.998807<br>
325 	0.974300 	0.996894<br>
350 	0.960700 	0.994098<br>
375 	0.956000 	0.991491<br>
400 	0.940500 	0.988697<br>
425 	0.949100 	0.987253<br>
450 	0.940600 	0.986425 <-- Picked checkpoint<br>
475 	0.888300 	0.994204<br>
500 	0.881700 	0.994897<br>

### Model Description


- **Developed by:** Me
- **Funded by:** Me
- **Model type:** Instruct
- **Language(s) (NLP):** Swedish
- **License:** llama-3
- **Finetuned from model:** Llama3 Instruct 8b

## Model Card Contact

[email protected]