Text Generation
Transformers
PyTorch
Safetensors
Finnish
llama
finnish
text-generation-inference
RASMUS nielsr HF Staff commited on
Commit
acb0a01
·
verified ·
1 Parent(s): 8e77c94

Improve model card with library name and primary paper link (#3)

Browse files

- Improve model card with library name and primary paper link (2429b4407e5c2a8d9666c3f6d604ec7b61483206)


Co-authored-by: Niels Rogge <[email protected]>

Files changed (1) hide show
  1. README.md +24 -22
README.md CHANGED
@@ -1,21 +1,23 @@
1
  ---
2
- language:
3
- - fi
4
- license: apache-2.0
5
- tags:
6
- - finnish
7
- - llama
8
  datasets:
9
  - Finnish-NLP/mc4_3.1.0_fi_cleaned
10
  - Finnish-NLP/oscar_2301_fi_cleaned
11
  - Finnish-NLP/Reddit_fi_2006_2022
12
  - Finnish-NLP/wikipedia_20230501_fi_cleaned
13
  - intfloat/multilingual_cc_news
14
- inference: false
 
 
15
  pipeline_tag: text-generation
16
-
 
 
 
 
17
  ---
18
 
 
 
19
  # Llama-3b for Finnish
20
 
21
  Pretrained Llama model on Finnish language using a causal language modeling (CLM) objective. Llama model was introduced in
@@ -106,18 +108,18 @@ The final training dataset had 19 billion words and the evaluation dataset had 2
106
 
107
  |Dataset | Words | Ratio |
108
  |------------------------------|-------------|-------------|
109
- |mc4_3.1.0_fi_cleaned | 11.462B | 60.7\% |
110
- |oscar_2301_fi_cleaned | 3.295B | 17.4\% |
111
- |Suomi24 | 3.045B | 16.1\% |
112
- |multilingual_cc_news | 0.295B | 1.6\% |
113
- |STT | 0.249B | 1.3\% |
114
- |Yle | 0.201B | 1.1\% |
115
- |Reddit_fi_2006_2022 | 0.138B | 0.7\% |
116
- |wikipedia_20230501_fi_cleaned | 0.096B | 0.5\% |
117
- |Project Lönnrot | 0.078B | 0.4\% |
118
- |Finnish parliament speeches | 0.021B | 0.1\% |
119
- |fi-news-corpus | 0.004B | 0.1\% |
120
- |**TOTAL** | **18.884B** | **100.0\%** |
121
 
122
  ## Training procedure
123
 
@@ -180,7 +182,7 @@ This model was evaluated using [FIN-bench by TurkuNLP](https://github.com/TurkuN
180
  [llama-7b-finnish](https://huggingface.co/Finnish-NLP/llama-7b-finnish):
181
 
182
  | Task |Version| Metric |Value | |Stderr|
183
- |------------------------------------------------|------:|---------------------|-----:|---|-----:|
184
  |bigbench_analogies | 0|multiple_choice_grade|0.2692|± |0.0391|
185
  |bigbench_arithmetic_1_digit_addition | 0|multiple_choice_grade|0.2600|± |0.0441|
186
  |bigbench_arithmetic_1_digit_division | 0|multiple_choice_grade|0.3043|± |0.0981|
@@ -229,4 +231,4 @@ This project would not have been possible without compute generously provided by
229
  - Aapo Tanskanen, [Hugging Face profile](https://huggingface.co/aapot), [LinkedIn profile](https://www.linkedin.com/in/aapotanskanen/)
230
  - Rasmus Toivanen, [Hugging Face profile](https://huggingface.co/RASMUS), [LinkedIn profile](https://www.linkedin.com/in/rasmustoivanen/)
231
 
232
- Feel free to contact us for more details 🤗
 
1
  ---
 
 
 
 
 
 
2
  datasets:
3
  - Finnish-NLP/mc4_3.1.0_fi_cleaned
4
  - Finnish-NLP/oscar_2301_fi_cleaned
5
  - Finnish-NLP/Reddit_fi_2006_2022
6
  - Finnish-NLP/wikipedia_20230501_fi_cleaned
7
  - intfloat/multilingual_cc_news
8
+ language:
9
+ - fi
10
+ license: apache-2.0
11
  pipeline_tag: text-generation
12
+ library_name: transformers
13
+ tags:
14
+ - finnish
15
+ - llama
16
+ inference: false
17
  ---
18
 
19
+ This is the Llama-3b for Finnish model described in the paper [Scaling Data-Constrained Language Models](https://huggingface.co/papers/2305.16264).
20
+
21
  # Llama-3b for Finnish
22
 
23
  Pretrained Llama model on Finnish language using a causal language modeling (CLM) objective. Llama model was introduced in
 
108
 
109
  |Dataset | Words | Ratio |
110
  |------------------------------|-------------|-------------|
111
+ |mc4_3.1.0_fi_cleaned | 11.462B | 60.7% |
112
+ |oscar_2301_fi_cleaned | 3.295B | 17.4% |
113
+ |Suomi24 | 3.045B | 16.1% |
114
+ |multilingual_cc_news | 0.295B | 1.6% |
115
+ |STT | 0.249B | 1.3% |
116
+ |Yle | 0.201B | 1.1% |
117
+ |Reddit_fi_2006_2022 | 0.138B | 0.7% |
118
+ |wikipedia_20230501_fi_cleaned | 0.096B | 0.5% |
119
+ |Project Lönnrot | 0.078B | 0.4% |
120
+ |Finnish parliament speeches | 0.021B | 0.1% |
121
+ |fi-news-corpus | 0.004B | 0.1% |
122
+ |**TOTAL** | **18.884B** | **100.0%** |
123
 
124
  ## Training procedure
125
 
 
182
  [llama-7b-finnish](https://huggingface.co/Finnish-NLP/llama-7b-finnish):
183
 
184
  | Task |Version| Metric |Value | |Stderr|
185
+ |------------------------------------------------|------:|---------------------|-----:|---|-----:|\
186
  |bigbench_analogies | 0|multiple_choice_grade|0.2692|± |0.0391|
187
  |bigbench_arithmetic_1_digit_addition | 0|multiple_choice_grade|0.2600|± |0.0441|
188
  |bigbench_arithmetic_1_digit_division | 0|multiple_choice_grade|0.3043|± |0.0981|
 
231
  - Aapo Tanskanen, [Hugging Face profile](https://huggingface.co/aapot), [LinkedIn profile](https://www.linkedin.com/in/aapotanskanen/)
232
  - Rasmus Toivanen, [Hugging Face profile](https://huggingface.co/RASMUS), [LinkedIn profile](https://www.linkedin.com/in/rasmustoivanen/)
233
 
234
+ Feel free to contact us for more details 🤗