ModelSpace commited on
Commit
ad93063
·
verified ·
1 Parent(s): 39d0bc4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +39 -12
README.md CHANGED
@@ -1,37 +1,64 @@
1
  ---
2
- license: other
3
  license_name: license
4
  license_link: LICENSE
5
  base_model:
6
  - google/gemma-2-2b
7
  pipeline_tag: translation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  ---
9
 
10
- # Model Card for GemmaX2-28
11
 
12
- ## Model Details
13
 
14
- ### Model Description
15
-
16
- GemmaX2-28-2B-Pretrain is a language model that results from continual pretraining of Gemma2-2B on a mix of 56 billion tokens of monolingual and parallel data in 28 different languages — Arabic, Bengali, Czech, German, English, Spanish, Persian, French, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Burmese, Dutch, polish, Portuguese, Russian, Thai, Tagalog, Turkish, Urdu, Vietnamese, Chinese.
17
 
18
  - **Developed by:** Xiaomi
19
- - **Model type:** A 2B parameter model base on Gemma2-2B, we obtained GemmaX2-28-2B-Pretrain by continuing pre-training on a large amount of monolingual and parallel data.
20
  - **Languages:** Arabic, Bengali, Czech, German, English, Spanish, Persian, French, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Burmese, Dutch, polish, Portuguese, Russian, Thai, Tagalog, Turkish, Urdu, Vietnamese, Chinese.
21
- - **License:** gemma
22
 
23
- ### Model Source
24
 
25
  - paper: [Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study](https://arxiv.org/pdf/2502.02481)
26
 
27
- ### Model Performance
28
 
29
  ![Experimental Result](main.png)
30
 
31
 
32
- ### Training Data
 
 
33
 
34
- We collected monolingual data from [CulturaX](https://huggingface.co/datasets/uonlp/CulturaX) and [MADLAD-400](https://huggingface.co/datasets/allenai/MADLAD-400). For parallel data, we collected all Chinese-centric and English-centric parallel dataset from the [OPUS](https://opus.nlpl.eu/) collection up to Auguest 2024 and underwent a series of filtering processes, such as language detection, semantic duplication filtering, quality filtering, and more.
35
 
36
  ## Citation
37
 
 
1
  ---
2
+ license: gemma
3
  license_name: license
4
  license_link: LICENSE
5
  base_model:
6
  - google/gemma-2-2b
7
  pipeline_tag: translation
8
+ library_name: transformers
9
+ language:
10
+ - ar
11
+ - bn
12
+ - cs
13
+ - de
14
+ - en
15
+ - es
16
+ - fa
17
+ - fr
18
+ - he
19
+ - hi
20
+ - id
21
+ - it
22
+ - ja
23
+ - km
24
+ - ko
25
+ - lo
26
+ - ms
27
+ - my
28
+ - nl
29
+ - pl
30
+ - pt
31
+ - ru
32
+ - th
33
+ - tl
34
+ - tr
35
+ - ur
36
+ - vi
37
+ - zh
38
  ---
39
 
 
40
 
 
41
 
42
+ ## Model Description
43
+ GemmaX2-28-2B-Pretrain is a language model developed through continual pretraining of Gemma2-2B using a mix of 56 billion tokens from both monolingual and parallel data across 28 different languages. Please find more details in our paper: [Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study](https://arxiv.org/pdf/2502.02481).
 
44
 
45
  - **Developed by:** Xiaomi
46
+ - **Model type:** GemmaX2-28-2B-Pretrain is obtained by continually pretraining Gemma2-2B on a large amount of monolingual and parallel data. Subsequently, GemmaX2-28-2B-v0.1 is derived through supervised finetuning on a small set of high-quality translation instruction data.
47
  - **Languages:** Arabic, Bengali, Czech, German, English, Spanish, Persian, French, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Burmese, Dutch, polish, Portuguese, Russian, Thai, Tagalog, Turkish, Urdu, Vietnamese, Chinese.
 
48
 
49
+ ## Model Source
50
 
51
  - paper: [Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study](https://arxiv.org/pdf/2502.02481)
52
 
53
+ ## Model Performance
54
 
55
  ![Experimental Result](main.png)
56
 
57
 
58
+ ## Training Data
59
+
60
+ We collect monolingual data from [CulturaX](https://huggingface.co/datasets/uonlp/CulturaX) and [MADLAD-400](https://huggingface.co/datasets/allenai/MADLAD-400). For parallel data, we collect all Chinese-centric and English-centric parallel datasets from the [OPUS](https://opus.nlpl.eu/) collection up to August 2024 and conduct a series of filtering processes, such as language identification, semantic duplication filtering, quality filtering, and more.
61
 
 
62
 
63
  ## Citation
64