transformers pytorch m2m_100 text2text-generation translation ak am ar awa azj bm ban be bem bn bho bjn bug bg ca ceb cs cjk ckb crh da de dik dyu el en eo et ee fo fj fi fon fr fur ff gaz gd ga gl gn gu ht ha he hi hne hr hu hy ig ilo id is it jv ja kab kac kam kn ks ka kk kbp kea mn km ki rw ky kmb kmr kr kg ko lo lij li ln lt lmo ltg lb lua lg luo lus lv mag mai ml mr min mk mt mni mos mi my nl nb ne nso nus ny oc ory pag pa pap pbt fa plt pl pt prs qu ro rn ru sg sa sat scn shn si sk sl sm sn sd so st es sc sr ss su sv sw szl ta taq tt te tg tl ti tpi tn ts tk tum tr tw tzm ug uk umb ur uz vec vi war wo xh yi yo zh ms zu dataset:slone/nllb-200-10M-sample license:cc-by-nc-4.0 autotrain_compatible endpoints_compatible region:us
---
datasets:
- slone/nllb-200-10M-sample
language:
- ak
- am
- ar
- awa
- azj
- bm
- ban
- be
- bem
- bn
- bho
- bjn
- bug
- bg
- ca
- ceb
- cs
- cjk
- ckb
- crh
- da
- de
- dik
- dyu
- el
- en
- eo
- et
- ee
- fo
- fj
- fi
- fon
- fr
- fur
- ff
- gaz
- gd
- ga
- gl
- gn
- gu
- ht
- ha
- he
- hi
- hne
- hr
- hu
- hy
- ig
- ilo
- id
- is
- it
- jv
- ja
- kab
- kac
- kam
- kn
- ks
- ks
- ka
- kk
- kbp
- kea
- mn
- km
- ki
- rw
- ky
- kmb
- kmr
- kr
- kr
- kg
- ko
- lo
- lij
- li
- ln
- lt
- lmo
- ltg
- lb
- lua
- lg
- luo
- lus
- lv
- mag
- mai
- ml
- mr
- min
- mk
- mt
- mni
- mos
- mi
- my
- nl
- nb
- ne
- nso
- nus
- ny
- oc
- ory
- pag
- pa
- pap
- pbt
- fa
- plt
- pl
- pt
- prs
- qu
- ro
- rn
- ru
- sg
- sa
- sat
- scn
- shn
- si
- sk
- sl
- sm
- sn
- sd
- so
- st
- es
- sc
- sr
- ss
- su
- sv
- sw
- szl
- ta
- taq
- tt
- te
- tg
- tl
- ti
- tpi
- tn
- ts
- tk
- tum
- tr
- tw
- tzm
- ug
- uk
- umb
- ur
- uz
- vec
- vi
- war
- wo
- xh
- yi
- yo
- zh
- zh
- ms
- zu
license: cc-by-nc-4.0
pipeline_tag: translation
---

It is a truncated version of [NLLB-200-600M](https://huggingface.co/facebook/nllb-200-distilled-600M) model 
(6 layers instead of 12, 512 hidden dimensions instead of 1024) with 175M parameters (131M of which are token embeddings).

This model was fine-tuned on the [slone/nllb-200-10M-sample](https://huggingface.co/datasets/slone/nllb-200-10M-sample) subset of 
the [NLLB dataset](https://huggingface.co/datasets/allenai/nllb) with 175 languages, using only the samples with BLASER score above 3.5.

Because of its small size, it is really bad at translation, but can serve as a base model for further fine-tuning for a small number of languages.
It is recommended to [prune the vocabulary of this model](https://towardsdatascience.com/how-to-adapt-a-multilingual-t5-model-for-a-single-language-b9f94f3d9c90) 
before fine-tuning, to preserve only the tokens used with the intended languages.