safetensors m2m_100 nllb translation ace acm acq aeb af ajp ak als am apc ar ars ary arz as ast awa ayr azb azj ba bm ban be bem bn bho bjn bo bs bug bg ca ceb cs cjk ckb crh cy da de dik dyu dz el en eo et eu ee fo fj fi fon fr fur fuv gaz gd ga gl gn gu ht ha he hi hne hr hu hy ig ilo id is it jv ja kab kac kam kn ks ka kk kbd kbp kea khk km ki rw ky kmb kmr knc kg ko lo lij li ln lt lmo ltg lb lua lg luo lus lvs mag mai ml mar min mk mt mni mos mi my nl nn nb npi nso nus ny oc ory pag pa pap pbt pes plt pl pt prs quy ro rn ru sg sa sat scn shn si sk sl sm sn sd so st es sc sr ss su sv swh szl ta taq tt te tg tl th ti tpi tn ts tk tum tr tw tzm ug uk umb ur uzn vec vi war wo xh ydd yo yue zh zsm zu dataset:flores-200 arxiv:2207.04672 base_model:facebook/nllb-200-distilled-600M base_model:finetune:facebook/nllb-200-distilled-600M license:cc-by-nc-4.0 region:us
---
base_model:
- facebook/nllb-200-distilled-600M
datasets:
- flores-200
language:
- ace
- acm
- acq
- aeb
- af
- ajp
- ak
- als
- am
- apc
- ar
- ars
- ary
- arz
- as
- ast
- awa
- ayr
- azb
- azj
- ba
- bm
- ban
- be
- bem
- bn
- bho
- bjn
- bo
- bs
- bug
- bg
- ca
- ceb
- cs
- cjk
- ckb
- crh
- cy
- da
- de
- dik
- dyu
- dz
- el
- en
- eo
- et
- eu
- ee
- fo
- fj
- fi
- fon
- fr
- fur
- fuv
- gaz
- gd
- ga
- gl
- gn
- gu
- ht
- ha
- he
- hi
- hne
- hr
- hu
- hy
- ig
- ilo
- id
- is
- it
- jv
- ja
- kab
- kac
- kam
- kn
- ks
- ka
- kk
- kbd
- kbp
- kea
- khk
- km
- ki
- rw
- ky
- kmb
- kmr
- knc
- kg
- ko
- lo
- lij
- li
- ln
- lt
- lmo
- ltg
- lb
- lua
- lg
- luo
- lus
- lvs
- mag
- mai
- ml
- mar
- min
- mk
- mt
- mni
- mos
- mi
- my
- nl
- nn
- nb
- npi
- nso
- nus
- ny
- oc
- ory
- pag
- pa
- pap
- pbt
- pes
- plt
- pl
- pt
- prs
- quy
- ro
- rn
- ru
- sg
- sa
- sat
- scn
- shn
- si
- sk
- sl
- sm
- sn
- sd
- so
- st
- es
- sc
- sr
- ss
- su
- sv
- swh
- szl
- ta
- taq
- tt
- te
- tg
- tl
- th
- ti
- tpi
- tn
- ts
- tk
- tum
- tr
- tw
- tzm
- ug
- uk
- umb
- ur
- uzn
- vec
- vi
- war
- wo
- xh
- ydd
- yo
- yue
- zh
- zsm
- zu
license: cc-by-nc-4.0
metrics:
- bleu
- spbleu
- chrf++
pipeline_tag: translation
tags:
- nllb
language_details: ace_Arab, ace_Latn, acm_Arab, acq_Arab, aeb_Arab, afr_Latn, ajp_Arab,
  aka_Latn, amh_Ethi, apc_Arab, arb_Arab, ars_Arab, ary_Arab, arz_Arab, asm_Beng,
  ast_Latn, awa_Deva, ayr_Latn, azb_Arab, azj_Latn, bak_Cyrl, bam_Latn, ban_Latn,bel_Cyrl,
  bem_Latn, ben_Beng, bho_Deva, bjn_Arab, bjn_Latn, bod_Tibt, bos_Latn, bug_Latn,
  bul_Cyrl, cat_Latn, ceb_Latn, ces_Latn, cjk_Latn, ckb_Arab, crh_Latn, cym_Latn,
  dan_Latn, deu_Latn, dik_Latn, dyu_Latn, dzo_Tibt, ell_Grek, eng_Latn, epo_Latn,
  est_Latn, eus_Latn, ewe_Latn, fao_Latn, pes_Arab, fij_Latn, fin_Latn, fon_Latn,
  fra_Latn, fur_Latn, fuv_Latn, gla_Latn, gle_Latn, glg_Latn, grn_Latn, guj_Gujr,
  hat_Latn, hau_Latn, heb_Hebr, hin_Deva, hne_Deva, hrv_Latn, hun_Latn, hye_Armn,
  ibo_Latn, ilo_Latn, ind_Latn, isl_Latn, ita_Latn, jav_Latn, jpn_Jpan, kab_Latn,
  kac_Latn, kam_Latn, kan_Knda, kas_Arab, kas_Deva, kat_Geor, knc_Arab, knc_Latn,
  kaz_Cyrl, kbd_Cyrl, kbp_Latn, kea_Latn, khm_Khmr, kik_Latn, kin_Latn, kir_Cyrl,
  kmb_Latn, kon_Latn, kor_Hang, kmr_Latn, lao_Laoo, lvs_Latn, lij_Latn, lim_Latn,
  lin_Latn, lit_Latn, lmo_Latn, ltg_Latn, ltz_Latn, lua_Latn, lug_Latn, luo_Latn,
  lus_Latn, mag_Deva, mai_Deva, mal_Mlym, mar_Deva, min_Latn, mkd_Cyrl, plt_Latn,
  mlt_Latn, mni_Beng, khk_Cyrl, mos_Latn, mri_Latn, zsm_Latn, mya_Mymr, nld_Latn,
  nno_Latn, nob_Latn, npi_Deva, nso_Latn, nus_Latn, nya_Latn, oci_Latn, gaz_Latn,
  ory_Orya, pag_Latn, pan_Guru, pap_Latn, pol_Latn, por_Latn, prs_Arab, pbt_Arab,
  quy_Latn, ron_Latn, run_Latn, rus_Cyrl, sag_Latn, san_Deva, sat_Beng, scn_Latn,
  shn_Mymr, sin_Sinh, slk_Latn, slv_Latn, smo_Latn, sna_Latn, snd_Arab, som_Latn,
  sot_Latn, spa_Latn, als_Latn, srd_Latn, srp_Cyrl, ssw_Latn, sun_Latn, swe_Latn,
  swh_Latn, szl_Latn, tam_Taml, tat_Cyrl, tel_Telu, tgk_Cyrl, tgl_Latn, tha_Thai,
  tir_Ethi, taq_Latn, taq_Tfng, tpi_Latn, tsn_Latn, tso_Latn, tuk_Latn, tum_Latn,
  tur_Latn, twi_Latn, tzm_Tfng, uig_Arab, ukr_Cyrl, umb_Latn, urd_Arab, uzn_Latn,
  vec_Latn, vie_Latn, war_Latn, wol_Latn, xho_Latn, ydd_Hebr, yor_Latn, yue_Hant,
  zho_Hans, zho_Hant, zul_Latn
inference: false
---

# NLLB-200 Distilled 600M Model Fine-tuned for Kabardian Translation

## Model Details

- **Model Name:** nllb-200-distilled-600M-kbd-v0.1
- **Base Model:** NLLB-200 Distilled 600M
- **Model Type:** Translation
- **Language(s):** Kabardian and others from NLLB-200 (200 languages)
- **Parameters:** 600 million (distilled from larger NLLB models)
- **License:** CC-BY-NC (inherited from base model)
- **Developer:** panagoa (fine-tuning), Meta AI (base model)
- **Last Updated:** February 10, 2025 (updated 19 days ago)
- **Paper:** [NLLB Team et al, No Language Left Behind: Scaling Human-Centered Machine Translation, Arxiv, 2022](https://arxiv.org/abs/2207.04672)

## Model Description

This model is a fine-tuned version of the NLLB-200 Distilled 600M model, specifically optimized for Kabardian language translation. Unlike the larger 1.3B parameter models in this collection, this distilled variant offers a more efficient alternative with approximately half the parameters. Knowledge distillation techniques have been used to preserve translation quality while significantly reducing model size, making it more suitable for deployment in resource-constrained environments or applications requiring faster inference.

## Intended Uses

- Efficient machine translation to and from Kabardian language
- Mobile and edge device deployment where model size matters
- Real-time translation applications with lower latency requirements
- Embedded systems and applications with limited computational resources
- NLP applications for the Kabardian language requiring balance between performance and efficiency
- Cultural and linguistic accessibility tools that need to work on consumer hardware
- Educational applications and resources for Kabardian speakers

## Training Data

This model has been fine-tuned on specialized Kabardian language datasets, building upon the original NLLB-200 Distilled 600M model. The distillation process in the base model likely used the larger NLLB models as teachers, transferring knowledge while reducing parameter count. The fine-tuning process for Kabardian language has been optimized to maintain translation quality despite the reduced model size.

## Performance and Limitations

- Offers a favorable balance between translation quality and computational efficiency
- Reduced parameter count (600M vs 1.3B) enables deployment in more resource-constrained environments
- May show some quality degradation compared to the larger 1.3B models, particularly for complex or nuanced translations
- Knowledge distillation helps preserve much of the translation capability of larger models
- Inherits limitations from the base NLLB-200 architecture:
  - Research model not intended for critical production deployments without proper evaluation
  - Not optimized for specialized domains (medical, legal, technical)
  - Limited to input sequences not exceeding 512 tokens
  - Translations should not be used as certified translations
- May have additional limitations specific to distilled models:
  - Potentially reduced ability to handle rare words or expressions
  - May show less consistency across diverse language pairs
  - Could exhibit less nuanced understanding of context-dependent translations

## Usage Example

```python
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_name = "panagoa/nllb-200-distilled-600M-kbd-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

def translate(text, src_lang='eng_Latn', tgt_lang='kbd_Cyrl', a=16, b=1.5, max_input_length=64, **kwargs):
    tokenizer.src_lang = src_lang
    tokenizer.tgt_lang = tgt_lang
    inputs = tokenizer(text, return_tensors='pt', padding=True, truncation=True, max_length=max_input_length)

    result = model.generate(
        **inputs.to(model.device),
        forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang),
        max_new_tokens=int(a + b * inputs.input_ids.shape[1]),
        **{
            'num_beams': 4,
            'temperature': 0.2,
            'top_p': 0.9,
            'length_penalty': 1.1,
            'repetition_penalty': 1.2,
            'do_sample': True,
            'early_stopping': True
        },
        **kwargs
    )
    return tokenizer.batch_decode(result, skip_special_tokens=True)

translate('A big car needs a lot of fuel.')
['Машинэшхуэм бензин куэд хуейщ.']
```

## Ethical Considerations

As noted for the base NLLB-200 model and applicable to this distilled version:
- This work prioritizes human users and aims to minimize risks transferred to them
- Translation access for low-resource languages like Kabardian can improve education and information access
- A smaller, more efficient model enables broader deployment across diverse hardware environments, potentially increasing accessibility
- The efficiency-focused nature of this model may help reduce computational resource requirements and associated environmental impacts
- Potential risks include:
  - Making groups with lower digital literacy vulnerable to misinformation
  - Mistranslations could have adverse impacts, especially in critical contexts
  - Reduced model size may amplify certain biases or limitations present in the training data
- Despite extensive data cleaning, personally identifiable information may not be entirely eliminated from training data

## Caveats and Recommendations

- This distilled model is recommended for applications where efficiency and resource constraints are important factors
- For highest translation quality with no resource constraints, consider the larger 1.3B models in this collection
- Performance will vary across different domains, contexts, and language pairs
- Users should conduct thorough evaluation for their specific use cases prior to deployment
- Consider performance-quality tradeoffs when choosing between this distilled model and larger alternatives
- This model could be particularly valuable for mobile applications, embedded systems, or when serving many simultaneous translation requests

## Additional Information

This model is part of panagoa's collection of NLLB models fine-tuned for Kabardian language translation. It represents an efficiency-focused alternative to the larger 1.3B parameter models, offering a different balance point in the tradeoff between model size and translation quality.