madlad400-finetuned-nya-eng

This model is a fine-tuned version of jbochi/madlad400-3b-mt for translation from Chichewa (Nyanja) to English.

Model details

• Developed by: SIL Global
• Finetuned from model: jbochi/madlad400-3b-mt
• Model type: Translation
• Source language: Chichewa (Nyanja) (nya)
• Target language: English (eng)
• License: closed/private

Datasets

The model was trained on a parallel corpus of plain text files:

Chichewa (Nyanja) :

• Biblica® Open Godʼs Word in Contemporary Chichewa (OCCL)
• License: CC-BY-SA 4.0 International

English:

• Berean Standard Bible in English (BSB)
• License: Public Domain

Usage

You can use this model with the transformers library like this:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("sil-ai/madlad400-finetuned-nya-eng")
model = AutoModelForSeq2SeqLM.from_pretrained("sil-ai/madlad400-finetuned-nya-eng")

inputs = tokenizer("Your input text here", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))

madlad400-finetuned-nya-eng

This model is a fine-tuned version of jbochi/madlad400-3b-mt on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0458
• Chrf: 88.3618

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 4
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Chrf
0.4876	1.6471	1600	0.3650	58.1259
0.3378	3.2943	3200	0.2283	66.4104
0.2706	4.9414	4800	0.1546	72.3427
0.192	6.5886	6400	0.0984	79.4187
0.1247	8.2357	8000	0.0614	85.2251
0.0963	9.8829	9600	0.0459	88.3687

Framework versions

• PEFT 0.12.0
• Transformers 4.44.2
• Pytorch 2.4.1+cu124
• Datasets 2.21.0
• Tokenizers 0.19.1