madlad400-finetuned-eng-nya

This model is a fine-tuned version of jbochi/madlad400-3b-mt for translation from English to Chichewa (Nyanja) .

Model details

• Developed by: SIL Global
• Finetuned from model: jbochi/madlad400-3b-mt
• Model type: Translation
• Source language: English (eng)
• Target language: Chichewa (Nyanja) (nya)
• License: closed/private

Datasets

The model was trained on a parallel corpus of plain text files:

English:

• Berean Standard Bible in English (BSB)
• License: Public Domain

Chichewa (Nyanja) :

• Biblica® Open Godʼs Word in Contemporary Chichewa (OCCL)
• License: CC-BY-SA 4.0 International

Usage

You can use this model with the transformers library like this:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("sil-ai/madlad400-finetuned-eng-nya")
model = AutoModelForSeq2SeqLM.from_pretrained("sil-ai/madlad400-finetuned-eng-nya")

inputs = tokenizer("Your input text here", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))

madlad400-finetuned-eng-nya

This model is a fine-tuned version of jbochi/madlad400-3b-mt on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0949
• Chrf: 79.6025

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 4
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Chrf
0.6176	1.6471	1600	0.4814	57.3589
0.4201	3.2943	3200	0.2982	63.8099
0.3462	4.9414	4800	0.2161	67.9079
0.2508	6.5886	6400	0.1566	72.5912
0.1995	8.2357	8000	0.1136	76.9187
0.1593	9.8829	9600	0.0950	79.5868

Framework versions

• PEFT 0.12.0
• Transformers 4.44.2
• Pytorch 2.4.1+cu124
• Datasets 2.21.0
• Tokenizers 0.19.1