safetensors region:us
---
{}
---
# Fine-Tuned BloomZ Model

**Repository Name:** `sprab4/bloomz_fine_tuned_model`

## Model Description
This repository contains a fine-tuned version of the **BloomZ** model for generating text in **Haitian Creole**. The BloomZ architecture is optimized for multilingual tasks, and this fine-tuning adapts it to generate fluent and contextually accurate text specifically in Haitian Creole.

## Training Details
- **Base Model:** [bigscience/bloomz](https://huggingface.co/bigscience/bloomz)
- **Dataset:** Haitian Creole dataset from Hugging Face which follows the alpaca format of instructions, input, and output.
- **Fine-Tuning Process:**
  - Trained for **2 epochs** using the Hugging Face Trainer.
  - Validation set was used to evaluate performance periodically during fine-tuning.
  - Instruction-response formats (Alpaca-style) were prioritized in the training dataset.
- **Hyperparameters:**
  - learning_rate=5e-5
  - per_device_train_batch_size=2
  - gradient_accumulation_steps=8
  - num_train_epochs=2
  - fp16=False
  - save_steps=500
  - save_total_limit=2
  - logging_steps=50
  - optim="adamw_hf"  
  - weight_decay=0.1
  - warmup_steps=500

## Results

### Quantitative Metrics

**BERTScore**
Varies based on the response for each prompt

BERTScore for the below prompt:

**Prompt:** Solèy la ap kouche ak yon bèl solèy kouche vizib nan orizon an, dekri li nan de fraz

English Translation: The sun is setting and a beautiful sunset is visible on the horizon, describe it in two sentences

- BERTScore Precision: 0.7526
- BERTScore Recall: 0.7445
- BERTScore F1: 0.7485

### Example Output
 
**Prompt:** Solèy la ap kouche ak yon bèl solèy kouche vizib nan orizon an, dekri li nan de fraz

English Translation: The sun is setting and a beautiful sunset is visible on the horizon, describe it in two sentences

**Prediction:** Solèy la ap kouche ak yon bèl solèy kouche vizib nan orizon an, dekri li nan de fraz
  Kouchè la se koupyè nan ouviyon an. Li se fè kouyèl nan fòmèl an a. Lè se vèk la a, ak li se pèmèt pou fòm la. Kouche la te kòt pou kòm la, li te pòmè pou vòk li. La te fèt li, ki te lès pou lè li anpè.

English Translation: The sun is setting and a beautiful sunset is visible on the horizon, describe it in two sentences
  The couch is the cup of the oven. It is done in a formal manner. Time is here, and it is allowed for the form. Lying there was the coast for the as there, it was pomer for his voice. There he was born, which was east for his peaceful time.

