peft safetensors llama unsloth generated_from_trainer dataset:Sadiya025/english-hindi-dataset base_model:unsloth/tinyllama-chat-bnb-4bit base_model:adapter:unsloth/tinyllama-chat-bnb-4bit license:apache-2.0 4-bit bitsandbytes region:us
---
base_model: unsloth/tinyllama-chat-bnb-4bit
datasets:
- Sadiya025/english-hindi-dataset
library_name: peft
license: apache-2.0
tags:
- unsloth
- generated_from_trainer
model-index:
- name: english-hindi-translator
  results: []
---

# english-hindi-translator

This model is a fine-tuned version of [unsloth/tinyllama-chat-bnb-4bit](https://huggingface.co/unsloth/tinyllama-chat-bnb-4bit) on the English-Hindi Dataset (Sadiya025/english-hindi-dataset).
It achieves the following results on the evaluation set:
- Loss: 11.8701

## Model description

- Base Model: unsloth/tinyllama-chat-bnb-4bit
- Dataset: Sadiya025/english-hindi-dataset
- Task: English-to-Hindi Colloquial Translation
- Quantization: 4-bit for optimized inference

## Training and evaluation data

The model was fine-tuned on the [Sadiya025/english-hindi-dataset](https://huggingface.co/datasets/Sadiya025/english-hindi-dataset), which consists of English sentences and their corresponding colloquial Hindi translations.  

## Training procedure

- **Base Model**: [unsloth/tinyllama-chat-bnb-4bit](https://huggingface.co/unsloth/tinyllama-chat-bnb-4bit)  
- **Frameworks**:  
  - PEFT 0.14.0  
  - Transformers 4.49.0  
  - PyTorch 2.6.0+cu124  
  - Datasets 3.2.0  
  - Tokenizers 0.21.0  
- **Fine-tuning Method**: Parameter-Efficient Fine-Tuning (PEFT)  
- **Precision**: 4-bit quantized model for memory efficiency

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 15
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step | Validation Loss |
|:-------------:|:-------:|:----:|:---------------:|
| 3.5793        | 1.0     | 462  | 12.4256         |
| 3.3523        | 2.0     | 924  | 12.0382         |
| 3.4293        | 3.0     | 1386 | 11.9843         |
| 3.4061        | 4.0     | 1848 | 12.0250         |
| 3.1875        | 5.0     | 2310 | 11.9591         |
| 3.3198        | 6.0     | 2772 | 11.9267         |
| 3.4656        | 7.0     | 3234 | 11.9042         |
| 3.4545        | 8.0     | 3696 | 11.9408         |
| 3.2797        | 9.0     | 4158 | 11.9250         |
| 3.2077        | 10.0    | 4620 | 11.9467         |
| 3.3623        | 11.0    | 5082 | 11.8813         |
| 3.1617        | 12.0    | 5544 | 11.8757         |
| 3.1431        | 13.0    | 6006 | 11.8749         |
| 3.1546        | 14.0    | 6468 | 11.8825         |
| 3.2779        | 14.9686 | 6915 | 11.8701         |


### Framework versions

- PEFT 0.14.0
- Transformers 4.49.0
- Pytorch 2.6.0+cu124
- Datasets 3.2.0
- Tokenizers 0.21.0