peft safetensors unsloth generated_from_trainer base_model:unsloth/tinyllama-chat-bnb-4bit base_model:adapter:unsloth/tinyllama-chat-bnb-4bit license:apache-2.0 model-index region:us
---
base_model: unsloth/tinyllama-chat-bnb-4bit
library_name: peft
license: apache-2.0
tags:
- unsloth
- generated_from_trainer
model-index:
- name: english-telugu-colloquial-translator
  results:
  - task:
      type: translation
      name: English-to-Telugu Colloquial Translation
    dataset:
      name: SujathaL/English_Telugu_Colloquial1
      type: custom
    metrics:
    - type: loss
      value: 5.2763
      name: Validation Loss
---

Model Overview:
This model is a fine-tuned version of unsloth/tinyllama-chat-bnb-4bit, designed for translating 
English text into Telugu while preserving colloquial expressions. The model helps generate
natural and contextually relevant translations for everyday conversations.

Model Description:
Base Model: unsloth/tinyllama-chat-bnb-4bit
Objective: English to Telugu translation with a focus on colloquial speech.
Fine-tuning Dataset: Custom dataset (details provided below).
Framework: PyTorch with Hugging Face transformers and PEFT.
Quantization: 4-bit for efficient inference.

Intended Uses & Limitations
Intended Uses
✅ Translation of English text to Telugu while maintaining natural conversational flow.
✅ Useful for chatbots, customer support, and informal communication.
✅ Can assist language learners in understanding Telugu colloquialisms.

Limitations
⚠️ May not perform well on highly technical or formal translations.
⚠️ Might struggle with complex sentence structures or idiomatic expressions.
⚠️ Requires further fine-tuning for domain-specific translations.

Training and Evaluation Data:
The model was fine-tuned on a custom dataset containing parallel English-Telugu colloquial sentences. The dataset includes:

-Conversations from social media platforms.
-Informal discussions and day-to-day dialogues.
-Commonly used Telugu expressions and their English equivalents.

Training Procedure:
Training Hyperparameters:

Hyperparameter                     Value

Learning Rate	                  0.0003
Training Batch Size	                   4
Evaluation Batch Size	               4
Gradient Accumulation Steps            2
Total Train Batch Size	               8
Optimizer	                       AdamW
LR Scheduler	                   Linear
Warmup Steps	                        2
Epochs	                               10
Mixed Precision Training	   Native AMP

Training Results:
The model achieved the following performance on the evaluation set:

Training Loss	  Epoch	     Step	   Validation Loss
8.9699	            0.2	       2	      5.5126
4.0118	            2.0	      20	      5.4684
1.1042	            5.2	      52	      5.1425
0.9898	           10.0	     100	      5.2763


Model Performance and Evaluation:
The model was evaluated using a validation dataset, and the loss metric was used to measure performance.

-Lower loss values indicate better performance.
-The model shows improvement over training epochs, stabilizing around a loss of 5.27.

Framework Versions:

PEFT: 0.14.0
Transformers: 4.49.0
PyTorch: 2.6.0+cu124
Datasets: 3.2.0
Tokenizers: 0.21.0

How to Use the Model:
You can use this model with Hugging Face’s transformers library
model_name = "SujathaL/english-telugu-colloquial-translator"

Limitations & Future Improvements:

Possible Enhancements:
Fine-tune on a larger, more diverse dataset.
Implement reinforcement learning for better colloquial accuracy.
Test on real-world conversational datasets.

Known Issues:
May require post-processing for fluency.
Some idioms may not translate accurately.

Contributors
Sujatha Lakkoju (Model Developer)
Dataset Sources (ChatGPT)

License:
This model is released under the MIT License.

Contact: If you have any queries, feel free to reach out at sujathalakkoju9@gmail.com


