t5 region:us
---
{}
---


# Instruction-Following Multilingual LLM (English + Azerbaijani)

## Overview
This model is a fine-tuned version of **mT5 (multilingual T5)** designed for instruction-following tasks in English and Azerbaijani. It effectively understands and generates responses in the same language as the input, making it highly suitable for multilingual NLP applications.

---

## Model Details
- **Base Model**: mT5
- **Supported Languages**: English, Azerbaijani
- **Primary Task**: Instruction following
- **Fine-tuned Dataset**: Custom dataset with diverse instructions and outputs in both languages
- **Dataset Size**: Approximately 118,000 examples
- **Applications**: Multilingual assistants, text generation, and language-specific QA systems

---

## Training Information
### Dataset
The fine-tuning dataset is in JSON format, with fields structured as follows:
- **instruction**: The task to be performed.
- **input**: Additional context or input (optional).
- **output**: The expected response.

**Example**:
```json
{
  "instruction": "Translate the following to Azerbaijani: How are you?",
  "input": "",
  "output": "Necəsən?"
}
```

### Training Parameters
- **Epochs**: 3  
- **Batch Size**: 16  
- **Learning Rate**: 5e-5  
- **Optimizer**: AdamW  
- **Max Input Length**: 512 tokens  
- **Max Output Length**: 512 tokens  

---

## Evaluation
### Evaluation Strategy
The model was assessed using both quantitative and qualitative methods:

#### Quantitative Metrics
- **Average Loss**: 2.15  
- **BLEU Score**: ~30 (for translation tasks)  
- **Exact Match (EM)**: ~75%  

#### Qualitative Analysis
- Reviewed sample predictions for accuracy in both languages.
- Verified translations using reliable references like Google Translate.

---

## Example Predictions
### English Example:
**Instruction**: "Summarize the following article: ..."  
**Prediction**: "The article discusses ..."  
**Reference**: "The article is about ..."  

### Azerbaijani Example:
**Instruction**: "Translate the following to Azerbaijani: How are you?"  
**Prediction**: "Necəsən?"  
**Reference**: "Necəsən?"

---

## Model Performance
- **Training Loss Curve**: [Insert visualization if available]
- **Evaluation Loss Curve**: [Insert visualization if available]

---

## Usage
### Loading the Model
```python
from transformers import T5ForConditionalGeneration, T5Tokenizer

# Load the model and tokenizer
model = T5ForConditionalGeneration.from_pretrained("your-username/instruction-following-mT5-finetuned-english-azerbaijani")
tokenizer = T5Tokenizer.from_pretrained("your-username/instruction-following-mT5-finetuned-english-azerbaijani")

# Prepare input
instruction = "Translate the following to Azerbaijani: How are you?"
input_text = tokenizer(instruction, return_tensors="pt", truncation=True, padding="max_length", max_length=512)

# Generate output
outputs = model.generate(input_text["input_ids"])
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

## Limitations
- Performance may degrade for very complex or out-of-scope instructions.
- Dataset biases could affect accuracy in certain domains.
- Currently limited to English and Azerbaijani languages.

---

## Citation
If you use this model, please cite:

```bibtex
@article{yourname2024instructionfollowing,
  title={Fine-tuned Instruction Following Model for English and Azerbaijani},
  author={Your Name},
  year={2024}
}
```

---

## Contact
For questions or support, please reach out to: **venkateshnaidu588@gmail.com**  

--- 

