transformers safetensors distilbert text-classification generated_from_trainer base_model:distilbert/distilbert-base-uncased base_model:finetune:distilbert/distilbert-base-uncased license:apache-2.0 autotrain_compatible endpoints_compatible region:us
---
base_model: distilbert-base-uncased
library_name: transformers
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: emotions-classifier-1.0
  results: []
---

# emotions-classifier-1.0

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the [dair-ai/emotion](https://huggingface.co/datasets/dair-ai/emotion) dataset.

## Model Description

This model is a fine-tuned version of DistilBERT for emotion classification tasks. It is trained on the `dair-ai/emotion` dataset, which contains short text samples categorized into six emotions:

- **Anger**
- **Fear**
- **Joy**
- **Love**
- **Sadness**
- **Surprise**

The model uses the lightweight DistilBERT architecture, making it efficient for deployment while maintaining strong performance on text classification tasks.

---

## Limitations & Biases

- **Generalization**: The model may not generalize well to non-English text, noisy inputs, or texts significantly different from the training data. It truncates inputs longer than 128 tokens, which may affect performance on long texts.
- **Ambiguity**: Struggles with mixed emotions, sarcasm, or nuanced contexts. It’s limited to single-label classification for six predefined emotions.
- **Dataset Bias**: Performance may be skewed due to class imbalances or cultural biases in the training dataset (`dair-ai/emotion`).
- **Model Bias**: Inherits potential biases from the pre-trained DistilBERT, including gender, racial, or socioeconomic biases.



## Evaluation & Training
-**Training Dataset: dair-ai/emotion** (16,000 examples)
- **Highest Validation Accuracy**: 94.05%  
- **Final Validation Loss**: 0.158
- **Test Accuracy** 92.3%
- **Metrics Used**:
  - **Accuracy**: Percentage of correctly classified labels in the validation set.
  - **Loss**: Cross-entropy loss on the validation set, used to measure the model's confidence in predictions.

The model achieved a strong performance on the `dair-ai/emotion` validation dataset, indicating its ability to classify text into six emotion categories effectively.

---

## Training Hyperparameters

The following hyperparameters were used during training:
- **Learning Rate**: 2e-05
- **Train Batch Size**: 16
- **Eval Batch Size**: 32
- **Seed**: 42
- **Optimizer**: `adamw_torch` with `betas=(0.9, 0.999)` and `epsilon=1e-08`  
- **Learning Rate Scheduler Type**: Linear  
- **Number of Epochs**: 3  
- **Weight Decay**: 0.05  

---

## Example Usage Code

```python
from transformers import pipeline

# Load the fine-tuned model
classifier = pipeline("text-classification", model="moeMachineLearning/emotion-classifier-1.0")

# Test texts
texts = [
    "I am so happy today!",
    "This is the worst day of my life.",
    "I'm feeling a bit nervous about tomorrow's event.",
]

# Run predictions
for text in texts:
    result = classifier(text)
    print(f"Text: {text}")
    print(f"Prediction: {result}\n")
