transformers safetensors bert text-classification bert-base-uncased en dataset:disham993/ElectricalDeviceFeedbackBalanced arxiv:2412.13663 base_model:google-bert/bert-base-uncased base_model:finetune:google-bert/bert-base-uncased license:mit autotrain_compatible endpoints_compatible region:us
---
base_model: google-bert/bert-base-uncased
datasets:
- disham993/ElectricalDeviceFeedbackBalanced
language: en
library_name: transformers
license: mit
metrics:
- epoch: 5
- eval_f1: 0.8835
- eval_accuracy: 0.8846
- eval_runtime: 0.8674
- eval_samples_per_second: 1558.742
- eval_steps_per_second: 25.364
tags:
- text-classification
- bert-base-uncased
---

# electrical-classification-bert-base-uncased

## Model description

This model is fine-tuned from [google-bert/bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased) for text classification tasks, specifically sentiment analysis of customer feedback on electrical devices - circuit breakers, transformers, smart meters, inverters, solar panels, power strips etc. The model has been optimized to classify sentiments into categories such as Positive, Negative, Neutral, and Mixed with high f1 and accuracy scores, making it ideal for analyzing product reviews, customer surveys, and other feedback to derive actionable insights.

## Training Data

The model was trained on the [disham993/ElectricalDeviceFeedbackBalanced](https://huggingface.co/datasets/disham993/ElectricalDeviceFeedbackBalanced) dataset, which has been carefully balanced to address class imbalances effectively. Original dataset which is imbalanced: [disham993/ElectricalDeviceFeedback](https://huggingface.co/datasets/disham993/ElectricalDeviceFeedback).

## Model Details
- **Base Model:** [google-bert/bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased)
- **Task:** text-classification
- **Language:** en
- **Dataset:** [disham993/ElectricalDeviceFeedbackBalanced](https://huggingface.co/datasets/disham993/ElectricalDeviceFeedbackBalanced) 

## Training procedure

### Training hyperparameters

- **Evaluation Strategy:** epoch
- **Learning Rate:** 1e-5
- **Batch Size:** 64 (for both training and evaluation)
- **Number of Epochs:** 5
- **Weight Decay:** 0.01

## Evaluation results

The following metrics were achieved during evaluation:

- **F1 Score:** 0.8835
- **Accuracy:** 0.8846
- **eval_runtime**: 0.8674
- **eval_samples_per_second**: 1558.742
- **eval_steps_per_second**: 25.364

## Usage

You can use this model for Sentiment Analysis of the Electrical Device Feedback as follows:

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

model_name = "disham993/electrical-classification-bert-base"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
nlp = pipeline("text-classification", model=model, tokenizer=tokenizer)

text = "The new washing machine is efficient but produces a bit of noise."
classification_results = nlp(text)
print(classification_results)
```
## Limitations and bias

The dataset includes synthetic data generated using Llama 3.1:8b, and despite careful optimization and prompt engineering, the model is not immune to errors in labeling. Additionally, as LLM technology is still in its early stages, there may be inherent inaccuracies or biases in the generated data that can impact the model's performance.

This model is intended for research and educational purposes only, and users are encouraged to validate results before applying them to critical applications.

## Training Infrastructure

For a complete guide covering the entire process - from data tokenization to pushing the model to the Hugging Face Hub - please refer to the [GitHub repository](https://github.com/di37/classification-electrical-feedback-finetuning).

## Last update

2025-01-05

## Citation 

```
@misc{modernbert,
      title={Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference}, 
      author={Benjamin Warner and Antoine Chaffin and Benjamin Clavié and Orion Weller and Oskar Hallström and Said Taghadouini and Alexis Gallagher and Raja Biswas and Faisal Ladhak and Tom Aarsen and Nathan Cooper and Griffin Adams and Jeremy Howard and Iacopo Poli},
      year={2024},
      eprint={2412.13663},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2412.13663}, 
}
```