README.md

---
library_name: transformers
tags: []
---

# Model Card for DistilBERT Text Classification

This is a DistilBERT model fine-tuned for text classification tasks.

## Model Details

### Model Description

This DistilBERT model is fine-tuned for text classification tasks. It is designed to classify texts into different categories based on the provided dataset.

- **Developed by:** Thiago Adriano
- **Model type:** DistilBERT for Sequence Classification
- **Language(s) (NLP):** Portuguese
- **License:** MIT License
- **Finetuned from model:** distilbert-base-uncased

### Model Sources

- **Repository:** [Link to your repository](https://huggingface.co/tadrianonet/distilbert-text-classification)


## How to Get Started with the Model

Use the code below to get started with the model.

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("tadrianonet/distilbert-text-classification")
model = AutoModelForSequenceClassification.from_pretrained("tadrianonet/distilbert-text-classification")

inputs = tokenizer("Sample text for classification", return_tensors="pt")
outputs = model(**inputs)
```

## Training Details

### Training Data

The training data consists of text-label pairs in Portuguese. The data is preprocessed to tokenize the text and convert labels to numerical format.

### Training Procedure

The model is fine-tuned using the Hugging Face `Trainer` API with the following hyperparameters:

- **Training regime:** fp32
- **Learning rate:** 2e-5
- **Batch size:** 16
- **Epochs:** 3

#### Speeds, Sizes, Times

- **Training time:** Approximately 10 minutes on a single GPU

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data

The testing data is a separate set of text-label pairs used to evaluate the model's performance.

#### Factors

The evaluation is disaggregated by accuracy and loss.

#### Metrics

- **Accuracy:** Measures the proportion of correct predictions
- **Loss:** Measures the error in the model's predictions

### Results

- **Evaluation Results:**
  - **Loss:** 0.692
  - **Accuracy:** 50%

#### Summary

The model achieves 50% accuracy on the evaluation dataset, indicating that further fine-tuning and evaluation on a more diverse dataset may be necessary.

## Model Examination

[More Information Needed]

## Environmental Impact

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** GPU
- **Hours used:** 0.2 hours
- **Cloud Provider:** [More Information Needed]
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** [More Information Needed]

## Technical Specifications

### Model Architecture and Objective

The model is based on DistilBERT, a smaller, faster, and cheaper version of BERT, designed for efficient text classification.

### Compute Infrastructure

#### Hardware

- **Hardware Type:** Single GPU
- **GPU Model:** [More Information Needed]

#### Software

- **Framework:** Transformers 4.x
- **Library:** PyTorch

## Citation

**BibTeX:**

1 ```bibtex
@misc{thiago_adriano_2024_distilbert,
  author = {Thiago Adriano},
  title = {DistilBERT Text Classification},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/tadrianonet/distilbert-text-classification}},
}
1 ```

**APA:**

Thiago Adriano. (2024). DistilBERT Text Classification. Hugging Face. https://huggingface.co/tadrianonet/distilbert-text-classification

## More Information

For more details, visit the [Hugging Face model page](https://huggingface.co/tadrianonet/distilbert-text-classification).

## Model Card Authors

Thiago Adriano

## Model Card Contact

For more information, contact Thiago Adriano at [tadriano.dev@gmail.com]