---
license: apache-2.0
language:
- en
- vi
metrics:
- f1
base_model:
- distilbert/distilbert-base-multilingual-cased
pipeline_tag: text-classification
tags:
- finance
- esg
- financial-text-analysis
- bert
library_name: transformers
widget:
- text: >-
    Over three chapters, it covers a range of topics from energy efficiency and
    renewable energy to the circular economy and sustainable transportation.
datasets:
- nguyen599/ViEn-ESG-100
---

ESG analysis can help investors determine a business' long-term sustainability and identify associated risks. ViDistilBERT-ESG-base is a [distilbert/distilbert-base-multilingual-cased](https://huggingface.co/distilbert/distilbert-base-multilingual-cased) model fine-tuned on [ViEn-ESG-100](https://huggingface.co/datasets/nguyen599/ViEn-ESG-100) dataset, include 100,000 annotated sentences from Vietnam, English news and ESG reports.  

**Input**: A financial text.

**Output**: Environmental, Social, Governance or None.

**Language support**: English, Vietnamese

# How to use 
You can use this model with Transformers pipeline for ESG classification.
```python
# tested in transformers==4.51.0 
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

esgbert = AutoModelForSequenceClassification.from_pretrained('nguyen599/ViDistilBERT-ESG-base',num_labels=4)
tokenizer = AutoTokenizer.from_pretrained('nguyen599/ViDistilBERT-ESG-base')
nlp = pipeline("text-classification", model=esgbert, tokenizer=tokenizer)
results = nlp('Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation.')
print(results) # [{'label': 'Environment', 'score': 0.9206041026115417}]

```

# Benchmark

F1 scores of models on each ESG category in the English ViEn-ESG-100 dataset.

<div align="center">

| **Model** | **Backbone** | **Param** | **E** | **S** | **G** | **N** |
| :------------ | :------------ | :------------: | :------------:  | :------------: | :------------: | :------------: |
| **SEC-BERT-ft** | **SEC-BERT-base** | 109M | 83.12 | 66.77 | 66.53 | 60.30 |
| **FinBERT-ESG** | **FinBERT** | 109M | 92.67 | 84.90 | 86.25 | 87.26 |
| **FinBERT-ESG-9-class** | **FinBERT** | 109M | 92.16 | 89.01 | 91.35 | 86.89 |
| **ESGify** | **MPNet-base** | 109M | 67.72 | 30.20 | 50.76 | 43.44 |
| **EnvironmentBERT** | **DistilRoBERTa** | 82M | 92.15 | - | - | 92.76 |
| **SocialBERT** | **DistilRoBERTa** | 82M | - | 76.81 | - | 81.23 |
| **GovernanceBERT** | **DistilRoBERTa** | 82M | - | - | 64.46 | 80.06 |
| **ViBERT-ESG(Our)** | **BERT-base-cased** | 168M | 93.76 | 94.53 | 94.98 | **94.15** |
| **ViRoBERTa-ESG(Our)** | **RoBERTa-base** | 124M | 95.43 | 94.06 | 95.01 | 91.32 |
| **ViXLMRoBERTa-ESG(Our)** | **XLM-RoBERTa-base** | 278M | 95.00 | 95.00 | **95.47** | 92.19 |
| **ViDeBERTa-ESG(Our)** | **DeBERTa-v3-base** | 184M | **95.50** | 94.49 | 94.81 | 91.48 |
| **ViDeBERTa-small-ESG(Our)** | **DeBERTa-v3-small** | 141M | 94.55 | 94.85 | 94.58 | 90.19 |
| **ViDistilBERT-ESG(Our)** | **DistilBERT-base-cased** | 135M | 95.15 | **95.19** | 94.33 | 91.75 |
| **ViBERT-Env(Our)** | **BERT-base-cased** | 168M | 94.62 | - | - | 92.13 |
| **ViBERT-Soc(Our)** | **BERT-base-cased** | 168M | - | 94.86 | - | 92.22 |
| **ViBERT-Gov(Our)** | **BERT-base-cased** | 168M | - | - | 93.47 | 93.82 |

</div>


F1 scores of models on each ESG category in the Vietnamese ViEn-ESG-100 dataset.

<div align="center">

| **Model**      | **Backbone**   | **Param**   | **E** | **S** | **G** | **N** |
| :------------ | :------------ | :------------: | :------------:  | :------------: | :------------: | :------------: |
| **ViBERT-ESG** | **BERT-base-cased** | 168M | 93.50 | 89.73 | 91.77 | **91.78** |
| **ViRoBERTa-ESG** | **RoBERTa-base** | 124M | 93.41 | 91.49 | 89.93 | 84.32 |
| **ViXLMRoBERTa-ESG** | **XLM-RoBERTa-base** | 278M | 93.45 | 91.02 | 91.69 | 90.41 |
| **ViDeBERTa-ESG** | **DeBERTa-v3-base** | 184M | **95.24** | 89.36 | **93.18** | 85.23 |
| **ViDeBERTa-small-ESG** | **DeBERTa-v3-small** | 141M | 92.90 | 87.79 | 90.63 | 81.48 |
| **ViDistilBERT-ESG** | **DistilBERT-base-cased** | 135M | 93.87 | **91.98** | 90.63 | 87.17 |
| **ViBERT-Env** | **BERT-base-cased** | 168M | 94.87 | - | - | 91.15 |
| **ViBERT-Soc** | **BERT-base-cased** | 168M | - | 91.07 | - | 90.29 |
| **ViBERT-Gov** | **BERT-base-cased** | 168M | - | - | 92.62 | 90.11 |

</div>