MHCTDS/visage

Model Card for Model ID

Best model between those tested on the VISAGE paper. Made for violence severity classification for social media messages in Brazilian Portuguese, according to an adaption of the Quantification of Violence Scale (QOVS)

Model Details

Has a total of 5 classes: No violence, low violence, medium violence, high violence and very high violence

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Matheus Henrique Cajueiro Tobias de Souza
• Funded by : Grant E-26/210.936/2024 by CNPq, CAPES, and FAPERJ
• Model type: BERT
• Language(s) (NLP): Brazilian Portuguese
• License: Apache 2.0
• Finetuned from model: BERTimbau

Model Sources [optional]

• Repository: https://github.com/ufrrj-labweb/visage
• Paper: Will soon be vailable

Direct Use

Predict violence severity of social media messages on Brazilian Portuguese based on the Quantification of Violence Scale (QOVS)

Downstream Use [optional]

General violence prediction or improved social media violence prediction for Portuguese and maybe Spanish

Out-of-Scope Use

Things not mentioned in the use cases, other form of written texts outside of social media without fine tuning.

Bias, Risks, and Limitations

Most of the data labels were no violence, high violence or very high violence, so the model tends to have difficulties with differentiating low violence from null and medium from high.

We trained it based on social media posts collected on X for major violent events happening especially in Rio de Janeiro. If the model is not fine tuned for your data and is used on a region with very different writing patterns or slang, the model might have difficulties.

Recommendations

Always fine tune it for your data, the model is pretty light so it should always be feasible.

If you don't want to accidentally overtune it set a low learning rate, but that shouldn't be an issue with proper fine tuning techniques.

How to Get Started with the Model

Use the code below to get started with the model.

model = BertForSequenceClassification.from_pretrained("MHCTDS/visage")

The github includes a BERT class, it contains functions for training the model and converting pandas dataframes into pytorch dataframes.

Accelerate can be used to parallelize it on any computer, including multiple GPUs or NPU setups (I don't know why you would need multiple GPUs for a 0.5 Gigabyte model though).

Just remember to create an accelerator object before the trainer object is used and use trainer=accelerator.prepare(trainer) on the trainer. Same with the model itself.

Training Details

Training Data

Trained on social media posts collected on X for major violent events happening especially in Rio de Janeiro.

Over 2k posts were labeled by 2 students based on the Quantification of Violence Scale (QOVS).

1758 of those entries (non-duplicates) were used on a 70/30 training-validation split.

For more information consult the paper or github. We plan on releasing the data on a dataset paper in the future.

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

• Training regime: [More Information Needed]

Speeds, Sizes, Times

Reached up to 120k entries per hour on a mac studio m2 max.

Only 1 size available for now. Conversion and use on mobile devices should be possible through TFlite, but untested as of this moment.

Evaluation

Evaluation class used is available on the github.

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

88% accuracy, 88% recall, 88% F1 and 87% precision on the current model

The past model had 87% recall, 86% F1, 85% Precision, 87% Accuracy, 92% ROC AUC and 78% APS

For comparison, on our dataset, the results with a dummy model were: 68% recall, 55% F1, 46% Precision, 68% Accuracy, 80% ROC AUC and 53% APS

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: Mac studio m2 max
• Hours used: 100
• Carbon Emitted: ~0.3kg | A negligible amount compared to a dishwasher in a year

Technical Specifications

Hardware

Needs at least 0.5G of ram

Software

Can run on any OS. Make sure to follow the instruction on the github, but I already tested that python 3.9 can run everything perfectly too.

Citation [optional]

Will be updated when the paper is up.

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Model Card Authors [optional]

Matheus Henrique Cajueiro Tobias de Souza, Eliel Roger da Silva, Tiago Cruz de França, Jonice Oliveira

Model Card Contact

[email protected], [email protected], [email protected], [email protected]