Te-BERT_WR: Telugu-BERT Sentiment Classification Model (With Rationale)

Model Overview

Te-BERT_WR is a Telugu sentiment classification model based on Telugu-BERT (L3Cube-Telugu-BERT), a transformer-based BERT model pre-trained specifically on Telugu text (OSCAR, Wikipedia, news) by the L3Cube Pune research group for the Masked Language Modeling (MLM) task. The "WR" in the model name stands for "With Rationale", meaning this model is trained using both sentiment labels and human-annotated rationales from the TeSent_Benchmark-Dataset.

---

Model Details

• Architecture: L3Cube-Telugu-BERT (BERT-base, pre-trained on Telugu)
• Pretraining Data: Telugu OSCAR, Wikipedia, and news articles
• Fine-tuning Data: TeSent_Benchmark-Dataset, using both sentence-level sentiment labels (positive, negative, neutral) and rationale annotations
• Task: Sentence-level sentiment classification (3-way)
• Rationale Usage: Used during training and/or inference ("WR" = With Rationale)

---

Intended Use

• Primary Use: Benchmarking Telugu sentiment classification on the TeSent_Benchmark-Dataset, especially as a baseline for models trained with and without rationales
• Research Setting: Ideal for researchers working on pure Telugu text analysis where sufficient labeled data and rationales exist for fine-tuning
• Academic Utility: Especially suitable for explainable NLP research in Telugu

---

Why Telugu-BERT?

Telugu-BERT is tailored for Telugu and excels in capturing the vocabulary, syntax, and semantics of the language. It recognizes nuanced expressions, idioms, and sentiments that are often poorly represented in multilingual models like mBERT and XLM-R. This makes Te-BERT_WR an excellent choice for sentiment analysis tasks and other Telugu NLP applications requiring strong language-specific representation and explainability through rationales.

---

Performance and Limitations

Strengths:

• Superior understanding of Telugu language specifics compared to multilingual models
• Capable of capturing nuanced and idiomatic expressions in sentiment analysis
• Provides explicit rationales for its predictions, aiding explainability
• Robust baseline for Telugu sentiment classification tasks

Limitations:

• Applicability limited to Telugu; not suitable for multilingual or cross-lingual tasks
• Requires sufficient labeled Telugu data and rationale annotations for best performance

---

Training Data

• Dataset: TeSent_Benchmark-Dataset
• Data Used: The Content (Telugu sentence), Label (sentiment label), and Rationale (human-annotated rationale) columns are used for Te-BERT_WR training

---

Language Coverage

• Language: Telugu (te)
• Model Scope: This implementation and evaluation focus strictly on Telugu sentiment classification

---

Citation and More Details

For detailed experimental setup, evaluation metrics, and comparisons with rationale-based models, please refer to our paper.

---

License

Released under CC BY 4.0.