Model Card for crpatel/mistral_prompt_tuning_tweet_classifier

Model Details

Model Description

This model is a fine-tuned version of Mistral, specifically trained for classifying tweets as complaints or non-complaints. The fine-tuning has been done using PEFT (Parameter-Efficient Fine-Tuning) and prompt tuning techniques. The model leverages the "ought/raft" dataset, specifically the "twitter_complaints" subset.

Developed by: crpatel
Shared by [optional]: crpatel
Model type: Causal Language Model (Fine-tuned with PEFT)
Language(s) (NLP): English
License: Apache 2.0
Finetuned from model: Mistral
Base Model Credit: The base model, Mistral, was developed by Mistral AI.

Model Sources

Repository: Hugging Face Model Hub
Dataset: RAFT - Twitter Complaints
Base Model: Mistral AI

Uses

Direct Use

This model can be used for classifying tweets as complaints or non-complaints, which can be useful for customer service automation, sentiment analysis, and social media monitoring.

Downstream Use

The model can be fine-tuned further for other social media classification tasks or sentiment analysis applications in customer support systems.

Out-of-Scope Use

This model is not designed for general sentiment analysis outside of complaint detection. Misuse for legal or high-stakes decision-making without validation is discouraged.

Bias, Risks, and Limitations

Recommendations

The model should be evaluated for biases before deployment.
Users should verify results against human-labeled datasets.
The model may not generalize well to tweets outside the training distribution.

How to Get Started with the Model

from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
from datasets import load_dataset
import torch

dataset = load_dataset("ought/raft", "twitter_complaints")
peft_model_id = "crpatel/mistral_prompt_tuning_tweet_classifier"
device = "cuda" if torch.cuda.is_available() else "cpu"
text_column = "Tweet text"
label_column = "text_label"

config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, torch_dtype=torch.float16).to(device)
model = PeftModel.from_pretrained(model, peft_model_id)
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)

Training Details

Training Data

Dataset: Ought/RAFT - Twitter Complaints
Preprocessing: Tokenization with AutoTokenizer
Filtering: Preprocessing steps included lowercasing and cleaning tweet text

Training Procedure

Optimization Algorithm: AdamW
Precision: FP16 for memory efficiency
Batch Size: 16
Learning Rate: 5e-5

Model Architecture and Objective

The model is based on Mistral and fine-tuned using PEFT to optimize efficiency in tweet classification.

Software

Transformers Library: 🤗 Transformers
Training Framework: PyTorch
Fine-Tuning: PEFT (Parameter-Efficient Fine-Tuning)

Citation

If you use this model, please cite:

@article{crpatel2024,
  author = {C.R. Patel},
  title = {Mistral Prompt Tuning Tweet Classifier},
  year = {2024},
  publisher = {Hugging Face},
  journal = {Hugging Face Model Hub}
}

Additionally, credit the base model:

@article{mistral2023,
  author = {Mistral AI},
  title = {Mistral Language Model},
  year = {2023},
  publisher = {Mistral AI},
  journal = {Mistral AI Model Hub}
}