NLPGenius/deepseekLora-social-media-detector

DeepSeek Social Media Target Detection Model

This model is a fine-tuned version of deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B for detecting potential targets in social media posts using PEFT (LoRA) technique.

Model Details

• Base Model: DeepSeek-R1-Distill-Qwen-1.5B (1.5B parameters)
• Fine-tuning Method: PEFT (Parameter Efficient Fine-Tuning) with LoRA
• Task: Multi-class Text Classification
• Languages: English, Urdu
• Dataset: Private curated dataset
• Number of Classes: Multi-class classification
• Model Size: Only LoRA adapters (~2-10MB) instead of full 1.5B model

Target Categories

Target Categories

The model can classify social media posts into multiple categories for security and content analysis purposes.

Note: Specific category details are kept private for privacy reasons.

Key Features

🎯 Multi-class Detection: Identifies various types of targets and content categories
🌍 Multilingual: Supports English and Urdu text
⚡ Efficient: Uses PEFT/LoRA for fast inference and small model size
🔒 Security Focused: Specifically trained for content analysis
🎛️ Configurable: Includes confidence-based filtering for production use

Usage

Quick Start

from transformers import AutoTokenizer, AutoModelForSequenceClassification
from peft import PeftModel
import torch

# Load base model and tokenizer
base_model_name = "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B"
model = AutoModelForSequenceClassification.from_pretrained(
    base_model_name, 
    num_labels=NUM_CLASSES  # Replace with your number of classes
)
tokenizer = AutoTokenizer.from_pretrained(base_model_name)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, "NLPGenius/deepseekLora-social-media-detector")

# Make prediction
def predict_target(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
    with torch.no_grad():
        outputs = model(**inputs)
    predicted_class_id = torch.argmax(outputs.logits, dim=-1).item()
    return predicted_class_id

# Example
text = "Your social media post here"
prediction = predict_target(text)
print(f"Predicted class ID: {prediction}")

Advanced Usage with Confidence Filtering

def predict_with_confidence(text, confidence_threshold=0.6):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
    with torch.no_grad():
        outputs = model(**inputs)

    probabilities = torch.softmax(outputs.logits, dim=-1)
    confidence = torch.max(probabilities).item()
    predicted_class = torch.argmax(probabilities).item()

    if confidence >= confidence_threshold:
        return predicted_class, confidence, True
    else:
        return "UNCERTAIN", confidence, False

# Filter out low-confidence predictions
text = "Ambiguous social media post"
pred_class, confidence, is_confident = predict_with_confidence(text)
print(f"Prediction: {pred_class}, Confidence: {confidence:.3f}")

Training Details

• Training Data: Curated dataset of social media posts
• Validation Split: 10% of training data
• Training Method: PEFT with LoRA (rank=16, alpha=32)
• Quantization: 4-bit quantization for memory efficiency
• Optimizer: 8-bit AdamW with weight decay
• Learning Rate: 1e-4
• Epochs: 5
• Batch Size: 2 (with gradient accumulation)

Performance

The model achieves strong performance on social media target detection while using only a fraction of the memory required for full fine-tuning:

• Memory Usage: 60-80% reduction compared to full fine-tuning
• Training Speed: 2-3x faster than traditional fine-tuning
• Model Size: Only LoRA adapters (~2-10MB) vs full model (>1GB)
• Accuracy: Maintains 95-99% of full fine-tuning performance

Intended Use

This model is designed for:

• ✅ Research on social media content analysis
• ✅ Educational purposes in NLP and security studies
• ✅ Development of content moderation systems
• ✅ Threat detection in social media monitoring

⚠️ Important: This model should be used responsibly and in compliance with applicable laws and regulations.

Limitations and Bias

• Performance may vary on content significantly different from training data
• Requires validation for specific domains or new languages
• May need threshold tuning for different use cases
• Potential biases from training data should be considered

Model Architecture

Base Model: DeepSeek-R1-Distill-Qwen-1.5B
├── Transformer Layers (with LoRA adapters)
├── Classification Head (multi-class)
└── PEFT Configuration:
    ├── LoRA Rank: 16
    ├── LoRA Alpha: 32
    ├── Target Modules: attention + MLP layers
    └── Trainable Parameters: <1% of base model

Citation

If you use this model in your research, please cite:

@misc{deepseek-social-media-detector-2025,
  title={DeepSeek LoRA Social Media Target Detection Model},
  author={NLPGenius},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/NLPGenius/deepseekLora-social-media-detector}
}

Acknowledgments

• Base model: DeepSeek-R1-Distill-Qwen-1.5B
• PEFT library: Hugging Face PEFT
• Training framework: Transformers

---

For questions or issues, please open a discussion on this model's page.