Qwen2.5-7B-Instruct Fine-Tuned for Mental Health Counseling
Model Overview
This is a fine-tuned version of unsloth/Qwen2.5-7B-Instruct-bnb-4bit
, optimized for mental health counseling conversations. The model is trained to provide supportive, empathetic, and informative responses for individuals seeking mental health support.
Dataset
The model is fine-tuned on a combination of:
- Amod/mental_health_counseling_conversations (cleaned version:
arafatanam/Mental-Health-Counseling
) - 2752 rows - chillies/student-mental-health-counseling-vn (translated version:
arafatanam/Student-Mental-Health-Counseling-10K
) - 7500 rows - Total dataset size: 10,252 rows
Training Details
- Hardware: Kaggle Notebooks (GPU T4 x2)
- Fine-tuning framework:
Unsloth
withLoRA
- Training settings:
max_seq_length = 512
batch_size = 8
gradient_accumulation_steps = 4
num_train_epochs = 2
learning_rate = 5e-5
optimizer = adamw_8bit
lr_scheduler = cosine
Training Results
- Final training loss:
1.3034
- Total steps:
640
- Trainable parameters:
0.58%
of the model` - Validation loss:
1.245
- Evaluation metric (perplexity):
3.42
Usage
This model can be used for:
- Providing mental health counseling support
- Generating empathetic responses to mental health queries
- Assisting in chatbot development for mental health applications
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for arafatanam/Student-Support-Qwen2.5-7B-Instruct
Base model
Qwen/Qwen2.5-7B
Finetuned
Qwen/Qwen2.5-7B-Instruct
Quantized
unsloth/Qwen2.5-7B-Instruct-bnb-4bit