Qwen2.5-7B-Instruct Fine-Tuned for Mental Health Counseling

Model Overview

This is a fine-tuned version of unsloth/Qwen2.5-7B-Instruct-bnb-4bit, optimized for mental health counseling conversations. The model is trained to provide supportive, empathetic, and informative responses for individuals seeking mental health support.

Dataset

The model is fine-tuned on a combination of:

Training Details

  • Hardware: Kaggle Notebooks (GPU T4 x2)
  • Fine-tuning framework: Unsloth with LoRA
  • Training settings:
    • max_seq_length = 512
    • batch_size = 8
    • gradient_accumulation_steps = 4
    • num_train_epochs = 2
    • learning_rate = 5e-5
    • optimizer = adamw_8bit
    • lr_scheduler = cosine

Training Results

  • Final training loss: 1.3034
  • Total steps: 640
  • Trainable parameters: 0.58% of the model`
  • Validation loss: 1.245
  • Evaluation metric (perplexity): 3.42

Usage

This model can be used for:

  • Providing mental health counseling support
  • Generating empathetic responses to mental health queries
  • Assisting in chatbot development for mental health applications
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for arafatanam/Student-Support-Qwen2.5-7B-Instruct

Base model

Qwen/Qwen2.5-7B
Finetuned
(52)
this model

Datasets used to train arafatanam/Student-Support-Qwen2.5-7B-Instruct