mhGPT: A Lightweight Generative Pre-Trained Transformer for Mental Health Text Analysis
<|reference_start|>mhGPT: A Lightweight Generative Pre-Trained Transformer for Mental Health Text Analysis: This paper introduces mhGPT, a lightweight generative pre-trained transformer trained on mental health-related social media and PubMed articles. Fine-tuned for specific mental health tasks, mhGPT was evaluated under limited hardware constraints and compared with state-of-the-art models like MentaLLaMA and Gemma. Despite having only 1.98 billion parameters and using just 5% of the dataset, mhGPT outperformed larger models and matched the performance of models trained on significantly more data. The key contributions include integrating diverse mental health data, creating a custom tokenizer, and optimizing a smaller architecture for low-resource settings. This research could advance AI-driven mental health care, especially in areas with limited computing power.<|reference_end|>
@article{kim2024mhgpt:,
    title={mhGPT: A Lightweight Generative Pre-Trained Transformer for Mental
  Health Text Analysis},
    author={Dae-young Kim (1), Rebecca Hwa (2), Muhammad Mahbubur Rahman (1) ((1)
  Children's National Hospital, Washington, DC, (2) George Washington
  University, Washington, DC)},
    journal={arXiv preprint arXiv:2408.08261},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.08261},
    primaryClass={cs.CL}
}