README.md

metadata

license: apache-2.0
base_model: meta-llama/Llama-3.2-1B
tags:
  - pubmedqa
  - llama3
  - qlora
  - sequence-classification
  - 4bit
  - peft

LLaMA3-2.1B QLoRA fine-tuned on PubMedQA

This model is a 4-bit quantized, QLoRA fine-tuned version of meta-llama/Llama-3.2-1B, trained on the PubMedQA dataset for medical question classification (yes, no, maybe). It was optimized using PEFT with LoRA adapters, and is designed for efficient inference on resource-constrained hardware.

Training Details

• Base model: meta-llama/Llama-3.2-1B
• Dataset: pubmed_qa/pqa_labeled
• Method: QLoRA (4-bit NF4)
• LoRA target modules: q_proj, v_proj
• Epochs: 5
• Batch size: 4