---
license: apache-2.0
base_model: meta-llama/Llama-3.2-1B
tags:
  - pubmedqa
  - llama3
  - qlora
  - sequence-classification
  - 4bit
  - peft
---

# LLaMA3-2.1B QLoRA fine-tuned on PubMedQA

This model is a 4-bit quantized, QLoRA fine-tuned version of `meta-llama/Llama-3.2-1B`, trained on the PubMedQA dataset for medical question classification (`yes`, `no`, `maybe`). It was optimized using PEFT with LoRA adapters, and is designed for efficient inference on resource-constrained hardware.

## Training Details
- **Base model**: `meta-llama/Llama-3.2-1B`
- **Dataset**: `pubmed_qa/pqa_labeled`
- **Method**: QLoRA (4-bit NF4)
- **LoRA target modules**: `q_proj`, `v_proj`
- **Epochs**: 5
- **Batch size**: 4