metadata
license: mit
datasets:
- miriad/miriad-4.4M
language:
- en
metrics:
- accuracy
base_model:
- google/gemma-3-270m
pipeline_tag: question-answering
library_name: transformers
tags:
- medical
- biology
- chemistry
馃┖ MedGemma-270M
MedGemma-270M is a 270M-parameter Gemma 3 model fine-tuned with LoRA on the MIRIAD-4.4M medical Q&A dataset.
This model is designed for fast, domain-specialized inference on small GPUs and CPUs.
Model Details
- Base Model: google/gemma-3-270m
- Parameters: 270M
- Fine-tuning Method: LoRA (r=8, alpha=16, dropout=0.0)
- Framework: Unsloth for efficient training
- Dataset: miriad-4.4M
- Task: Medical question answering & clinical reasoning
Training Configuration
- Epochs: 1
- Max Steps: 600
- Batch Size: 1 (grad_acc=24)
- Max Seq Length: 384
- Optimizer: AdamW 8-bit
- Precision: float16 (fp16)
Usage
Inference (Transformers)
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "OmerShah/medgemma-270m"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")
prompt = "What are the common symptoms of iron deficiency anemia?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))