Model Card for medgemma-4b-it-sft-lora-stroke

This model is a fine-tuned version of google/medgemma-4b-it. It has been trained using TRL.

Quick start

from transformers import pipeline
from PIL import Image
import requests
import torch

pipe = pipeline(
    "image-text-to-text",
    model="BTX24/medgemma-4b-it-sft-lora-stroke",
    torch_dtype=torch.bfloat16,
    device="cuda",
)

# Image attribution: Stillwaterising, CC0, via Wikimedia Commons
image_url = "https://storage.googleapis.com/kagglesdsdata/datasets/6652053/10729545/%C4%B0NME%20VER%C4%B0%20SET%C4%B0/Kanama/PNG/10033.png?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=gcp-kaggle-com%40kaggle-161607.iam.gserviceaccount.com%2F20250605%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20250605T135753Z&X-Goog-Expires=259200&X-Goog-SignedHeaders=host&X-Goog-Signature=2e6408958b200e1b573c9f62f7b97a9c8cbdc8fc49c84e112a8c148226ec813c74c2da88404689d1563efac8f7759f86b14a579f3a15126b48fb74a6e3776d9fdf2c14471f6bca7b8a6c92f97af3678fb132f968e0eb318a32eef45794e3da8f1227b37aab36c3040cbb9f59c20116b498ce36c1ac19a75c5678dbbbb85bc00c91435f68935be48b50382750d3f5cdd3bd2dca06bce90b7b3f76d54ba454d7cf54ebd94b540e9d67127a7277e856a27d725dbed4ab9482b6b9c5e5ba6422182a5afbedf7b5ed2a3a4670b181b9b08fc538dce2743e3d8bfe39f4cb5bc9d78fb3205697b325ea8c37a70bd5350afcc9b41aee0a859462cf26b476db206e073513"
image = Image.open(requests.get(image_url, headers={"User-Agent": "example"}, stream=True).raw)

messages = [
    {
        "role": "system",
        "content": [{"type": "text", "text": "You are an expert radiologist."}]
    },
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this X-ray"}
            {"type": "image", "image": image},
        ]
    }
]

output = pipe(text=messages, max_new_tokens=200)
print(output[0]["generated_text"][-1]["content"])

Training procedure

This model was trained with SFT.

Framework versions

• TRL: 0.18.1
• Transformers: 4.52.4
• Pytorch: 2.6.0+cu124
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citations

Cite TRL as:

@misc{vonwerra2022trl,
    title        = {{TRL: Transformer Reinforcement Learning}},
    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
    year         = 2020,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/huggingface/trl}}
}