RetFiner: A Vision-Language Refinement Scheme for Retinal Foundation
  Models
The rise of imaging techniques such as optical coherence tomography (OCT) and
advances in deep learning (DL) have enabled clinicians and researchers to
streamline retinal disease staging. A popular DL approach is self-supervised
learning (SSL), where models learn from vast amounts of unlabeled data,
avoiding costly annotation. SSL has allowed the development of foundation
models (FMs), large models that can be used for a variety of downstream tasks.
However, existing FMs for OCT, trained solely on image data, lack a
comprehensive and robust semantic understanding of images, as evidenced by
their downstream performance (especially for complex tasks), and thus require
supervised fine-tuning (which may be unfeasible) to better adapt to specific
applications and populations. To address this, we propose RetFiner, an SSL
vision-language refinement scheme that improves the representations of existing
FMs and enables their efficient and direct adaptation to specific populations
for improved downstream performance. Our method uses a diverse set of training
objectives which take advantage of the rich supervisory signal found in textual
data. We tested RetFiner on the retinal FMs RETFound, UrFound, and VisionFM,
showing significant improvements in linear probing performance on seven highly
diverse OCT classification tasks, with an average increase of 5.8, 3.9, and 2.1
percentage points over their baselines, respectively. Our code and model
weights are publicly available at https://github.com/ronnief1/RetFiner.
68625f5d9e7509383d29ac62
Ronald Fecso
68625f5d9e7509383d29ac63
José Morano
claimed_verified
2025-07-02T07:41:11.408Z
655b3383ed8df831286969f0
/avatars/ab47ac437d9e978a7480cb9d82fa7638.svg
José Morano
user
j-morano
68625f5d9e7509383d29ac64
Ursula Schmidt-Erfurth
68625f5d9e7509383d29ac65
Hrvoje Bogunović
RetFiner: A Vision-Language Refinement Scheme for Retinal Foundation
  Models
The rise of imaging techniques such as optical coherence tomography (OCT) and
advances in deep learning (DL) have enabled clinicians and researchers to
streamline retinal disease staging. A popular DL approach is self-supervised
learning (SSL), where models learn from vast amounts of unlabeled data,
avoiding costly annotation. SSL has allowed the development of foundation
models (FMs), large models that can be used for a variety of downstream tasks.
However, existing FMs for OCT, trained solely on image data, lack a
comprehensive and robust semantic understanding of images, as evidenced by
their downstream performance (especially for complex tasks), and thus require
supervised fine-tuning (which may be unfeasible) to better adapt to specific
applications and populations. To address this, we propose RetFiner, an SSL
vision-language refinement scheme that improves the representations of existing
FMs and enables their efficient and direct adaptation to specific populations
for improved downstream performance. Our method uses a diverse set of training
objectives which take advantage of the rich supervisory signal found in textual
data. We tested RetFiner on the retinal FMs RETFound, UrFound, and VisionFM,
showing significant improvements in linear probing performance on seven highly
diverse OCT classification tasks, with an average increase of 5.8, 3.9, and 2.1
percentage points over their baselines, respectively. Our code and model
weights are publicly available at https://github.com/ronnief1/RetFiner.
RetFiner, a vision-language refinement scheme, enhances self-supervised foundation models for OCT by leveraging textual data, improving their downstream performance in retinal disease classification tasks.
optical coherence tomography (OCT) deep learning (DL) self-supervised learning (SSL) foundation models (FMs) supervised fine-tuning RetFiner vision-language refinement linear probing performance