|
|
---
|
|
|
license: mit
|
|
|
---
|
|
|
|
|
|
# Med-VAE
|
|
|
|
|
|
Med-VAE is a family of six large-scale, generalizable 2D and 3D variational autoencoders (VAEs) designed for medical imaging. It is trained on over one million medical images across multiple anatomical regions and modalities. Med-VAE autoencoders encode medical images as downsized latent representations and decode latent representations back to high-resolution images. Across diverse tasks obtained from 20 medical image datasets, we demonstrate that utilizing MedVAE latent representations in place of high-resolution images when training downstream models can lead to efficiency benefits (up to 70x improvement in throughput) while simultaneously preserving clinically-relevant features.
|
|
|
|
|
|
[💻 Github](https://github.com/StanfordMIMI/MedVAE)
|
|
|
|
|
|
## Model Description
|
|
|
| Total Compression Factor | Channels | Dimensions | Modalities | Anatomies | Config File | Model File |
|
|
|
|----------|----------|----------|----------|----------|----------|----------|
|
|
|
| 16 | 1 | 2D | X-ray | Chest, Breast (FFDM) | [medvae_4x1.yaml ](model_weights/medvae_4x1.yaml)| [vae_4x_1c_2D.ckpt](model_weights/vae_4x_1c_2D.ckpt)
|
|
|
| 16 | 3 | 2D | X-ray | Chest, Breast (FFDM) | [medvae_4x3.yaml](model_weights/medvae_4x3.yaml) | [vae_4x_3c_2D.ckpt](model_weights/vae_4x_3c_2D.ckpt)
|
|
|
| 64 | 1 | 2D | X-ray | Chest, Breast (FFDM) | [medvae_8x1.yaml](model_weights/medvae_8x1.yaml) | [vae_8x_1c_2D.ckpt](model_weights/vae_8x_1c_2D.ckpt)
|
|
|
| 64 | 3 | 2D | X-ray | Chest, Breast (FFDM) | [medvae_8x4.yaml](model_weights/medvae_8x4.yaml) | [vae_8x_4c_2D.ckpt](model_weights/vae_8x_4c_2D.ckpt)
|
|
|
| 64 | 1 | 3D | MRI, CT | Whole-Body | [medvae_4x1.yaml ](model_weights/medvae_4x1.yaml) | [vae_4x_1c_3D.ckpt](model_weights/vae_4x_1c_3D.ckpt)
|
|
|
| 512 | 1 | 3D | MRI, CT | Whole-Body | [medvae_8x1.yaml](model_weights/medvae_8x1.yaml) | [vae_8x_1c_3D.ckpt](model_weights/vae_8x_1c_3D.ckpt)
|
|
|
|
|
|
Note: Model weights and checkpoints are located in the `model_weights` folder.
|
|
|
|
|
|
## Usage Instructions
|
|
|
|
|
|
|
|
|
|
|
|
## Citation
|
|
|
If you use Med-VAE, please cite the original paper:
|
|
|
|
|
|
```bibtex
|
|
|
@article{varma2025medvae,
|
|
|
title = {Med-VAE: --},
|
|
|
author = {Maya Varma, Ashwin Kumar, Rogier van der Sluijs, Sophie Ostmeier, Louis Blankemeier, Pierre Chambon, Christian Bluethgen, Jip Prince, Curtis Langlotz, Akshay Chaudhari},
|
|
|
year = {2025},
|
|
|
publisher = {Arxiv},
|
|
|
journal = {Arvix},
|
|
|
howpublished = {TODO}
|
|
|
}
|
|
|
```
|
|
|
|
|
|
For questions, please place a Github Issues message.
|
|
|
|
|
|
|
