---
license: mit
---
This repository contains the composer training checkpoints for BioClinical ModernBERT [base](https://huggingface.co/thomas-sounack/BioClinical-ModernBERT-base) and [large](https://huggingface.co/thomas-sounack/BioClinical-ModernBERT-large).

The training checkpoints for Bio ModernBERT [base](https://huggingface.co/thomas-sounack/Bio-ModernBERT-base) and [large](https://huggingface.co/thomas-sounack/Bio-ModernBERT-large) are also included.

If you use BioClinical ModernBERT in your work, please cite our [preprint](https://arxiv.org/abs/2506.10896):

```
@misc{sounack2025bioclinicalmodernbertstateoftheartlongcontext,
      title={BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP}, 
      author={Thomas Sounack and Joshua Davis and Brigitte Durieux and Antoine Chaffin and Tom J. Pollard and Eric Lehman and Alistair E. W. Johnson and Matthew McDermott and Tristan Naumann and Charlotta Lindvall},
      year={2025},
      eprint={2506.10896},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2506.10896}, 
}
```