readme: ## RadBERT-RoBERTa-4m
  
  This is one variant of our RadBERT models trained with 4 million deidentified medical reports from US VA hospital, which achieves stronger medical language understanding performance than previous medical domain models such as BioBERT, Clinical-BERT, BLUE-BERT and BioMed-RoBERTa. 
  
  Performances are evaluated on three tasks: 
  (a) abnormal sentence classification: sentence classification in radiology reports as reporting abnormal or normal findings; 
  (b) report coding: Assign a diagnostic code to a given radiology report for five different coding systems; 
  (c) report summarization: given the findings section of a radiology report, extractively select key sentences that summarized the findings.
  
  For details, check out the paper here: 
  [RadBERT: Adapting transformer-based language models to radiology](https://pubs.rsna.org/doi/abs/10.1148/ryai.210258)
  
  Code for the paper is released at [this GitHub repo](https://github.com/zzxslp/RadBERT).
  
  ### How to use
  
  Here is an example of how to use this model to extract the features of a given text in PyTorch:
  
  ```python
  from transformers import AutoConfig, AutoTokenizer, AutoModel
  config = AutoConfig.from_pretrained('zzxslp/RadBERT-RoBERTa-4m')
  tokenizer = AutoTokenizer.from_pretrained('zzxslp/RadBERT-RoBERTa-4m')
  model = AutoModel.from_pretrained('zzxslp/RadBERT-RoBERTa-4m', config=config)
  text = "Replace me by any medical text you'd like."
  encoded_input = tokenizer(text, return_tensors='pt')
  output = model(**encoded_input)
  ```
  
  ### BibTeX entry and citation info
  
  If you use the model, please cite our paper:
  
  ```bibtex
  @article{yan2022radbert,
    title={RadBERT: Adapting transformer-based language models to radiology},
    author={Yan, An and McAuley, Julian and Lu, Xing and Du, Jiang and Chang, Eric Y and Gentili, Amilcare and Hsu, Chun-Nan},
    journal={Radiology: Artificial Intelligence},
    volume={4},
    number={4},
    pages={e210258},
    year={2022},
    publisher={Radiological Society of North America}
  }
  ```