readme: ---
  license: apache-2.0
  pipeline_tag: image-to-text
  ---
  
  ## generate-cxr
  
  This BlipForConditionalGeneration model generates realistic radiology reports given an chest X-ray and a clinical indication (e.g. 'RLL crackles, eval for pneumonia').
  
  - **Developed by:** Nathan Sutton
  - **Model type:** BLIP
  - **Language(s) (NLP):** English
  - **License:** Apache 2.0
  - **Finetuned from model:** Salesforce/blip-image-captioning-large
  
  ## Model Sources
  
  - **Repository:** https://github.com/nathansutton/prerad
  - **Paper:** https://medium.com/@nasutton/a-new-generative-model-for-radiology-b687a993cbb
  - **Demo:** https://nathansutton-prerad.hf.space/
  
  
  ## Out-of-Scope Use
  
  Any medical application.
  
  ## How to Get Started with the Model
  
  ```
  from PIL import Image
  from transformers import BlipForConditionalGeneration, BlipProcessor
  
  # read in the model
  processor = BlipProcessor.from_pretrained("nathansutton/generate-cxr")
  model = BlipForConditionalGeneration.from_pretrained("nathansutton/generate-cxr")
  
  # your data
  my_image = 'my-chest-x-ray.jpg'
  my_indication = 'RLL crackles, eval for pneumonia'
  
  # process the inputs
  inputs = processor(
      images=Image.open(my_image), 
      text='indication:' + my_indication,
      return_tensors="pt"
  )
  
  # generate an entire radiology report
  output = model.generate(**inputs,max_length=512)
  report = processor.decode(output[0], skip_special_tokens=True)
  
  ```
  
  # Training Details
  
  This model was trained by cross-referencing the radiology reports in MIMIC-CXR with the images in the MIMIC-CXR-JPG.  None are available here and require a data usage agreement with physionet.
  