readme: ---
  license: bigscience-bloom-rail-1.0
  datasets:
  - pubmed
  widget: 
  - text: 'Photosynthesis is'
  ---
  
  # Model Card for BioMedLM 2.7B
  
  Note: This model was previously known as PubMedGPT 2.7B, but we have changed it due to a request from the NIH which holds the trademark for "PubMed".
  
  
  BioMedLM 2.7B is new language model trained exclusively on biomedical abstracts and papers from [The Pile](https://pile.eleuther.ai/). This GPT-style model can achieve strong results on a variety of biomedical NLP tasks, including a new state of the art performance of 50.3% accuracy on the MedQA biomedical question answering task.
  
  As an autoregressive language model, BioMedLM 2.7B is also capable of natural language generation. However, we have only begun to explore the generation capabilities and limitations of this model, and we emphasize that this model’s generation capabilities are for research purposes only and not suitable for production. In releasing this model, we hope to advance both the development of biomedical NLP applications and best practices for responsibly training and utilizing domain-specific language models; issues of reliability, truthfulness, and explainability are top of mind for us.
  
  This model was a joint collaboration of [Stanford CRFM](https://crfm.stanford.edu/) and [MosaicML](https://www.mosaicml.com/).
  
  #  Table of Contents
  
  - [Model Card for BioMedLM 2.7B](#model-card-for--model_id-)
  - [Table of Contents](#table-of-contents)
  - [Model Details](#model-details)
    - [Model Description](#model-description)
  - [Uses](#uses)
    - [Downstream Use](#downstream-use)
    - [Out-of-Scope Use](#out-of-scope-use)
  - [Bias, Risks, and Limitations](#bias-risks-and-limitations)
    - [Recommendations](#recommendations)
  - [Training Details](#training-details)
    - [Training Data](#training-data)
    - [Training Procedure](#training-procedure)
      - [Preprocessing](#preprocessing)
  - [Environmental Impact](#environmental-impact)
  - [Technical Specifications](#technical-specifications)
    - [Model Architecture and Objective](#model-architecture-and-objective)
    - [Compute Infrastructure](#compute-infrastructure)
  
  # Model Details
  
  ## Model Description
  
  <!-- Provide a longer summary of what this model is/does. -->
  BioMedLM 2.7B is new language model trained exclusively on biomedical abstracts and papers from [The Pile](https://pile.eleuther.ai/). This GPT-style model can achieve strong results on a variety of biomedical NLP tasks, including a new state of the art performance of 50.3% accuracy on the MedQA biomedical question answering task.
  
  As an autoregressive language model, BioMedLM 2.7B is also capable of natural language generation. However, we have only begun to explore the generation capabilities and limitations of this model, and we emphasize that this model’s generation capabilities are for research purposes only and not suitable for production. In releasing this model, we hope to advance both the development of biomedical NLP applications and best practices for responsibly training and utilizing domain-specific language models; issues of reliability, truthfulness, and explainability are top of mind for us.
  
  This model was a joint collaboration of [Stanford CRFM](https://crfm.stanford.edu/) and [MosaicML](https://www.mosaicml.com/).
  
  
  - **Developed by:** Stanford CRFM, MosaicML
  - **Shared by:** Stanford CRFM
  - **Model type:** Language model
  - **Language(s) (NLP):** en
  - **License:**  [bigscience-bloom-rail-1.0](https://huggingface.co/spaces/bigscience/license)
  
  # Uses
  
  This model is licensed under the terms of [BigScience Open RAIL-M license](https://huggingface.co/spaces/bigscience/license) used for [BLOOM](https://huggingface.co/bigscience/bloom-1b1). Please note that, among other restrictions, this license forbids use of the model (or derivatives thereof)
  "To provide medical advice and medical results interpretation." If you are concerned that your use case would follow under the "letter" of this restriction, but not the "spirit," you can contact us to discuss.
  
  ## Direct Use
  
  <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
  <!-- If the user enters content, print that. If not, but they enter a task in the list, use that. If neither, say "more info needed." -->
  It is possible to use this model to generate text, which is useful for experimentation and understanding its capabilities. It should not be directly used for production or work that may directly impact people.
  
  ## Downstream Use
  
  <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
  The main way we have used this model is finetuning for downstream question answering tasks, and we recommend using this model that way.
   
  ## Out-of-Scope Use
  
  <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
  We do not recommend using this model for natural language generation in a production environment, finetuned or 