readme: ---
  language: en
  license: bsd-3-clause
  ---
  
  # ctrl
  
  #  Table of Contents
  
  1. [Model Details](#model-details)
  2. [Uses](#uses)
  3. [Bias, Risks, and Limitations](#bias-risks-and-limitations)
  4. [Training](#training)
  5. [Evaluation](#evaluation)
  6. [Environmental Impact](#environmental-impact)
  7. [Technical Specifications](#technical-specifications)
  8. [Citation](#citation)
  9. [Model Card Authors](#model-card-authors)
  10. [How To Get Started With the Model](#how-to-get-started-with-the-model)
  
  
  # Model Details
  
  ## Model Description
  
  The CTRL model was proposed in [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://arxiv.org/abs/1909.05858) by Nitish Shirish Keskar*, Bryan McCann*, Lav R. Varshney, Caiming Xiong and Richard Socher. It's a causal (unidirectional) transformer pre-trained using language modeling on a very large corpus of ~140 GB of text data with the first token reserved as a control code (such as Links, Books, Wikipedia etc.). The model developers released a model card for CTRL, available [here](https://github.com/salesforce/ctrl/blob/master/ModelCard.pdf).
  
  In their [model card](https://github.com/salesforce/ctrl/blob/master/ModelCard.pdf), the developers write: 
  
  > The CTRL Language Model analyzed in this card generates text conditioned on control codes that specify domain, style, topics, dates, entities, relationships between entities, plot points, and task-related behavior.
  
  - **Developed by:** See [associated paper](https://arxiv.org/abs/1909.05858) from Salesforce Research
  - **Model type:** Transformer-based language model
  - **Language(s) (NLP):** Primarily English, some German, Spanish, French
  - **License:** [BSD 3-Clause](https://github.com/salesforce/ctrl/blob/master/LICENSE.txt); also see [Code of Conduct](https://github.com/salesforce/ctrl)
  - **Related Models:** More information needed
      - **Parent Model:** More information needed
  - **Resources for more information:** 
    - [Associated paper](https://arxiv.org/abs/1909.05858)
    - [GitHub repo](https://github.com/salesforce/ctrl)
    - [Developer Model Card](https://github.com/salesforce/ctrl/blob/master/ModelCard.pdf)
    - [Blog post](https://blog.salesforceairesearch.com/introducing-a-conditional-transformer-language-model-for-controllable-generation/)
  
  # Uses
  
  ## Direct Use
  
  The model is a language model. The model can be used for text generation. 
  
  ## Downstream Use
  
  In their [model card](https://github.com/salesforce/ctrl/blob/master/ModelCard.pdf), the developers write that the primary intended users are general audiences and NLP Researchers, and that the primary intended uses are:
  
  > 1. Generating artificial text in collaboration with a human, including but not limited to:
  >   - Creative writing
  >   - Automating repetitive writing tasks
  >   - Formatting specific text types
  >   - Creating contextualized marketing materials
  > 2. Improvement of other NLP applications through fine-tuning (on another task or other data, e.g. fine-tuning CTRL to learn new kinds of language like product descriptions)
  > 3. Enhancement in the field of natural language understanding to push towards a better understanding of artificial text generation, including how to detect it and work toward control, understanding, and potentially combating potentially negative consequences of such models.
  
  ## Out-of-Scope Use
  
  In their [model card](https://github.com/salesforce/ctrl/blob/master/ModelCard.pdf), the developers write: 
  
  > - CTRL should not be used for generating artificial text without collaboration with a human.
  > - It should not be used to make normative or prescriptive claims.
  > - This software should not be used to promote or profit from:
  >   - violence, hate, and division;
  >   - environmental destruction;
  >   - abuse of human rights; or
  >   - the destruction of people's physical and mental health.
  
  # Bias, Risks, and Limitations
  
  Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.
  
  In their [model card](https://github.com/salesforce/ctrl/blob/master/ModelCard.pdf), the developers write: 
  
  > We recognize the potential for misuse or abuse, including use by bad actors who could manipulate the system to act maliciously and generate text to influence decision-making in political, economic, and social settings. False attribution could also harm individuals, organizations, or other entities. To address these concerns, the model was evaluated internally as well as externally by third parties, including the Partnership on AI, prior to release.
  
  > To mitigate potential misuse to the extent possible, we stripped out all detectable training data from undesirable sour