readme: ---
  language:
  - en
  license: mit
  tags:
  - autogenerated-modelcard
  datasets:
  - multi_nli
  - wikipedia
  - bookcorpus
  ---
  
  # roberta-large-mnli
  
  ## Table of Contents
  - [Model Details](#model-details)
  - [How To Get Started With the Model](#how-to-get-started-with-the-model)
  - [Uses](#uses)
  - [Risks, Limitations and Biases](#risks-limitations-and-biases)
  - [Training](#training)
  - [Evaluation](#evaluation-results)
  - [Environmental Impact](#environmental-impact)
  - [Technical Specifications](#technical-specifications)
  - [Citation Information](#citation-information)
  - [Model Card Authors](#model-card-author)
  
  ## Model Details
  
  **Model Description:** roberta-large-mnli is the [RoBERTa large model](https://huggingface.co/roberta-large) fine-tuned on the [Multi-Genre Natural Language Inference (MNLI)](https://huggingface.co/datasets/multi_nli) corpus. The model is a pretrained model on English language text using a masked language modeling (MLM) objective.
  
  - **Developed by:** See [GitHub Repo](https://github.com/facebookresearch/fairseq/tree/main/examples/roberta) for model developers
  - **Model Type:** Transformer-based language model
  - **Language(s):** English
  - **License:** MIT 
  - **Parent Model:** This model is a fine-tuned version of the RoBERTa large model. Users should see the [RoBERTa large model card](https://huggingface.co/roberta-large) for relevant information.
  - **Resources for more information:**
    - [Research Paper](https://arxiv.org/abs/1907.11692)
    - [GitHub Repo](https://github.com/facebookresearch/fairseq/tree/main/examples/roberta)
  
  ## How to Get Started with the Model 
  
  Use the code below to get started with the model. The model can be loaded with the zero-shot-classification pipeline like so:
  
  ```python
  from transformers import pipeline
  classifier = pipeline('zero-shot-classification', model='roberta-large-mnli')
  ```
  
  You can then use this pipeline to classify sequences into any of the class names you specify. For example:
  
  ```python
  sequence_to_classify = "one day I will see the world"
  candidate_labels = ['travel', 'cooking', 'dancing']
  classifier(sequence_to_classify, candidate_labels)
  ```
  
  ## Uses
  
  #### Direct Use
  
  This fine-tuned model can be used for zero-shot classification tasks, including zero-shot sentence-pair classification (see the [GitHub repo](https://github.com/facebookresearch/fairseq/tree/main/examples/roberta) for examples) and zero-shot sequence classification.
  
  #### Misuse and Out-of-scope Use
  
  The model should not be used to intentionally create hostile or alienating environments for people. In addition, the model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
  
  ## Risks, Limitations and Biases
  
  **CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propogate historical and current stereotypes.**
  
  Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). The [RoBERTa large model card](https://huggingface.co/roberta-large) notes that: "The training data used for this model contains a lot of unfiltered content from the internet, which is far from neutral." 
  
  Predictions generated by the model can include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups. For example:
  
  ```python
  sequence_to_classify = "The CEO had a strong handshake."
  candidate_labels = ['male', 'female']
  hypothesis_template = "This text speaks about a {} profession."
  classifier(sequence_to_classify, candidate_labels, hypothesis_template=hypothesis_template)
  ```
  
  Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
  
  ## Training
  
  #### Training Data
  
  This model was fine-tuned on the [Multi-Genre Natural Language Inference (MNLI)](https://cims.nyu.edu/~sbowman/multinli/) corpus. Also see the [MNLI data card](https://huggingface.co/datasets/multi_nli) for more information. 
  
  As described in the [RoBERTa large model card](https://huggingface.co/roberta-large): 
  
  > The RoBERTa model was pretrained on the reunion of five datasets:
  > 
  > - [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books;
  > - [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers) ;
  > - [CC-News](https://commoncrawl.org/2016/10/news-dataset-available/), a dataset containing 63 millions English news articles crawled between September 2016 and February 2019.
  > - [OpenWebText](https://github.com/jcpeterson/openwebtext), an opensource recreation of the WebText dataset used to train GPT-2,
  > - [Stories](https://arxiv.org/abs/1806.02847), a dataset containing a subs