readme: ---
  language: en
  inference: false
  tags:
  - text-generation
  - opt
  
  license: other
  commercial: false
  ---
  
  # OPT : Open Pre-trained Transformer Language Models
  
  OPT was first introduced in [Open Pre-trained Transformer Language Models](https://arxiv.org/abs/2205.01068) and first released in [metaseq's repository](https://github.com/facebookresearch/metaseq) on May 3rd 2022 by Meta AI.
  
  **Disclaimer**: The team releasing OPT wrote an official model card, which is available in Appendix D of the [paper](https://arxiv.org/pdf/2205.01068.pdf). 
  Content from **this** model card has been written by the Hugging Face team.
  
  ## Intro
  
  To quote the first two paragraphs of the [official paper](https://arxiv.org/abs/2205.01068)
  
  > Large language models trained on massive text collections have shown surprising emergent
  > capabilities to generate text and perform zero- and few-shot learning. While in some cases the public
  > can interact with these models through paid APIs, full model access is currently limited to only a
  > few highly resourced labs. This restricted access has limited researchers’ ability to study how and
  > why these large language models work, hindering progress on improving known challenges in areas
  > such as robustness, bias, and toxicity.
  
  > We present Open Pretrained Transformers (OPT), a suite of decoder-only pre-trained transformers ranging from 125M
  > to 175B parameters, which we aim to fully and responsibly share with interested researchers. We train the OPT models to roughly match 
  > the performance and sizes of the GPT-3 class of models, while also applying the latest best practices in data
  > collection and efficient training. Our aim in developing this suite of OPT models is to enable reproducible and responsible research at scale, and
  > to bring more voices to the table in studying the impact of these LLMs. Definitions of risk, harm, bias, and toxicity, etc., should be articulated by the
  > collective research community as a whole, which is only possible when models are available for study.
  
  ## Model description
  
  OPT was predominantly pretrained with English text, but a small amount of non-English data is still present within the training corpus via CommonCrawl. The model was pretrained using a causal language modeling (CLM) objective.
  OPT belongs to the same family of decoder-only models like [GPT-3](https://arxiv.org/abs/2005.14165). As such, it was pretrained using the self-supervised causal language modedling objective.
  
  For evaluation, OPT follows [GPT-3](https://arxiv.org/abs/2005.14165) by using their prompts and overall experimental setup. For more details, please read 
  the [official paper](https://arxiv.org/abs/2205.01068).
  ## Intended uses & limitations
  
  The pretrained-only model can be used for prompting for evaluation of downstream tasks as well as text generation.
  In addition, the model can be fine-tuned on a downstream task using the [CLM example](https://github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling). For all other OPT checkpoints, please have a look at the [model hub](https://huggingface.co/models?filter=opt).
  
  ### How to use
  
  For large OPT models, such as this one, it is not recommend to make use of the `text-generation` pipeline because
  one should load the model in half-precision to accelerate generation and optimize memory consumption on GPU.
  It is recommended to directly call the [`generate`](https://huggingface.co/docs/transformers/main/en/main_classes/text_generation#transformers.generation_utils.GenerationMixin.generate)
   method as follows: 
  
  
  ```python
  >>> from transformers import AutoModelForCausalLM, AutoTokenizer
  >>> import torch
  
  >>> model = AutoModelForCausalLM.from_pretrained("facebook/opt-66b", torch_dtype=torch.float16).cuda()
  
  >>> # the fast tokenizer currently does not work correctly
  >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-66b", use_fast=False)
  
  >>> prompt = "Hello, I am conscious and"
  
  
  >>> input_ids = tokenizer(prompt, return_tensors="pt").input_ids.cuda()
  
  >>> generated_ids = model.generate(input_ids)
  
  >>> tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
  ['Hello, I am conscious and I am here.\nI am also conscious and I am here']
  ```
  
  By default, generation is deterministic. In order to use the top-k sampling, please set `do_sample` to `True`. 
  
  ```python
  >>> from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed
  >>> import torch
  
  >>> model = AutoModelForCausalLM.from_pretrained("facebook/opt-66b", torch_dtype=torch.float16).cuda()
  
  >>> # the fast tokenizer currently does not work correctly
  >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-66b", use_fast=False)
  
  >>> prompt = "Hello, I am conscious and"
  
  >>> input_ids = tokenizer(prompt, return_tensors="pt").input_ids.cuda()
  
  >>> set_seed(32)
  >>> generated_ids = model.generate(input_ids, do_sample=True)
  
  >>> tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
  ['Hello, I am conscious and aware that you have y