readme: ---
  language: en
  license: mit
  ---
  
  # GPT-2 Medium
  
  ## Model Details
  
  **Model Description:** GPT-2 Medium is the **355M parameter** version of GPT-2, a transformer-based language model created and released by OpenAI. The model is a pretrained model on English language using a causal language modeling (CLM) objective. 
  
  - **Developed by:** OpenAI, see [associated research paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and [GitHub repo](https://github.com/openai/gpt-2) for model developers.
  - **Model Type:** Transformer-based language model
  - **Language(s):** English
  - **License:** [Modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE)
  - **Related Models:** [GPT2](https://huggingface.co/gpt2), [GPT2-Large](https://huggingface.co/gpt2-large) and [GPT2-XL](https://huggingface.co/gpt2-xl)
  - **Resources for more information:**
    - [Research Paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)
    - [OpenAI Blog Post](https://openai.com/blog/better-language-models/)
    - [GitHub Repo](https://github.com/openai/gpt-2)
    - [OpenAI Model Card for GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md)
    - Test the full generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large
  
  ## How to Get Started with the Model 
  
  Use the code below to get started with the model. You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we
  set a seed for reproducibility:
  
  ```python
  >>> from transformers import pipeline, set_seed
  >>> generator = pipeline('text-generation', model='gpt2-medium')
  >>> set_seed(42)
  >>> generator("Hello, I'm a language model,", max_length=30, num_return_sequences=5)
  
  [{'generated_text': "Hello, I'm a language model, I'm a language. I'm a compiler, I'm a parser, I'm a server process. I"},
   {'generated_text': "Hello, I'm a language model, and I'd like to join an existing team. What can I do to get started?\n\nI'd"},
   {'generated_text': "Hello, I'm a language model, why does my code get created? Can't I just copy it? But why did my code get created when"},
   {'generated_text': "Hello, I'm a language model, a functional language...\n\nI'm a functional language. Is it hard? A little, yes. But"},
   {'generated_text': "Hello, I'm a language model, not an object model.\n\nIn a nutshell, I need to give me objects from which I can get"}]
  ```
  
  Here is how to use this model to get the features of a given text in PyTorch:
  
  ```python
  from transformers import GPT2Tokenizer, GPT2Model
  tokenizer = GPT2Tokenizer.from_pretrained('gpt2-medium')
  model = GPT2Model.from_pretrained('gpt2-medium')
  text = "Replace me by any text you'd like."
  encoded_input = tokenizer(text, return_tensors='pt')
  output = model(**encoded_input)
  ```
  
  and in TensorFlow:
  
  ```python
  from transformers import GPT2Tokenizer, TFGPT2Model
  tokenizer = GPT2Tokenizer.from_pretrained('gpt2-medium')
  model = TFGPT2Model.from_pretrained('gpt2-medium')
  text = "Replace me by any text you'd like."
  encoded_input = tokenizer(text, return_tensors='tf')
  output = model(encoded_input)
  ```
  
  ## Uses
  
  #### Direct Use
  
  In their [model card about GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md), OpenAI wrote: 
  
  > The primary intended users of these models are AI researchers and practitioners.
  > 
  > We primarily imagine these language models will be used by researchers to better understand the behaviors, capabilities, biases, and constraints of large-scale generative language models.
  
  #### Downstream Use
  
  In their [model card about GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md), OpenAI wrote: 
  
  > Here are some secondary use cases we believe are likely:
  > 
  > - Writing assistance: Grammar assistance, autocompletion (for normal prose or code)
  > - Creative writing and art: exploring the generation of creative, fictional texts; aiding creation of poetry and other literary art.
  > - Entertainment: Creation of games, chat bots, and amusing generations.
  
  #### Misuse and Out-of-scope Use
  
  In their [model card about GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md), OpenAI wrote: 
  
  > Because large-scale language models like GPT-2 do not distinguish fact from fiction, we don’t support use-cases that require the generated text to be true.
  > 
  > Additionally, language models like GPT-2 reflect the biases inherent to the systems they were trained on, so we do not recommend that they be deployed into systems that interact with humans unless the deployers first carry out a study of biases relevant to the intended use-case. We found no statistically significant difference in gender, race, and religious bias probes between 774M and 1.5B, implying all versions of GPT-2 should be approached with similar levels of caution around use cases that are sensitive to biases around human attributes.
  
  ## Risks, Limi