readme: ---
  language: 
  - en
  - sp
  - ja
  - pe
  - hi
  - fr
  - ch
  - be
  - gu
  - ge
  - te
  - it
  - ar
  - po
  - ta
  - ma
  - ma
  - or
  - pa
  - po
  - ur
  - ga
  - he
  - ko
  - ca
  - th
  - du
  - in
  - vi
  - bu
  - fi
  - ce
  - la
  - tu
  - ru
  - cr
  - sw
  - yo
  - ku
  - bu
  - ma
  - cz
  - fi
  - so
  - ta
  - sw
  - si
  - ka
  - zh
  - ig
  - xh
  - ro
  - ha
  - es
  - sl
  - li
  - gr
  - ne
  - as
  - no
  
  tags:
  - text2text-generation
  
  widget:
  - text: "Translate to German:  My name is Arthur"
    example_title: "Translation"
  - text: "Please answer to the following question. Who is going to be the next Ballon d'or?"
    example_title: "Question Answering"
  - text: "Q: Can Geoffrey Hinton have a conversation with George Washington? Give the rationale before answering."
    example_title: "Logical reasoning"
  - text: "Please answer the following question. What is the boiling point of Nitrogen?"
    example_title: "Scientific knowledge"
  - text: "Answer the following yes/no question. Can you write a whole Haiku in a single tweet?"
    example_title: "Yes/no question"
  - text: "Answer the following yes/no question by reasoning step-by-step. Can you write a whole Haiku in a single tweet?"
    example_title: "Reasoning task"
  - text: "Q: ( False or not False or False ) is? A: Let's think step by step"
    example_title: "Boolean Expressions"
  - text: "The square root of x is the cube root of y. What is y to the power of 2, if x = 4?"
    example_title: "Math reasoning"
  - text: "Premise:  At my age you will probably have learnt one lesson. Hypothesis:  It's not certain how many lessons you'll learn by your thirties. Does the premise entail the hypothesis?"
    example_title: "Premise and hypothesis"
  
  datasets:
  - svakulenk0/qrecc
  - taskmaster2
  - djaym7/wiki_dialog
  - deepmind/code_contests
  - lambada
  - gsm8k
  - aqua_rat
  - esnli
  - quasc
  - qed
  
  
  license: apache-2.0
  ---
  
  # Model Card for FLAN-T5 base
  
  ![model image](https://s3.amazonaws.com/moonup/production/uploads/1666363435475-62441d1d9fdefb55a0b7d12c.png)
  
  #  Table of Contents
  
  0. [TL;DR](#TL;DR)
  1. [Model Details](#model-details)
  2. [Usage](#usage)
  3. [Uses](#uses)
  4. [Bias, Risks, and Limitations](#bias-risks-and-limitations)
  5. [Training Details](#training-details)
  6. [Evaluation](#evaluation)
  7. [Environmental Impact](#environmental-impact)
  8. [Citation](#citation)
  9. [Model Card Authors](#model-card-authors)
  
  # TL;DR
  
  If you already know T5, FLAN-T5 is just better at everything. For the same number of parameters, these models have been fine-tuned on more than 1000 additional tasks covering also more languages. 
  As mentioned in the first few lines of the abstract : 
  >  Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints,1 which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models.
  
  **Disclaimer**: Content from **this** model card has been written by the Hugging Face team, and parts of it were copy pasted from the [T5 model card](https://huggingface.co/t5-large).
  
  # Model Details
  
  ## Model Description
  
  
  - **Model type:** Language model
  - **Language(s) (NLP):** English, Spanish, Japanese, Persian, Hindi, French, Chinese, Bengali, Gujarati, German, Telugu, Italian, Arabic, Polish, Tamil, Marathi, Malayalam, Oriya, Panjabi, Portuguese, Urdu, Galician, Hebrew, Korean, Catalan, Thai, Dutch, Indonesian, Vietnamese, Bulgarian, Filipino, Central Khmer, Lao, Turkish, Russian, Croatian, Swedish, Yoruba, Kurdish, Burmese, Malay, Czech, Finnish, Somali, Tagalog, Swahili, Sinhala, Kannada, Zhuang, Igbo, Xhosa, Romanian, Haitian, Estonian, Slovak, Lithuanian, Greek, Nepali, Assamese, Norwegian
  - **License:** Apache 2.0
  - **Related Models:** [All FLAN-T5 Checkpoints](https://huggingface.co/models?search=flan-t5)
  - **Original Checkpoints:** [All Original FLAN-T5 Checkpoints](https://github.com/google-research/t5x/blob/main/docs/models.md#flan-t5-checkpoints)
  - **Resources for more information:**
    - [Research paper](https://arxiv.org/pdf/2210.11416.pdf)
    - [GitHub Repo](https://github.com/google-research/t5x)
    - [Hugging Face FLAN-T5 Docs (Similar to T5) ](https://huggingface.co/docs/transformers/model_doc/t5)
  
  # Usage
  
  Find below some example scripts on how to use the model in `transformers`:
  
  ## Using the Pytorch model
  
  ### Running the model on a CPU
  
  <details>
  <summary> Click to expand </summary>
  
  ```python
  
  from transformers import T5Tokenizer, T5ForConditionalGeneration
  
  tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-base")
  model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-base")
  
  input_text = "translate English to German: How old are you?"
  input_ids = tokenizer(input_text, return_tensors="pt").input_ids
  
  outputs = model.generate(input_ids)
  print(tokenizer.decode(outputs[0]))
  ```
  
  </details>
  
  ### Running the model on a GPU
  
  <details>
  <summary> Click to expand </summary>
  
  ```python
  # pip install accelera