---
local: mt5
sections:
- local: overview
  title: Overview
- local: transformers.MT5Config
  title: MT5Config
- local: transformers.T5Tokenizer
  title: MT5Tokenizer
- local: transformers.T5TokenizerFast
  title: MT5TokenizerFast
- local: transformers.MT5Model
  title: MT5Model
- local: transformers.MT5ForConditionalGeneration
  title: MT5ForConditionalGeneration
- local: transformers.MT5EncoderModel
  title: MT5EncoderModel
- local: transformers.TFMT5Model
  title: TFMT5Model
- local: transformers.TFMT5ForConditionalGeneration
  title: TFMT5ForConditionalGeneration
- local: transformers.TFMT5EncoderModel
  title: TFMT5EncoderModel
- local: transformers.FlaxMT5Model
  title: FlaxMT5Model
- local: transformers.FlaxMT5ForConditionalGeneration
  title: FlaxMT5ForConditionalGeneration
title: mT5
---
<script>
import Tip from "./Tip.svelte";
import Youtube from "./Youtube.svelte";
import Docstring from "./Docstring.svelte";
import CodeBlock from "./CodeBlock.svelte";
import CodeBlockFw from "./CodeBlockFw.svelte";
import DocNotebookDropdown from "./DocNotebookDropdown.svelte";
import IconCopyLink from "./IconCopyLink.svelte";
export let fw: "pt" | "tf"
</script>
<!--Copyright 2020 The HuggingFace Team. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

<h1 id="mt5">mT5</h1>

<h2 id="overview">Overview</h2>

The mT5 model was presented in [mT5: A massively multilingual pre-trained text-to-text transformer](https://arxiv.org/abs/2010.11934) by Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya
Siddhant, Aditya Barua, Colin Raffel.

The abstract from the paper is the following:

*The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified text-to-text format and scale to attain
state-of-the-art results on a wide variety of English-language NLP tasks. In this paper, we introduce mT5, a
multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages. We detail
the design and modified training of mT5 and demonstrate its state-of-the-art performance on many multilingual
benchmarks. We also describe a simple technique to prevent "accidental translation" in the zero-shot setting, where a
generative model chooses to (partially) translate its prediction into the wrong language. All of the code and model
checkpoints used in this work are publicly available.*

Note: mT5 was only pre-trained on [mC4](https://huggingface.co/datasets/mc4) excluding any supervised training.
Therefore, this model has to be fine-tuned before it is useable on a downstream task, unlike the original T5 model.
Since mT5 was pre-trained unsupervisedly, there's no real advantage to using a task prefix during single-task
fine-tuning. If you are doing multi-task fine-tuning, you should use a prefix.

Google has released the following variants:

- [google/mt5-small](https://huggingface.co/google/mt5-small)

- [google/mt5-base](https://huggingface.co/google/mt5-base)

- [google/mt5-large](https://huggingface.co/google/mt5-large)

- [google/mt5-xl](https://huggingface.co/google/mt5-xl)

- [google/mt5-xxl](https://huggingface.co/google/mt5-xxl).

This model was contributed by [patrickvonplaten](https://huggingface.co/patrickvonplaten). The original code can be
found [here](https://github.com/google-research/multilingual-t5).

<h2 id="transformers.MT5Config">MT5Config</h2>

<div class="docstring">

<docstring><name>class transformers.MT5Config</name><anchor>transformers.MT5Config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/mt5/configuration_mt5.py#L24</source><parameters>[{"name": "vocab_size", "val": " = 250112"}, {"name": "d_model", "val": " = 512"}, {"name": "d_kv", "val": " = 64"}, {"name": "d_ff", "val": " = 1024"}, {"name": "num_layers", "val": " = 8"}, {"name": "num_decoder_layers", "val": " = None"}, {"name": "num_heads", "val": " = 6"}, {"name": "relative_attention_num_buckets", "val": " = 32"}, {"name": "dropout_rate", "val": " = 0.1"}, {"name": "layer_norm_epsilon", "val": " = 1e-06"}, {"name": "initializer_factor", "val": " = 1.0"}, {"name": "feed_forward_proj", "val": " = 'gated-gelu'"}, {"name": "is_encoder_decoder", "val": " = True"}, {"name": "use_cache", "val": " = True"}, {"name": "tokenizer_class", "val": " = 'T5Tokenizer'"}, {"name": "tie_word_embeddings", "val": " = False"}, {"name": "pad_token_id", "val": " = 0"}, {"name": "eos_token_id", "val": " = 1"}, {"name": "decoder_start_token_id", "val": " = 0"}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **vocab_size** (`int`, *optional*, defaults to 250112) --
  Vocabulary size of the T5 model. Defines the number of different tokens that can be represented by the
  `inputs_ids` passed when calling [T5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Model) or [TFT5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.TFT5Model).
- **d_model** (`int`, *optional*, defaults to 512) --
  Size of the encoder layers and the pooler layer.
- **d_kv** (`int`, *optional*, defaults to 64) --
  Size of the key, query, value projections per attention head. `d_kv` has to be equal to `d_model //
  num_heads`.
- **d_ff** (`int`, *optional*, defaults to 1024) --
  Size of the intermediate feed forward layer in each `T5Block`.
- **num_layers** (`int`, *optional*, defaults to 8) --
  Number of hidden layers in the Transformer encoder.
- **num_decoder_layers** (`int`, *optional*) --
  Number of hidden layers in the Transformer decoder. Will use the same value as `num_layers` if not set.
- **num_heads** (`int`, *optional*, defaults to 6) --
  Number of attention heads for each attention layer in the Transformer encoder.
- **relative_attention_num_buckets** (`int`, *optional*, defaults to 32) --
  The number of buckets to use for each attention layer.
- **dropout_rate** (`float`, *optional*, defaults to 0.1) --
  The ratio for all dropout layers.
- **layer_norm_eps** (`float`, *optional*, defaults to 1e-6) --
  The epsilon used by the layer normalization layers.
- **initializer_factor** (`float`, *optional*, defaults to 1) --
  A factor for initializing all weight matrices (should be kept to 1, used internally for initialization
  testing).
- **feed_forward_proj** (`string`, *optional*, defaults to `"gated-gelu"`) --
  Type of feed forward layer to be used. Should be one of `"relu"` or `"gated-gelu"`.
- **use_cache** (`bool`, *optional*, defaults to `True`) --
  Whether or not the model should return the last key/values attentions (not used by all models).</paramsdesc><paramgroups>0</paramgroups></docstring>

This is the configuration class to store the configuration of a [MT5Model](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Model) or a [TFMT5Model](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.TFMT5Model). It is used to
instantiate a mT5 model according to the specified arguments, defining the model architecture. Instantiating a
configuration with the defaults will yield a similar configuration to that of the mT5
[google/mt5-small](https://huggingface.co/google/mt5-small) architecture.

Configuration objects inherit from [PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig) and can be used to control the model outputs. Read the
documentation from [PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig) for more information.




</div>

<h2 id="transformers.T5Tokenizer">MT5Tokenizer</h2>

<div class="docstring">

<docstring><name>class transformers.T5Tokenizer</name><anchor>transformers.T5Tokenizer</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/t5/tokenization_t5.py#L53</source><parameters>[{"name": "vocab_file", "val": ""}, {"name": "eos_token", "val": " = '</s>'"}, {"name": "unk_token", "val": " = '<unk>'"}, {"name": "pad_token", "val": " = '<pad>'"}, {"name": "extra_ids", "val": " = 100"}, {"name": "additional_special_tokens", "val": " = None"}, {"name": "sp_model_kwargs", "val": ": typing.Union[typing.Dict[str, typing.Any], NoneType] = None"}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **vocab_file** (`str`) --
[SentencePiece](https://github.com/google/sentencepiece) file (generally has a *.spm* extension) that
contains the vocabulary necessary to instantiate a tokenizer.
- **eos_token** (`str`, *optional*, defaults to `"&amp;lt;/s>"`) --
  The end of sequence token.

  <Tip>

  When building a sequence using special tokens, this is not the token that is used for the end of sequence.
  The token used is the `sep_token`.

  </Tip>

- **unk_token** (`str`, *optional*, defaults to `"&amp;lt;unk>"`) --
  The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
  token instead.
- **pad_token** (`str`, *optional*, defaults to `"&amp;lt;pad>"`) --
  The token used for padding, for example when batching sequences of different lengths.
- **extra_ids** (`int`, *optional*, defaults to 100) --
  Add a number of extra ids added to the end of the vocabulary for use as sentinels. These tokens are
  accessible as "&amp;lt;extra_id_&amp;lcub;%d}>" where "&amp;lcub;%d}" is a number between 0 and extra_ids-1. Extra tokens are
  indexed from the end of the vocabulary up to beginning ("&amp;lt;extra_id_0>" is the last token in the vocabulary
  like in T5 preprocessing see
  [here](https://github.com/google-research/text-to-text-transfer-transformer/blob/9fd7b14a769417be33bc6c850f9598764913c833/t5/data/preprocessors.py#L2117)).
- **additional_special_tokens** (`List[str]`, *optional*) --
  Additional special tokens used by the tokenizer.
- **sp_model_kwargs** (`dict`, *optional*) --
  Will be passed to the `SentencePieceProcessor.__init__()` method. The [Python wrapper for
  SentencePiece](https://github.com/google/sentencepiece/tree/master/python) can be used, among other things,
  to set:

  - `enable_sampling`: Enable subword regularization.
  - `nbest_size`: Sampling parameters for unigram. Invalid for BPE-Dropout.

    - `nbest_size = &amp;lcub;0,1}`: No sampling is performed.
    - `nbest_size > 1`: samples from the nbest_size results.
    - `nbest_size &amp;lt; 0`: assuming that nbest_size is infinite and samples from the all hypothesis (lattice)
      using forward-filtering-and-backward-sampling algorithm.

  - `alpha`: Smoothing parameter for unigram sampling, and dropout probability of merge operations for
    BPE-dropout.
- **sp_model** (`SentencePieceProcessor`) --
  The *SentencePiece* processor that is used for every conversion (string, tokens and IDs).</paramsdesc><paramgroups>0</paramgroups></docstring>

Construct a T5 tokenizer. Based on [SentencePiece](https://github.com/google/sentencepiece).

This tokenizer inherits from [PreTrainedTokenizer](/docs/transformers/pr_15592/en/main_classes/tokenizer#transformers.PreTrainedTokenizer) which contains most of the main methods. Users should refer to
this superclass for more information regarding those methods.









<div class="docstring">
<docstring><name>build\_inputs\_with\_special\_tokens</name><anchor>transformers.T5Tokenizer.build_inputs_with_special_tokens</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/t5/tokenization_t5.py#L223</source><parameters>[{"name": "token_ids_0", "val": ": typing.List[int]"}, {"name": "token_ids_1", "val": ": typing.Optional[typing.List[int]] = None"}]</parameters><paramsdesc>- **token_ids_0** (`List[int]`) --
  List of IDs to which the special tokens will be added.
- **token_ids_1** (`List[int]`, *optional*) --
  Optional second list of IDs for sequence pairs.</paramsdesc><paramgroups>0</paramgroups><rettype>`List[int]`</rettype><retdesc>List of [input IDs](../glossary#input-ids) with the appropriate special tokens.</retdesc></docstring>

Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and
adding special tokens. A sequence has the following format:

- single sequence: `X &amp;lt;/s>`
- pair of sequences: `A &amp;lt;/s> B &amp;lt;/s>`








</div>
<div class="docstring">
<docstring><name>convert\_tokens\_to\_string</name><anchor>transformers.T5Tokenizer.convert_tokens_to_string</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/t5/tokenization_t5.py#L284</source><parameters>[{"name": "tokens", "val": ""}]</parameters></docstring>
Converts a sequence of tokens (string) in a single string.

</div>
<div class="docstring">
<docstring><name>create\_token\_type\_ids\_from\_sequences</name><anchor>transformers.T5Tokenizer.create_token_type_ids_from_sequences</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/t5/tokenization_t5.py#L201</source><parameters>[{"name": "token_ids_0", "val": ": typing.List[int]"}, {"name": "token_ids_1", "val": ": typing.Optional[typing.List[int]] = None"}]</parameters><paramsdesc>- **token_ids_0** (`List[int]`) --
  List of IDs.
- **token_ids_1** (`List[int]`, *optional*) --
  Optional second list of IDs for sequence pairs.</paramsdesc><paramgroups>0</paramgroups><rettype>`List[int]`</rettype><retdesc>List of zeros.</retdesc></docstring>

Create a mask from the two sequences passed to be used in a sequence-pair classification task. T5 does not make
use of token type ids, therefore a list of zeros is returned.








</div>
<div class="docstring">
<docstring><name>get\_special\_tokens\_mask</name><anchor>transformers.T5Tokenizer.get_special_tokens_mask</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/t5/tokenization_t5.py#L163</source><parameters>[{"name": "token_ids_0", "val": ": typing.List[int]"}, {"name": "token_ids_1", "val": ": typing.Optional[typing.List[int]] = None"}, {"name": "already_has_special_tokens", "val": ": bool = False"}]</parameters><paramsdesc>- **token_ids_0** (`List[int]`) --
  List of IDs.
- **token_ids_1** (`List[int]`, *optional*) --
  Optional second list of IDs for sequence pairs.
- **already_has_special_tokens** (`bool`, *optional*, defaults to `False`) --
  Whether or not the token list is already formatted with special tokens for the model.</paramsdesc><paramgroups>0</paramgroups><rettype>`List[int]`</rettype><retdesc>A list of integers in the range [0, 1]: 1 for a special token, 0 for a sequence token.</retdesc></docstring>

Retrieve sequence ids from a token list that has no special tokens added. This method is called when adding
special tokens using the tokenizer `prepare_for_model` method.








</div></div>

See [T5Tokenizer](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.T5Tokenizer) for all details.


<h2 id="transformers.T5TokenizerFast">MT5TokenizerFast</h2>

<div class="docstring">

<docstring><name>class transformers.T5TokenizerFast</name><anchor>transformers.T5TokenizerFast</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/t5/tokenization_t5_fast.py#L63</source><parameters>[{"name": "vocab_file", "val": " = None"}, {"name": "tokenizer_file", "val": " = None"}, {"name": "eos_token", "val": " = '</s>'"}, {"name": "unk_token", "val": " = '<unk>'"}, {"name": "pad_token", "val": " = '<pad>'"}, {"name": "extra_ids", "val": " = 100"}, {"name": "additional_special_tokens", "val": " = None"}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **vocab_file** (`str`) --
  [SentencePiece](https://github.com/google/sentencepiece) file (generally has a *.spm* extension) that
  contains the vocabulary necessary to instantiate a tokenizer.
- **eos_token** (`str`, *optional*, defaults to `"&amp;lt;/s>"`) --
  The end of sequence token.

  <Tip>

  When building a sequence using special tokens, this is not the token that is used for the end of sequence.
  The token used is the `sep_token`.

  </Tip>

- **unk_token** (`str`, *optional*, defaults to `"&amp;lt;unk>"`) --
  The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
  token instead.
- **pad_token** (`str`, *optional*, defaults to `"&amp;lt;pad>"`) --
  The token used for padding, for example when batching sequences of different lengths.
- **extra_ids** (`int`, *optional*, defaults to 100) --
  Add a number of extra ids added to the end of the vocabulary for use as sentinels. These tokens are
  accessible as "&amp;lt;extra_id_&amp;lcub;%d}>" where "&amp;lcub;%d}" is a number between 0 and extra_ids-1. Extra tokens are
  indexed from the end of the vocabulary up to beginning ("&amp;lt;extra_id_0>" is the last token in the vocabulary
  like in T5 preprocessing see
  [here](https://github.com/google-research/text-to-text-transfer-transformer/blob/9fd7b14a769417be33bc6c850f9598764913c833/t5/data/preprocessors.py#L2117)).
- **additional_special_tokens** (`List[str]`, *optional*) --
  Additional special tokens used by the tokenizer.</paramsdesc><paramgroups>0</paramgroups></docstring>

Construct a "fast" T5 tokenizer (backed by HuggingFace's *tokenizers* library). Based on
[Unigram](https://huggingface.co/docs/tokenizers/python/latest/components.html?highlight=unigram#models).

This tokenizer inherits from [PreTrainedTokenizerFast](/docs/transformers/pr_15592/en/main_classes/tokenizer#transformers.PreTrainedTokenizerFast) which contains most of the main methods. Users should
refer to this superclass for more information regarding those methods.





<div class="docstring">
<docstring><name>build\_inputs\_with\_special\_tokens</name><anchor>transformers.T5TokenizerFast.build_inputs_with_special_tokens</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/t5/tokenization_t5_fast.py#L166</source><parameters>[{"name": "token_ids_0", "val": ": typing.List[int]"}, {"name": "token_ids_1", "val": ": typing.Optional[typing.List[int]] = None"}]</parameters><paramsdesc>- **token_ids_0** (`List[int]`) --
  List of IDs to which the special tokens will be added.
- **token_ids_1** (`List[int]`, *optional*) --
  Optional second list of IDs for sequence pairs.</paramsdesc><paramgroups>0</paramgroups><rettype>`List[int]`</rettype><retdesc>List of [input IDs](../glossary#input-ids) with the appropriate special tokens.</retdesc></docstring>

Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and
adding special tokens. A sequence has the following format:

- single sequence: `X &amp;lt;/s>`
- pair of sequences: `A &amp;lt;/s> B &amp;lt;/s>`








</div>
<div class="docstring">
<docstring><name>create\_token\_type\_ids\_from\_sequences</name><anchor>transformers.T5TokenizerFast.create_token_type_ids_from_sequences</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/t5/tokenization_t5_fast.py#L192</source><parameters>[{"name": "token_ids_0", "val": ": typing.List[int]"}, {"name": "token_ids_1", "val": ": typing.Optional[typing.List[int]] = None"}]</parameters><paramsdesc>- **token_ids_0** (`List[int]`) --
  List of IDs.
- **token_ids_1** (`List[int]`, *optional*) --
  Optional second list of IDs for sequence pairs.</paramsdesc><paramgroups>0</paramgroups><rettype>`List[int]`</rettype><retdesc>List of zeros.</retdesc></docstring>

Create a mask from the two sequences passed to be used in a sequence-pair classification task. T5 does not make
use of token type ids, therefore a list of zeros is returned.








</div></div>

See [T5TokenizerFast](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.T5TokenizerFast) for all details.


<h2 id="transformers.MT5Model">MT5Model</h2>

<div class="docstring">

<docstring><name>class transformers.MT5Model</name><anchor>transformers.MT5Model</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/mt5/modeling_mt5.py#L28</source><parameters>[{"name": "config", "val": ": T5Config"}]</parameters></docstring>

This class overrides [T5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Model). Please check the superclass for the appropriate documentation alongside usage
examples.

Examples:

```python
>>> from transformers import MT5Model, T5Tokenizer

>>> model = MT5Model.from_pretrained("google/mt5-small")
>>> tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")
>>> article = "UN Offizier sagt, dass weiter verhandelt werden muss in Syrien."
>>> summary = "Weiter Verhandlung in Syrien."
>>> inputs = tokenizer(article, return_tensors="pt")
>>> with tokenizer.as_target_tokenizer():
...     labels = tokenizer(summary, return_tensors="pt")

>>> outputs = model(input_ids=inputs["input_ids"], decoder_input_ids=labels["input_ids"])
>>> hidden_states = outputs.last_hidden_state
```

</div>

<h2 id="transformers.MT5ForConditionalGeneration">MT5ForConditionalGeneration</h2>

<div class="docstring">

<docstring><name>class transformers.MT5ForConditionalGeneration</name><anchor>transformers.MT5ForConditionalGeneration</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/mt5/modeling_mt5.py#L62</source><parameters>[{"name": "config", "val": ""}]</parameters></docstring>

This class overrides [T5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5ForConditionalGeneration). Please check the superclass for the appropriate documentation
alongside usage examples.

Examples:

```python
>>> from transformers import MT5ForConditionalGeneration, T5Tokenizer

>>> model = MT5ForConditionalGeneration.from_pretrained("google/mt5-small")
>>> tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")
>>> article = "UN Offizier sagt, dass weiter verhandelt werden muss in Syrien."
>>> summary = "Weiter Verhandlung in Syrien."
>>> inputs = tokenizer(article, return_tensors="pt")
>>> with tokenizer.as_target_tokenizer():
...     labels = tokenizer(summary, return_tensors="pt")

>>> outputs = model(**inputs, labels=labels["input_ids"])
>>> loss = outputs.loss
```

</div>

<h2 id="transformers.MT5EncoderModel">MT5EncoderModel</h2>

<div class="docstring">

<docstring><name>class transformers.MT5EncoderModel</name><anchor>transformers.MT5EncoderModel</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/mt5/modeling_mt5.py#L94</source><parameters>[{"name": "config", "val": ": T5Config"}]</parameters></docstring>

This class overrides [T5EncoderModel](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5EncoderModel). Please check the superclass for the appropriate documentation alongside
usage examples.

Examples:

```python
>>> from transformers import MT5EncoderModel, T5Tokenizer

>>> model = MT5EncoderModel.from_pretrained("google/mt5-small")
>>> tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")
>>> article = "UN Offizier sagt, dass weiter verhandelt werden muss in Syrien."
>>> input_ids = tokenizer(article, return_tensors="pt").input_ids
>>> outputs = model(input_ids)
>>> hidden_state = outputs.last_hidden_state
```

</div>

<h2 id="transformers.TFMT5Model">TFMT5Model</h2>

<div class="docstring">

<docstring><name>class transformers.TFMT5Model</name><anchor>transformers.TFMT5Model</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/mt5/modeling_tf_mt5.py#L28</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This class overrides [TFT5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.TFT5Model). Please check the superclass for the appropriate documentation alongside usage
examples.

Examples:

```python
>>> from transformers import TFMT5Model, T5Tokenizer

>>> model = TFMT5Model.from_pretrained("google/mt5-small")
>>> tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")
>>> article = "UN Offizier sagt, dass weiter verhandelt werden muss in Syrien."
>>> summary = "Weiter Verhandlung in Syrien."
>>> inputs = tokenizer(article, return_tensors="tf")
>>> with tokenizer.as_target_tokenizer():
...     labels = tokenizer(summary, return_tensors="tf")

>>> outputs = model(input_ids=inputs["input_ids"], decoder_input_ids=labels["input_ids"])
>>> hidden_states = outputs.last_hidden_state
```

</div>

<h2 id="transformers.TFMT5ForConditionalGeneration">TFMT5ForConditionalGeneration</h2>

<div class="docstring">

<docstring><name>class transformers.TFMT5ForConditionalGeneration</name><anchor>transformers.TFMT5ForConditionalGeneration</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/mt5/modeling_tf_mt5.py#L53</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This class overrides [TFT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.TFT5ForConditionalGeneration). Please check the superclass for the appropriate
documentation alongside usage examples.

Examples:

```python
>>> from transformers import TFMT5ForConditionalGeneration, T5Tokenizer

>>> model = TFMT5ForConditionalGeneration.from_pretrained("google/mt5-small")
>>> tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")
>>> article = "UN Offizier sagt, dass weiter verhandelt werden muss in Syrien."
>>> summary = "Weiter Verhandlung in Syrien."
>>> inputs = tokenizer(article, return_tensors="tf")
>>> with tokenizer.as_target_tokenizer():
...     labels = tokenizer(summary, return_tensors="tf")

>>> outputs = model(**inputs, labels=labels["input_ids"])
>>> loss = outputs.loss
```

</div>

<h2 id="transformers.TFMT5EncoderModel">TFMT5EncoderModel</h2>

<div class="docstring">

<docstring><name>class transformers.TFMT5EncoderModel</name><anchor>transformers.TFMT5EncoderModel</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/mt5/modeling_tf_mt5.py#L79</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This class overrides [TFT5EncoderModel](/docs/transformers/pr_15592/en/model_doc/t5#transformers.TFT5EncoderModel). Please check the superclass for the appropriate documentation alongside
usage examples.

Examples:

```python
>>> from transformers import TFMT5EncoderModel, T5Tokenizer

>>> model = TFMT5EncoderModel.from_pretrained("google/mt5-small")
>>> tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")
>>> article = "UN Offizier sagt, dass weiter verhandelt werden muss in Syrien."
>>> input_ids = tokenizer(article, return_tensors="tf").input_ids
>>> outputs = model(input_ids)
>>> hidden_state = outputs.last_hidden_state
```

</div>

<h2 id="transformers.FlaxMT5Model">FlaxMT5Model</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxMT5Model</name><anchor>transformers.FlaxMT5Model</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/mt5/modeling_flax_mt5.py#L28</source><parameters>[{"name": "config", "val": ": T5Config"}, {"name": "input_shape", "val": ": typing.Tuple[int] = (1, 1)"}, {"name": "seed", "val": ": int = 0"}, {"name": "dtype", "val": ": dtype = <class 'jax._src.numpy.lax_numpy.float32'>"}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This class overrides [FlaxT5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.FlaxT5Model). Please check the superclass for the appropriate documentation alongside usage
examples.

Examples:

```python
>>> from transformers import FlaxMT5Model, T5Tokenizer

>>> model = FlaxMT5Model.from_pretrained("google/mt5-small")
>>> tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")

>>> article = "UN Offizier sagt, dass weiter verhandelt werden muss in Syrien."
>>> summary = "Weiter Verhandlung in Syrien."
>>> inputs = tokenizer(article, return_tensors="np")

>>> with tokenizer.as_target_tokenizer():
...     decoder_input_ids = tokenizer(summary, return_tensors="np").input_ids

>>> outputs = model(input_ids=inputs["input_ids"], decoder_input_ids=decoder_input_ids)
>>> hidden_states = outputs.last_hidden_state
```

</div>

<h2 id="transformers.FlaxMT5ForConditionalGeneration">FlaxMT5ForConditionalGeneration</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxMT5ForConditionalGeneration</name><anchor>transformers.FlaxMT5ForConditionalGeneration</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/mt5/modeling_flax_mt5.py#L55</source><parameters>[{"name": "config", "val": ": T5Config"}, {"name": "input_shape", "val": ": typing.Tuple[int] = (1, 1)"}, {"name": "seed", "val": ": int = 0"}, {"name": "dtype", "val": ": dtype = <class 'jax._src.numpy.lax_numpy.float32'>"}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This class overrides [FlaxT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.FlaxT5ForConditionalGeneration). Please check the superclass for the appropriate
documentation alongside usage examples.

Examples:

```python
>>> from transformers import FlaxMT5ForConditionalGeneration, T5Tokenizer

>>> model = FlaxMT5ForConditionalGeneration.from_pretrained("google/mt5-small")
>>> tokenizer = T5Tokenizer.from_pretrained("google/mt5-small")

>>> article = "UN Offizier sagt, dass weiter verhandelt werden muss in Syrien."
>>> summary = "Weiter Verhandlung in Syrien."
>>> inputs = tokenizer(article, return_tensors="np")

>>> with tokenizer.as_target_tokenizer():
...     decoder_input_ids = tokenizer(summary, return_tensors="np").input_ids

>>> outputs = model(**inputs, decoder_input_ids=decoder_input_ids)
>>> logits = outputs.logits
```

</div>
