---
local: auto-classes
sections:
- local: extending-the-auto-classes
  title: Extending the Auto Classes
- local: transformers.AutoConfig
  title: AutoConfig
- local: transformers.AutoTokenizer
  title: AutoTokenizer
- local: transformers.AutoFeatureExtractor
  title: AutoFeatureExtractor
- local: transformers.AutoProcessor
  title: AutoProcessor
- local: transformers.AutoModel
  title: AutoModel
- local: transformers.AutoModelForPreTraining
  title: AutoModelForPreTraining
- local: transformers.AutoModelForCausalLM
  title: AutoModelForCausalLM
- local: transformers.AutoModelForMaskedLM
  title: AutoModelForMaskedLM
- local: transformers.AutoModelForSeq2SeqLM
  title: AutoModelForSeq2SeqLM
- local: transformers.AutoModelForSequenceClassification
  title: AutoModelForSequenceClassification
- local: transformers.AutoModelForMultipleChoice
  title: AutoModelForMultipleChoice
- local: transformers.AutoModelForNextSentencePrediction
  title: AutoModelForNextSentencePrediction
- local: transformers.AutoModelForTokenClassification
  title: AutoModelForTokenClassification
- local: transformers.AutoModelForQuestionAnswering
  title: AutoModelForQuestionAnswering
- local: transformers.AutoModelForTableQuestionAnswering
  title: AutoModelForTableQuestionAnswering
- local: transformers.AutoModelForImageClassification
  title: AutoModelForImageClassification
- local: transformers.AutoModelForVision2Seq
  title: AutoModelForVision2Seq
- local: transformers.AutoModelForAudioClassification
  title: AutoModelForAudioClassification
- local: transformers.AutoModelForAudioFrameClassification
  title: AutoModelForAudioFrameClassification
- local: transformers.AutoModelForCTC
  title: AutoModelForCTC
- local: transformers.AutoModelForSpeechSeq2Seq
  title: AutoModelForSpeechSeq2Seq
- local: transformers.AutoModelForAudioXVector
  title: AutoModelForAudioXVector
- local: transformers.AutoModelForMaskedImageModeling
  title: AutoModelForMaskedImageModeling
- local: transformers.AutoModelForObjectDetection
  title: AutoModelForObjectDetection
- local: transformers.AutoModelForImageSegmentation
  title: AutoModelForImageSegmentation
- local: transformers.AutoModelForSemanticSegmentation
  title: AutoModelForSemanticSegmentation
- local: transformers.TFAutoModel
  title: TFAutoModel
- local: transformers.TFAutoModelForPreTraining
  title: TFAutoModelForPreTraining
- local: transformers.TFAutoModelForCausalLM
  title: TFAutoModelForCausalLM
- local: transformers.TFAutoModelForImageClassification
  title: TFAutoModelForImageClassification
- local: transformers.TFAutoModelForMaskedLM
  title: TFAutoModelForMaskedLM
- local: transformers.TFAutoModelForSeq2SeqLM
  title: TFAutoModelForSeq2SeqLM
- local: transformers.TFAutoModelForSequenceClassification
  title: TFAutoModelForSequenceClassification
- local: transformers.TFAutoModelForMultipleChoice
  title: TFAutoModelForMultipleChoice
- local: transformers.TFAutoModelForTableQuestionAnswering
  title: TFAutoModelForTableQuestionAnswering
- local: transformers.TFAutoModelForTokenClassification
  title: TFAutoModelForTokenClassification
- local: transformers.TFAutoModelForQuestionAnswering
  title: TFAutoModelForQuestionAnswering
- local: transformers.TFAutoModelForVision2Seq
  title: TFAutoModelForVision2Seq
- local: transformers.TFAutoModelForSpeechSeq2Seq
  title: TFAutoModelForSpeechSeq2Seq
- local: transformers.FlaxAutoModel
  title: FlaxAutoModel
- local: transformers.FlaxAutoModelForCausalLM
  title: FlaxAutoModelForCausalLM
- local: transformers.FlaxAutoModelForPreTraining
  title: FlaxAutoModelForPreTraining
- local: transformers.FlaxAutoModelForMaskedLM
  title: FlaxAutoModelForMaskedLM
- local: transformers.FlaxAutoModelForSeq2SeqLM
  title: FlaxAutoModelForSeq2SeqLM
- local: transformers.FlaxAutoModelForSequenceClassification
  title: FlaxAutoModelForSequenceClassification
- local: transformers.FlaxAutoModelForQuestionAnswering
  title: FlaxAutoModelForQuestionAnswering
- local: transformers.FlaxAutoModelForTokenClassification
  title: FlaxAutoModelForTokenClassification
- local: transformers.FlaxAutoModelForMultipleChoice
  title: FlaxAutoModelForMultipleChoice
- local: transformers.FlaxAutoModelForNextSentencePrediction
  title: FlaxAutoModelForNextSentencePrediction
- local: transformers.FlaxAutoModelForImageClassification
  title: FlaxAutoModelForImageClassification
- local: transformers.FlaxAutoModelForVision2Seq
  title: FlaxAutoModelForVision2Seq
title: Auto Classes
---
<script>
import Tip from "./Tip.svelte";
import Youtube from "./Youtube.svelte";
import Docstring from "./Docstring.svelte";
import CodeBlock from "./CodeBlock.svelte";
import CodeBlockFw from "./CodeBlockFw.svelte";
import DocNotebookDropdown from "./DocNotebookDropdown.svelte";
import IconCopyLink from "./IconCopyLink.svelte";
export let fw: "pt" | "tf"
</script>
<!--Copyright 2020 The HuggingFace Team. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

<h1 id="auto-classes">Auto Classes</h1>

In many cases, the architecture you want to use can be guessed from the name or the path of the pretrained model you
are supplying to the `from_pretrained()` method. AutoClasses are here to do this job for you so that you
automatically retrieve the relevant model given the name/path to the pretrained weights/config/vocabulary.

Instantiating one of [AutoConfig](/docs/transformers/pr_15592/en/model_doc/auto#transformers.AutoConfig), [AutoModel](/docs/transformers/pr_15592/en/model_doc/auto#transformers.AutoModel), and
[AutoTokenizer](/docs/transformers/pr_15592/en/model_doc/auto#transformers.AutoTokenizer) will directly create a class of the relevant architecture. For instance


```python
model = AutoModel.from_pretrained("bert-base-cased")
```

will create a model that is an instance of [BertModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertModel).

There is one class of `AutoModel` for each task, and for each backend (PyTorch, TensorFlow, or Flax).

<h2 id="extending-the-auto-classes">Extending the Auto Classes</h2>

Each of the auto classes has a method to be extended with your custom classes. For instance, if you have defined a
custom class of model `NewModel`, make sure you have a `NewModelConfig` then you can add those to the auto
classes like this:

```python
from transformers import AutoConfig, AutoModel

AutoConfig.register("new-model", NewModelConfig)
AutoModel.register(NewModelConfig, NewModel)
```

You will then be able to use the auto classes like you would usually do!

<Tip warning=&amp;lcub;true}>

If your `NewModelConfig` is a subclass of `PretrainedConfig`, make sure its
`model_type` attribute is set to the same key you use when registering the config (here `"new-model"`).

Likewise, if your `NewModel` is a subclass of [PreTrainedModel](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel), make sure its
`config_class` attribute is set to the same class you use when registering the model (here
`NewModelConfig`).

</Tip>

<h2 id="transformers.AutoConfig">AutoConfig</h2>

<div class="docstring">

<docstring><name>class transformers.AutoConfig</name><anchor>transformers.AutoConfig</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/configuration_auto.py#L515</source><parameters>[]</parameters></docstring>

This is a generic configuration class that will be instantiated as one of the configuration classes of the library
when created with the [from_pretrained()](/docs/transformers/pr_15592/en/model_doc/auto#transformers.AutoConfig.from_pretrained) class method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.AutoConfig.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/configuration_auto.py#L538</source><parameters>[{"name": "pretrained_model_name_or_path", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model configuration hosted inside a model repo on
    huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or
    namespaced under a user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing a configuration file saved using the
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.save_pretrained) method, or the [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) method,
    e.g., `./my_model_directory/`.
  - A path or url to a saved configuration JSON *file*, e.g.,
    `./my_model_directory/configuration.json`.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download the model weights and configuration files and override the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **return_unused_kwargs** (`bool`, *optional*, defaults to `False`) --
  If `False`, then this function returns just the final configuration object.

  If `True`, then this functions returns a `Tuple(config, unused_kwargs)` where *unused_kwargs* is a
  dictionary consisting of the key/value pairs whose keys are not configuration attributes: i.e., the
  part of `kwargs` which has not been used to update `config` and is otherwise ignored.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs(additional** keyword arguments, *optional*) --
  The values in kwargs of any keys which are configuration attributes will be used to override the loaded
  values. Behavior concerning key/value pairs whose keys are *not* configuration attributes is controlled
  by the `return_unused_kwargs` keyword parameter.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the configuration classes of the library from a pretrained model configuration.

The configuration class to instantiate is selected based on the `model_type` property of the config object that
is loaded, or when it's missing, by falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) (ALBERT model)
- **bart** -- [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) (BART model)
- **beit** -- [BeitConfig](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitConfig) (BEiT model)
- **bert** -- [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) (BERT model)
- **bert-generation** -- [BertGenerationConfig](/docs/transformers/pr_15592/en/model_doc/bert-generation#transformers.BertGenerationConfig) (Bert Generation model)
- **big_bird** -- [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) (BigBird model)
- **bigbird_pegasus** -- [BigBirdPegasusConfig](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusConfig) (BigBirdPegasus model)
- **blenderbot** -- [BlenderbotConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotConfig) (Blenderbot model)
- **blenderbot-small** -- [BlenderbotSmallConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallConfig) (BlenderbotSmall model)
- **camembert** -- [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) (CamemBERT model)
- **canine** -- [CanineConfig](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineConfig) (Canine model)
- **clip** -- [CLIPConfig](/docs/transformers/pr_15592/en/model_doc/clip#transformers.CLIPConfig) (CLIP model)
- **convbert** -- [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) (ConvBERT model)
- **convnext** -- [ConvNextConfig](/docs/transformers/pr_15592/en/model_doc/convnext#transformers.ConvNextConfig) (ConvNext model)
- **ctrl** -- [CTRLConfig](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLConfig) (CTRL model)
- **deberta** -- [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) (DeBERTa model)
- **deberta-v2** -- [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) (DeBERTa-v2 model)
- **deit** -- [DeiTConfig](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTConfig) (DeiT model)
- **detr** -- [DetrConfig](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrConfig) (DETR model)
- **distilbert** -- [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) (DistilBERT model)
- **dpr** -- [DPRConfig](/docs/transformers/pr_15592/en/model_doc/dpr#transformers.DPRConfig) (DPR model)
- **electra** -- [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) (ELECTRA model)
- **encoder-decoder** -- [EncoderDecoderConfig](/docs/transformers/pr_15592/en/model_doc/encoder-decoder#transformers.EncoderDecoderConfig) (Encoder decoder model)
- **flaubert** -- [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) (FlauBERT model)
- **fnet** -- [FNetConfig](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetConfig) (FNet model)
- **fsmt** -- [FSMTConfig](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTConfig) (FairSeq Machine-Translation model)
- **funnel** -- [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) (Funnel Transformer model)
- **gpt2** -- [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) (OpenAI GPT-2 model)
- **gpt_neo** -- [GPTNeoConfig](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoConfig) (GPT Neo model)
- **gptj** -- [GPTJConfig](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJConfig) (GPT-J model)
- **hubert** -- [HubertConfig](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertConfig) (Hubert model)
- **ibert** -- [IBertConfig](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertConfig) (I-BERT model)
- **imagegpt** -- [ImageGPTConfig](/docs/transformers/pr_15592/en/model_doc/imagegpt#transformers.ImageGPTConfig) (ImageGPT model)
- **layoutlm** -- [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) (LayoutLM model)
- **layoutlmv2** -- [LayoutLMv2Config](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2Config) (LayoutLMv2 model)
- **led** -- [LEDConfig](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDConfig) (LED model)
- **longformer** -- [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) (Longformer model)
- **luke** -- [LukeConfig](/docs/transformers/pr_15592/en/model_doc/luke#transformers.LukeConfig) (LUKE model)
- **lxmert** -- [LxmertConfig](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertConfig) (LXMERT model)
- **m2m_100** -- [M2M100Config](/docs/transformers/pr_15592/en/model_doc/m2m_100#transformers.M2M100Config) (M2M100 model)
- **marian** -- [MarianConfig](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianConfig) (Marian model)
- **mbart** -- [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) (mBART model)
- **megatron-bert** -- [MegatronBertConfig](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertConfig) (MegatronBert model)
- **mobilebert** -- [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) (MobileBERT model)
- **mpnet** -- [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) (MPNet model)
- **mt5** -- [MT5Config](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Config) (mT5 model)
- **nystromformer** -- [NystromformerConfig](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerConfig) (Nystromformer model)
- **openai-gpt** -- [OpenAIGPTConfig](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTConfig) (OpenAI GPT model)
- **pegasus** -- [PegasusConfig](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusConfig) (Pegasus model)
- **perceiver** -- [PerceiverConfig](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverConfig) (Perceiver model)
- **plbart** -- [PLBartConfig](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartConfig) (PLBart model)
- **poolformer** -- [PoolFormerConfig](/docs/transformers/pr_15592/en/model_doc/poolformer#transformers.PoolFormerConfig) (PoolFormer model)
- **prophetnet** -- [ProphetNetConfig](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetConfig) (ProphetNet model)
- **qdqbert** -- [QDQBertConfig](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertConfig) (QDQBert model)
- **rag** -- [RagConfig](/docs/transformers/pr_15592/en/model_doc/rag#transformers.RagConfig) (RAG model)
- **realm** -- [RealmConfig](/docs/transformers/pr_15592/en/model_doc/realm#transformers.RealmConfig) (Realm model)
- **reformer** -- [ReformerConfig](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerConfig) (Reformer model)
- **rembert** -- [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) (RemBERT model)
- **retribert** -- [RetriBertConfig](/docs/transformers/pr_15592/en/model_doc/retribert#transformers.RetriBertConfig) (RetriBERT model)
- **roberta** -- [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) (RoBERTa model)
- **roformer** -- [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) (RoFormer model)
- **segformer** -- [SegformerConfig](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerConfig) (SegFormer model)
- **sew** -- [SEWConfig](/docs/transformers/pr_15592/en/model_doc/sew#transformers.SEWConfig) (SEW model)
- **sew-d** -- [SEWDConfig](/docs/transformers/pr_15592/en/model_doc/sew-d#transformers.SEWDConfig) (SEW-D model)
- **speech-encoder-decoder** -- [SpeechEncoderDecoderConfig](/docs/transformers/pr_15592/en/model_doc/speech-encoder-decoder#transformers.SpeechEncoderDecoderConfig) (Speech Encoder decoder model)
- **speech_to_text** -- [Speech2TextConfig](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextConfig) (Speech2Text model)
- **speech_to_text_2** -- [Speech2Text2Config](/docs/transformers/pr_15592/en/model_doc/speech_to_text_2#transformers.Speech2Text2Config) (Speech2Text2 model)
- **splinter** -- [SplinterConfig](/docs/transformers/pr_15592/en/model_doc/splinter#transformers.SplinterConfig) (Splinter model)
- **squeezebert** -- [SqueezeBertConfig](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertConfig) (SqueezeBERT model)
- **swin** -- [SwinConfig](/docs/transformers/pr_15592/en/model_doc/swin#transformers.SwinConfig) (Swin model)
- **t5** -- [T5Config](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Config) (T5 model)
- **tapas** -- [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) (TAPAS model)
- **transfo-xl** -- [TransfoXLConfig](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLConfig) (Transformer-XL model)
- **trocr** -- [TrOCRConfig](/docs/transformers/pr_15592/en/model_doc/trocr#transformers.TrOCRConfig) (TrOCR model)
- **unispeech** -- [UniSpeechConfig](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechConfig) (UniSpeech model)
- **unispeech-sat** -- [UniSpeechSatConfig](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatConfig) (UniSpeechSat model)
- **vilt** -- [ViltConfig](/docs/transformers/pr_15592/en/model_doc/vilt#transformers.ViltConfig) (ViLT model)
- **vision-encoder-decoder** -- [VisionEncoderDecoderConfig](/docs/transformers/pr_15592/en/model_doc/vision-encoder-decoder#transformers.VisionEncoderDecoderConfig) (Vision Encoder decoder model)
- **vision-text-dual-encoder** -- [VisionTextDualEncoderConfig](/docs/transformers/pr_15592/en/model_doc/vision-text-dual-encoder#transformers.VisionTextDualEncoderConfig) (VisionTextDualEncoder model)
- **visual_bert** -- [VisualBertConfig](/docs/transformers/pr_15592/en/model_doc/visual_bert#transformers.VisualBertConfig) (VisualBert model)
- **vit** -- [ViTConfig](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTConfig) (ViT model)
- **vit_mae** -- [ViTMAEConfig](/docs/transformers/pr_15592/en/model_doc/vit_mae#transformers.ViTMAEConfig) (ViTMAE model)
- **wav2vec2** -- [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) (Wav2Vec2 model)
- **wavlm** -- [WavLMConfig](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMConfig) (WavLM model)
- **xglm** -- [XGLMConfig](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMConfig) (XGLM model)
- **xlm** -- [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) (XLM model)
- **xlm-prophetnet** -- [XLMProphetNetConfig](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetConfig) (XLMProphetNet model)
- **xlm-roberta** -- [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) (XLM-RoBERTa model)
- **xlm-roberta-xl** -- [XLMRobertaXLConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLConfig) (XLM-RoBERTa-XL model)
- **xlnet** -- [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) (XLNet model)
- **yoso** -- [YosoConfig](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoConfig) (YOSO model)



Examples:

```python
>>> from transformers import AutoConfig

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-uncased")

>>> # Download configuration from huggingface.co (user-uploaded) and cache.
>>> config = AutoConfig.from_pretrained("dbmdz/bert-base-german-cased")

>>> # If configuration file is in a directory (e.g., was saved using *save_pretrained('./test/saved_model/')*).
>>> config = AutoConfig.from_pretrained("./test/bert_saved_model/")

>>> # Load a specific configuration file.
>>> config = AutoConfig.from_pretrained("./test/bert_saved_model/my_configuration.json")

>>> # Change some config attributes when loading a pretrained config.
>>> config = AutoConfig.from_pretrained("bert-base-uncased", output_attentions=True, foo=False)
>>> config.output_attentions
True

>>> config, unused_kwargs = AutoConfig.from_pretrained(
...     "bert-base-uncased", output_attentions=True, foo=False, return_unused_kwargs=True
... )
>>> config.output_attentions
True

>>> config.unused_kwargs
&amp;lcub;'foo': False}
```

</div>
<div class="docstring">
<docstring><name>register</name><anchor>transformers.AutoConfig.register</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/configuration_auto.py#L660</source><parameters>[{"name": "model_type", "val": ""}, {"name": "config", "val": ""}]</parameters><paramsdesc>- **model_type** (`str`) -- The model type like "bert" or "gpt".
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) -- The config to register.</paramsdesc><paramgroups>0</paramgroups></docstring>

Register a new configuration for this class.




</div></div>

<h2 id="transformers.AutoTokenizer">AutoTokenizer</h2>

<div class="docstring">

<docstring><name>class transformers.AutoTokenizer</name><anchor>transformers.AutoTokenizer</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/tokenization_auto.py#L351</source><parameters>[]</parameters></docstring>

This is a generic tokenizer class that will be instantiated as one of the tokenizer classes of the library when
created with the [AutoTokenizer.from_pretrained()](/docs/transformers/pr_15592/en/model_doc/auto#transformers.AutoTokenizer.from_pretrained) class method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.AutoTokenizer.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/tokenization_auto.py#L365</source><parameters>[{"name": "pretrained_model_name_or_path", "val": ""}, {"name": "*inputs", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a predefined tokenizer hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing vocabulary files required by the tokenizer, for instance saved
    using the [save_pretrained()](/docs/transformers/pr_15592/en/internal/tokenization_utils#transformers.PreTrainedTokenizerBase.save_pretrained) method, e.g., `./my_model_directory/`.
  - A path or url to a single saved vocabulary file if and only if the tokenizer only requires a
    single vocabulary file (like Bert or XLNet), e.g.: `./my_model_directory/vocab.txt`. (Not
    applicable to all derived classes)
- **inputs** (additional positional arguments, *optional*) --
  Will be passed along to the Tokenizer `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  The configuration object used to dertermine the tokenizer class to instantiate.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download the model weights and configuration files and override the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **subfolder** (`str`, *optional*) --
  In case the relevant files are located inside a subfolder of the model repo on huggingface.co (e.g. for
  facebook/rag-token-base), specify it here.
- **use_fast** (`bool`, *optional*, defaults to `True`) --
  Whether or not to try to load the fast version of the tokenizer.
- **tokenizer_type** (`str`, *optional*) --
  Tokenizer type to be loaded.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Will be passed to the Tokenizer `__init__()` method. Can be used to set special tokens like
  `bos_token`, `eos_token`, `unk_token`, `sep_token`, `pad_token`, `cls_token`, `mask_token`,
  `additional_special_tokens`. See parameters in the `__init__()` for more details.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the tokenizer classes of the library from a pretrained model vocabulary.

The tokenizer class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [AlbertTokenizer](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertTokenizer) or [AlbertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertTokenizerFast) (ALBERT model)
- **bart** -- [BartTokenizer](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartTokenizer) or [BartTokenizerFast](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartTokenizerFast) (BART model)
- **barthez** -- [BarthezTokenizer](/docs/transformers/pr_15592/en/model_doc/barthez#transformers.BarthezTokenizer) or [BarthezTokenizerFast](/docs/transformers/pr_15592/en/model_doc/barthez#transformers.BarthezTokenizerFast) (BARThez model)
- **bartpho** -- [BartphoTokenizer](/docs/transformers/pr_15592/en/model_doc/bartpho#transformers.BartphoTokenizer) (BARTpho model)
- **bert** -- [BertTokenizer](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertTokenizer) or [BertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertTokenizerFast) (BERT model)
- **bert-generation** -- [BertGenerationTokenizer](/docs/transformers/pr_15592/en/model_doc/bert-generation#transformers.BertGenerationTokenizer) (Bert Generation model)
- **bert-japanese** -- [BertJapaneseTokenizer](/docs/transformers/pr_15592/en/model_doc/bert-japanese#transformers.BertJapaneseTokenizer) (BertJapanese model)
- **bertweet** -- [BertweetTokenizer](/docs/transformers/pr_15592/en/model_doc/bertweet#transformers.BertweetTokenizer) (Bertweet model)
- **big_bird** -- [BigBirdTokenizer](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdTokenizer) or [BigBirdTokenizerFast](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdTokenizerFast) (BigBird model)
- **bigbird_pegasus** -- [PegasusTokenizer](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusTokenizer) or [PegasusTokenizerFast](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusTokenizerFast) (BigBirdPegasus model)
- **blenderbot** -- [BlenderbotTokenizer](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotTokenizer) or [BlenderbotTokenizerFast](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotTokenizerFast) (Blenderbot model)
- **blenderbot-small** -- [BlenderbotSmallTokenizer](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallTokenizer) (BlenderbotSmall model)
- **byt5** -- [ByT5Tokenizer](/docs/transformers/pr_15592/en/model_doc/byt5#transformers.ByT5Tokenizer) (ByT5 model)
- **camembert** -- [CamembertTokenizer](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertTokenizer) or [CamembertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertTokenizerFast) (CamemBERT model)
- **canine** -- [CanineTokenizer](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineTokenizer) (Canine model)
- **clip** -- [CLIPTokenizer](/docs/transformers/pr_15592/en/model_doc/clip#transformers.CLIPTokenizer) or [CLIPTokenizerFast](/docs/transformers/pr_15592/en/model_doc/clip#transformers.CLIPTokenizerFast) (CLIP model)
- **convbert** -- [ConvBertTokenizer](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertTokenizer) or [ConvBertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertTokenizerFast) (ConvBERT model)
- **cpm** -- [CpmTokenizer](/docs/transformers/pr_15592/en/model_doc/cpm#transformers.CpmTokenizer) or `CpmTokenizerFast` (CPM model)
- **ctrl** -- [CTRLTokenizer](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLTokenizer) (CTRL model)
- **deberta** -- [DebertaTokenizer](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaTokenizer) or [DebertaTokenizerFast](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaTokenizerFast) (DeBERTa model)
- **deberta-v2** -- [DebertaV2Tokenizer](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Tokenizer) (DeBERTa-v2 model)
- **distilbert** -- [DistilBertTokenizer](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertTokenizer) or [DistilBertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertTokenizerFast) (DistilBERT model)
- **dpr** -- [DPRQuestionEncoderTokenizer](/docs/transformers/pr_15592/en/model_doc/dpr#transformers.DPRQuestionEncoderTokenizer) or [DPRQuestionEncoderTokenizerFast](/docs/transformers/pr_15592/en/model_doc/dpr#transformers.DPRQuestionEncoderTokenizerFast) (DPR model)
- **electra** -- [ElectraTokenizer](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraTokenizer) or [ElectraTokenizerFast](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraTokenizerFast) (ELECTRA model)
- **flaubert** -- [FlaubertTokenizer](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertTokenizer) (FlauBERT model)
- **fnet** -- [FNetTokenizer](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetTokenizer) or [FNetTokenizerFast](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetTokenizerFast) (FNet model)
- **fsmt** -- [FSMTTokenizer](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTTokenizer) (FairSeq Machine-Translation model)
- **funnel** -- [FunnelTokenizer](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelTokenizer) or [FunnelTokenizerFast](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelTokenizerFast) (Funnel Transformer model)
- **gpt2** -- [GPT2Tokenizer](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Tokenizer) or [GPT2TokenizerFast](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2TokenizerFast) (OpenAI GPT-2 model)
- **gpt_neo** -- [GPT2Tokenizer](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Tokenizer) or [GPT2TokenizerFast](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2TokenizerFast) (GPT Neo model)
- **herbert** -- [HerbertTokenizer](/docs/transformers/pr_15592/en/model_doc/herbert#transformers.HerbertTokenizer) or [HerbertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/herbert#transformers.HerbertTokenizerFast) (HerBERT model)
- **hubert** -- [Wav2Vec2CTCTokenizer](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2CTCTokenizer) (Hubert model)
- **ibert** -- [RobertaTokenizer](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaTokenizer) or [RobertaTokenizerFast](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaTokenizerFast) (I-BERT model)
- **layoutlm** -- [LayoutLMTokenizer](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMTokenizer) or [LayoutLMTokenizerFast](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMTokenizerFast) (LayoutLM model)
- **layoutlmv2** -- [LayoutLMv2Tokenizer](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2Tokenizer) or [LayoutLMv2TokenizerFast](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2TokenizerFast) (LayoutLMv2 model)
- **layoutxlm** -- [LayoutXLMTokenizer](/docs/transformers/pr_15592/en/model_doc/layoutxlm#transformers.LayoutXLMTokenizer) or [LayoutXLMTokenizerFast](/docs/transformers/pr_15592/en/model_doc/layoutxlm#transformers.LayoutXLMTokenizerFast) (LayoutXLM model)
- **led** -- [LEDTokenizer](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDTokenizer) or [LEDTokenizerFast](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDTokenizerFast) (LED model)
- **longformer** -- [LongformerTokenizer](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerTokenizer) or [LongformerTokenizerFast](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerTokenizerFast) (Longformer model)
- **luke** -- [LukeTokenizer](/docs/transformers/pr_15592/en/model_doc/luke#transformers.LukeTokenizer) (LUKE model)
- **lxmert** -- [LxmertTokenizer](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertTokenizer) or [LxmertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertTokenizerFast) (LXMERT model)
- **m2m_100** -- [M2M100Tokenizer](/docs/transformers/pr_15592/en/model_doc/m2m_100#transformers.M2M100Tokenizer) (M2M100 model)
- **marian** -- [MarianTokenizer](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianTokenizer) (Marian model)
- **mbart** -- [MBartTokenizer](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartTokenizer) or [MBartTokenizerFast](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartTokenizerFast) (mBART model)
- **mbart50** -- [MBart50Tokenizer](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBart50Tokenizer) or [MBart50TokenizerFast](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBart50TokenizerFast) (mBART-50 model)
- **mluke** -- [MLukeTokenizer](/docs/transformers/pr_15592/en/model_doc/mluke#transformers.MLukeTokenizer) (mLUKE model)
- **mobilebert** -- [MobileBertTokenizer](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertTokenizer) or [MobileBertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertTokenizerFast) (MobileBERT model)
- **mpnet** -- [MPNetTokenizer](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetTokenizer) or [MPNetTokenizerFast](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetTokenizerFast) (MPNet model)
- **mt5** -- [MT5Tokenizer](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.T5Tokenizer) or [MT5TokenizerFast](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.T5TokenizerFast) (mT5 model)
- **openai-gpt** -- [OpenAIGPTTokenizer](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTTokenizer) or [OpenAIGPTTokenizerFast](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTTokenizerFast) (OpenAI GPT model)
- **pegasus** -- [PegasusTokenizer](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusTokenizer) or [PegasusTokenizerFast](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusTokenizerFast) (Pegasus model)
- **perceiver** -- [PerceiverTokenizer](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverTokenizer) (Perceiver model)
- **phobert** -- [PhobertTokenizer](/docs/transformers/pr_15592/en/model_doc/phobert#transformers.PhobertTokenizer) (PhoBERT model)
- **plbart** -- [PLBartTokenizer](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartTokenizer) (PLBart model)
- **prophetnet** -- [ProphetNetTokenizer](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetTokenizer) (ProphetNet model)
- **qdqbert** -- [BertTokenizer](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertTokenizer) or [BertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertTokenizerFast) (QDQBert model)
- **rag** -- [RagTokenizer](/docs/transformers/pr_15592/en/model_doc/rag#transformers.RagTokenizer) (RAG model)
- **reformer** -- [ReformerTokenizer](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerTokenizer) or [ReformerTokenizerFast](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerTokenizerFast) (Reformer model)
- **rembert** -- [RemBertTokenizer](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertTokenizer) or [RemBertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertTokenizerFast) (RemBERT model)
- **retribert** -- [RetriBertTokenizer](/docs/transformers/pr_15592/en/model_doc/retribert#transformers.RetriBertTokenizer) or [RetriBertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/retribert#transformers.RetriBertTokenizerFast) (RetriBERT model)
- **roberta** -- [RobertaTokenizer](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaTokenizer) or [RobertaTokenizerFast](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaTokenizerFast) (RoBERTa model)
- **roformer** -- [RoFormerTokenizer](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerTokenizer) or [RoFormerTokenizerFast](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerTokenizerFast) (RoFormer model)
- **speech_to_text** -- [Speech2TextTokenizer](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextTokenizer) (Speech2Text model)
- **speech_to_text_2** -- [Speech2Text2Tokenizer](/docs/transformers/pr_15592/en/model_doc/speech_to_text_2#transformers.Speech2Text2Tokenizer) (Speech2Text2 model)
- **splinter** -- [SplinterTokenizer](/docs/transformers/pr_15592/en/model_doc/splinter#transformers.SplinterTokenizer) or [SplinterTokenizerFast](/docs/transformers/pr_15592/en/model_doc/splinter#transformers.SplinterTokenizerFast) (Splinter model)
- **squeezebert** -- [SqueezeBertTokenizer](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertTokenizer) or [SqueezeBertTokenizerFast](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertTokenizerFast) (SqueezeBERT model)
- **t5** -- [T5Tokenizer](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.T5Tokenizer) or [T5TokenizerFast](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.T5TokenizerFast) (T5 model)
- **tapas** -- [TapasTokenizer](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasTokenizer) (TAPAS model)
- **transfo-xl** -- [TransfoXLTokenizer](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLTokenizer) (Transformer-XL model)
- **wav2vec2** -- [Wav2Vec2CTCTokenizer](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2CTCTokenizer) (Wav2Vec2 model)
- **wav2vec2_phoneme** -- [Wav2Vec2PhonemeCTCTokenizer](/docs/transformers/pr_15592/en/model_doc/wav2vec2_phoneme#transformers.Wav2Vec2PhonemeCTCTokenizer) (Wav2Vec2Phoneme model)
- **xglm** -- [XGLMTokenizer](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMTokenizer) or [XGLMTokenizerFast](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMTokenizerFast) (XGLM model)
- **xlm** -- [XLMTokenizer](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMTokenizer) (XLM model)
- **xlm-prophetnet** -- [XLMProphetNetTokenizer](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetTokenizer) (XLMProphetNet model)
- **xlm-roberta** -- [XLMRobertaTokenizer](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaTokenizer) or [XLMRobertaTokenizerFast](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaTokenizerFast) (XLM-RoBERTa model)
- **xlnet** -- [XLNetTokenizer](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetTokenizer) or [XLNetTokenizerFast](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetTokenizerFast) (XLNet model)



Examples:

```python
>>> from transformers import AutoTokenizer

>>> # Download vocabulary from huggingface.co and cache.
>>> tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

>>> # Download vocabulary from huggingface.co (user-uploaded) and cache.
>>> tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-german-cased")

>>> # If vocabulary files are in a directory (e.g. tokenizer was saved using *save_pretrained('./test/saved_model/')*)
>>> tokenizer = AutoTokenizer.from_pretrained("./test/bert_saved_model/")
```

</div>
<div class="docstring">
<docstring><name>register</name><anchor>transformers.AutoTokenizer.register</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/tokenization_auto.py#L561</source><parameters>[{"name": "config_class", "val": ""}, {"name": "slow_tokenizer_class", "val": " = None"}, {"name": "fast_tokenizer_class", "val": " = None"}]</parameters><paramsdesc>- **config_class** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The configuration corresponding to the model to register.
- **slow_tokenizer_class** (`PretrainedTokenizer`, *optional*) --
  The slow tokenizer to register.
- **slow_tokenizer_class** (`PretrainedTokenizerFast`, *optional*) --
  The fast tokenizer to register.</paramsdesc><paramgroups>0</paramgroups></docstring>

Register a new tokenizer in this mapping.





</div></div>

<h2 id="transformers.AutoFeatureExtractor">AutoFeatureExtractor</h2>

<div class="docstring">

<docstring><name>class transformers.AutoFeatureExtractor</name><anchor>transformers.AutoFeatureExtractor</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/feature_extraction_auto.py#L169</source><parameters>[]</parameters></docstring>

This is a generic feature extractor class that will be instantiated as one of the feature extractor classes of the
library when created with the [AutoFeatureExtractor.from_pretrained()](/docs/transformers/pr_15592/en/model_doc/auto#transformers.AutoFeatureExtractor.from_pretrained) class method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.AutoFeatureExtractor.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/feature_extraction_auto.py#L183</source><parameters>[{"name": "pretrained_model_name_or_path", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  This can be either:

  - a string, the *model id* of a pretrained feature_extractor hosted inside a model repo on
    huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or
    namespaced under a user or organization name, like `dbmdz/bert-base-german-cased`.
  - a path to a *directory* containing a feature extractor file saved using the
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/feature_extractor#transformers.FeatureExtractionMixin.save_pretrained) method, e.g.,
    `./my_model_directory/`.
  - a path or url to a saved feature extractor JSON *file*, e.g.,
    `./my_model_directory/preprocessor_config.json`.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model feature extractor should be cached if the
  standard cache should not be used.
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force to (re-)download the feature extractor files and override the cached versions
  if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received file. Attempts to resume the download if such a file
  exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request.
- **use_auth_token** (`str` or *bool*, *optional*) --
  The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
  when running `transformers-cli login` (stored in `~/.huggingface`).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **return_unused_kwargs** (`bool`, *optional*, defaults to `False`) --
  If `False`, then this function returns just the final feature extractor object. If `True`, then this
  functions returns a `Tuple(feature_extractor, unused_kwargs)` where *unused_kwargs* is a dictionary
  consisting of the key/value pairs whose keys are not feature extractor attributes: i.e., the part of
  `kwargs` which has not been used to update `feature_extractor` and is otherwise ignored.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (`Dict[str, Any]`, *optional*) --
  The values in kwargs of any keys which are feature extractor attributes will be used to override the
  loaded values. Behavior concerning key/value pairs whose keys are *not* feature extractor attributes is
  controlled by the `return_unused_kwargs` keyword parameter.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the feature extractor classes of the library from a pretrained model vocabulary.

The feature extractor class to instantiate is selected based on the `model_type` property of the config object
(either passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's
missing, by falling back to using pattern matching on `pretrained_model_name_or_path`:

- **beit** -- [BeitFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitFeatureExtractor) (BEiT model)
- **clip** -- [CLIPFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/clip#transformers.CLIPFeatureExtractor) (CLIP model)
- **convnext** -- [ConvNextFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/convnext#transformers.ConvNextFeatureExtractor) (ConvNext model)
- **deit** -- [DeiTFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTFeatureExtractor) (DeiT model)
- **detr** -- [DetrFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrFeatureExtractor) (DETR model)
- **hubert** -- [Wav2Vec2FeatureExtractor](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2FeatureExtractor) (Hubert model)
- **layoutlmv2** -- [LayoutLMv2FeatureExtractor](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2FeatureExtractor) (LayoutLMv2 model)
- **perceiver** -- [PerceiverFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverFeatureExtractor) (Perceiver model)
- **poolformer** -- [PoolFormerFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/poolformer#transformers.PoolFormerFeatureExtractor) (PoolFormer model)
- **segformer** -- [SegformerFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerFeatureExtractor) (SegFormer model)
- **speech_to_text** -- [Speech2TextFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextFeatureExtractor) (Speech2Text model)
- **swin** -- [ViTFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTFeatureExtractor) (Swin model)
- **vit** -- [ViTFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTFeatureExtractor) (ViT model)
- **vit_mae** -- [ViTFeatureExtractor](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTFeatureExtractor) (ViTMAE model)
- **wav2vec2** -- [Wav2Vec2FeatureExtractor](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2FeatureExtractor) (Wav2Vec2 model)



<Tip>

Passing `use_auth_token=True` is required when you want to use a private model.

</Tip>

Examples:

```python
>>> from transformers import AutoFeatureExtractor

>>> # Download feature extractor from huggingface.co and cache.
>>> feature_extractor = AutoFeatureExtractor.from_pretrained("facebook/wav2vec2-base-960h")

>>> # If feature extractor files are in a directory (e.g. feature extractor was saved using *save_pretrained('./test/saved_model/')*)
>>> feature_extractor = AutoFeatureExtractor.from_pretrained("./test/saved_model/")
```

</div>
<div class="docstring">
<docstring><name>register</name><anchor>transformers.AutoFeatureExtractor.register</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/feature_extraction_auto.py#L310</source><parameters>[{"name": "config_class", "val": ""}, {"name": "feature_extractor_class", "val": ""}]</parameters><paramsdesc>- **config_class** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The configuration corresponding to the model to register.
- **feature_extractor_class** (`FeatureExtractorMixin`) -- The feature extractor to register.</paramsdesc><paramgroups>0</paramgroups></docstring>

Register a new feature extractor for this class.




</div></div>

<h2 id="transformers.AutoProcessor">AutoProcessor</h2>

<div class="docstring">

<docstring><name>class transformers.AutoProcessor</name><anchor>transformers.AutoProcessor</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/processing_auto.py#L71</source><parameters>[]</parameters></docstring>

This is a generic processor class that will be instantiated as one of the processor classes of the library when
created with the [AutoProcessor.from_pretrained()](/docs/transformers/pr_15592/en/model_doc/auto#transformers.AutoProcessor.from_pretrained) class method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.AutoProcessor.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/processing_auto.py#L85</source><parameters>[{"name": "pretrained_model_name_or_path", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  This can be either:

  - a string, the *model id* of a pretrained feature_extractor hosted inside a model repo on
    huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or
    namespaced under a user or organization name, like `dbmdz/bert-base-german-cased`.
  - a path to a *directory* containing a processor files saved using the `save_pretrained()` method,
    e.g., `./my_model_directory/`.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model feature extractor should be cached if the
  standard cache should not be used.
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force to (re-)download the feature extractor files and override the cached versions
  if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received file. Attempts to resume the download if such a file
  exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request.
- **use_auth_token** (`str` or *bool*, *optional*) --
  The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
  when running `transformers-cli login` (stored in `~/.huggingface`).
- **revision** (`str`, *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **return_unused_kwargs** (`bool`, *optional*, defaults to `False`) --
  If `False`, then this function returns just the final feature extractor object. If `True`, then this
  functions returns a `Tuple(feature_extractor, unused_kwargs)` where *unused_kwargs* is a dictionary
  consisting of the key/value pairs whose keys are not feature extractor attributes: i.e., the part of
  `kwargs` which has not been used to update `feature_extractor` and is otherwise ignored.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (`Dict[str, Any]`, *optional*) --
  The values in kwargs of any keys which are feature extractor attributes will be used to override the
  loaded values. Behavior concerning key/value pairs whose keys are *not* feature extractor attributes is
  controlled by the `return_unused_kwargs` keyword parameter.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the processor classes of the library from a pretrained model vocabulary.

The processor class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible):

- **clip** -- [CLIPProcessor](/docs/transformers/pr_15592/en/model_doc/clip#transformers.CLIPProcessor) (CLIP model)
- **layoutlmv2** -- [LayoutLMv2Processor](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2Processor) (LayoutLMv2 model)
- **layoutxlm** -- [LayoutXLMProcessor](/docs/transformers/pr_15592/en/model_doc/layoutxlm#transformers.LayoutXLMProcessor) (LayoutXLM model)
- **speech_to_text** -- [Speech2TextProcessor](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextProcessor) (Speech2Text model)
- **speech_to_text_2** -- [Speech2Text2Processor](/docs/transformers/pr_15592/en/model_doc/speech_to_text_2#transformers.Speech2Text2Processor) (Speech2Text2 model)
- **trocr** -- [TrOCRProcessor](/docs/transformers/pr_15592/en/model_doc/trocr#transformers.TrOCRProcessor) (TrOCR model)
- **vision-text-dual-encoder** -- [VisionTextDualEncoderProcessor](/docs/transformers/pr_15592/en/model_doc/vision-text-dual-encoder#transformers.VisionTextDualEncoderProcessor) (VisionTextDualEncoder model)
- **wav2vec2** -- [Wav2Vec2Processor](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Processor) (Wav2Vec2 model)



<Tip>

Passing `use_auth_token=True` is required when you want to use a private model.

</Tip>

Examples:

```python
>>> from transformers import AutoProcessor

>>> # Download processor from huggingface.co and cache.
>>> processor = AutoProcessor.from_pretrained("facebook/wav2vec2-base-960h")

>>> # If processor files are in a directory (e.g. processor was saved using *save_pretrained('./test/saved_model/')*)
>>> processor = AutoProcessor.from_pretrained("./test/saved_model/")
```

</div>
<div class="docstring">
<docstring><name>register</name><anchor>transformers.AutoProcessor.register</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/processing_auto.py#L238</source><parameters>[{"name": "config_class", "val": ""}, {"name": "processor_class", "val": ""}]</parameters><paramsdesc>- **config_class** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The configuration corresponding to the model to register.
- **processor_class** (`FeatureExtractorMixin`) -- The processor to register.</paramsdesc><paramgroups>0</paramgroups></docstring>

Register a new processor for this class.




</div></div>

<h2 id="transformers.AutoModel">AutoModel</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModel</name><anchor>transformers.AutoModel</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L672</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the base model classes of the library when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [AlbertModel](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertModel) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [BartModel](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartModel) (BART model)
  - [BeitConfig](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitConfig) configuration class: [BeitModel](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitModel) (BEiT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [BertModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertModel) (BERT model)
  - [BertGenerationConfig](/docs/transformers/pr_15592/en/model_doc/bert-generation#transformers.BertGenerationConfig) configuration class: [BertGenerationEncoder](/docs/transformers/pr_15592/en/model_doc/bert-generation#transformers.BertGenerationEncoder) (Bert Generation model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [BigBirdModel](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdModel) (BigBird model)
  - [BigBirdPegasusConfig](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusConfig) configuration class: [BigBirdPegasusModel](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusModel) (BigBirdPegasus model)
  - [BlenderbotConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotConfig) configuration class: [BlenderbotModel](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotModel) (Blenderbot model)
  - [BlenderbotSmallConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallConfig) configuration class: [BlenderbotSmallModel](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallModel) (BlenderbotSmall model)
  - [CLIPConfig](/docs/transformers/pr_15592/en/model_doc/clip#transformers.CLIPConfig) configuration class: [CLIPModel](/docs/transformers/pr_15592/en/model_doc/clip#transformers.CLIPModel) (CLIP model)
  - [CTRLConfig](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLConfig) configuration class: [CTRLModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLModel) (CTRL model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [CamembertModel](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertModel) (CamemBERT model)
  - [CanineConfig](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineConfig) configuration class: [CanineModel](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineModel) (Canine model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [ConvBertModel](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertModel) (ConvBERT model)
  - [ConvNextConfig](/docs/transformers/pr_15592/en/model_doc/convnext#transformers.ConvNextConfig) configuration class: [ConvNextModel](/docs/transformers/pr_15592/en/model_doc/convnext#transformers.ConvNextModel) (ConvNext model)
  - [DPRConfig](/docs/transformers/pr_15592/en/model_doc/dpr#transformers.DPRConfig) configuration class: [DPRQuestionEncoder](/docs/transformers/pr_15592/en/model_doc/dpr#transformers.DPRQuestionEncoder) (DPR model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [DebertaModel](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaModel) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [DebertaV2Model](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Model) (DeBERTa-v2 model)
  - [DeiTConfig](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTConfig) configuration class: [DeiTModel](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTModel) (DeiT model)
  - [DetrConfig](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrConfig) configuration class: [DetrModel](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrModel) (DETR model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [DistilBertModel](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertModel) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [ElectraModel](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraModel) (ELECTRA model)
  - [FNetConfig](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetConfig) configuration class: [FNetModel](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetModel) (FNet model)
  - [FSMTConfig](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTConfig) configuration class: [FSMTModel](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTModel) (FairSeq Machine-Translation model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [FlaubertModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertModel) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [FunnelModel](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelModel) or [FunnelBaseModel](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelBaseModel) (Funnel Transformer model)
  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [GPT2Model](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Model) (OpenAI GPT-2 model)
  - [GPTJConfig](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJConfig) configuration class: [GPTJModel](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJModel) (GPT-J model)
  - [GPTNeoConfig](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoConfig) configuration class: [GPTNeoModel](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoModel) (GPT Neo model)
  - [HubertConfig](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertConfig) configuration class: [HubertModel](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertModel) (Hubert model)
  - [IBertConfig](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertConfig) configuration class: [IBertModel](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertModel) (I-BERT model)
  - [ImageGPTConfig](/docs/transformers/pr_15592/en/model_doc/imagegpt#transformers.ImageGPTConfig) configuration class: [ImageGPTModel](/docs/transformers/pr_15592/en/model_doc/imagegpt#transformers.ImageGPTModel) (ImageGPT model)
  - [LEDConfig](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDConfig) configuration class: [LEDModel](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDModel) (LED model)
  - [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) configuration class: [LayoutLMModel](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMModel) (LayoutLM model)
  - [LayoutLMv2Config](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2Config) configuration class: [LayoutLMv2Model](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2Model) (LayoutLMv2 model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [LongformerModel](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerModel) (Longformer model)
  - [LukeConfig](/docs/transformers/pr_15592/en/model_doc/luke#transformers.LukeConfig) configuration class: [LukeModel](/docs/transformers/pr_15592/en/model_doc/luke#transformers.LukeModel) (LUKE model)
  - [LxmertConfig](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertConfig) configuration class: [LxmertModel](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertModel) (LXMERT model)
  - [M2M100Config](/docs/transformers/pr_15592/en/model_doc/m2m_100#transformers.M2M100Config) configuration class: [M2M100Model](/docs/transformers/pr_15592/en/model_doc/m2m_100#transformers.M2M100Model) (M2M100 model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [MBartModel](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartModel) (mBART model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [MPNetModel](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetModel) (MPNet model)
  - [MT5Config](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Config) configuration class: [MT5Model](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Model) (mT5 model)
  - [MarianConfig](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianConfig) configuration class: [MarianModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianModel) (Marian model)
  - [MegatronBertConfig](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertConfig) configuration class: [MegatronBertModel](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertModel) (MegatronBert model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [MobileBertModel](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertModel) (MobileBERT model)
  - [NystromformerConfig](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerConfig) configuration class: [NystromformerModel](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerModel) (Nystromformer model)
  - [OpenAIGPTConfig](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTConfig) configuration class: [OpenAIGPTModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTModel) (OpenAI GPT model)
  - [PLBartConfig](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartConfig) configuration class: [PLBartModel](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartModel) (PLBart model)
  - [PegasusConfig](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusConfig) configuration class: [PegasusModel](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusModel) (Pegasus model)
  - [PerceiverConfig](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverConfig) configuration class: [PerceiverModel](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverModel) (Perceiver model)
  - [PoolFormerConfig](/docs/transformers/pr_15592/en/model_doc/poolformer#transformers.PoolFormerConfig) configuration class: [PoolFormerModel](/docs/transformers/pr_15592/en/model_doc/poolformer#transformers.PoolFormerModel) (PoolFormer model)
  - [ProphetNetConfig](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetConfig) configuration class: [ProphetNetModel](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetModel) (ProphetNet model)
  - [QDQBertConfig](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertConfig) configuration class: [QDQBertModel](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertModel) (QDQBert model)
  - [ReformerConfig](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerConfig) configuration class: [ReformerModel](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerModel) (Reformer model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [RemBertModel](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertModel) (RemBERT model)
  - [RetriBertConfig](/docs/transformers/pr_15592/en/model_doc/retribert#transformers.RetriBertConfig) configuration class: [RetriBertModel](/docs/transformers/pr_15592/en/model_doc/retribert#transformers.RetriBertModel) (RetriBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [RoFormerModel](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerModel) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [RobertaModel](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaModel) (RoBERTa model)
  - [SEWConfig](/docs/transformers/pr_15592/en/model_doc/sew#transformers.SEWConfig) configuration class: [SEWModel](/docs/transformers/pr_15592/en/model_doc/sew#transformers.SEWModel) (SEW model)
  - [SEWDConfig](/docs/transformers/pr_15592/en/model_doc/sew-d#transformers.SEWDConfig) configuration class: [SEWDModel](/docs/transformers/pr_15592/en/model_doc/sew-d#transformers.SEWDModel) (SEW-D model)
  - [SegformerConfig](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerConfig) configuration class: [SegformerModel](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerModel) (SegFormer model)
  - [Speech2TextConfig](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextConfig) configuration class: [Speech2TextModel](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextModel) (Speech2Text model)
  - [SplinterConfig](/docs/transformers/pr_15592/en/model_doc/splinter#transformers.SplinterConfig) configuration class: [SplinterModel](/docs/transformers/pr_15592/en/model_doc/splinter#transformers.SplinterModel) (Splinter model)
  - [SqueezeBertConfig](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertConfig) configuration class: [SqueezeBertModel](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertModel) (SqueezeBERT model)
  - [SwinConfig](/docs/transformers/pr_15592/en/model_doc/swin#transformers.SwinConfig) configuration class: [SwinModel](/docs/transformers/pr_15592/en/model_doc/swin#transformers.SwinModel) (Swin model)
  - [T5Config](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Config) configuration class: [T5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Model) (T5 model)
  - [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) configuration class: [TapasModel](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasModel) (TAPAS model)
  - [TransfoXLConfig](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLConfig) configuration class: [TransfoXLModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLModel) (Transformer-XL model)
  - [UniSpeechConfig](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechConfig) configuration class: [UniSpeechModel](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechModel) (UniSpeech model)
  - [UniSpeechSatConfig](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatConfig) configuration class: [UniSpeechSatModel](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatModel) (UniSpeechSat model)
  - [ViTConfig](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTConfig) configuration class: [ViTModel](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTModel) (ViT model)
  - [ViTMAEConfig](/docs/transformers/pr_15592/en/model_doc/vit_mae#transformers.ViTMAEConfig) configuration class: [ViTMAEModel](/docs/transformers/pr_15592/en/model_doc/vit_mae#transformers.ViTMAEModel) (ViTMAE model)
  - [ViltConfig](/docs/transformers/pr_15592/en/model_doc/vilt#transformers.ViltConfig) configuration class: [ViltModel](/docs/transformers/pr_15592/en/model_doc/vilt#transformers.ViltModel) (ViLT model)
  - [VisionTextDualEncoderConfig](/docs/transformers/pr_15592/en/model_doc/vision-text-dual-encoder#transformers.VisionTextDualEncoderConfig) configuration class: [VisionTextDualEncoderModel](/docs/transformers/pr_15592/en/model_doc/vision-text-dual-encoder#transformers.VisionTextDualEncoderModel) (VisionTextDualEncoder model)
  - [VisualBertConfig](/docs/transformers/pr_15592/en/model_doc/visual_bert#transformers.VisualBertConfig) configuration class: [VisualBertModel](/docs/transformers/pr_15592/en/model_doc/visual_bert#transformers.VisualBertModel) (VisualBert model)
  - [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) configuration class: [Wav2Vec2Model](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Model) (Wav2Vec2 model)
  - [WavLMConfig](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMConfig) configuration class: [WavLMModel](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMModel) (WavLM model)
  - [XGLMConfig](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMConfig) configuration class: [XGLMModel](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMModel) (XGLM model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [XLMModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMModel) (XLM model)
  - [XLMProphetNetConfig](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetConfig) configuration class: [XLMProphetNetModel](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetModel) (XLMProphetNet model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [XLMRobertaModel](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaModel) (XLM-RoBERTa model)
  - [XLMRobertaXLConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLConfig) configuration class: [XLMRobertaXLModel](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLModel) (XLM-RoBERTa-XL model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [XLNetModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetModel) (XLNet model)
  - [YosoConfig](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoConfig) configuration class: [YosoModel](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoModel) (YOSO model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the base model classes of the library from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModel

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModel.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the base model classes of the library from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [AlbertModel](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertModel) (ALBERT model)
- **bart** -- [BartModel](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartModel) (BART model)
- **beit** -- [BeitModel](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitModel) (BEiT model)
- **bert** -- [BertModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertModel) (BERT model)
- **bert-generation** -- [BertGenerationEncoder](/docs/transformers/pr_15592/en/model_doc/bert-generation#transformers.BertGenerationEncoder) (Bert Generation model)
- **big_bird** -- [BigBirdModel](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdModel) (BigBird model)
- **bigbird_pegasus** -- [BigBirdPegasusModel](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusModel) (BigBirdPegasus model)
- **blenderbot** -- [BlenderbotModel](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotModel) (Blenderbot model)
- **blenderbot-small** -- [BlenderbotSmallModel](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallModel) (BlenderbotSmall model)
- **camembert** -- [CamembertModel](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertModel) (CamemBERT model)
- **canine** -- [CanineModel](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineModel) (Canine model)
- **clip** -- [CLIPModel](/docs/transformers/pr_15592/en/model_doc/clip#transformers.CLIPModel) (CLIP model)
- **convbert** -- [ConvBertModel](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertModel) (ConvBERT model)
- **convnext** -- [ConvNextModel](/docs/transformers/pr_15592/en/model_doc/convnext#transformers.ConvNextModel) (ConvNext model)
- **ctrl** -- [CTRLModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLModel) (CTRL model)
- **deberta** -- [DebertaModel](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaModel) (DeBERTa model)
- **deberta-v2** -- [DebertaV2Model](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Model) (DeBERTa-v2 model)
- **deit** -- [DeiTModel](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTModel) (DeiT model)
- **detr** -- [DetrModel](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrModel) (DETR model)
- **distilbert** -- [DistilBertModel](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertModel) (DistilBERT model)
- **dpr** -- [DPRQuestionEncoder](/docs/transformers/pr_15592/en/model_doc/dpr#transformers.DPRQuestionEncoder) (DPR model)
- **electra** -- [ElectraModel](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraModel) (ELECTRA model)
- **flaubert** -- [FlaubertModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertModel) (FlauBERT model)
- **fnet** -- [FNetModel](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetModel) (FNet model)
- **fsmt** -- [FSMTModel](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTModel) (FairSeq Machine-Translation model)
- **funnel** -- [FunnelModel](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelModel) or [FunnelBaseModel](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelBaseModel) (Funnel Transformer model)
- **gpt2** -- [GPT2Model](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Model) (OpenAI GPT-2 model)
- **gpt_neo** -- [GPTNeoModel](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoModel) (GPT Neo model)
- **gptj** -- [GPTJModel](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJModel) (GPT-J model)
- **hubert** -- [HubertModel](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertModel) (Hubert model)
- **ibert** -- [IBertModel](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertModel) (I-BERT model)
- **imagegpt** -- [ImageGPTModel](/docs/transformers/pr_15592/en/model_doc/imagegpt#transformers.ImageGPTModel) (ImageGPT model)
- **layoutlm** -- [LayoutLMModel](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMModel) (LayoutLM model)
- **layoutlmv2** -- [LayoutLMv2Model](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2Model) (LayoutLMv2 model)
- **led** -- [LEDModel](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDModel) (LED model)
- **longformer** -- [LongformerModel](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerModel) (Longformer model)
- **luke** -- [LukeModel](/docs/transformers/pr_15592/en/model_doc/luke#transformers.LukeModel) (LUKE model)
- **lxmert** -- [LxmertModel](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertModel) (LXMERT model)
- **m2m_100** -- [M2M100Model](/docs/transformers/pr_15592/en/model_doc/m2m_100#transformers.M2M100Model) (M2M100 model)
- **marian** -- [MarianModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianModel) (Marian model)
- **mbart** -- [MBartModel](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartModel) (mBART model)
- **megatron-bert** -- [MegatronBertModel](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertModel) (MegatronBert model)
- **mobilebert** -- [MobileBertModel](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertModel) (MobileBERT model)
- **mpnet** -- [MPNetModel](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetModel) (MPNet model)
- **mt5** -- [MT5Model](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Model) (mT5 model)
- **nystromformer** -- [NystromformerModel](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerModel) (Nystromformer model)
- **openai-gpt** -- [OpenAIGPTModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTModel) (OpenAI GPT model)
- **pegasus** -- [PegasusModel](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusModel) (Pegasus model)
- **perceiver** -- [PerceiverModel](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverModel) (Perceiver model)
- **plbart** -- [PLBartModel](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartModel) (PLBart model)
- **poolformer** -- [PoolFormerModel](/docs/transformers/pr_15592/en/model_doc/poolformer#transformers.PoolFormerModel) (PoolFormer model)
- **prophetnet** -- [ProphetNetModel](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetModel) (ProphetNet model)
- **qdqbert** -- [QDQBertModel](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertModel) (QDQBert model)
- **reformer** -- [ReformerModel](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerModel) (Reformer model)
- **rembert** -- [RemBertModel](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertModel) (RemBERT model)
- **retribert** -- [RetriBertModel](/docs/transformers/pr_15592/en/model_doc/retribert#transformers.RetriBertModel) (RetriBERT model)
- **roberta** -- [RobertaModel](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaModel) (RoBERTa model)
- **roformer** -- [RoFormerModel](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerModel) (RoFormer model)
- **segformer** -- [SegformerModel](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerModel) (SegFormer model)
- **sew** -- [SEWModel](/docs/transformers/pr_15592/en/model_doc/sew#transformers.SEWModel) (SEW model)
- **sew-d** -- [SEWDModel](/docs/transformers/pr_15592/en/model_doc/sew-d#transformers.SEWDModel) (SEW-D model)
- **speech_to_text** -- [Speech2TextModel](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextModel) (Speech2Text model)
- **splinter** -- [SplinterModel](/docs/transformers/pr_15592/en/model_doc/splinter#transformers.SplinterModel) (Splinter model)
- **squeezebert** -- [SqueezeBertModel](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertModel) (SqueezeBERT model)
- **swin** -- [SwinModel](/docs/transformers/pr_15592/en/model_doc/swin#transformers.SwinModel) (Swin model)
- **t5** -- [T5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Model) (T5 model)
- **tapas** -- [TapasModel](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasModel) (TAPAS model)
- **transfo-xl** -- [TransfoXLModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLModel) (Transformer-XL model)
- **unispeech** -- [UniSpeechModel](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechModel) (UniSpeech model)
- **unispeech-sat** -- [UniSpeechSatModel](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatModel) (UniSpeechSat model)
- **vilt** -- [ViltModel](/docs/transformers/pr_15592/en/model_doc/vilt#transformers.ViltModel) (ViLT model)
- **vision-text-dual-encoder** -- [VisionTextDualEncoderModel](/docs/transformers/pr_15592/en/model_doc/vision-text-dual-encoder#transformers.VisionTextDualEncoderModel) (VisionTextDualEncoder model)
- **visual_bert** -- [VisualBertModel](/docs/transformers/pr_15592/en/model_doc/visual_bert#transformers.VisualBertModel) (VisualBert model)
- **vit** -- [ViTModel](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTModel) (ViT model)
- **vit_mae** -- [ViTMAEModel](/docs/transformers/pr_15592/en/model_doc/vit_mae#transformers.ViTMAEModel) (ViTMAE model)
- **wav2vec2** -- [Wav2Vec2Model](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Model) (Wav2Vec2 model)
- **wavlm** -- [WavLMModel](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMModel) (WavLM model)
- **xglm** -- [XGLMModel](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMModel) (XGLM model)
- **xlm** -- [XLMModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMModel) (XLM model)
- **xlm-prophetnet** -- [XLMProphetNetModel](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetModel) (XLMProphetNet model)
- **xlm-roberta** -- [XLMRobertaModel](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaModel) (XLM-RoBERTa model)
- **xlm-roberta-xl** -- [XLMRobertaXLModel](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLModel) (XLM-RoBERTa-XL model)
- **xlnet** -- [XLNetModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetModel) (XLNet model)
- **yoso** -- [YosoModel](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoModel) (YOSO model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModel

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModel.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModel.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModel.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForPreTraining">AutoModelForPreTraining</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForPreTraining</name><anchor>transformers.AutoModelForPreTraining</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L679</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a pretraining head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [AlbertForPreTraining](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForPreTraining) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [BartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForConditionalGeneration) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [BertForPreTraining](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForPreTraining) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [BigBirdForPreTraining](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForPreTraining) (BigBird model)
  - [CTRLConfig](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLConfig) configuration class: [CTRLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLLMHeadModel) (CTRL model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [CamembertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForMaskedLM) (CamemBERT model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [DebertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaForMaskedLM) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [DebertaV2ForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2ForMaskedLM) (DeBERTa-v2 model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [DistilBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForMaskedLM) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [ElectraForPreTraining](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForPreTraining) (ELECTRA model)
  - [FNetConfig](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetConfig) configuration class: [FNetForPreTraining](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForPreTraining) (FNet model)
  - [FSMTConfig](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTConfig) configuration class: [FSMTForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTForConditionalGeneration) (FairSeq Machine-Translation model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [FlaubertWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertWithLMHeadModel) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [FunnelForPreTraining](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForPreTraining) (Funnel Transformer model)
  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [GPT2LMHeadModel](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2LMHeadModel) (OpenAI GPT-2 model)
  - [IBertConfig](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertConfig) configuration class: [IBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForMaskedLM) (I-BERT model)
  - [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) configuration class: [LayoutLMForMaskedLM](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMForMaskedLM) (LayoutLM model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [LongformerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForMaskedLM) (Longformer model)
  - [LxmertConfig](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertConfig) configuration class: [LxmertForPreTraining](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertForPreTraining) (LXMERT model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [MPNetForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForMaskedLM) (MPNet model)
  - [MegatronBertConfig](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertConfig) configuration class: [MegatronBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForPreTraining) (MegatronBert model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [MobileBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForPreTraining) (MobileBERT model)
  - [OpenAIGPTConfig](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTConfig) configuration class: [OpenAIGPTLMHeadModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTLMHeadModel) (OpenAI GPT model)
  - [RetriBertConfig](/docs/transformers/pr_15592/en/model_doc/retribert#transformers.RetriBertConfig) configuration class: [RetriBertModel](/docs/transformers/pr_15592/en/model_doc/retribert#transformers.RetriBertModel) (RetriBERT model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [RobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForMaskedLM) (RoBERTa model)
  - [SqueezeBertConfig](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertConfig) configuration class: [SqueezeBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForMaskedLM) (SqueezeBERT model)
  - [T5Config](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Config) configuration class: [T5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5ForConditionalGeneration) (T5 model)
  - [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) configuration class: [TapasForMaskedLM](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasForMaskedLM) (TAPAS model)
  - [TransfoXLConfig](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLConfig) configuration class: [TransfoXLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLLMHeadModel) (Transformer-XL model)
  - [UniSpeechConfig](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechConfig) configuration class: [UniSpeechForPreTraining](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechForPreTraining) (UniSpeech model)
  - [UniSpeechSatConfig](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatConfig) configuration class: [UniSpeechSatForPreTraining](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatForPreTraining) (UniSpeechSat model)
  - [ViTMAEConfig](/docs/transformers/pr_15592/en/model_doc/vit_mae#transformers.ViTMAEConfig) configuration class: [ViTMAEForPreTraining](/docs/transformers/pr_15592/en/model_doc/vit_mae#transformers.ViTMAEForPreTraining) (ViTMAE model)
  - [VisualBertConfig](/docs/transformers/pr_15592/en/model_doc/visual_bert#transformers.VisualBertConfig) configuration class: [VisualBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/visual_bert#transformers.VisualBertForPreTraining) (VisualBert model)
  - [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) configuration class: [Wav2Vec2ForPreTraining](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2ForPreTraining) (Wav2Vec2 model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [XLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMWithLMHeadModel) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [XLMRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForMaskedLM) (XLM-RoBERTa model)
  - [XLMRobertaXLConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLConfig) configuration class: [XLMRobertaXLForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForMaskedLM) (XLM-RoBERTa-XL model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [XLNetLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetLMHeadModel) (XLNet model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a pretraining head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForPreTraining

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForPreTraining.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a pretraining head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [AlbertForPreTraining](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForPreTraining) (ALBERT model)
- **bart** -- [BartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForConditionalGeneration) (BART model)
- **bert** -- [BertForPreTraining](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForPreTraining) (BERT model)
- **big_bird** -- [BigBirdForPreTraining](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForPreTraining) (BigBird model)
- **camembert** -- [CamembertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForMaskedLM) (CamemBERT model)
- **ctrl** -- [CTRLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLLMHeadModel) (CTRL model)
- **deberta** -- [DebertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaForMaskedLM) (DeBERTa model)
- **deberta-v2** -- [DebertaV2ForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2ForMaskedLM) (DeBERTa-v2 model)
- **distilbert** -- [DistilBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForMaskedLM) (DistilBERT model)
- **electra** -- [ElectraForPreTraining](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForPreTraining) (ELECTRA model)
- **flaubert** -- [FlaubertWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertWithLMHeadModel) (FlauBERT model)
- **fnet** -- [FNetForPreTraining](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForPreTraining) (FNet model)
- **fsmt** -- [FSMTForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTForConditionalGeneration) (FairSeq Machine-Translation model)
- **funnel** -- [FunnelForPreTraining](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForPreTraining) (Funnel Transformer model)
- **gpt2** -- [GPT2LMHeadModel](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2LMHeadModel) (OpenAI GPT-2 model)
- **ibert** -- [IBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForMaskedLM) (I-BERT model)
- **layoutlm** -- [LayoutLMForMaskedLM](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMForMaskedLM) (LayoutLM model)
- **longformer** -- [LongformerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForMaskedLM) (Longformer model)
- **lxmert** -- [LxmertForPreTraining](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertForPreTraining) (LXMERT model)
- **megatron-bert** -- [MegatronBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForPreTraining) (MegatronBert model)
- **mobilebert** -- [MobileBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForPreTraining) (MobileBERT model)
- **mpnet** -- [MPNetForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForMaskedLM) (MPNet model)
- **openai-gpt** -- [OpenAIGPTLMHeadModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTLMHeadModel) (OpenAI GPT model)
- **retribert** -- [RetriBertModel](/docs/transformers/pr_15592/en/model_doc/retribert#transformers.RetriBertModel) (RetriBERT model)
- **roberta** -- [RobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForMaskedLM) (RoBERTa model)
- **squeezebert** -- [SqueezeBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForMaskedLM) (SqueezeBERT model)
- **t5** -- [T5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5ForConditionalGeneration) (T5 model)
- **tapas** -- [TapasForMaskedLM](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasForMaskedLM) (TAPAS model)
- **transfo-xl** -- [TransfoXLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLLMHeadModel) (Transformer-XL model)
- **unispeech** -- [UniSpeechForPreTraining](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechForPreTraining) (UniSpeech model)
- **unispeech-sat** -- [UniSpeechSatForPreTraining](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatForPreTraining) (UniSpeechSat model)
- **visual_bert** -- [VisualBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/visual_bert#transformers.VisualBertForPreTraining) (VisualBert model)
- **vit_mae** -- [ViTMAEForPreTraining](/docs/transformers/pr_15592/en/model_doc/vit_mae#transformers.ViTMAEForPreTraining) (ViTMAE model)
- **wav2vec2** -- [Wav2Vec2ForPreTraining](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2ForPreTraining) (Wav2Vec2 model)
- **xlm** -- [XLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMWithLMHeadModel) (XLM model)
- **xlm-roberta** -- [XLMRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForMaskedLM) (XLM-RoBERTa model)
- **xlm-roberta-xl** -- [XLMRobertaXLForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForMaskedLM) (XLM-RoBERTa-XL model)
- **xlnet** -- [XLNetLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetLMHeadModel) (XLNet model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForPreTraining

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForPreTraining.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForPreTraining.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForPreTraining.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForCausalLM">AutoModelForCausalLM</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForCausalLM</name><anchor>transformers.AutoModelForCausalLM</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L694</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a causal language modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [BartForCausalLM](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForCausalLM) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [BertLMHeadModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertLMHeadModel) (BERT model)
  - [BertGenerationConfig](/docs/transformers/pr_15592/en/model_doc/bert-generation#transformers.BertGenerationConfig) configuration class: [BertGenerationDecoder](/docs/transformers/pr_15592/en/model_doc/bert-generation#transformers.BertGenerationDecoder) (Bert Generation model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [BigBirdForCausalLM](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForCausalLM) (BigBird model)
  - [BigBirdPegasusConfig](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusConfig) configuration class: [BigBirdPegasusForCausalLM](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusForCausalLM) (BigBirdPegasus model)
  - [BlenderbotConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotConfig) configuration class: [BlenderbotForCausalLM](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotForCausalLM) (Blenderbot model)
  - [BlenderbotSmallConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallConfig) configuration class: [BlenderbotSmallForCausalLM](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallForCausalLM) (BlenderbotSmall model)
  - [CTRLConfig](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLConfig) configuration class: [CTRLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLLMHeadModel) (CTRL model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [CamembertForCausalLM](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForCausalLM) (CamemBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [ElectraForCausalLM](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForCausalLM) (ELECTRA model)
  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [GPT2LMHeadModel](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2LMHeadModel) (OpenAI GPT-2 model)
  - [GPTJConfig](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJConfig) configuration class: [GPTJForCausalLM](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJForCausalLM) (GPT-J model)
  - [GPTNeoConfig](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoConfig) configuration class: [GPTNeoForCausalLM](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoForCausalLM) (GPT Neo model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [MBartForCausalLM](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartForCausalLM) (mBART model)
  - [MarianConfig](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianConfig) configuration class: [MarianForCausalLM](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianForCausalLM) (Marian model)
  - [MegatronBertConfig](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertConfig) configuration class: [MegatronBertForCausalLM](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForCausalLM) (MegatronBert model)
  - [OpenAIGPTConfig](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTConfig) configuration class: [OpenAIGPTLMHeadModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTLMHeadModel) (OpenAI GPT model)
  - [PLBartConfig](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartConfig) configuration class: [PLBartForCausalLM](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartForCausalLM) (PLBart model)
  - [PegasusConfig](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusConfig) configuration class: [PegasusForCausalLM](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusForCausalLM) (Pegasus model)
  - [ProphetNetConfig](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetConfig) configuration class: [ProphetNetForCausalLM](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetForCausalLM) (ProphetNet model)
  - [QDQBertConfig](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertConfig) configuration class: [QDQBertLMHeadModel](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertLMHeadModel) (QDQBert model)
  - [ReformerConfig](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerConfig) configuration class: [ReformerModelWithLMHead](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerModelWithLMHead) (Reformer model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [RemBertForCausalLM](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForCausalLM) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [RoFormerForCausalLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForCausalLM) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [RobertaForCausalLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForCausalLM) (RoBERTa model)
  - [Speech2Text2Config](/docs/transformers/pr_15592/en/model_doc/speech_to_text_2#transformers.Speech2Text2Config) configuration class: [Speech2Text2ForCausalLM](/docs/transformers/pr_15592/en/model_doc/speech_to_text_2#transformers.Speech2Text2ForCausalLM) (Speech2Text2 model)
  - [TrOCRConfig](/docs/transformers/pr_15592/en/model_doc/trocr#transformers.TrOCRConfig) configuration class: [TrOCRForCausalLM](/docs/transformers/pr_15592/en/model_doc/trocr#transformers.TrOCRForCausalLM) (TrOCR model)
  - [TransfoXLConfig](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLConfig) configuration class: [TransfoXLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLLMHeadModel) (Transformer-XL model)
  - [XGLMConfig](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMConfig) configuration class: [XGLMForCausalLM](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMForCausalLM) (XGLM model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [XLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMWithLMHeadModel) (XLM model)
  - [XLMProphetNetConfig](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetConfig) configuration class: [XLMProphetNetForCausalLM](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetForCausalLM) (XLMProphetNet model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [XLMRobertaForCausalLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForCausalLM) (XLM-RoBERTa model)
  - [XLMRobertaXLConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLConfig) configuration class: [XLMRobertaXLForCausalLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForCausalLM) (XLM-RoBERTa-XL model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [XLNetLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetLMHeadModel) (XLNet model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a causal language modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForCausalLM

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForCausalLM.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a causal language modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **bart** -- [BartForCausalLM](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForCausalLM) (BART model)
- **bert** -- [BertLMHeadModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertLMHeadModel) (BERT model)
- **bert-generation** -- [BertGenerationDecoder](/docs/transformers/pr_15592/en/model_doc/bert-generation#transformers.BertGenerationDecoder) (Bert Generation model)
- **big_bird** -- [BigBirdForCausalLM](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForCausalLM) (BigBird model)
- **bigbird_pegasus** -- [BigBirdPegasusForCausalLM](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusForCausalLM) (BigBirdPegasus model)
- **blenderbot** -- [BlenderbotForCausalLM](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotForCausalLM) (Blenderbot model)
- **blenderbot-small** -- [BlenderbotSmallForCausalLM](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallForCausalLM) (BlenderbotSmall model)
- **camembert** -- [CamembertForCausalLM](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForCausalLM) (CamemBERT model)
- **ctrl** -- [CTRLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLLMHeadModel) (CTRL model)
- **electra** -- [ElectraForCausalLM](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForCausalLM) (ELECTRA model)
- **gpt2** -- [GPT2LMHeadModel](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2LMHeadModel) (OpenAI GPT-2 model)
- **gpt_neo** -- [GPTNeoForCausalLM](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoForCausalLM) (GPT Neo model)
- **gptj** -- [GPTJForCausalLM](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJForCausalLM) (GPT-J model)
- **marian** -- [MarianForCausalLM](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianForCausalLM) (Marian model)
- **mbart** -- [MBartForCausalLM](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartForCausalLM) (mBART model)
- **megatron-bert** -- [MegatronBertForCausalLM](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForCausalLM) (MegatronBert model)
- **openai-gpt** -- [OpenAIGPTLMHeadModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTLMHeadModel) (OpenAI GPT model)
- **pegasus** -- [PegasusForCausalLM](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusForCausalLM) (Pegasus model)
- **plbart** -- [PLBartForCausalLM](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartForCausalLM) (PLBart model)
- **prophetnet** -- [ProphetNetForCausalLM](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetForCausalLM) (ProphetNet model)
- **qdqbert** -- [QDQBertLMHeadModel](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertLMHeadModel) (QDQBert model)
- **reformer** -- [ReformerModelWithLMHead](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerModelWithLMHead) (Reformer model)
- **rembert** -- [RemBertForCausalLM](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForCausalLM) (RemBERT model)
- **roberta** -- [RobertaForCausalLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForCausalLM) (RoBERTa model)
- **roformer** -- [RoFormerForCausalLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForCausalLM) (RoFormer model)
- **speech_to_text_2** -- [Speech2Text2ForCausalLM](/docs/transformers/pr_15592/en/model_doc/speech_to_text_2#transformers.Speech2Text2ForCausalLM) (Speech2Text2 model)
- **transfo-xl** -- [TransfoXLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLLMHeadModel) (Transformer-XL model)
- **trocr** -- [TrOCRForCausalLM](/docs/transformers/pr_15592/en/model_doc/trocr#transformers.TrOCRForCausalLM) (TrOCR model)
- **xglm** -- [XGLMForCausalLM](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMForCausalLM) (XGLM model)
- **xlm** -- [XLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMWithLMHeadModel) (XLM model)
- **xlm-prophetnet** -- [XLMProphetNetForCausalLM](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetForCausalLM) (XLMProphetNet model)
- **xlm-roberta** -- [XLMRobertaForCausalLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForCausalLM) (XLM-RoBERTa model)
- **xlm-roberta-xl** -- [XLMRobertaXLForCausalLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForCausalLM) (XLM-RoBERTa-XL model)
- **xlnet** -- [XLNetLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetLMHeadModel) (XLNet model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForCausalLM

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForCausalLM.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForCausalLM.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForCausalLM.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForMaskedLM">AutoModelForMaskedLM</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForMaskedLM</name><anchor>transformers.AutoModelForMaskedLM</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L701</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a masked language modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [AlbertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForMaskedLM) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [BartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForConditionalGeneration) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [BertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForMaskedLM) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [BigBirdForMaskedLM](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForMaskedLM) (BigBird model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [CamembertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForMaskedLM) (CamemBERT model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [ConvBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertForMaskedLM) (ConvBERT model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [DebertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaForMaskedLM) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [DebertaV2ForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2ForMaskedLM) (DeBERTa-v2 model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [DistilBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForMaskedLM) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [ElectraForMaskedLM](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForMaskedLM) (ELECTRA model)
  - [FNetConfig](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetConfig) configuration class: [FNetForMaskedLM](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForMaskedLM) (FNet model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [FlaubertWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertWithLMHeadModel) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [FunnelForMaskedLM](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForMaskedLM) (Funnel Transformer model)
  - [IBertConfig](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertConfig) configuration class: [IBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForMaskedLM) (I-BERT model)
  - [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) configuration class: [LayoutLMForMaskedLM](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMForMaskedLM) (LayoutLM model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [LongformerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForMaskedLM) (Longformer model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [MBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartForConditionalGeneration) (mBART model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [MPNetForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForMaskedLM) (MPNet model)
  - [MegatronBertConfig](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertConfig) configuration class: [MegatronBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForMaskedLM) (MegatronBert model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [MobileBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForMaskedLM) (MobileBERT model)
  - [NystromformerConfig](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerConfig) configuration class: [NystromformerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerForMaskedLM) (Nystromformer model)
  - [PerceiverConfig](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverConfig) configuration class: [PerceiverForMaskedLM](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverForMaskedLM) (Perceiver model)
  - [QDQBertConfig](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertConfig) configuration class: [QDQBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForMaskedLM) (QDQBert model)
  - [ReformerConfig](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerConfig) configuration class: [ReformerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerForMaskedLM) (Reformer model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [RemBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForMaskedLM) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [RoFormerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForMaskedLM) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [RobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForMaskedLM) (RoBERTa model)
  - [SqueezeBertConfig](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertConfig) configuration class: [SqueezeBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForMaskedLM) (SqueezeBERT model)
  - [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) configuration class: [TapasForMaskedLM](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasForMaskedLM) (TAPAS model)
  - [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) configuration class: `Wav2Vec2ForMaskedLM`(Wav2Vec2 model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [XLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMWithLMHeadModel) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [XLMRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForMaskedLM) (XLM-RoBERTa model)
  - [XLMRobertaXLConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLConfig) configuration class: [XLMRobertaXLForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForMaskedLM) (XLM-RoBERTa-XL model)
  - [YosoConfig](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoConfig) configuration class: [YosoForMaskedLM](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoForMaskedLM) (YOSO model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a masked language modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForMaskedLM

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForMaskedLM.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a masked language modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [AlbertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForMaskedLM) (ALBERT model)
- **bart** -- [BartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForConditionalGeneration) (BART model)
- **bert** -- [BertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForMaskedLM) (BERT model)
- **big_bird** -- [BigBirdForMaskedLM](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForMaskedLM) (BigBird model)
- **camembert** -- [CamembertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForMaskedLM) (CamemBERT model)
- **convbert** -- [ConvBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertForMaskedLM) (ConvBERT model)
- **deberta** -- [DebertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaForMaskedLM) (DeBERTa model)
- **deberta-v2** -- [DebertaV2ForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2ForMaskedLM) (DeBERTa-v2 model)
- **distilbert** -- [DistilBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForMaskedLM) (DistilBERT model)
- **electra** -- [ElectraForMaskedLM](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForMaskedLM) (ELECTRA model)
- **flaubert** -- [FlaubertWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertWithLMHeadModel) (FlauBERT model)
- **fnet** -- [FNetForMaskedLM](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForMaskedLM) (FNet model)
- **funnel** -- [FunnelForMaskedLM](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForMaskedLM) (Funnel Transformer model)
- **ibert** -- [IBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForMaskedLM) (I-BERT model)
- **layoutlm** -- [LayoutLMForMaskedLM](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMForMaskedLM) (LayoutLM model)
- **longformer** -- [LongformerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForMaskedLM) (Longformer model)
- **mbart** -- [MBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartForConditionalGeneration) (mBART model)
- **megatron-bert** -- [MegatronBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForMaskedLM) (MegatronBert model)
- **mobilebert** -- [MobileBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForMaskedLM) (MobileBERT model)
- **mpnet** -- [MPNetForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForMaskedLM) (MPNet model)
- **nystromformer** -- [NystromformerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerForMaskedLM) (Nystromformer model)
- **perceiver** -- [PerceiverForMaskedLM](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverForMaskedLM) (Perceiver model)
- **qdqbert** -- [QDQBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForMaskedLM) (QDQBert model)
- **reformer** -- [ReformerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerForMaskedLM) (Reformer model)
- **rembert** -- [RemBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForMaskedLM) (RemBERT model)
- **roberta** -- [RobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForMaskedLM) (RoBERTa model)
- **roformer** -- [RoFormerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForMaskedLM) (RoFormer model)
- **squeezebert** -- [SqueezeBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForMaskedLM) (SqueezeBERT model)
- **tapas** -- [TapasForMaskedLM](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasForMaskedLM) (TAPAS model)
- **wav2vec2** -- `Wav2Vec2ForMaskedLM`(Wav2Vec2 model)
- **xlm** -- [XLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMWithLMHeadModel) (XLM model)
- **xlm-roberta** -- [XLMRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForMaskedLM) (XLM-RoBERTa model)
- **xlm-roberta-xl** -- [XLMRobertaXLForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForMaskedLM) (XLM-RoBERTa-XL model)
- **yoso** -- [YosoForMaskedLM](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoForMaskedLM) (YOSO model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForMaskedLM

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForMaskedLM.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForMaskedLM.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForMaskedLM.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForSeq2SeqLM">AutoModelForSeq2SeqLM</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForSeq2SeqLM</name><anchor>transformers.AutoModelForSeq2SeqLM</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L708</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence-to-sequence language modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [BartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForConditionalGeneration) (BART model)
  - [BigBirdPegasusConfig](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusConfig) configuration class: [BigBirdPegasusForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusForConditionalGeneration) (BigBirdPegasus model)
  - [BlenderbotConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotConfig) configuration class: [BlenderbotForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotForConditionalGeneration) (Blenderbot model)
  - [BlenderbotSmallConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallConfig) configuration class: [BlenderbotSmallForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallForConditionalGeneration) (BlenderbotSmall model)
  - [EncoderDecoderConfig](/docs/transformers/pr_15592/en/model_doc/encoder-decoder#transformers.EncoderDecoderConfig) configuration class: [EncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/encoder-decoder#transformers.EncoderDecoderModel) (Encoder decoder model)
  - [FSMTConfig](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTConfig) configuration class: [FSMTForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTForConditionalGeneration) (FairSeq Machine-Translation model)
  - [LEDConfig](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDConfig) configuration class: [LEDForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDForConditionalGeneration) (LED model)
  - [M2M100Config](/docs/transformers/pr_15592/en/model_doc/m2m_100#transformers.M2M100Config) configuration class: [M2M100ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/m2m_100#transformers.M2M100ForConditionalGeneration) (M2M100 model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [MBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartForConditionalGeneration) (mBART model)
  - [MT5Config](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Config) configuration class: [MT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5ForConditionalGeneration) (mT5 model)
  - [MarianConfig](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianConfig) configuration class: [MarianMTModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianMTModel) (Marian model)
  - [PLBartConfig](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartConfig) configuration class: [PLBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartForConditionalGeneration) (PLBart model)
  - [PegasusConfig](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusConfig) configuration class: [PegasusForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusForConditionalGeneration) (Pegasus model)
  - [ProphetNetConfig](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetConfig) configuration class: [ProphetNetForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetForConditionalGeneration) (ProphetNet model)
  - [T5Config](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Config) configuration class: [T5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5ForConditionalGeneration) (T5 model)
  - [XLMProphetNetConfig](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetConfig) configuration class: [XLMProphetNetForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetForConditionalGeneration) (XLMProphetNet model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a sequence-to-sequence language modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForSeq2SeqLM

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("t5-base")
>>> model = AutoModelForSeq2SeqLM.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a sequence-to-sequence language modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **bart** -- [BartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForConditionalGeneration) (BART model)
- **bigbird_pegasus** -- [BigBirdPegasusForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusForConditionalGeneration) (BigBirdPegasus model)
- **blenderbot** -- [BlenderbotForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotForConditionalGeneration) (Blenderbot model)
- **blenderbot-small** -- [BlenderbotSmallForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallForConditionalGeneration) (BlenderbotSmall model)
- **encoder-decoder** -- [EncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/encoder-decoder#transformers.EncoderDecoderModel) (Encoder decoder model)
- **fsmt** -- [FSMTForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/fsmt#transformers.FSMTForConditionalGeneration) (FairSeq Machine-Translation model)
- **led** -- [LEDForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDForConditionalGeneration) (LED model)
- **m2m_100** -- [M2M100ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/m2m_100#transformers.M2M100ForConditionalGeneration) (M2M100 model)
- **marian** -- [MarianMTModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianMTModel) (Marian model)
- **mbart** -- [MBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartForConditionalGeneration) (mBART model)
- **mt5** -- [MT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5ForConditionalGeneration) (mT5 model)
- **pegasus** -- [PegasusForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusForConditionalGeneration) (Pegasus model)
- **plbart** -- [PLBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartForConditionalGeneration) (PLBart model)
- **prophetnet** -- [ProphetNetForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/prophetnet#transformers.ProphetNetForConditionalGeneration) (ProphetNet model)
- **t5** -- [T5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5ForConditionalGeneration) (T5 model)
- **xlm-prophetnet** -- [XLMProphetNetForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/xlm-prophetnet#transformers.XLMProphetNetForConditionalGeneration) (XLMProphetNet model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForSeq2SeqLM

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")

>>> # Update configuration during loading
>>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/t5_tf_model_config.json")
>>> model = AutoModelForSeq2SeqLM.from_pretrained(
...     "./tf_model/t5_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForSequenceClassification">AutoModelForSequenceClassification</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForSequenceClassification</name><anchor>transformers.AutoModelForSequenceClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L717</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [AlbertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForSequenceClassification) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [BartForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForSequenceClassification) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [BertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForSequenceClassification) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [BigBirdForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForSequenceClassification) (BigBird model)
  - [BigBirdPegasusConfig](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusConfig) configuration class: [BigBirdPegasusForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusForSequenceClassification) (BigBirdPegasus model)
  - [CTRLConfig](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLConfig) configuration class: [CTRLForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLForSequenceClassification) (CTRL model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [CamembertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForSequenceClassification) (CamemBERT model)
  - [CanineConfig](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineConfig) configuration class: [CanineForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineForSequenceClassification) (Canine model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [ConvBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertForSequenceClassification) (ConvBERT model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [DebertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaForSequenceClassification) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [DebertaV2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2ForSequenceClassification) (DeBERTa-v2 model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [DistilBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForSequenceClassification) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [ElectraForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForSequenceClassification) (ELECTRA model)
  - [FNetConfig](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetConfig) configuration class: [FNetForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForSequenceClassification) (FNet model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [FlaubertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertForSequenceClassification) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [FunnelForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForSequenceClassification) (Funnel Transformer model)
  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [GPT2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2ForSequenceClassification) (OpenAI GPT-2 model)
  - [GPTJConfig](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJConfig) configuration class: [GPTJForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJForSequenceClassification) (GPT-J model)
  - [GPTNeoConfig](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoConfig) configuration class: [GPTNeoForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoForSequenceClassification) (GPT Neo model)
  - [IBertConfig](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertConfig) configuration class: [IBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForSequenceClassification) (I-BERT model)
  - [LEDConfig](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDConfig) configuration class: [LEDForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDForSequenceClassification) (LED model)
  - [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) configuration class: [LayoutLMForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMForSequenceClassification) (LayoutLM model)
  - [LayoutLMv2Config](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2Config) configuration class: [LayoutLMv2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2ForSequenceClassification) (LayoutLMv2 model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [LongformerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForSequenceClassification) (Longformer model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [MBartForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartForSequenceClassification) (mBART model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [MPNetForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForSequenceClassification) (MPNet model)
  - [MegatronBertConfig](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertConfig) configuration class: [MegatronBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForSequenceClassification) (MegatronBert model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [MobileBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForSequenceClassification) (MobileBERT model)
  - [NystromformerConfig](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerConfig) configuration class: [NystromformerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerForSequenceClassification) (Nystromformer model)
  - [OpenAIGPTConfig](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTConfig) configuration class: [OpenAIGPTForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTForSequenceClassification) (OpenAI GPT model)
  - [PLBartConfig](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartConfig) configuration class: [PLBartForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartForSequenceClassification) (PLBart model)
  - [PerceiverConfig](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverConfig) configuration class: [PerceiverForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverForSequenceClassification) (Perceiver model)
  - [QDQBertConfig](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertConfig) configuration class: [QDQBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForSequenceClassification) (QDQBert model)
  - [ReformerConfig](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerConfig) configuration class: [ReformerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerForSequenceClassification) (Reformer model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [RemBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForSequenceClassification) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [RoFormerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForSequenceClassification) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [RobertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForSequenceClassification) (RoBERTa model)
  - [SqueezeBertConfig](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertConfig) configuration class: [SqueezeBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForSequenceClassification) (SqueezeBERT model)
  - [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) configuration class: [TapasForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasForSequenceClassification) (TAPAS model)
  - [TransfoXLConfig](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLConfig) configuration class: [TransfoXLForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLForSequenceClassification) (Transformer-XL model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [XLMForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMForSequenceClassification) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [XLMRobertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForSequenceClassification) (XLM-RoBERTa model)
  - [XLMRobertaXLConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLConfig) configuration class: [XLMRobertaXLForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForSequenceClassification) (XLM-RoBERTa-XL model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [XLNetForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetForSequenceClassification) (XLNet model)
  - [YosoConfig](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoConfig) configuration class: [YosoForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoForSequenceClassification) (YOSO model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a sequence classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForSequenceClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForSequenceClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a sequence classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [AlbertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForSequenceClassification) (ALBERT model)
- **bart** -- [BartForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForSequenceClassification) (BART model)
- **bert** -- [BertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForSequenceClassification) (BERT model)
- **big_bird** -- [BigBirdForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForSequenceClassification) (BigBird model)
- **bigbird_pegasus** -- [BigBirdPegasusForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusForSequenceClassification) (BigBirdPegasus model)
- **camembert** -- [CamembertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForSequenceClassification) (CamemBERT model)
- **canine** -- [CanineForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineForSequenceClassification) (Canine model)
- **convbert** -- [ConvBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertForSequenceClassification) (ConvBERT model)
- **ctrl** -- [CTRLForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLForSequenceClassification) (CTRL model)
- **deberta** -- [DebertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaForSequenceClassification) (DeBERTa model)
- **deberta-v2** -- [DebertaV2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2ForSequenceClassification) (DeBERTa-v2 model)
- **distilbert** -- [DistilBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForSequenceClassification) (DistilBERT model)
- **electra** -- [ElectraForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForSequenceClassification) (ELECTRA model)
- **flaubert** -- [FlaubertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertForSequenceClassification) (FlauBERT model)
- **fnet** -- [FNetForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForSequenceClassification) (FNet model)
- **funnel** -- [FunnelForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForSequenceClassification) (Funnel Transformer model)
- **gpt2** -- [GPT2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2ForSequenceClassification) (OpenAI GPT-2 model)
- **gpt_neo** -- [GPTNeoForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoForSequenceClassification) (GPT Neo model)
- **gptj** -- [GPTJForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJForSequenceClassification) (GPT-J model)
- **ibert** -- [IBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForSequenceClassification) (I-BERT model)
- **layoutlm** -- [LayoutLMForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMForSequenceClassification) (LayoutLM model)
- **layoutlmv2** -- [LayoutLMv2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2ForSequenceClassification) (LayoutLMv2 model)
- **led** -- [LEDForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDForSequenceClassification) (LED model)
- **longformer** -- [LongformerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForSequenceClassification) (Longformer model)
- **mbart** -- [MBartForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartForSequenceClassification) (mBART model)
- **megatron-bert** -- [MegatronBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForSequenceClassification) (MegatronBert model)
- **mobilebert** -- [MobileBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForSequenceClassification) (MobileBERT model)
- **mpnet** -- [MPNetForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForSequenceClassification) (MPNet model)
- **nystromformer** -- [NystromformerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerForSequenceClassification) (Nystromformer model)
- **openai-gpt** -- [OpenAIGPTForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTForSequenceClassification) (OpenAI GPT model)
- **perceiver** -- [PerceiverForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverForSequenceClassification) (Perceiver model)
- **plbart** -- [PLBartForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/plbart#transformers.PLBartForSequenceClassification) (PLBart model)
- **qdqbert** -- [QDQBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForSequenceClassification) (QDQBert model)
- **reformer** -- [ReformerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerForSequenceClassification) (Reformer model)
- **rembert** -- [RemBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForSequenceClassification) (RemBERT model)
- **roberta** -- [RobertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForSequenceClassification) (RoBERTa model)
- **roformer** -- [RoFormerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForSequenceClassification) (RoFormer model)
- **squeezebert** -- [SqueezeBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForSequenceClassification) (SqueezeBERT model)
- **tapas** -- [TapasForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasForSequenceClassification) (TAPAS model)
- **transfo-xl** -- [TransfoXLForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLForSequenceClassification) (Transformer-XL model)
- **xlm** -- [XLMForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMForSequenceClassification) (XLM model)
- **xlm-roberta** -- [XLMRobertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForSequenceClassification) (XLM-RoBERTa model)
- **xlm-roberta-xl** -- [XLMRobertaXLForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForSequenceClassification) (XLM-RoBERTa-XL model)
- **xlnet** -- [XLNetForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetForSequenceClassification) (XLNet model)
- **yoso** -- [YosoForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoForSequenceClassification) (YOSO model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForSequenceClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForSequenceClassification.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForMultipleChoice">AutoModelForMultipleChoice</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForMultipleChoice</name><anchor>transformers.AutoModelForMultipleChoice</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L751</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a multiple choice head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [AlbertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForMultipleChoice) (ALBERT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [BertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForMultipleChoice) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [BigBirdForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForMultipleChoice) (BigBird model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [CamembertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForMultipleChoice) (CamemBERT model)
  - [CanineConfig](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineConfig) configuration class: [CanineForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineForMultipleChoice) (Canine model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [ConvBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertForMultipleChoice) (ConvBERT model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [DistilBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForMultipleChoice) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [ElectraForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForMultipleChoice) (ELECTRA model)
  - [FNetConfig](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetConfig) configuration class: [FNetForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForMultipleChoice) (FNet model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [FlaubertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertForMultipleChoice) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [FunnelForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForMultipleChoice) (Funnel Transformer model)
  - [IBertConfig](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertConfig) configuration class: [IBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForMultipleChoice) (I-BERT model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [LongformerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForMultipleChoice) (Longformer model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [MPNetForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForMultipleChoice) (MPNet model)
  - [MegatronBertConfig](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertConfig) configuration class: [MegatronBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForMultipleChoice) (MegatronBert model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [MobileBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForMultipleChoice) (MobileBERT model)
  - [NystromformerConfig](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerConfig) configuration class: [NystromformerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerForMultipleChoice) (Nystromformer model)
  - [QDQBertConfig](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertConfig) configuration class: [QDQBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForMultipleChoice) (QDQBert model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [RemBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForMultipleChoice) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [RoFormerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForMultipleChoice) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [RobertaForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForMultipleChoice) (RoBERTa model)
  - [SqueezeBertConfig](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertConfig) configuration class: [SqueezeBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForMultipleChoice) (SqueezeBERT model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [XLMForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMForMultipleChoice) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [XLMRobertaForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForMultipleChoice) (XLM-RoBERTa model)
  - [XLMRobertaXLConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLConfig) configuration class: [XLMRobertaXLForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForMultipleChoice) (XLM-RoBERTa-XL model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [XLNetForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetForMultipleChoice) (XLNet model)
  - [YosoConfig](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoConfig) configuration class: [YosoForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoForMultipleChoice) (YOSO model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a multiple choice head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForMultipleChoice

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForMultipleChoice.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a multiple choice head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [AlbertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForMultipleChoice) (ALBERT model)
- **bert** -- [BertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForMultipleChoice) (BERT model)
- **big_bird** -- [BigBirdForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForMultipleChoice) (BigBird model)
- **camembert** -- [CamembertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForMultipleChoice) (CamemBERT model)
- **canine** -- [CanineForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineForMultipleChoice) (Canine model)
- **convbert** -- [ConvBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertForMultipleChoice) (ConvBERT model)
- **distilbert** -- [DistilBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForMultipleChoice) (DistilBERT model)
- **electra** -- [ElectraForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForMultipleChoice) (ELECTRA model)
- **flaubert** -- [FlaubertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertForMultipleChoice) (FlauBERT model)
- **fnet** -- [FNetForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForMultipleChoice) (FNet model)
- **funnel** -- [FunnelForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForMultipleChoice) (Funnel Transformer model)
- **ibert** -- [IBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForMultipleChoice) (I-BERT model)
- **longformer** -- [LongformerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForMultipleChoice) (Longformer model)
- **megatron-bert** -- [MegatronBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForMultipleChoice) (MegatronBert model)
- **mobilebert** -- [MobileBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForMultipleChoice) (MobileBERT model)
- **mpnet** -- [MPNetForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForMultipleChoice) (MPNet model)
- **nystromformer** -- [NystromformerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerForMultipleChoice) (Nystromformer model)
- **qdqbert** -- [QDQBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForMultipleChoice) (QDQBert model)
- **rembert** -- [RemBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForMultipleChoice) (RemBERT model)
- **roberta** -- [RobertaForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForMultipleChoice) (RoBERTa model)
- **roformer** -- [RoFormerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForMultipleChoice) (RoFormer model)
- **squeezebert** -- [SqueezeBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForMultipleChoice) (SqueezeBERT model)
- **xlm** -- [XLMForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMForMultipleChoice) (XLM model)
- **xlm-roberta** -- [XLMRobertaForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForMultipleChoice) (XLM-RoBERTa model)
- **xlm-roberta-xl** -- [XLMRobertaXLForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForMultipleChoice) (XLM-RoBERTa-XL model)
- **xlnet** -- [XLNetForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetForMultipleChoice) (XLNet model)
- **yoso** -- [YosoForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoForMultipleChoice) (YOSO model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForMultipleChoice

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForMultipleChoice.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForMultipleChoice.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForMultipleChoice.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForNextSentencePrediction">AutoModelForNextSentencePrediction</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForNextSentencePrediction</name><anchor>transformers.AutoModelForNextSentencePrediction</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L758</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a next sentence prediction head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [BertForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForNextSentencePrediction) (BERT model)
  - [FNetConfig](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetConfig) configuration class: [FNetForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForNextSentencePrediction) (FNet model)
  - [MegatronBertConfig](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertConfig) configuration class: [MegatronBertForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForNextSentencePrediction) (MegatronBert model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [MobileBertForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForNextSentencePrediction) (MobileBERT model)
  - [QDQBertConfig](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertConfig) configuration class: [QDQBertForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForNextSentencePrediction) (QDQBert model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a next sentence prediction head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForNextSentencePrediction

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForNextSentencePrediction.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a next sentence prediction head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **bert** -- [BertForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForNextSentencePrediction) (BERT model)
- **fnet** -- [FNetForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForNextSentencePrediction) (FNet model)
- **megatron-bert** -- [MegatronBertForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForNextSentencePrediction) (MegatronBert model)
- **mobilebert** -- [MobileBertForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForNextSentencePrediction) (MobileBERT model)
- **qdqbert** -- [QDQBertForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForNextSentencePrediction) (QDQBert model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForNextSentencePrediction

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForNextSentencePrediction.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForNextSentencePrediction.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForNextSentencePrediction.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForTokenClassification">AutoModelForTokenClassification</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForTokenClassification</name><anchor>transformers.AutoModelForTokenClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L744</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a token classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [AlbertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForTokenClassification) (ALBERT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [BertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForTokenClassification) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [BigBirdForTokenClassification](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForTokenClassification) (BigBird model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [CamembertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForTokenClassification) (CamemBERT model)
  - [CanineConfig](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineConfig) configuration class: [CanineForTokenClassification](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineForTokenClassification) (Canine model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [ConvBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertForTokenClassification) (ConvBERT model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [DebertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaForTokenClassification) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [DebertaV2ForTokenClassification](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2ForTokenClassification) (DeBERTa-v2 model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [DistilBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForTokenClassification) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [ElectraForTokenClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForTokenClassification) (ELECTRA model)
  - [FNetConfig](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetConfig) configuration class: [FNetForTokenClassification](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForTokenClassification) (FNet model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [FlaubertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertForTokenClassification) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [FunnelForTokenClassification](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForTokenClassification) (Funnel Transformer model)
  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [GPT2ForTokenClassification](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2ForTokenClassification) (OpenAI GPT-2 model)
  - [IBertConfig](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertConfig) configuration class: [IBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForTokenClassification) (I-BERT model)
  - [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) configuration class: [LayoutLMForTokenClassification](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMForTokenClassification) (LayoutLM model)
  - [LayoutLMv2Config](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2Config) configuration class: [LayoutLMv2ForTokenClassification](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2ForTokenClassification) (LayoutLMv2 model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [LongformerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForTokenClassification) (Longformer model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [MPNetForTokenClassification](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForTokenClassification) (MPNet model)
  - [MegatronBertConfig](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertConfig) configuration class: [MegatronBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForTokenClassification) (MegatronBert model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [MobileBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForTokenClassification) (MobileBERT model)
  - [NystromformerConfig](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerConfig) configuration class: [NystromformerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerForTokenClassification) (Nystromformer model)
  - [QDQBertConfig](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertConfig) configuration class: [QDQBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForTokenClassification) (QDQBert model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [RemBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForTokenClassification) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [RoFormerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForTokenClassification) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [RobertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForTokenClassification) (RoBERTa model)
  - [SqueezeBertConfig](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertConfig) configuration class: [SqueezeBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForTokenClassification) (SqueezeBERT model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [XLMForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMForTokenClassification) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [XLMRobertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForTokenClassification) (XLM-RoBERTa model)
  - [XLMRobertaXLConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLConfig) configuration class: [XLMRobertaXLForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForTokenClassification) (XLM-RoBERTa-XL model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [XLNetForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetForTokenClassification) (XLNet model)
  - [YosoConfig](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoConfig) configuration class: [YosoForTokenClassification](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoForTokenClassification) (YOSO model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a token classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForTokenClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForTokenClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a token classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [AlbertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForTokenClassification) (ALBERT model)
- **bert** -- [BertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForTokenClassification) (BERT model)
- **big_bird** -- [BigBirdForTokenClassification](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForTokenClassification) (BigBird model)
- **camembert** -- [CamembertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForTokenClassification) (CamemBERT model)
- **canine** -- [CanineForTokenClassification](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineForTokenClassification) (Canine model)
- **convbert** -- [ConvBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertForTokenClassification) (ConvBERT model)
- **deberta** -- [DebertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaForTokenClassification) (DeBERTa model)
- **deberta-v2** -- [DebertaV2ForTokenClassification](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2ForTokenClassification) (DeBERTa-v2 model)
- **distilbert** -- [DistilBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForTokenClassification) (DistilBERT model)
- **electra** -- [ElectraForTokenClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForTokenClassification) (ELECTRA model)
- **flaubert** -- [FlaubertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertForTokenClassification) (FlauBERT model)
- **fnet** -- [FNetForTokenClassification](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForTokenClassification) (FNet model)
- **funnel** -- [FunnelForTokenClassification](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForTokenClassification) (Funnel Transformer model)
- **gpt2** -- [GPT2ForTokenClassification](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2ForTokenClassification) (OpenAI GPT-2 model)
- **ibert** -- [IBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForTokenClassification) (I-BERT model)
- **layoutlm** -- [LayoutLMForTokenClassification](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMForTokenClassification) (LayoutLM model)
- **layoutlmv2** -- [LayoutLMv2ForTokenClassification](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2ForTokenClassification) (LayoutLMv2 model)
- **longformer** -- [LongformerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForTokenClassification) (Longformer model)
- **megatron-bert** -- [MegatronBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForTokenClassification) (MegatronBert model)
- **mobilebert** -- [MobileBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForTokenClassification) (MobileBERT model)
- **mpnet** -- [MPNetForTokenClassification](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForTokenClassification) (MPNet model)
- **nystromformer** -- [NystromformerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerForTokenClassification) (Nystromformer model)
- **qdqbert** -- [QDQBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForTokenClassification) (QDQBert model)
- **rembert** -- [RemBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForTokenClassification) (RemBERT model)
- **roberta** -- [RobertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForTokenClassification) (RoBERTa model)
- **roformer** -- [RoFormerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForTokenClassification) (RoFormer model)
- **squeezebert** -- [SqueezeBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForTokenClassification) (SqueezeBERT model)
- **xlm** -- [XLMForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMForTokenClassification) (XLM model)
- **xlm-roberta** -- [XLMRobertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForTokenClassification) (XLM-RoBERTa model)
- **xlm-roberta-xl** -- [XLMRobertaXLForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForTokenClassification) (XLM-RoBERTa-XL model)
- **xlnet** -- [XLNetForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetForTokenClassification) (XLNet model)
- **yoso** -- [YosoForTokenClassification](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoForTokenClassification) (YOSO model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForTokenClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForTokenClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForTokenClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForTokenClassification.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForQuestionAnswering">AutoModelForQuestionAnswering</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForQuestionAnswering</name><anchor>transformers.AutoModelForQuestionAnswering</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L726</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a question answering head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [AlbertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForQuestionAnswering) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [BartForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForQuestionAnswering) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [BertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForQuestionAnswering) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [BigBirdForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForQuestionAnswering) (BigBird model)
  - [BigBirdPegasusConfig](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusConfig) configuration class: [BigBirdPegasusForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusForQuestionAnswering) (BigBirdPegasus model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [CamembertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForQuestionAnswering) (CamemBERT model)
  - [CanineConfig](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineConfig) configuration class: [CanineForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineForQuestionAnswering) (Canine model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [ConvBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertForQuestionAnswering) (ConvBERT model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [DebertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaForQuestionAnswering) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [DebertaV2ForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2ForQuestionAnswering) (DeBERTa-v2 model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [DistilBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForQuestionAnswering) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [ElectraForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForQuestionAnswering) (ELECTRA model)
  - [FNetConfig](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetConfig) configuration class: [FNetForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForQuestionAnswering) (FNet model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [FlaubertForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertForQuestionAnsweringSimple) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [FunnelForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForQuestionAnswering) (Funnel Transformer model)
  - [GPTJConfig](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJConfig) configuration class: [GPTJForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJForQuestionAnswering) (GPT-J model)
  - [IBertConfig](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertConfig) configuration class: [IBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForQuestionAnswering) (I-BERT model)
  - [LEDConfig](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDConfig) configuration class: [LEDForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDForQuestionAnswering) (LED model)
  - [LayoutLMv2Config](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2Config) configuration class: [LayoutLMv2ForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2ForQuestionAnswering) (LayoutLMv2 model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [LongformerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForQuestionAnswering) (Longformer model)
  - [LxmertConfig](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertConfig) configuration class: [LxmertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertForQuestionAnswering) (LXMERT model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [MBartForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartForQuestionAnswering) (mBART model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [MPNetForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForQuestionAnswering) (MPNet model)
  - [MegatronBertConfig](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertConfig) configuration class: [MegatronBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForQuestionAnswering) (MegatronBert model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [MobileBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForQuestionAnswering) (MobileBERT model)
  - [NystromformerConfig](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerConfig) configuration class: [NystromformerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerForQuestionAnswering) (Nystromformer model)
  - [QDQBertConfig](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertConfig) configuration class: [QDQBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForQuestionAnswering) (QDQBert model)
  - [ReformerConfig](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerConfig) configuration class: [ReformerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerForQuestionAnswering) (Reformer model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [RemBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForQuestionAnswering) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [RoFormerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForQuestionAnswering) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [RobertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForQuestionAnswering) (RoBERTa model)
  - [SplinterConfig](/docs/transformers/pr_15592/en/model_doc/splinter#transformers.SplinterConfig) configuration class: [SplinterForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/splinter#transformers.SplinterForQuestionAnswering) (Splinter model)
  - [SqueezeBertConfig](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertConfig) configuration class: [SqueezeBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForQuestionAnswering) (SqueezeBERT model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [XLMForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMForQuestionAnsweringSimple) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [XLMRobertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForQuestionAnswering) (XLM-RoBERTa model)
  - [XLMRobertaXLConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLConfig) configuration class: [XLMRobertaXLForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForQuestionAnswering) (XLM-RoBERTa-XL model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [XLNetForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetForQuestionAnsweringSimple) (XLNet model)
  - [YosoConfig](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoConfig) configuration class: [YosoForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoForQuestionAnswering) (YOSO model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a question answering head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForQuestionAnswering

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForQuestionAnswering.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a question answering head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [AlbertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertForQuestionAnswering) (ALBERT model)
- **bart** -- [BartForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartForQuestionAnswering) (BART model)
- **bert** -- [BertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertForQuestionAnswering) (BERT model)
- **big_bird** -- [BigBirdForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdForQuestionAnswering) (BigBird model)
- **bigbird_pegasus** -- [BigBirdPegasusForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bigbird_pegasus#transformers.BigBirdPegasusForQuestionAnswering) (BigBirdPegasus model)
- **camembert** -- [CamembertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertForQuestionAnswering) (CamemBERT model)
- **canine** -- [CanineForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/canine#transformers.CanineForQuestionAnswering) (Canine model)
- **convbert** -- [ConvBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertForQuestionAnswering) (ConvBERT model)
- **deberta** -- [DebertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaForQuestionAnswering) (DeBERTa model)
- **deberta-v2** -- [DebertaV2ForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2ForQuestionAnswering) (DeBERTa-v2 model)
- **distilbert** -- [DistilBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertForQuestionAnswering) (DistilBERT model)
- **electra** -- [ElectraForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraForQuestionAnswering) (ELECTRA model)
- **flaubert** -- [FlaubertForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertForQuestionAnsweringSimple) (FlauBERT model)
- **fnet** -- [FNetForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/fnet#transformers.FNetForQuestionAnswering) (FNet model)
- **funnel** -- [FunnelForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelForQuestionAnswering) (Funnel Transformer model)
- **gptj** -- [GPTJForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJForQuestionAnswering) (GPT-J model)
- **ibert** -- [IBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/ibert#transformers.IBertForQuestionAnswering) (I-BERT model)
- **layoutlmv2** -- [LayoutLMv2ForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/layoutlmv2#transformers.LayoutLMv2ForQuestionAnswering) (LayoutLMv2 model)
- **led** -- [LEDForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDForQuestionAnswering) (LED model)
- **longformer** -- [LongformerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerForQuestionAnswering) (Longformer model)
- **lxmert** -- [LxmertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertForQuestionAnswering) (LXMERT model)
- **mbart** -- [MBartForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartForQuestionAnswering) (mBART model)
- **megatron-bert** -- [MegatronBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/megatron-bert#transformers.MegatronBertForQuestionAnswering) (MegatronBert model)
- **mobilebert** -- [MobileBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertForQuestionAnswering) (MobileBERT model)
- **mpnet** -- [MPNetForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetForQuestionAnswering) (MPNet model)
- **nystromformer** -- [NystromformerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/nystromformer#transformers.NystromformerForQuestionAnswering) (Nystromformer model)
- **qdqbert** -- [QDQBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/qdqbert#transformers.QDQBertForQuestionAnswering) (QDQBert model)
- **reformer** -- [ReformerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/reformer#transformers.ReformerForQuestionAnswering) (Reformer model)
- **rembert** -- [RemBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertForQuestionAnswering) (RemBERT model)
- **roberta** -- [RobertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaForQuestionAnswering) (RoBERTa model)
- **roformer** -- [RoFormerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerForQuestionAnswering) (RoFormer model)
- **splinter** -- [SplinterForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/splinter#transformers.SplinterForQuestionAnswering) (Splinter model)
- **squeezebert** -- [SqueezeBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/squeezebert#transformers.SqueezeBertForQuestionAnswering) (SqueezeBERT model)
- **xlm** -- [XLMForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMForQuestionAnsweringSimple) (XLM model)
- **xlm-roberta** -- [XLMRobertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaForQuestionAnswering) (XLM-RoBERTa model)
- **xlm-roberta-xl** -- [XLMRobertaXLForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/xlm-roberta-xl#transformers.XLMRobertaXLForQuestionAnswering) (XLM-RoBERTa-XL model)
- **xlnet** -- [XLNetForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetForQuestionAnsweringSimple) (XLNet model)
- **yoso** -- [YosoForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/yoso#transformers.YosoForQuestionAnswering) (YOSO model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForQuestionAnswering

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForQuestionAnswering.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForQuestionAnswering.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForQuestionAnswering.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForTableQuestionAnswering">AutoModelForTableQuestionAnswering</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForTableQuestionAnswering</name><anchor>transformers.AutoModelForTableQuestionAnswering</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L733</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a table question answering head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) configuration class: [TapasForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasForQuestionAnswering) (TAPAS model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a table question answering head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForTableQuestionAnswering

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("google/tapas-base-finetuned-wtq")
>>> model = AutoModelForTableQuestionAnswering.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a table question answering head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **tapas** -- [TapasForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasForQuestionAnswering) (TAPAS model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForTableQuestionAnswering

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForTableQuestionAnswering.from_pretrained("google/tapas-base-finetuned-wtq")

>>> # Update configuration during loading
>>> model = AutoModelForTableQuestionAnswering.from_pretrained("google/tapas-base-finetuned-wtq", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/tapas_tf_model_config.json")
>>> model = AutoModelForTableQuestionAnswering.from_pretrained(
...     "./tf_model/tapas_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForImageClassification">AutoModelForImageClassification</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForImageClassification</name><anchor>transformers.AutoModelForImageClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L767</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a image classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [BeitConfig](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitConfig) configuration class: [BeitForImageClassification](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitForImageClassification) (BEiT model)
  - [ConvNextConfig](/docs/transformers/pr_15592/en/model_doc/convnext#transformers.ConvNextConfig) configuration class: [ConvNextForImageClassification](/docs/transformers/pr_15592/en/model_doc/convnext#transformers.ConvNextForImageClassification) (ConvNext model)
  - [DeiTConfig](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTConfig) configuration class: [DeiTForImageClassification](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTForImageClassification) or [DeiTForImageClassificationWithTeacher](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTForImageClassificationWithTeacher) (DeiT model)
  - [ImageGPTConfig](/docs/transformers/pr_15592/en/model_doc/imagegpt#transformers.ImageGPTConfig) configuration class: [ImageGPTForImageClassification](/docs/transformers/pr_15592/en/model_doc/imagegpt#transformers.ImageGPTForImageClassification) (ImageGPT model)
  - [PerceiverConfig](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverConfig) configuration class: [PerceiverForImageClassificationLearned](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverForImageClassificationLearned) or [PerceiverForImageClassificationFourier](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverForImageClassificationFourier) or [PerceiverForImageClassificationConvProcessing](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverForImageClassificationConvProcessing) (Perceiver model)
  - [PoolFormerConfig](/docs/transformers/pr_15592/en/model_doc/poolformer#transformers.PoolFormerConfig) configuration class: [PoolFormerForImageClassification](/docs/transformers/pr_15592/en/model_doc/poolformer#transformers.PoolFormerForImageClassification) (PoolFormer model)
  - [SegformerConfig](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerConfig) configuration class: [SegformerForImageClassification](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerForImageClassification) (SegFormer model)
  - [SwinConfig](/docs/transformers/pr_15592/en/model_doc/swin#transformers.SwinConfig) configuration class: [SwinForImageClassification](/docs/transformers/pr_15592/en/model_doc/swin#transformers.SwinForImageClassification) (Swin model)
  - [ViTConfig](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTConfig) configuration class: [ViTForImageClassification](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTForImageClassification) (ViT model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a image classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForImageClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForImageClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a image classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **beit** -- [BeitForImageClassification](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitForImageClassification) (BEiT model)
- **convnext** -- [ConvNextForImageClassification](/docs/transformers/pr_15592/en/model_doc/convnext#transformers.ConvNextForImageClassification) (ConvNext model)
- **deit** -- [DeiTForImageClassification](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTForImageClassification) or [DeiTForImageClassificationWithTeacher](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTForImageClassificationWithTeacher) (DeiT model)
- **imagegpt** -- [ImageGPTForImageClassification](/docs/transformers/pr_15592/en/model_doc/imagegpt#transformers.ImageGPTForImageClassification) (ImageGPT model)
- **perceiver** -- [PerceiverForImageClassificationLearned](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverForImageClassificationLearned) or [PerceiverForImageClassificationFourier](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverForImageClassificationFourier) or [PerceiverForImageClassificationConvProcessing](/docs/transformers/pr_15592/en/model_doc/perceiver#transformers.PerceiverForImageClassificationConvProcessing) (Perceiver model)
- **poolformer** -- [PoolFormerForImageClassification](/docs/transformers/pr_15592/en/model_doc/poolformer#transformers.PoolFormerForImageClassification) (PoolFormer model)
- **segformer** -- [SegformerForImageClassification](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerForImageClassification) (SegFormer model)
- **swin** -- [SwinForImageClassification](/docs/transformers/pr_15592/en/model_doc/swin#transformers.SwinForImageClassification) (Swin model)
- **vit** -- [ViTForImageClassification](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTForImageClassification) (ViT model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForImageClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForImageClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForImageClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForImageClassification.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForVision2Seq">AutoModelForVision2Seq</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForVision2Seq</name><anchor>transformers.AutoModelForVision2Seq</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L797</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a vision-to-text modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [VisionEncoderDecoderConfig](/docs/transformers/pr_15592/en/model_doc/vision-encoder-decoder#transformers.VisionEncoderDecoderConfig) configuration class: [VisionEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/vision-encoder-decoder#transformers.VisionEncoderDecoderModel) (Vision Encoder decoder model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a vision-to-text modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForVision2Seq

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForVision2Seq.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a vision-to-text modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **vision-encoder-decoder** -- [VisionEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/vision-encoder-decoder#transformers.VisionEncoderDecoderModel) (Vision Encoder decoder model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForVision2Seq

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForVision2Seq.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForVision2Seq.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForVision2Seq.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForAudioClassification">AutoModelForAudioClassification</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForAudioClassification</name><anchor>transformers.AutoModelForAudioClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L804</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a audio classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [HubertConfig](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertConfig) configuration class: [HubertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertForSequenceClassification) (Hubert model)
  - [SEWConfig](/docs/transformers/pr_15592/en/model_doc/sew#transformers.SEWConfig) configuration class: [SEWForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/sew#transformers.SEWForSequenceClassification) (SEW model)
  - [SEWDConfig](/docs/transformers/pr_15592/en/model_doc/sew-d#transformers.SEWDConfig) configuration class: [SEWDForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/sew-d#transformers.SEWDForSequenceClassification) (SEW-D model)
  - [UniSpeechConfig](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechConfig) configuration class: [UniSpeechForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechForSequenceClassification) (UniSpeech model)
  - [UniSpeechSatConfig](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatConfig) configuration class: [UniSpeechSatForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatForSequenceClassification) (UniSpeechSat model)
  - [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) configuration class: [Wav2Vec2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2ForSequenceClassification) (Wav2Vec2 model)
  - [WavLMConfig](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMConfig) configuration class: [WavLMForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMForSequenceClassification) (WavLM model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a audio classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForAudioClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForAudioClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a audio classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **hubert** -- [HubertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertForSequenceClassification) (Hubert model)
- **sew** -- [SEWForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/sew#transformers.SEWForSequenceClassification) (SEW model)
- **sew-d** -- [SEWDForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/sew-d#transformers.SEWDForSequenceClassification) (SEW-D model)
- **unispeech** -- [UniSpeechForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechForSequenceClassification) (UniSpeech model)
- **unispeech-sat** -- [UniSpeechSatForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatForSequenceClassification) (UniSpeechSat model)
- **wav2vec2** -- [Wav2Vec2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2ForSequenceClassification) (Wav2Vec2 model)
- **wavlm** -- [WavLMForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMForSequenceClassification) (WavLM model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForAudioClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForAudioClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForAudioClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForAudioClassification.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForAudioFrameClassification">AutoModelForAudioFrameClassification</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForAudioFrameClassification</name><anchor>transformers.AutoModelForAudioFrameClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L827</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a audio frame (token) classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [UniSpeechSatConfig](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatConfig) configuration class: [UniSpeechSatForAudioFrameClassification](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatForAudioFrameClassification) (UniSpeechSat model)
  - [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) configuration class: [Wav2Vec2ForAudioFrameClassification](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2ForAudioFrameClassification) (Wav2Vec2 model)
  - [WavLMConfig](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMConfig) configuration class: [WavLMForAudioFrameClassification](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMForAudioFrameClassification) (WavLM model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a audio frame (token) classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForAudioFrameClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForAudioFrameClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a audio frame (token) classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **unispeech-sat** -- [UniSpeechSatForAudioFrameClassification](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatForAudioFrameClassification) (UniSpeechSat model)
- **wav2vec2** -- [Wav2Vec2ForAudioFrameClassification](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2ForAudioFrameClassification) (Wav2Vec2 model)
- **wavlm** -- [WavLMForAudioFrameClassification](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMForAudioFrameClassification) (WavLM model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForAudioFrameClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForAudioFrameClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForAudioFrameClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForAudioFrameClassification.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForCTC">AutoModelForCTC</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForCTC</name><anchor>transformers.AutoModelForCTC</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L811</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a connectionist temporal classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [HubertConfig](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertConfig) configuration class: [HubertForCTC](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertForCTC) (Hubert model)
  - [SEWConfig](/docs/transformers/pr_15592/en/model_doc/sew#transformers.SEWConfig) configuration class: [SEWForCTC](/docs/transformers/pr_15592/en/model_doc/sew#transformers.SEWForCTC) (SEW model)
  - [SEWDConfig](/docs/transformers/pr_15592/en/model_doc/sew-d#transformers.SEWDConfig) configuration class: [SEWDForCTC](/docs/transformers/pr_15592/en/model_doc/sew-d#transformers.SEWDForCTC) (SEW-D model)
  - [UniSpeechConfig](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechConfig) configuration class: [UniSpeechForCTC](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechForCTC) (UniSpeech model)
  - [UniSpeechSatConfig](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatConfig) configuration class: [UniSpeechSatForCTC](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatForCTC) (UniSpeechSat model)
  - [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) configuration class: [Wav2Vec2ForCTC](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2ForCTC) (Wav2Vec2 model)
  - [WavLMConfig](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMConfig) configuration class: [WavLMForCTC](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMForCTC) (WavLM model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a connectionist temporal classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForCTC

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForCTC.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a connectionist temporal classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **hubert** -- [HubertForCTC](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertForCTC) (Hubert model)
- **sew** -- [SEWForCTC](/docs/transformers/pr_15592/en/model_doc/sew#transformers.SEWForCTC) (SEW model)
- **sew-d** -- [SEWDForCTC](/docs/transformers/pr_15592/en/model_doc/sew-d#transformers.SEWDForCTC) (SEW-D model)
- **unispeech** -- [UniSpeechForCTC](/docs/transformers/pr_15592/en/model_doc/unispeech#transformers.UniSpeechForCTC) (UniSpeech model)
- **unispeech-sat** -- [UniSpeechSatForCTC](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatForCTC) (UniSpeechSat model)
- **wav2vec2** -- [Wav2Vec2ForCTC](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2ForCTC) (Wav2Vec2 model)
- **wavlm** -- [WavLMForCTC](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMForCTC) (WavLM model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForCTC

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForCTC.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForCTC.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForCTC.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForSpeechSeq2Seq">AutoModelForSpeechSeq2Seq</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForSpeechSeq2Seq</name><anchor>transformers.AutoModelForSpeechSeq2Seq</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L818</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence-to-sequence speech-to-text modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [Speech2TextConfig](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextConfig) configuration class: [Speech2TextForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextForConditionalGeneration) (Speech2Text model)
  - [SpeechEncoderDecoderConfig](/docs/transformers/pr_15592/en/model_doc/speech-encoder-decoder#transformers.SpeechEncoderDecoderConfig) configuration class: [SpeechEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/speech-encoder-decoder#transformers.SpeechEncoderDecoderModel) (Speech Encoder decoder model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a sequence-to-sequence speech-to-text modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForSpeechSeq2Seq

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForSpeechSeq2Seq.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a sequence-to-sequence speech-to-text modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **speech-encoder-decoder** -- [SpeechEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/speech-encoder-decoder#transformers.SpeechEncoderDecoderModel) (Speech Encoder decoder model)
- **speech_to_text** -- [Speech2TextForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextForConditionalGeneration) (Speech2Text model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForSpeechSeq2Seq

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForSpeechSeq2Seq.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForSpeechSeq2Seq.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForSpeechSeq2Seq.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForAudioXVector">AutoModelForAudioXVector</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForAudioXVector</name><anchor>transformers.AutoModelForAudioXVector</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L836</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a audio retrieval via x-vector head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [UniSpeechSatConfig](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatConfig) configuration class: [UniSpeechSatForXVector](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatForXVector) (UniSpeechSat model)
  - [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) configuration class: [Wav2Vec2ForXVector](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2ForXVector) (Wav2Vec2 model)
  - [WavLMConfig](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMConfig) configuration class: [WavLMForXVector](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMForXVector) (WavLM model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a audio retrieval via x-vector head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForAudioXVector

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForAudioXVector.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a audio retrieval via x-vector head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **unispeech-sat** -- [UniSpeechSatForXVector](/docs/transformers/pr_15592/en/model_doc/unispeech-sat#transformers.UniSpeechSatForXVector) (UniSpeechSat model)
- **wav2vec2** -- [Wav2Vec2ForXVector](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2ForXVector) (Wav2Vec2 model)
- **wavlm** -- [WavLMForXVector](/docs/transformers/pr_15592/en/model_doc/wavlm#transformers.WavLMForXVector) (WavLM model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForAudioXVector

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForAudioXVector.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForAudioXVector.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForAudioXVector.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForMaskedImageModeling">AutoModelForMaskedImageModeling</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForMaskedImageModeling</name><anchor>transformers.AutoModelForMaskedImageModeling</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L843</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a masked image modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [DeiTConfig](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTConfig) configuration class: [DeiTForMaskedImageModeling](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTForMaskedImageModeling) (DeiT model)
  - [SwinConfig](/docs/transformers/pr_15592/en/model_doc/swin#transformers.SwinConfig) configuration class: [SwinForMaskedImageModeling](/docs/transformers/pr_15592/en/model_doc/swin#transformers.SwinForMaskedImageModeling) (Swin model)
  - [ViTConfig](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTConfig) configuration class: [ViTForMaskedImageModeling](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTForMaskedImageModeling) (ViT model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a masked image modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForMaskedImageModeling

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForMaskedImageModeling.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a masked image modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **deit** -- [DeiTForMaskedImageModeling](/docs/transformers/pr_15592/en/model_doc/deit#transformers.DeiTForMaskedImageModeling) (DeiT model)
- **swin** -- [SwinForMaskedImageModeling](/docs/transformers/pr_15592/en/model_doc/swin#transformers.SwinForMaskedImageModeling) (Swin model)
- **vit** -- [ViTForMaskedImageModeling](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTForMaskedImageModeling) (ViT model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForMaskedImageModeling

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForMaskedImageModeling.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForMaskedImageModeling.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForMaskedImageModeling.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForObjectDetection">AutoModelForObjectDetection</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForObjectDetection</name><anchor>transformers.AutoModelForObjectDetection</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L790</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a object detection head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [DetrConfig](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrConfig) configuration class: [DetrForObjectDetection](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrForObjectDetection) (DETR model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a object detection head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForObjectDetection

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForObjectDetection.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a object detection head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **detr** -- [DetrForObjectDetection](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrForObjectDetection) (DETR model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForObjectDetection

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForObjectDetection.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForObjectDetection.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForObjectDetection.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForImageSegmentation">AutoModelForImageSegmentation</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForImageSegmentation</name><anchor>transformers.AutoModelForImageSegmentation</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L774</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a image segmentation head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [DetrConfig](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrConfig) configuration class: [DetrForSegmentation](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrForSegmentation) (DETR model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a image segmentation head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForImageSegmentation

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForImageSegmentation.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a image segmentation head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **detr** -- [DetrForSegmentation](/docs/transformers/pr_15592/en/model_doc/detr#transformers.DetrForSegmentation) (DETR model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForImageSegmentation

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForImageSegmentation.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForImageSegmentation.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForImageSegmentation.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.AutoModelForSemanticSegmentation">AutoModelForSemanticSegmentation</h2>

<div class="docstring">

<docstring><name>class transformers.AutoModelForSemanticSegmentation</name><anchor>transformers.AutoModelForSemanticSegmentation</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_auto.py#L781</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a semantic segmentation head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [BeitConfig](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitConfig) configuration class: [BeitForSemanticSegmentation](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitForSemanticSegmentation) (BEiT model)
  - [SegformerConfig](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerConfig) configuration class: [SegformerForSemanticSegmentation](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerForSemanticSegmentation) (SegFormer model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a semantic segmentation head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForSemanticSegmentation

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = AutoModelForSemanticSegmentation.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *tensorflow index checkpoint file* (e.g, `./tf_model/model.ckpt.index`). In
    this case, `from_tf` should be set to `True` and a configuration object should be provided as
    `config` argument. This loading path is slower than converting the TensorFlow checkpoint in a
    PyTorch model using the provided conversion scripts and loading the PyTorch model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **state_dict** (*Dict[str, torch.Tensor]*, *optional*) --
  A state dictionary to use instead of a state dictionary loaded from saved weights file.

  This option can be used if you want to create a model from a pretrained configuration but load your own
  weights. In this case though, you should check if using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and
  [from_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) is not a simpler option.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_tf** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a TensorFlow checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a semantic segmentation head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **beit** -- [BeitForSemanticSegmentation](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitForSemanticSegmentation) (BEiT model)
- **segformer** -- [SegformerForSemanticSegmentation](/docs/transformers/pr_15592/en/model_doc/segformer#transformers.SegformerForSemanticSegmentation) (SegFormer model)

The model is set in evaluation mode by default using `model.eval()` (so for instance, dropout modules are
deactivated). To train the model, you should first set it back in training mode with `model.train()`



Examples:

```python
>>> from transformers import AutoConfig, AutoModelForSemanticSegmentation

>>> # Download model and configuration from huggingface.co and cache.
>>> model = AutoModelForSemanticSegmentation.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = AutoModelForSemanticSegmentation.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a TF checkpoint file instead of a PyTorch model (slower)
>>> config = AutoConfig.from_pretrained("./tf_model/bert_tf_model_config.json")
>>> model = AutoModelForSemanticSegmentation.from_pretrained(
...     "./tf_model/bert_tf_checkpoint.ckpt.index", from_tf=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModel">TFAutoModel</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModel</name><anchor>transformers.TFAutoModel</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L371</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the base model classes of the library when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [TFAlbertModel](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertModel) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [TFBartModel](/docs/transformers/pr_15592/en/model_doc/bart#transformers.TFBartModel) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [TFBertModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertModel) (BERT model)
  - [BlenderbotConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotConfig) configuration class: [TFBlenderbotModel](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.TFBlenderbotModel) (Blenderbot model)
  - [BlenderbotSmallConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallConfig) configuration class: [TFBlenderbotSmallModel](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.TFBlenderbotSmallModel) (BlenderbotSmall model)
  - [CLIPConfig](/docs/transformers/pr_15592/en/model_doc/clip#transformers.CLIPConfig) configuration class: [TFCLIPModel](/docs/transformers/pr_15592/en/model_doc/clip#transformers.TFCLIPModel) (CLIP model)
  - [CTRLConfig](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLConfig) configuration class: [TFCTRLModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.TFCTRLModel) (CTRL model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [TFCamembertModel](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertModel) (CamemBERT model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [TFConvBertModel](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertModel) (ConvBERT model)
  - [DPRConfig](/docs/transformers/pr_15592/en/model_doc/dpr#transformers.DPRConfig) configuration class: [TFDPRQuestionEncoder](/docs/transformers/pr_15592/en/model_doc/dpr#transformers.TFDPRQuestionEncoder) (DPR model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [TFDebertaModel](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.TFDebertaModel) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [TFDebertaV2Model](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.TFDebertaV2Model) (DeBERTa-v2 model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [TFDistilBertModel](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertModel) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [TFElectraModel](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraModel) (ELECTRA model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [TFFlaubertModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertModel) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [TFFunnelModel](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelModel) or [TFFunnelBaseModel](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelBaseModel) (Funnel Transformer model)
  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [TFGPT2Model](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.TFGPT2Model) (OpenAI GPT-2 model)
  - [HubertConfig](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.HubertConfig) configuration class: [TFHubertModel](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.TFHubertModel) (Hubert model)
  - [LEDConfig](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDConfig) configuration class: [TFLEDModel](/docs/transformers/pr_15592/en/model_doc/led#transformers.TFLEDModel) (LED model)
  - [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) configuration class: [TFLayoutLMModel](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.TFLayoutLMModel) (LayoutLM model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [TFLongformerModel](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerModel) (Longformer model)
  - [LxmertConfig](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertConfig) configuration class: [TFLxmertModel](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.TFLxmertModel) (LXMERT model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [TFMBartModel](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.TFMBartModel) (mBART model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [TFMPNetModel](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetModel) (MPNet model)
  - [MT5Config](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Config) configuration class: [TFMT5Model](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.TFMT5Model) (mT5 model)
  - [MarianConfig](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianConfig) configuration class: [TFMarianModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.TFMarianModel) (Marian model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [TFMobileBertModel](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertModel) (MobileBERT model)
  - [OpenAIGPTConfig](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTConfig) configuration class: [TFOpenAIGPTModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.TFOpenAIGPTModel) (OpenAI GPT model)
  - [PegasusConfig](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusConfig) configuration class: [TFPegasusModel](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.TFPegasusModel) (Pegasus model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [TFRemBertModel](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertModel) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [TFRoFormerModel](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerModel) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [TFRobertaModel](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaModel) (RoBERTa model)
  - [Speech2TextConfig](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextConfig) configuration class: [TFSpeech2TextModel](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.TFSpeech2TextModel) (Speech2Text model)
  - [T5Config](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Config) configuration class: [TFT5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.TFT5Model) (T5 model)
  - [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) configuration class: [TFTapasModel](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TFTapasModel) (TAPAS model)
  - [TransfoXLConfig](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLConfig) configuration class: [TFTransfoXLModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TFTransfoXLModel) (Transformer-XL model)
  - [ViTConfig](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTConfig) configuration class: [TFViTModel](/docs/transformers/pr_15592/en/model_doc/vit#transformers.TFViTModel) (ViT model)
  - [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) configuration class: [TFWav2Vec2Model](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.TFWav2Vec2Model) (Wav2Vec2 model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [TFXLMModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMModel) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [TFXLMRobertaModel](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaModel) (XLM-RoBERTa model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [TFXLNetModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetModel) (XLNet model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the base model classes of the library from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModel

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModel.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the base model classes of the library from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [TFAlbertModel](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertModel) (ALBERT model)
- **bart** -- [TFBartModel](/docs/transformers/pr_15592/en/model_doc/bart#transformers.TFBartModel) (BART model)
- **bert** -- [TFBertModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertModel) (BERT model)
- **blenderbot** -- [TFBlenderbotModel](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.TFBlenderbotModel) (Blenderbot model)
- **blenderbot-small** -- [TFBlenderbotSmallModel](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.TFBlenderbotSmallModel) (BlenderbotSmall model)
- **camembert** -- [TFCamembertModel](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertModel) (CamemBERT model)
- **clip** -- [TFCLIPModel](/docs/transformers/pr_15592/en/model_doc/clip#transformers.TFCLIPModel) (CLIP model)
- **convbert** -- [TFConvBertModel](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertModel) (ConvBERT model)
- **ctrl** -- [TFCTRLModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.TFCTRLModel) (CTRL model)
- **deberta** -- [TFDebertaModel](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.TFDebertaModel) (DeBERTa model)
- **deberta-v2** -- [TFDebertaV2Model](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.TFDebertaV2Model) (DeBERTa-v2 model)
- **distilbert** -- [TFDistilBertModel](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertModel) (DistilBERT model)
- **dpr** -- [TFDPRQuestionEncoder](/docs/transformers/pr_15592/en/model_doc/dpr#transformers.TFDPRQuestionEncoder) (DPR model)
- **electra** -- [TFElectraModel](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraModel) (ELECTRA model)
- **flaubert** -- [TFFlaubertModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertModel) (FlauBERT model)
- **funnel** -- [TFFunnelModel](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelModel) or [TFFunnelBaseModel](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelBaseModel) (Funnel Transformer model)
- **gpt2** -- [TFGPT2Model](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.TFGPT2Model) (OpenAI GPT-2 model)
- **hubert** -- [TFHubertModel](/docs/transformers/pr_15592/en/model_doc/hubert#transformers.TFHubertModel) (Hubert model)
- **layoutlm** -- [TFLayoutLMModel](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.TFLayoutLMModel) (LayoutLM model)
- **led** -- [TFLEDModel](/docs/transformers/pr_15592/en/model_doc/led#transformers.TFLEDModel) (LED model)
- **longformer** -- [TFLongformerModel](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerModel) (Longformer model)
- **lxmert** -- [TFLxmertModel](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.TFLxmertModel) (LXMERT model)
- **marian** -- [TFMarianModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.TFMarianModel) (Marian model)
- **mbart** -- [TFMBartModel](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.TFMBartModel) (mBART model)
- **mobilebert** -- [TFMobileBertModel](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertModel) (MobileBERT model)
- **mpnet** -- [TFMPNetModel](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetModel) (MPNet model)
- **mt5** -- [TFMT5Model](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.TFMT5Model) (mT5 model)
- **openai-gpt** -- [TFOpenAIGPTModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.TFOpenAIGPTModel) (OpenAI GPT model)
- **pegasus** -- [TFPegasusModel](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.TFPegasusModel) (Pegasus model)
- **rembert** -- [TFRemBertModel](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertModel) (RemBERT model)
- **roberta** -- [TFRobertaModel](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaModel) (RoBERTa model)
- **roformer** -- [TFRoFormerModel](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerModel) (RoFormer model)
- **speech_to_text** -- [TFSpeech2TextModel](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.TFSpeech2TextModel) (Speech2Text model)
- **t5** -- [TFT5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.TFT5Model) (T5 model)
- **tapas** -- [TFTapasModel](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TFTapasModel) (TAPAS model)
- **transfo-xl** -- [TFTransfoXLModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TFTransfoXLModel) (Transformer-XL model)
- **vit** -- [TFViTModel](/docs/transformers/pr_15592/en/model_doc/vit#transformers.TFViTModel) (ViT model)
- **wav2vec2** -- [TFWav2Vec2Model](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.TFWav2Vec2Model) (Wav2Vec2 model)
- **xlm** -- [TFXLMModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMModel) (XLM model)
- **xlm-roberta** -- [TFXLMRobertaModel](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaModel) (XLM-RoBERTa model)
- **xlnet** -- [TFXLNetModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetModel) (XLNet model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModel

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModel.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModel.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModel.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForPreTraining">TFAutoModelForPreTraining</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForPreTraining</name><anchor>transformers.TFAutoModelForPreTraining</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L378</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a pretraining head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [TFAlbertForPreTraining](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForPreTraining) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [TFBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.TFBartForConditionalGeneration) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [TFBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForPreTraining) (BERT model)
  - [CTRLConfig](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLConfig) configuration class: [TFCTRLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.TFCTRLLMHeadModel) (CTRL model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [TFCamembertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForMaskedLM) (CamemBERT model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [TFDistilBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForMaskedLM) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [TFElectraForPreTraining](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForPreTraining) (ELECTRA model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [TFFlaubertWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertWithLMHeadModel) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [TFFunnelForPreTraining](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForPreTraining) (Funnel Transformer model)
  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [TFGPT2LMHeadModel](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.TFGPT2LMHeadModel) (OpenAI GPT-2 model)
  - [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) configuration class: [TFLayoutLMForMaskedLM](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.TFLayoutLMForMaskedLM) (LayoutLM model)
  - [LxmertConfig](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.LxmertConfig) configuration class: [TFLxmertForPreTraining](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.TFLxmertForPreTraining) (LXMERT model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [TFMPNetForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForMaskedLM) (MPNet model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [TFMobileBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForPreTraining) (MobileBERT model)
  - [OpenAIGPTConfig](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTConfig) configuration class: [TFOpenAIGPTLMHeadModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.TFOpenAIGPTLMHeadModel) (OpenAI GPT model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [TFRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForMaskedLM) (RoBERTa model)
  - [T5Config](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Config) configuration class: [TFT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.TFT5ForConditionalGeneration) (T5 model)
  - [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) configuration class: [TFTapasForMaskedLM](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TFTapasForMaskedLM) (TAPAS model)
  - [TransfoXLConfig](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLConfig) configuration class: [TFTransfoXLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TFTransfoXLLMHeadModel) (Transformer-XL model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [TFXLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMWithLMHeadModel) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [TFXLMRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForMaskedLM) (XLM-RoBERTa model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [TFXLNetLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetLMHeadModel) (XLNet model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a pretraining head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForPreTraining

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForPreTraining.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a pretraining head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [TFAlbertForPreTraining](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForPreTraining) (ALBERT model)
- **bart** -- [TFBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.TFBartForConditionalGeneration) (BART model)
- **bert** -- [TFBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForPreTraining) (BERT model)
- **camembert** -- [TFCamembertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForMaskedLM) (CamemBERT model)
- **ctrl** -- [TFCTRLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.TFCTRLLMHeadModel) (CTRL model)
- **distilbert** -- [TFDistilBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForMaskedLM) (DistilBERT model)
- **electra** -- [TFElectraForPreTraining](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForPreTraining) (ELECTRA model)
- **flaubert** -- [TFFlaubertWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertWithLMHeadModel) (FlauBERT model)
- **funnel** -- [TFFunnelForPreTraining](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForPreTraining) (Funnel Transformer model)
- **gpt2** -- [TFGPT2LMHeadModel](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.TFGPT2LMHeadModel) (OpenAI GPT-2 model)
- **layoutlm** -- [TFLayoutLMForMaskedLM](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.TFLayoutLMForMaskedLM) (LayoutLM model)
- **lxmert** -- [TFLxmertForPreTraining](/docs/transformers/pr_15592/en/model_doc/lxmert#transformers.TFLxmertForPreTraining) (LXMERT model)
- **mobilebert** -- [TFMobileBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForPreTraining) (MobileBERT model)
- **mpnet** -- [TFMPNetForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForMaskedLM) (MPNet model)
- **openai-gpt** -- [TFOpenAIGPTLMHeadModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.TFOpenAIGPTLMHeadModel) (OpenAI GPT model)
- **roberta** -- [TFRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForMaskedLM) (RoBERTa model)
- **t5** -- [TFT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.TFT5ForConditionalGeneration) (T5 model)
- **tapas** -- [TFTapasForMaskedLM](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TFTapasForMaskedLM) (TAPAS model)
- **transfo-xl** -- [TFTransfoXLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TFTransfoXLLMHeadModel) (Transformer-XL model)
- **xlm** -- [TFXLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMWithLMHeadModel) (XLM model)
- **xlm-roberta** -- [TFXLMRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForMaskedLM) (XLM-RoBERTa model)
- **xlnet** -- [TFXLNetLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetLMHeadModel) (XLNet model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForPreTraining

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForPreTraining.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModelForPreTraining.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForPreTraining.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForCausalLM">TFAutoModelForCausalLM</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForCausalLM</name><anchor>transformers.TFAutoModelForCausalLM</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L393</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a causal language modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [TFBertLMHeadModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertLMHeadModel) (BERT model)
  - [CTRLConfig](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLConfig) configuration class: [TFCTRLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.TFCTRLLMHeadModel) (CTRL model)
  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [TFGPT2LMHeadModel](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.TFGPT2LMHeadModel) (OpenAI GPT-2 model)
  - [OpenAIGPTConfig](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTConfig) configuration class: [TFOpenAIGPTLMHeadModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.TFOpenAIGPTLMHeadModel) (OpenAI GPT model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [TFRemBertForCausalLM](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForCausalLM) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [TFRoFormerForCausalLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForCausalLM) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [TFRobertaForCausalLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForCausalLM) (RoBERTa model)
  - [TransfoXLConfig](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLConfig) configuration class: [TFTransfoXLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TFTransfoXLLMHeadModel) (Transformer-XL model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [TFXLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMWithLMHeadModel) (XLM model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [TFXLNetLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetLMHeadModel) (XLNet model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a causal language modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForCausalLM

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForCausalLM.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a causal language modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **bert** -- [TFBertLMHeadModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertLMHeadModel) (BERT model)
- **ctrl** -- [TFCTRLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.TFCTRLLMHeadModel) (CTRL model)
- **gpt2** -- [TFGPT2LMHeadModel](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.TFGPT2LMHeadModel) (OpenAI GPT-2 model)
- **openai-gpt** -- [TFOpenAIGPTLMHeadModel](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.TFOpenAIGPTLMHeadModel) (OpenAI GPT model)
- **rembert** -- [TFRemBertForCausalLM](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForCausalLM) (RemBERT model)
- **roberta** -- [TFRobertaForCausalLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForCausalLM) (RoBERTa model)
- **roformer** -- [TFRoFormerForCausalLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForCausalLM) (RoFormer model)
- **transfo-xl** -- [TFTransfoXLLMHeadModel](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TFTransfoXLLMHeadModel) (Transformer-XL model)
- **xlm** -- [TFXLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMWithLMHeadModel) (XLM model)
- **xlnet** -- [TFXLNetLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetLMHeadModel) (XLNet model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForCausalLM

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForCausalLM.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModelForCausalLM.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForCausalLM.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForImageClassification">TFAutoModelForImageClassification</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForImageClassification</name><anchor>transformers.TFAutoModelForImageClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L400</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a image classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [ViTConfig](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTConfig) configuration class: [TFViTForImageClassification](/docs/transformers/pr_15592/en/model_doc/vit#transformers.TFViTForImageClassification) (ViT model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a image classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForImageClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForImageClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a image classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **vit** -- [TFViTForImageClassification](/docs/transformers/pr_15592/en/model_doc/vit#transformers.TFViTForImageClassification) (ViT model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForImageClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForImageClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModelForImageClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForImageClassification.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForMaskedLM">TFAutoModelForMaskedLM</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForMaskedLM</name><anchor>transformers.TFAutoModelForMaskedLM</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L414</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a masked language modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [TFAlbertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForMaskedLM) (ALBERT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [TFBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForMaskedLM) (BERT model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [TFCamembertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForMaskedLM) (CamemBERT model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [TFConvBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertForMaskedLM) (ConvBERT model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [TFDebertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.TFDebertaForMaskedLM) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [TFDebertaV2ForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.TFDebertaV2ForMaskedLM) (DeBERTa-v2 model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [TFDistilBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForMaskedLM) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [TFElectraForMaskedLM](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForMaskedLM) (ELECTRA model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [TFFlaubertWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertWithLMHeadModel) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [TFFunnelForMaskedLM](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForMaskedLM) (Funnel Transformer model)
  - [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) configuration class: [TFLayoutLMForMaskedLM](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.TFLayoutLMForMaskedLM) (LayoutLM model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [TFLongformerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerForMaskedLM) (Longformer model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [TFMPNetForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForMaskedLM) (MPNet model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [TFMobileBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForMaskedLM) (MobileBERT model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [TFRemBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForMaskedLM) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [TFRoFormerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForMaskedLM) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [TFRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForMaskedLM) (RoBERTa model)
  - [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) configuration class: [TFTapasForMaskedLM](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TFTapasForMaskedLM) (TAPAS model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [TFXLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMWithLMHeadModel) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [TFXLMRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForMaskedLM) (XLM-RoBERTa model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a masked language modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForMaskedLM

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForMaskedLM.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a masked language modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [TFAlbertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForMaskedLM) (ALBERT model)
- **bert** -- [TFBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForMaskedLM) (BERT model)
- **camembert** -- [TFCamembertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForMaskedLM) (CamemBERT model)
- **convbert** -- [TFConvBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertForMaskedLM) (ConvBERT model)
- **deberta** -- [TFDebertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.TFDebertaForMaskedLM) (DeBERTa model)
- **deberta-v2** -- [TFDebertaV2ForMaskedLM](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.TFDebertaV2ForMaskedLM) (DeBERTa-v2 model)
- **distilbert** -- [TFDistilBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForMaskedLM) (DistilBERT model)
- **electra** -- [TFElectraForMaskedLM](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForMaskedLM) (ELECTRA model)
- **flaubert** -- [TFFlaubertWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertWithLMHeadModel) (FlauBERT model)
- **funnel** -- [TFFunnelForMaskedLM](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForMaskedLM) (Funnel Transformer model)
- **layoutlm** -- [TFLayoutLMForMaskedLM](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.TFLayoutLMForMaskedLM) (LayoutLM model)
- **longformer** -- [TFLongformerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerForMaskedLM) (Longformer model)
- **mobilebert** -- [TFMobileBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForMaskedLM) (MobileBERT model)
- **mpnet** -- [TFMPNetForMaskedLM](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForMaskedLM) (MPNet model)
- **rembert** -- [TFRemBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForMaskedLM) (RemBERT model)
- **roberta** -- [TFRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForMaskedLM) (RoBERTa model)
- **roformer** -- [TFRoFormerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForMaskedLM) (RoFormer model)
- **tapas** -- [TFTapasForMaskedLM](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TFTapasForMaskedLM) (TAPAS model)
- **xlm** -- [TFXLMWithLMHeadModel](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMWithLMHeadModel) (XLM model)
- **xlm-roberta** -- [TFXLMRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForMaskedLM) (XLM-RoBERTa model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForMaskedLM

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForMaskedLM.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModelForMaskedLM.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForMaskedLM.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForSeq2SeqLM">TFAutoModelForSeq2SeqLM</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForSeq2SeqLM</name><anchor>transformers.TFAutoModelForSeq2SeqLM</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L421</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence-to-sequence language modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [TFBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.TFBartForConditionalGeneration) (BART model)
  - [BlenderbotConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotConfig) configuration class: [TFBlenderbotForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.TFBlenderbotForConditionalGeneration) (Blenderbot model)
  - [BlenderbotSmallConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallConfig) configuration class: [TFBlenderbotSmallForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.TFBlenderbotSmallForConditionalGeneration) (BlenderbotSmall model)
  - [EncoderDecoderConfig](/docs/transformers/pr_15592/en/model_doc/encoder-decoder#transformers.EncoderDecoderConfig) configuration class: [TFEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/encoder-decoder#transformers.TFEncoderDecoderModel) (Encoder decoder model)
  - [LEDConfig](/docs/transformers/pr_15592/en/model_doc/led#transformers.LEDConfig) configuration class: [TFLEDForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/led#transformers.TFLEDForConditionalGeneration) (LED model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [TFMBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.TFMBartForConditionalGeneration) (mBART model)
  - [MT5Config](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Config) configuration class: [TFMT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.TFMT5ForConditionalGeneration) (mT5 model)
  - [MarianConfig](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianConfig) configuration class: [TFMarianMTModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.TFMarianMTModel) (Marian model)
  - [PegasusConfig](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusConfig) configuration class: [TFPegasusForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.TFPegasusForConditionalGeneration) (Pegasus model)
  - [T5Config](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Config) configuration class: [TFT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.TFT5ForConditionalGeneration) (T5 model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a sequence-to-sequence language modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForSeq2SeqLM

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("t5-base")
>>> model = TFAutoModelForSeq2SeqLM.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a sequence-to-sequence language modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **bart** -- [TFBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.TFBartForConditionalGeneration) (BART model)
- **blenderbot** -- [TFBlenderbotForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.TFBlenderbotForConditionalGeneration) (Blenderbot model)
- **blenderbot-small** -- [TFBlenderbotSmallForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.TFBlenderbotSmallForConditionalGeneration) (BlenderbotSmall model)
- **encoder-decoder** -- [TFEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/encoder-decoder#transformers.TFEncoderDecoderModel) (Encoder decoder model)
- **led** -- [TFLEDForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/led#transformers.TFLEDForConditionalGeneration) (LED model)
- **marian** -- [TFMarianMTModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.TFMarianMTModel) (Marian model)
- **mbart** -- [TFMBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.TFMBartForConditionalGeneration) (mBART model)
- **mt5** -- [TFMT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.TFMT5ForConditionalGeneration) (mT5 model)
- **pegasus** -- [TFPegasusForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.TFPegasusForConditionalGeneration) (Pegasus model)
- **t5** -- [TFT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.TFT5ForConditionalGeneration) (T5 model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForSeq2SeqLM

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForSeq2SeqLM.from_pretrained("t5-base")

>>> # Update configuration during loading
>>> model = TFAutoModelForSeq2SeqLM.from_pretrained("t5-base", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/t5_pt_model_config.json")
>>> model = TFAutoModelForSeq2SeqLM.from_pretrained(
...     "./pt_model/t5_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForSequenceClassification">TFAutoModelForSequenceClassification</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForSequenceClassification</name><anchor>transformers.TFAutoModelForSequenceClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L430</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [TFAlbertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForSequenceClassification) (ALBERT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [TFBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForSequenceClassification) (BERT model)
  - [CTRLConfig](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.CTRLConfig) configuration class: [TFCTRLForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.TFCTRLForSequenceClassification) (CTRL model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [TFCamembertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForSequenceClassification) (CamemBERT model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [TFConvBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertForSequenceClassification) (ConvBERT model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [TFDebertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.TFDebertaForSequenceClassification) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [TFDebertaV2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.TFDebertaV2ForSequenceClassification) (DeBERTa-v2 model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [TFDistilBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForSequenceClassification) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [TFElectraForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForSequenceClassification) (ELECTRA model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [TFFlaubertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertForSequenceClassification) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [TFFunnelForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForSequenceClassification) (Funnel Transformer model)
  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [TFGPT2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.TFGPT2ForSequenceClassification) (OpenAI GPT-2 model)
  - [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) configuration class: [TFLayoutLMForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.TFLayoutLMForSequenceClassification) (LayoutLM model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [TFLongformerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerForSequenceClassification) (Longformer model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [TFMPNetForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForSequenceClassification) (MPNet model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [TFMobileBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForSequenceClassification) (MobileBERT model)
  - [OpenAIGPTConfig](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.OpenAIGPTConfig) configuration class: [TFOpenAIGPTForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.TFOpenAIGPTForSequenceClassification) (OpenAI GPT model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [TFRemBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForSequenceClassification) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [TFRoFormerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForSequenceClassification) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [TFRobertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForSequenceClassification) (RoBERTa model)
  - [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) configuration class: [TFTapasForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TFTapasForSequenceClassification) (TAPAS model)
  - [TransfoXLConfig](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TransfoXLConfig) configuration class: [TFTransfoXLForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TFTransfoXLForSequenceClassification) (Transformer-XL model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [TFXLMForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMForSequenceClassification) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [TFXLMRobertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForSequenceClassification) (XLM-RoBERTa model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [TFXLNetForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetForSequenceClassification) (XLNet model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a sequence classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForSequenceClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForSequenceClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a sequence classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [TFAlbertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForSequenceClassification) (ALBERT model)
- **bert** -- [TFBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForSequenceClassification) (BERT model)
- **camembert** -- [TFCamembertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForSequenceClassification) (CamemBERT model)
- **convbert** -- [TFConvBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertForSequenceClassification) (ConvBERT model)
- **ctrl** -- [TFCTRLForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/ctrl#transformers.TFCTRLForSequenceClassification) (CTRL model)
- **deberta** -- [TFDebertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.TFDebertaForSequenceClassification) (DeBERTa model)
- **deberta-v2** -- [TFDebertaV2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.TFDebertaV2ForSequenceClassification) (DeBERTa-v2 model)
- **distilbert** -- [TFDistilBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForSequenceClassification) (DistilBERT model)
- **electra** -- [TFElectraForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForSequenceClassification) (ELECTRA model)
- **flaubert** -- [TFFlaubertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertForSequenceClassification) (FlauBERT model)
- **funnel** -- [TFFunnelForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForSequenceClassification) (Funnel Transformer model)
- **gpt2** -- [TFGPT2ForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.TFGPT2ForSequenceClassification) (OpenAI GPT-2 model)
- **layoutlm** -- [TFLayoutLMForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.TFLayoutLMForSequenceClassification) (LayoutLM model)
- **longformer** -- [TFLongformerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerForSequenceClassification) (Longformer model)
- **mobilebert** -- [TFMobileBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForSequenceClassification) (MobileBERT model)
- **mpnet** -- [TFMPNetForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForSequenceClassification) (MPNet model)
- **openai-gpt** -- [TFOpenAIGPTForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/openai-gpt#transformers.TFOpenAIGPTForSequenceClassification) (OpenAI GPT model)
- **rembert** -- [TFRemBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForSequenceClassification) (RemBERT model)
- **roberta** -- [TFRobertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForSequenceClassification) (RoBERTa model)
- **roformer** -- [TFRoFormerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForSequenceClassification) (RoFormer model)
- **tapas** -- [TFTapasForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TFTapasForSequenceClassification) (TAPAS model)
- **transfo-xl** -- [TFTransfoXLForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/transfo-xl#transformers.TFTransfoXLForSequenceClassification) (Transformer-XL model)
- **xlm** -- [TFXLMForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMForSequenceClassification) (XLM model)
- **xlm-roberta** -- [TFXLMRobertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForSequenceClassification) (XLM-RoBERTa model)
- **xlnet** -- [TFXLNetForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetForSequenceClassification) (XLNet model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForSequenceClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForSequenceClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModelForSequenceClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForSequenceClassification.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForMultipleChoice">TFAutoModelForMultipleChoice</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForMultipleChoice</name><anchor>transformers.TFAutoModelForMultipleChoice</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L466</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a multiple choice head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [TFAlbertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForMultipleChoice) (ALBERT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [TFBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForMultipleChoice) (BERT model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [TFCamembertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForMultipleChoice) (CamemBERT model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [TFConvBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertForMultipleChoice) (ConvBERT model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [TFDistilBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForMultipleChoice) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [TFElectraForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForMultipleChoice) (ELECTRA model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [TFFlaubertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertForMultipleChoice) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [TFFunnelForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForMultipleChoice) (Funnel Transformer model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [TFLongformerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerForMultipleChoice) (Longformer model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [TFMPNetForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForMultipleChoice) (MPNet model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [TFMobileBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForMultipleChoice) (MobileBERT model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [TFRemBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForMultipleChoice) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [TFRoFormerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForMultipleChoice) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [TFRobertaForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForMultipleChoice) (RoBERTa model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [TFXLMForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMForMultipleChoice) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [TFXLMRobertaForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForMultipleChoice) (XLM-RoBERTa model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [TFXLNetForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetForMultipleChoice) (XLNet model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a multiple choice head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForMultipleChoice

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForMultipleChoice.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a multiple choice head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [TFAlbertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForMultipleChoice) (ALBERT model)
- **bert** -- [TFBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForMultipleChoice) (BERT model)
- **camembert** -- [TFCamembertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForMultipleChoice) (CamemBERT model)
- **convbert** -- [TFConvBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertForMultipleChoice) (ConvBERT model)
- **distilbert** -- [TFDistilBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForMultipleChoice) (DistilBERT model)
- **electra** -- [TFElectraForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForMultipleChoice) (ELECTRA model)
- **flaubert** -- [TFFlaubertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertForMultipleChoice) (FlauBERT model)
- **funnel** -- [TFFunnelForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForMultipleChoice) (Funnel Transformer model)
- **longformer** -- [TFLongformerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerForMultipleChoice) (Longformer model)
- **mobilebert** -- [TFMobileBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForMultipleChoice) (MobileBERT model)
- **mpnet** -- [TFMPNetForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForMultipleChoice) (MPNet model)
- **rembert** -- [TFRemBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForMultipleChoice) (RemBERT model)
- **roberta** -- [TFRobertaForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForMultipleChoice) (RoBERTa model)
- **roformer** -- [TFRoFormerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForMultipleChoice) (RoFormer model)
- **xlm** -- [TFXLMForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMForMultipleChoice) (XLM model)
- **xlm-roberta** -- [TFXLMRobertaForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForMultipleChoice) (XLM-RoBERTa model)
- **xlnet** -- [TFXLNetForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetForMultipleChoice) (XLNet model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForMultipleChoice

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForMultipleChoice.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModelForMultipleChoice.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForMultipleChoice.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForTableQuestionAnswering">TFAutoModelForTableQuestionAnswering</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForTableQuestionAnswering</name><anchor>transformers.TFAutoModelForTableQuestionAnswering</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L446</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a table question answering head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [TapasConfig](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TapasConfig) configuration class: [TFTapasForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TFTapasForQuestionAnswering) (TAPAS model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a table question answering head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForTableQuestionAnswering

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("google/tapas-base-finetuned-wtq")
>>> model = TFAutoModelForTableQuestionAnswering.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a table question answering head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **tapas** -- [TFTapasForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/tapas#transformers.TFTapasForQuestionAnswering) (TAPAS model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForTableQuestionAnswering

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForTableQuestionAnswering.from_pretrained("google/tapas-base-finetuned-wtq")

>>> # Update configuration during loading
>>> model = TFAutoModelForTableQuestionAnswering.from_pretrained("google/tapas-base-finetuned-wtq", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/tapas_pt_model_config.json")
>>> model = TFAutoModelForTableQuestionAnswering.from_pretrained(
...     "./pt_model/tapas_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForTokenClassification">TFAutoModelForTokenClassification</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForTokenClassification</name><anchor>transformers.TFAutoModelForTokenClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L457</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a token classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [TFAlbertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForTokenClassification) (ALBERT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [TFBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForTokenClassification) (BERT model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [TFCamembertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForTokenClassification) (CamemBERT model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [TFConvBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertForTokenClassification) (ConvBERT model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [TFDebertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.TFDebertaForTokenClassification) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [TFDebertaV2ForTokenClassification](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.TFDebertaV2ForTokenClassification) (DeBERTa-v2 model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [TFDistilBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForTokenClassification) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [TFElectraForTokenClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForTokenClassification) (ELECTRA model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [TFFlaubertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertForTokenClassification) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [TFFunnelForTokenClassification](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForTokenClassification) (Funnel Transformer model)
  - [LayoutLMConfig](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.LayoutLMConfig) configuration class: [TFLayoutLMForTokenClassification](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.TFLayoutLMForTokenClassification) (LayoutLM model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [TFLongformerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerForTokenClassification) (Longformer model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [TFMPNetForTokenClassification](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForTokenClassification) (MPNet model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [TFMobileBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForTokenClassification) (MobileBERT model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [TFRemBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForTokenClassification) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [TFRoFormerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForTokenClassification) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [TFRobertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForTokenClassification) (RoBERTa model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [TFXLMForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMForTokenClassification) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [TFXLMRobertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForTokenClassification) (XLM-RoBERTa model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [TFXLNetForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetForTokenClassification) (XLNet model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a token classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForTokenClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForTokenClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a token classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [TFAlbertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForTokenClassification) (ALBERT model)
- **bert** -- [TFBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForTokenClassification) (BERT model)
- **camembert** -- [TFCamembertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForTokenClassification) (CamemBERT model)
- **convbert** -- [TFConvBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertForTokenClassification) (ConvBERT model)
- **deberta** -- [TFDebertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.TFDebertaForTokenClassification) (DeBERTa model)
- **deberta-v2** -- [TFDebertaV2ForTokenClassification](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.TFDebertaV2ForTokenClassification) (DeBERTa-v2 model)
- **distilbert** -- [TFDistilBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForTokenClassification) (DistilBERT model)
- **electra** -- [TFElectraForTokenClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForTokenClassification) (ELECTRA model)
- **flaubert** -- [TFFlaubertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertForTokenClassification) (FlauBERT model)
- **funnel** -- [TFFunnelForTokenClassification](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForTokenClassification) (Funnel Transformer model)
- **layoutlm** -- [TFLayoutLMForTokenClassification](/docs/transformers/pr_15592/en/model_doc/layoutlm#transformers.TFLayoutLMForTokenClassification) (LayoutLM model)
- **longformer** -- [TFLongformerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerForTokenClassification) (Longformer model)
- **mobilebert** -- [TFMobileBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForTokenClassification) (MobileBERT model)
- **mpnet** -- [TFMPNetForTokenClassification](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForTokenClassification) (MPNet model)
- **rembert** -- [TFRemBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForTokenClassification) (RemBERT model)
- **roberta** -- [TFRobertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForTokenClassification) (RoBERTa model)
- **roformer** -- [TFRoFormerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForTokenClassification) (RoFormer model)
- **xlm** -- [TFXLMForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMForTokenClassification) (XLM model)
- **xlm-roberta** -- [TFXLMRobertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForTokenClassification) (XLM-RoBERTa model)
- **xlnet** -- [TFXLNetForTokenClassification](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetForTokenClassification) (XLNet model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForTokenClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForTokenClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModelForTokenClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForTokenClassification.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForQuestionAnswering">TFAutoModelForQuestionAnswering</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForQuestionAnswering</name><anchor>transformers.TFAutoModelForQuestionAnswering</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L439</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a question answering head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [TFAlbertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForQuestionAnswering) (ALBERT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [TFBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForQuestionAnswering) (BERT model)
  - [CamembertConfig](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.CamembertConfig) configuration class: [TFCamembertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForQuestionAnswering) (CamemBERT model)
  - [ConvBertConfig](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.ConvBertConfig) configuration class: [TFConvBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertForQuestionAnswering) (ConvBERT model)
  - [DebertaConfig](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.DebertaConfig) configuration class: [TFDebertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.TFDebertaForQuestionAnswering) (DeBERTa model)
  - [DebertaV2Config](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.DebertaV2Config) configuration class: [TFDebertaV2ForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.TFDebertaV2ForQuestionAnswering) (DeBERTa-v2 model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [TFDistilBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForQuestionAnswering) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [TFElectraForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForQuestionAnswering) (ELECTRA model)
  - [FlaubertConfig](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.FlaubertConfig) configuration class: [TFFlaubertForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertForQuestionAnsweringSimple) (FlauBERT model)
  - [FunnelConfig](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.FunnelConfig) configuration class: [TFFunnelForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForQuestionAnswering) (Funnel Transformer model)
  - [LongformerConfig](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.LongformerConfig) configuration class: [TFLongformerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerForQuestionAnswering) (Longformer model)
  - [MPNetConfig](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.MPNetConfig) configuration class: [TFMPNetForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForQuestionAnswering) (MPNet model)
  - [MobileBertConfig](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.MobileBertConfig) configuration class: [TFMobileBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForQuestionAnswering) (MobileBERT model)
  - [RemBertConfig](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.RemBertConfig) configuration class: [TFRemBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForQuestionAnswering) (RemBERT model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [TFRoFormerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForQuestionAnswering) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [TFRobertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForQuestionAnswering) (RoBERTa model)
  - [XLMConfig](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.XLMConfig) configuration class: [TFXLMForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMForQuestionAnsweringSimple) (XLM model)
  - [XLMRobertaConfig](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.XLMRobertaConfig) configuration class: [TFXLMRobertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForQuestionAnswering) (XLM-RoBERTa model)
  - [XLNetConfig](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.XLNetConfig) configuration class: [TFXLNetForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetForQuestionAnsweringSimple) (XLNet model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a question answering head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForQuestionAnswering

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForQuestionAnswering.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a question answering head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [TFAlbertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/albert#transformers.TFAlbertForQuestionAnswering) (ALBERT model)
- **bert** -- [TFBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bert#transformers.TFBertForQuestionAnswering) (BERT model)
- **camembert** -- [TFCamembertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/camembert#transformers.TFCamembertForQuestionAnswering) (CamemBERT model)
- **convbert** -- [TFConvBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/convbert#transformers.TFConvBertForQuestionAnswering) (ConvBERT model)
- **deberta** -- [TFDebertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/deberta#transformers.TFDebertaForQuestionAnswering) (DeBERTa model)
- **deberta-v2** -- [TFDebertaV2ForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/deberta-v2#transformers.TFDebertaV2ForQuestionAnswering) (DeBERTa-v2 model)
- **distilbert** -- [TFDistilBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.TFDistilBertForQuestionAnswering) (DistilBERT model)
- **electra** -- [TFElectraForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/electra#transformers.TFElectraForQuestionAnswering) (ELECTRA model)
- **flaubert** -- [TFFlaubertForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/flaubert#transformers.TFFlaubertForQuestionAnsweringSimple) (FlauBERT model)
- **funnel** -- [TFFunnelForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/funnel#transformers.TFFunnelForQuestionAnswering) (Funnel Transformer model)
- **longformer** -- [TFLongformerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/longformer#transformers.TFLongformerForQuestionAnswering) (Longformer model)
- **mobilebert** -- [TFMobileBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mobilebert#transformers.TFMobileBertForQuestionAnswering) (MobileBERT model)
- **mpnet** -- [TFMPNetForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mpnet#transformers.TFMPNetForQuestionAnswering) (MPNet model)
- **rembert** -- [TFRemBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/rembert#transformers.TFRemBertForQuestionAnswering) (RemBERT model)
- **roberta** -- [TFRobertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.TFRobertaForQuestionAnswering) (RoBERTa model)
- **roformer** -- [TFRoFormerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.TFRoFormerForQuestionAnswering) (RoFormer model)
- **xlm** -- [TFXLMForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/xlm#transformers.TFXLMForQuestionAnsweringSimple) (XLM model)
- **xlm-roberta** -- [TFXLMRobertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/xlm-roberta#transformers.TFXLMRobertaForQuestionAnswering) (XLM-RoBERTa model)
- **xlnet** -- [TFXLNetForQuestionAnsweringSimple](/docs/transformers/pr_15592/en/model_doc/xlnet#transformers.TFXLNetForQuestionAnsweringSimple) (XLNet model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForQuestionAnswering

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForQuestionAnswering.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModelForQuestionAnswering.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForQuestionAnswering.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForVision2Seq">TFAutoModelForVision2Seq</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForVision2Seq</name><anchor>transformers.TFAutoModelForVision2Seq</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L407</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a vision-to-text modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [VisionEncoderDecoderConfig](/docs/transformers/pr_15592/en/model_doc/vision-encoder-decoder#transformers.VisionEncoderDecoderConfig) configuration class: [TFVisionEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/vision-encoder-decoder#transformers.TFVisionEncoderDecoderModel) (Vision Encoder decoder model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a vision-to-text modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForVision2Seq

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForVision2Seq.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a vision-to-text modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **vision-encoder-decoder** -- [TFVisionEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/vision-encoder-decoder#transformers.TFVisionEncoderDecoderModel) (Vision Encoder decoder model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForVision2Seq

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForVision2Seq.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModelForVision2Seq.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForVision2Seq.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.TFAutoModelForSpeechSeq2Seq">TFAutoModelForSpeechSeq2Seq</h2>

<div class="docstring">

<docstring><name>class transformers.TFAutoModelForSpeechSeq2Seq</name><anchor>transformers.TFAutoModelForSpeechSeq2Seq</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_tf_auto.py#L482</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence-to-sequence speech-to-text modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [Speech2TextConfig](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.Speech2TextConfig) configuration class: [TFSpeech2TextForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.TFSpeech2TextForConditionalGeneration) (Speech2Text model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a sequence-to-sequence speech-to-text modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForSpeechSeq2Seq

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModelForSpeechSeq2Seq.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a sequence-to-sequence speech-to-text modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **speech_to_text** -- [TFSpeech2TextForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/speech_to_text#transformers.TFSpeech2TextForConditionalGeneration) (Speech2Text model)



Examples:

```python
>>> from transformers import AutoConfig, TFAutoModelForSpeechSeq2Seq

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModelForSpeechSeq2Seq.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModelForSpeechSeq2Seq.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModelForSpeechSeq2Seq.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModel">FlaxAutoModel</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModel</name><anchor>transformers.FlaxAutoModel</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L220</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the base model classes of the library when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [FlaxAlbertModel](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertModel) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [FlaxBartModel](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartModel) (BART model)
  - [BeitConfig](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitConfig) configuration class: [FlaxBeitModel](/docs/transformers/pr_15592/en/model_doc/beit#transformers.FlaxBeitModel) (BEiT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [FlaxBertModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertModel) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [FlaxBigBirdModel](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdModel) (BigBird model)
  - [BlenderbotConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotConfig) configuration class: [FlaxBlenderbotModel](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.FlaxBlenderbotModel) (Blenderbot model)
  - [BlenderbotSmallConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallConfig) configuration class: [FlaxBlenderbotSmallModel](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.FlaxBlenderbotSmallModel) (BlenderbotSmall model)
  - [CLIPConfig](/docs/transformers/pr_15592/en/model_doc/clip#transformers.CLIPConfig) configuration class: [FlaxCLIPModel](/docs/transformers/pr_15592/en/model_doc/clip#transformers.FlaxCLIPModel) (CLIP model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [FlaxDistilBertModel](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertModel) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [FlaxElectraModel](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraModel) (ELECTRA model)
  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [FlaxGPT2Model](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.FlaxGPT2Model) (OpenAI GPT-2 model)
  - [GPTJConfig](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJConfig) configuration class: [FlaxGPTJModel](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.FlaxGPTJModel) (GPT-J model)
  - [GPTNeoConfig](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoConfig) configuration class: [FlaxGPTNeoModel](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.FlaxGPTNeoModel) (GPT Neo model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [FlaxMBartModel](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartModel) (mBART model)
  - [MT5Config](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Config) configuration class: [FlaxMT5Model](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.FlaxMT5Model) (mT5 model)
  - [MarianConfig](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianConfig) configuration class: [FlaxMarianModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.FlaxMarianModel) (Marian model)
  - [PegasusConfig](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusConfig) configuration class: [FlaxPegasusModel](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.FlaxPegasusModel) (Pegasus model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [FlaxRoFormerModel](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerModel) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [FlaxRobertaModel](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaModel) (RoBERTa model)
  - [T5Config](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Config) configuration class: [FlaxT5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.FlaxT5Model) (T5 model)
  - [ViTConfig](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTConfig) configuration class: [FlaxViTModel](/docs/transformers/pr_15592/en/model_doc/vit#transformers.FlaxViTModel) (ViT model)
  - [VisionTextDualEncoderConfig](/docs/transformers/pr_15592/en/model_doc/vision-text-dual-encoder#transformers.VisionTextDualEncoderConfig) configuration class: [FlaxVisionTextDualEncoderModel](/docs/transformers/pr_15592/en/model_doc/vision-text-dual-encoder#transformers.FlaxVisionTextDualEncoderModel) (VisionTextDualEncoder model)
  - [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) configuration class: [FlaxWav2Vec2Model](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.FlaxWav2Vec2Model) (Wav2Vec2 model)
  - [XGLMConfig](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMConfig) configuration class: [FlaxXGLMModel](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.FlaxXGLMModel) (XGLM model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the base model classes of the library from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModel

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModel.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the base model classes of the library from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [FlaxAlbertModel](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertModel) (ALBERT model)
- **bart** -- [FlaxBartModel](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartModel) (BART model)
- **beit** -- [FlaxBeitModel](/docs/transformers/pr_15592/en/model_doc/beit#transformers.FlaxBeitModel) (BEiT model)
- **bert** -- [FlaxBertModel](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertModel) (BERT model)
- **big_bird** -- [FlaxBigBirdModel](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdModel) (BigBird model)
- **blenderbot** -- [FlaxBlenderbotModel](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.FlaxBlenderbotModel) (Blenderbot model)
- **blenderbot-small** -- [FlaxBlenderbotSmallModel](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.FlaxBlenderbotSmallModel) (BlenderbotSmall model)
- **clip** -- [FlaxCLIPModel](/docs/transformers/pr_15592/en/model_doc/clip#transformers.FlaxCLIPModel) (CLIP model)
- **distilbert** -- [FlaxDistilBertModel](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertModel) (DistilBERT model)
- **electra** -- [FlaxElectraModel](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraModel) (ELECTRA model)
- **gpt2** -- [FlaxGPT2Model](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.FlaxGPT2Model) (OpenAI GPT-2 model)
- **gpt_neo** -- [FlaxGPTNeoModel](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.FlaxGPTNeoModel) (GPT Neo model)
- **gptj** -- [FlaxGPTJModel](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.FlaxGPTJModel) (GPT-J model)
- **marian** -- [FlaxMarianModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.FlaxMarianModel) (Marian model)
- **mbart** -- [FlaxMBartModel](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartModel) (mBART model)
- **mt5** -- [FlaxMT5Model](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.FlaxMT5Model) (mT5 model)
- **pegasus** -- [FlaxPegasusModel](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.FlaxPegasusModel) (Pegasus model)
- **roberta** -- [FlaxRobertaModel](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaModel) (RoBERTa model)
- **roformer** -- [FlaxRoFormerModel](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerModel) (RoFormer model)
- **t5** -- [FlaxT5Model](/docs/transformers/pr_15592/en/model_doc/t5#transformers.FlaxT5Model) (T5 model)
- **vision-text-dual-encoder** -- [FlaxVisionTextDualEncoderModel](/docs/transformers/pr_15592/en/model_doc/vision-text-dual-encoder#transformers.FlaxVisionTextDualEncoderModel) (VisionTextDualEncoder model)
- **vit** -- [FlaxViTModel](/docs/transformers/pr_15592/en/model_doc/vit#transformers.FlaxViTModel) (ViT model)
- **wav2vec2** -- [FlaxWav2Vec2Model](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.FlaxWav2Vec2Model) (Wav2Vec2 model)
- **xglm** -- [FlaxXGLMModel](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.FlaxXGLMModel) (XGLM model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModel

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModel.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModel.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModel.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForCausalLM">FlaxAutoModelForCausalLM</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForCausalLM</name><anchor>transformers.FlaxAutoModelForCausalLM</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L234</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a causal language modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [GPT2Config](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.GPT2Config) configuration class: [FlaxGPT2LMHeadModel](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.FlaxGPT2LMHeadModel) (OpenAI GPT-2 model)
  - [GPTJConfig](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.GPTJConfig) configuration class: [FlaxGPTJForCausalLM](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.FlaxGPTJForCausalLM) (GPT-J model)
  - [GPTNeoConfig](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.GPTNeoConfig) configuration class: [FlaxGPTNeoForCausalLM](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.FlaxGPTNeoForCausalLM) (GPT Neo model)
  - [XGLMConfig](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.XGLMConfig) configuration class: [FlaxXGLMForCausalLM](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.FlaxXGLMForCausalLM) (XGLM model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a causal language modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForCausalLM

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForCausalLM.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a causal language modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **gpt2** -- [FlaxGPT2LMHeadModel](/docs/transformers/pr_15592/en/model_doc/gpt2#transformers.FlaxGPT2LMHeadModel) (OpenAI GPT-2 model)
- **gpt_neo** -- [FlaxGPTNeoForCausalLM](/docs/transformers/pr_15592/en/model_doc/gpt_neo#transformers.FlaxGPTNeoForCausalLM) (GPT Neo model)
- **gptj** -- [FlaxGPTJForCausalLM](/docs/transformers/pr_15592/en/model_doc/gptj#transformers.FlaxGPTJForCausalLM) (GPT-J model)
- **xglm** -- [FlaxXGLMForCausalLM](/docs/transformers/pr_15592/en/model_doc/xglm#transformers.FlaxXGLMForCausalLM) (XGLM model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForCausalLM

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForCausalLM.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForCausalLM.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForCausalLM.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForPreTraining">FlaxAutoModelForPreTraining</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForPreTraining</name><anchor>transformers.FlaxAutoModelForPreTraining</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L227</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a pretraining head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [FlaxAlbertForPreTraining](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForPreTraining) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [FlaxBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartForConditionalGeneration) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [FlaxBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForPreTraining) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [FlaxBigBirdForPreTraining](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForPreTraining) (BigBird model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [FlaxElectraForPreTraining](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForPreTraining) (ELECTRA model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [FlaxMBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartForConditionalGeneration) (mBART model)
  - [MT5Config](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Config) configuration class: [FlaxMT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.FlaxMT5ForConditionalGeneration) (mT5 model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [FlaxRoFormerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForMaskedLM) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [FlaxRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForMaskedLM) (RoBERTa model)
  - [T5Config](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Config) configuration class: [FlaxT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.FlaxT5ForConditionalGeneration) (T5 model)
  - [Wav2Vec2Config](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.Wav2Vec2Config) configuration class: [FlaxWav2Vec2ForPreTraining](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.FlaxWav2Vec2ForPreTraining) (Wav2Vec2 model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a pretraining head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForPreTraining

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForPreTraining.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a pretraining head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [FlaxAlbertForPreTraining](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForPreTraining) (ALBERT model)
- **bart** -- [FlaxBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartForConditionalGeneration) (BART model)
- **bert** -- [FlaxBertForPreTraining](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForPreTraining) (BERT model)
- **big_bird** -- [FlaxBigBirdForPreTraining](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForPreTraining) (BigBird model)
- **electra** -- [FlaxElectraForPreTraining](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForPreTraining) (ELECTRA model)
- **mbart** -- [FlaxMBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartForConditionalGeneration) (mBART model)
- **mt5** -- [FlaxMT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.FlaxMT5ForConditionalGeneration) (mT5 model)
- **roberta** -- [FlaxRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForMaskedLM) (RoBERTa model)
- **roformer** -- [FlaxRoFormerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForMaskedLM) (RoFormer model)
- **t5** -- [FlaxT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.FlaxT5ForConditionalGeneration) (T5 model)
- **wav2vec2** -- [FlaxWav2Vec2ForPreTraining](/docs/transformers/pr_15592/en/model_doc/wav2vec2#transformers.FlaxWav2Vec2ForPreTraining) (Wav2Vec2 model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForPreTraining

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForPreTraining.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForPreTraining.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForPreTraining.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForMaskedLM">FlaxAutoModelForMaskedLM</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForMaskedLM</name><anchor>transformers.FlaxAutoModelForMaskedLM</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L241</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a masked language modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [FlaxAlbertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForMaskedLM) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [FlaxBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartForConditionalGeneration) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [FlaxBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForMaskedLM) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [FlaxBigBirdForMaskedLM](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForMaskedLM) (BigBird model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [FlaxDistilBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertForMaskedLM) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [FlaxElectraForMaskedLM](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForMaskedLM) (ELECTRA model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [FlaxMBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartForConditionalGeneration) (mBART model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [FlaxRoFormerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForMaskedLM) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [FlaxRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForMaskedLM) (RoBERTa model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a masked language modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForMaskedLM

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForMaskedLM.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a masked language modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [FlaxAlbertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForMaskedLM) (ALBERT model)
- **bart** -- [FlaxBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartForConditionalGeneration) (BART model)
- **bert** -- [FlaxBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForMaskedLM) (BERT model)
- **big_bird** -- [FlaxBigBirdForMaskedLM](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForMaskedLM) (BigBird model)
- **distilbert** -- [FlaxDistilBertForMaskedLM](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertForMaskedLM) (DistilBERT model)
- **electra** -- [FlaxElectraForMaskedLM](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForMaskedLM) (ELECTRA model)
- **mbart** -- [FlaxMBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartForConditionalGeneration) (mBART model)
- **roberta** -- [FlaxRobertaForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForMaskedLM) (RoBERTa model)
- **roformer** -- [FlaxRoFormerForMaskedLM](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForMaskedLM) (RoFormer model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForMaskedLM

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForMaskedLM.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForMaskedLM.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForMaskedLM.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForSeq2SeqLM">FlaxAutoModelForSeq2SeqLM</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForSeq2SeqLM</name><anchor>transformers.FlaxAutoModelForSeq2SeqLM</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L248</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence-to-sequence language modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [FlaxBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartForConditionalGeneration) (BART model)
  - [BlenderbotConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.BlenderbotConfig) configuration class: [FlaxBlenderbotForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.FlaxBlenderbotForConditionalGeneration) (Blenderbot model)
  - [BlenderbotSmallConfig](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.BlenderbotSmallConfig) configuration class: [FlaxBlenderbotSmallForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.FlaxBlenderbotSmallForConditionalGeneration) (BlenderbotSmall model)
  - [EncoderDecoderConfig](/docs/transformers/pr_15592/en/model_doc/encoder-decoder#transformers.EncoderDecoderConfig) configuration class: [FlaxEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/encoder-decoder#transformers.FlaxEncoderDecoderModel) (Encoder decoder model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [FlaxMBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartForConditionalGeneration) (mBART model)
  - [MT5Config](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.MT5Config) configuration class: [FlaxMT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.FlaxMT5ForConditionalGeneration) (mT5 model)
  - [MarianConfig](/docs/transformers/pr_15592/en/model_doc/marian#transformers.MarianConfig) configuration class: [FlaxMarianMTModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.FlaxMarianMTModel) (Marian model)
  - [PegasusConfig](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.PegasusConfig) configuration class: [FlaxPegasusForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.FlaxPegasusForConditionalGeneration) (Pegasus model)
  - [T5Config](/docs/transformers/pr_15592/en/model_doc/t5#transformers.T5Config) configuration class: [FlaxT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.FlaxT5ForConditionalGeneration) (T5 model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a sequence-to-sequence language modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForSeq2SeqLM

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("t5-base")
>>> model = FlaxAutoModelForSeq2SeqLM.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a sequence-to-sequence language modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **bart** -- [FlaxBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartForConditionalGeneration) (BART model)
- **blenderbot** -- [FlaxBlenderbotForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot#transformers.FlaxBlenderbotForConditionalGeneration) (Blenderbot model)
- **blenderbot-small** -- [FlaxBlenderbotSmallForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/blenderbot-small#transformers.FlaxBlenderbotSmallForConditionalGeneration) (BlenderbotSmall model)
- **encoder-decoder** -- [FlaxEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/encoder-decoder#transformers.FlaxEncoderDecoderModel) (Encoder decoder model)
- **marian** -- [FlaxMarianMTModel](/docs/transformers/pr_15592/en/model_doc/marian#transformers.FlaxMarianMTModel) (Marian model)
- **mbart** -- [FlaxMBartForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartForConditionalGeneration) (mBART model)
- **mt5** -- [FlaxMT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/mt5#transformers.FlaxMT5ForConditionalGeneration) (mT5 model)
- **pegasus** -- [FlaxPegasusForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/pegasus#transformers.FlaxPegasusForConditionalGeneration) (Pegasus model)
- **t5** -- [FlaxT5ForConditionalGeneration](/docs/transformers/pr_15592/en/model_doc/t5#transformers.FlaxT5ForConditionalGeneration) (T5 model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForSeq2SeqLM

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForSeq2SeqLM.from_pretrained("t5-base")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForSeq2SeqLM.from_pretrained("t5-base", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/t5_pt_model_config.json")
>>> model = FlaxAutoModelForSeq2SeqLM.from_pretrained(
...     "./pt_model/t5_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForSequenceClassification">FlaxAutoModelForSequenceClassification</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForSequenceClassification</name><anchor>transformers.FlaxAutoModelForSequenceClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L257</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a sequence classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [FlaxAlbertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForSequenceClassification) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [FlaxBartForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartForSequenceClassification) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [FlaxBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForSequenceClassification) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [FlaxBigBirdForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForSequenceClassification) (BigBird model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [FlaxDistilBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertForSequenceClassification) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [FlaxElectraForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForSequenceClassification) (ELECTRA model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [FlaxMBartForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartForSequenceClassification) (mBART model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [FlaxRoFormerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForSequenceClassification) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [FlaxRobertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForSequenceClassification) (RoBERTa model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a sequence classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForSequenceClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForSequenceClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a sequence classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [FlaxAlbertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForSequenceClassification) (ALBERT model)
- **bart** -- [FlaxBartForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartForSequenceClassification) (BART model)
- **bert** -- [FlaxBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForSequenceClassification) (BERT model)
- **big_bird** -- [FlaxBigBirdForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForSequenceClassification) (BigBird model)
- **distilbert** -- [FlaxDistilBertForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertForSequenceClassification) (DistilBERT model)
- **electra** -- [FlaxElectraForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForSequenceClassification) (ELECTRA model)
- **mbart** -- [FlaxMBartForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartForSequenceClassification) (mBART model)
- **roberta** -- [FlaxRobertaForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForSequenceClassification) (RoBERTa model)
- **roformer** -- [FlaxRoFormerForSequenceClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForSequenceClassification) (RoFormer model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForSequenceClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForSequenceClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForSequenceClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForSequenceClassification.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForQuestionAnswering">FlaxAutoModelForQuestionAnswering</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForQuestionAnswering</name><anchor>transformers.FlaxAutoModelForQuestionAnswering</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L266</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a question answering head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [FlaxAlbertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForQuestionAnswering) (ALBERT model)
  - [BartConfig](/docs/transformers/pr_15592/en/model_doc/bart#transformers.BartConfig) configuration class: [FlaxBartForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartForQuestionAnswering) (BART model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [FlaxBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForQuestionAnswering) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [FlaxBigBirdForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForQuestionAnswering) (BigBird model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [FlaxDistilBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertForQuestionAnswering) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [FlaxElectraForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForQuestionAnswering) (ELECTRA model)
  - [MBartConfig](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.MBartConfig) configuration class: [FlaxMBartForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartForQuestionAnswering) (mBART model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [FlaxRoFormerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForQuestionAnswering) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [FlaxRobertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForQuestionAnswering) (RoBERTa model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a question answering head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForQuestionAnswering

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForQuestionAnswering.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a question answering head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [FlaxAlbertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForQuestionAnswering) (ALBERT model)
- **bart** -- [FlaxBartForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bart#transformers.FlaxBartForQuestionAnswering) (BART model)
- **bert** -- [FlaxBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForQuestionAnswering) (BERT model)
- **big_bird** -- [FlaxBigBirdForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForQuestionAnswering) (BigBird model)
- **distilbert** -- [FlaxDistilBertForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertForQuestionAnswering) (DistilBERT model)
- **electra** -- [FlaxElectraForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForQuestionAnswering) (ELECTRA model)
- **mbart** -- [FlaxMBartForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/mbart#transformers.FlaxMBartForQuestionAnswering) (mBART model)
- **roberta** -- [FlaxRobertaForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForQuestionAnswering) (RoBERTa model)
- **roformer** -- [FlaxRoFormerForQuestionAnswering](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForQuestionAnswering) (RoFormer model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForQuestionAnswering

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForQuestionAnswering.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForQuestionAnswering.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForQuestionAnswering.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForTokenClassification">FlaxAutoModelForTokenClassification</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForTokenClassification</name><anchor>transformers.FlaxAutoModelForTokenClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L273</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a token classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [FlaxAlbertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForTokenClassification) (ALBERT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [FlaxBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForTokenClassification) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [FlaxBigBirdForTokenClassification](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForTokenClassification) (BigBird model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [FlaxDistilBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertForTokenClassification) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [FlaxElectraForTokenClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForTokenClassification) (ELECTRA model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [FlaxRoFormerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForTokenClassification) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [FlaxRobertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForTokenClassification) (RoBERTa model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a token classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForTokenClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForTokenClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a token classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [FlaxAlbertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForTokenClassification) (ALBERT model)
- **bert** -- [FlaxBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForTokenClassification) (BERT model)
- **big_bird** -- [FlaxBigBirdForTokenClassification](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForTokenClassification) (BigBird model)
- **distilbert** -- [FlaxDistilBertForTokenClassification](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertForTokenClassification) (DistilBERT model)
- **electra** -- [FlaxElectraForTokenClassification](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForTokenClassification) (ELECTRA model)
- **roberta** -- [FlaxRobertaForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForTokenClassification) (RoBERTa model)
- **roformer** -- [FlaxRoFormerForTokenClassification](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForTokenClassification) (RoFormer model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForTokenClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForTokenClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForTokenClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForTokenClassification.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForMultipleChoice">FlaxAutoModelForMultipleChoice</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForMultipleChoice</name><anchor>transformers.FlaxAutoModelForMultipleChoice</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L282</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a multiple choice head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [AlbertConfig](/docs/transformers/pr_15592/en/model_doc/albert#transformers.AlbertConfig) configuration class: [FlaxAlbertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForMultipleChoice) (ALBERT model)
  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [FlaxBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForMultipleChoice) (BERT model)
  - [BigBirdConfig](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.BigBirdConfig) configuration class: [FlaxBigBirdForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForMultipleChoice) (BigBird model)
  - [DistilBertConfig](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.DistilBertConfig) configuration class: [FlaxDistilBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertForMultipleChoice) (DistilBERT model)
  - [ElectraConfig](/docs/transformers/pr_15592/en/model_doc/electra#transformers.ElectraConfig) configuration class: [FlaxElectraForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForMultipleChoice) (ELECTRA model)
  - [RoFormerConfig](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.RoFormerConfig) configuration class: [FlaxRoFormerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForMultipleChoice) (RoFormer model)
  - [RobertaConfig](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.RobertaConfig) configuration class: [FlaxRobertaForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForMultipleChoice) (RoBERTa model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a multiple choice head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForMultipleChoice

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForMultipleChoice.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a multiple choice head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **albert** -- [FlaxAlbertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/albert#transformers.FlaxAlbertForMultipleChoice) (ALBERT model)
- **bert** -- [FlaxBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForMultipleChoice) (BERT model)
- **big_bird** -- [FlaxBigBirdForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/big_bird#transformers.FlaxBigBirdForMultipleChoice) (BigBird model)
- **distilbert** -- [FlaxDistilBertForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/distilbert#transformers.FlaxDistilBertForMultipleChoice) (DistilBERT model)
- **electra** -- [FlaxElectraForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/electra#transformers.FlaxElectraForMultipleChoice) (ELECTRA model)
- **roberta** -- [FlaxRobertaForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roberta#transformers.FlaxRobertaForMultipleChoice) (RoBERTa model)
- **roformer** -- [FlaxRoFormerForMultipleChoice](/docs/transformers/pr_15592/en/model_doc/roformer#transformers.FlaxRoFormerForMultipleChoice) (RoFormer model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForMultipleChoice

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForMultipleChoice.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForMultipleChoice.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForMultipleChoice.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForNextSentencePrediction">FlaxAutoModelForNextSentencePrediction</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForNextSentencePrediction</name><anchor>transformers.FlaxAutoModelForNextSentencePrediction</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L289</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a next sentence prediction head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [BertConfig](/docs/transformers/pr_15592/en/model_doc/bert#transformers.BertConfig) configuration class: [FlaxBertForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForNextSentencePrediction) (BERT model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a next sentence prediction head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForNextSentencePrediction

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForNextSentencePrediction.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a next sentence prediction head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **bert** -- [FlaxBertForNextSentencePrediction](/docs/transformers/pr_15592/en/model_doc/bert#transformers.FlaxBertForNextSentencePrediction) (BERT model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForNextSentencePrediction

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForNextSentencePrediction.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForNextSentencePrediction.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForNextSentencePrediction.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForImageClassification">FlaxAutoModelForImageClassification</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForImageClassification</name><anchor>transformers.FlaxAutoModelForImageClassification</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L298</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a image classification head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [BeitConfig](/docs/transformers/pr_15592/en/model_doc/beit#transformers.BeitConfig) configuration class: [FlaxBeitForImageClassification](/docs/transformers/pr_15592/en/model_doc/beit#transformers.FlaxBeitForImageClassification) (BEiT model)
  - [ViTConfig](/docs/transformers/pr_15592/en/model_doc/vit#transformers.ViTConfig) configuration class: [FlaxViTForImageClassification](/docs/transformers/pr_15592/en/model_doc/vit#transformers.FlaxViTForImageClassification) (ViT model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a image classification head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForImageClassification

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForImageClassification.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a image classification head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **beit** -- [FlaxBeitForImageClassification](/docs/transformers/pr_15592/en/model_doc/beit#transformers.FlaxBeitForImageClassification) (BEiT model)
- **vit** -- [FlaxViTForImageClassification](/docs/transformers/pr_15592/en/model_doc/vit#transformers.FlaxViTForImageClassification) (ViT model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForImageClassification

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForImageClassification.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForImageClassification.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForImageClassification.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>

<h2 id="transformers.FlaxAutoModelForVision2Seq">FlaxAutoModelForVision2Seq</h2>

<div class="docstring">

<docstring><name>class transformers.FlaxAutoModelForVision2Seq</name><anchor>transformers.FlaxAutoModelForVision2Seq</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/modeling_flax_auto.py#L307</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>

This is a generic model class that will be instantiated as one of the model classes of the library (with a vision-to-text modeling head) when created
with the `from_pretrained()`class method or the `from_config()`class
method.

This class cannot be instantiated directly using `__init__()` (throws an error).



<div class="docstring">
<docstring><name>from\_config</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_config</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L390</source><parameters>[{"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig)) --
  The model class to instantiate is selected based on the configuration class:

  - [VisionEncoderDecoderConfig](/docs/transformers/pr_15592/en/model_doc/vision-encoder-decoder#transformers.VisionEncoderDecoderConfig) configuration class: [FlaxVisionEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/vision-encoder-decoder#transformers.FlaxVisionEncoderDecoderModel) (Vision Encoder decoder model)</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiates one of the model classes of the library (with a vision-to-text modeling head) from a configuration.

Note:
Loading a model from its configuration file does **not** load the model weights. It only affects the
model's configuration. Use `from_pretrained()`to load the model weights.



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForVision2Seq

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = FlaxAutoModelForVision2Seq.from_config(config)
```


</div>
<div class="docstring">
<docstring><name>from\_pretrained</name><anchor>transformers.models.auto.auto_factory._BaseAutoModelClass.from_pretrained</anchor><source>https://github.com/huggingface/transformers/blob/pr_15592/src/transformers/models/auto/auto_factory.py#L418</source><parameters>[{"name": "*model_args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters><paramsdesc>- **pretrained_model_name_or_path** (`str` or `os.PathLike`) --
  Can be either:

  - A string, the *model id* of a pretrained model hosted inside a model repo on huggingface.co.
    Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a
    user or organization name, like `dbmdz/bert-base-german-cased`.
  - A path to a *directory* containing model weights saved using
    [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained), e.g., `./my_model_directory/`.
  - A path or url to a *PyTorch state_dict save file* (e.g, `./pt_model/pytorch_model.bin`). In this
    case, `from_pt` should be set to `True` and a configuration object should be provided as `config`
    argument. This loading path is slower than converting the PyTorch model in a TensorFlow model
    using the provided conversion scripts and loading the TensorFlow model afterwards.
- **model_args** (additional positional arguments, *optional*) --
  Will be passed along to the underlying model `__init__()` method.
- **config** ([PretrainedConfig](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig), *optional*) --
  Configuration for the model to use instead of an automatically loaded configuration. Configuration can
  be automatically loaded when:

  - The model is a model provided by the library (loaded with the *model id* string of a pretrained
    model).
  - The model was saved using [save_pretrained()](/docs/transformers/pr_15592/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) and is reloaded by supplying the
    save directory.
  - The model is loaded by supplying a local directory as `pretrained_model_name_or_path` and a
    configuration JSON file named *config.json* is found in the directory.
- **cache_dir** (`str` or `os.PathLike`, *optional*) --
  Path to a directory in which a downloaded pretrained model configuration should be cached if the
  standard cache should not be used.
- **from_pt** (`bool`, *optional*, defaults to `False`) --
  Load the model weights from a PyTorch checkpoint save file (see docstring of
  `pretrained_model_name_or_path` argument).
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **resume_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to delete incompletely received files. Will attempt to resume the download if such a
  file exists.
- **proxies** (`Dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, e.g., `&amp;lcub;'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether or not to only look at local files (e.g., not try downloading the model).
- **revision(`str`,** *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a
  git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any
  identifier allowed by git.
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom models defined on the Hub in their own modeling files. This option
  should only be set to `True` for repositories you trust and in which you have read the code, as it will
  execute code present on the Hub on your local machine.
- **kwargs** (additional keyword arguments, *optional*) --
  Can be used to update the configuration object (after it being loaded) and initiate the model (e.g.,
  `output_attentions=True`). Behaves differently depending on whether a `config` is provided or
  automatically loaded:

  - If a configuration is provided with `config`, `**kwargs` will be directly passed to the
    underlying model's `__init__` method (we assume all relevant updates to the configuration have
    already been done)
  - If a configuration is not provided, `kwargs` will be first passed to the configuration class
    initialization function ([from_pretrained()](/docs/transformers/pr_15592/en/main_classes/configuration#transformers.PretrainedConfig.from_pretrained)). Each key of `kwargs` that
    corresponds to a configuration attribute will be used to override said attribute with the
    supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute
    will be passed to the underlying model's `__init__` function.</paramsdesc><paramgroups>0</paramgroups></docstring>

Instantiate one of the model classes of the library (with a vision-to-text modeling head) from a pretrained model.

The model class to instantiate is selected based on the `model_type` property of the config object (either
passed as an argument or loaded from `pretrained_model_name_or_path` if possible), or when it's missing, by
falling back to using pattern matching on `pretrained_model_name_or_path`:

- **vision-encoder-decoder** -- [FlaxVisionEncoderDecoderModel](/docs/transformers/pr_15592/en/model_doc/vision-encoder-decoder#transformers.FlaxVisionEncoderDecoderModel) (Vision Encoder decoder model)



Examples:

```python
>>> from transformers import AutoConfig, FlaxAutoModelForVision2Seq

>>> # Download model and configuration from huggingface.co and cache.
>>> model = FlaxAutoModelForVision2Seq.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = FlaxAutoModelForVision2Seq.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = FlaxAutoModelForVision2Seq.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )
```


</div></div>
