<meta charset="utf-8" /><meta http-equiv="content-security-policy" content=""><meta name="hf:doc:metadata" content="{&quot;local&quot;:&quot;philosophy&quot;,&quot;sections&quot;:[{&quot;local&quot;:&quot;main-concepts&quot;,&quot;title&quot;:&quot;Main concepts&quot;}],&quot;title&quot;:&quot;Philosophy&quot;}" data-svelte="svelte-1phssyn">
	<link rel="modulepreload" href="/docs/transformers/pr_17170/en/_app/assets/pages/__layout.svelte-659a8e1b.css">
	<link rel="modulepreload" href="/docs/transformers/pr_17170/en/_app/start-7b834ccc.js">
	<link rel="modulepreload" href="/docs/transformers/pr_17170/en/_app/chunks/vendor-6b77c823.js">
	<link rel="modulepreload" href="/docs/transformers/pr_17170/en/_app/chunks/paths-4b3c6e7e.js">
	<link rel="modulepreload" href="/docs/transformers/pr_17170/en/_app/pages/__layout.svelte-3f22c1e5.js">
	<link rel="modulepreload" href="/docs/transformers/pr_17170/en/_app/pages/philosophy.mdx-1f9ff3b4.js">
	<link rel="modulepreload" href="/docs/transformers/pr_17170/en/_app/chunks/IconCopyLink-7a11ce68.js"> 






<h1 class="relative group"><a id="philosophy" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#philosophy"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>Philosophy
	</span></h1>

<p>🤗 Transformers is an opinionated library built for:</p>
<ul><li>NLP researchers and educators seeking to use/study/extend large-scale transformers models</li>
<li>hands-on practitioners who want to fine-tune those models and/or serve them in production</li>
<li>engineers who just want to download a pretrained model and use it to solve a given NLP task.</li></ul>
<p>The library was designed with two strong goals in mind:</p>
<ul><li><p>Be as easy and fast to use as possible:</p>
<ul><li>We strongly limited the number of user-facing abstractions to learn, in fact, there are almost no abstractions,
just three standard classes required to use each model: <a href="main_classes/configuration">configuration</a>,
<a href="main_classes/model">models</a> and <a href="main_classes/tokenizer">tokenizer</a>.</li>
<li>All of these classes can be initialized in a simple and unified way from pretrained instances by using a common
<code>from_pretrained()</code> instantiation method which will take care of downloading (if needed), caching and
loading the related class instance and associated data (configurations’ hyper-parameters, tokenizers’ vocabulary,
and models’ weights) from a pretrained checkpoint provided on <a href="https://huggingface.co/models" rel="nofollow">Hugging Face Hub</a> or your own saved checkpoint.</li>
<li>On top of those three base classes, the library provides two APIs: <a href="/docs/transformers/pr_17170/en/main_classes/pipelines#transformers.pipeline">pipeline()</a> for quickly
using a model (plus its associated tokenizer and configuration) on a given task and
<a href="/docs/transformers/pr_17170/en/main_classes/trainer#transformers.Trainer">Trainer</a>/<code>Keras.fit</code> to quickly train or fine-tune a given model.</li>
<li>As a consequence, this library is NOT a modular toolbox of building blocks for neural nets. If you want to
extend/build-upon the library, just use regular Python/PyTorch/TensorFlow/Keras modules and inherit from the base
classes of the library to reuse functionalities like model loading/saving.</li></ul></li>
<li><p>Provide state-of-the-art models with performances as close as possible to the original models:</p>
<ul><li>We provide at least one example for each architecture which reproduces a result provided by the official authors
of said architecture.</li>
<li>The code is usually as close to the original code base as possible which means some PyTorch code may be not as
<em>pytorchic</em> as it could be as a result of being converted TensorFlow code and vice versa.</li></ul></li></ul>
<p>A few other goals:</p>
<ul><li><p>Expose the models’ internals as consistently as possible:</p>
<ul><li>We give access, using a single API, to the full hidden-states and attention weights.</li>
<li>Tokenizer and base model’s API are standardized to easily switch between models.</li></ul></li>
<li><p>Incorporate a subjective selection of promising tools for fine-tuning/investigating these models:</p>
<ul><li>A simple/consistent way to add new tokens to the vocabulary and embeddings for fine-tuning.</li>
<li>Simple ways to mask and prune transformer heads.</li></ul></li>
<li><p>Switch easily between PyTorch and TensorFlow 2.0, allowing training using one framework and inference using another.</p></li></ul>
<h2 class="relative group"><a id="main-concepts" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#main-concepts"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>Main concepts
	</span></h2>

<p>The library is built around three types of classes for each model:</p>
<ul><li><strong>Model classes</strong> such as <a href="/docs/transformers/pr_17170/en/model_doc/bert#transformers.BertModel">BertModel</a>, which are 30+ PyTorch models (<a href="https://pytorch.org/docs/stable/nn.html#torch.nn.Module" rel="nofollow">torch.nn.Module</a>) or Keras models (<a href="https://www.tensorflow.org/api_docs/python/tf/keras/Model" rel="nofollow">tf.keras.Model</a>) that work with the pretrained weights provided in the
library.</li>
<li><strong>Configuration classes</strong> such as <a href="/docs/transformers/pr_17170/en/model_doc/bert#transformers.BertConfig">BertConfig</a>, which store all the parameters required to build
a model. You don’t always need to instantiate these yourself. In particular, if you are using a pretrained model
without any modification, creating the model will automatically take care of instantiating the configuration (which
is part of the model).</li>
<li><strong>Tokenizer classes</strong> such as <a href="/docs/transformers/pr_17170/en/model_doc/bert#transformers.BertTokenizer">BertTokenizer</a>, which store the vocabulary for each model and
provide methods for encoding/decoding strings in a list of token embeddings indices to be fed to a model.</li></ul>
<p>All these classes can be instantiated from pretrained instances and saved locally using two methods:</p>
<ul><li><code>from_pretrained()</code> lets you instantiate a model/configuration/tokenizer from a pretrained version either
provided by the library itself (the supported models can be found on the <a href="https://huggingface.co/models" rel="nofollow">Model Hub</a>) or
stored locally (or on a server) by the user,</li>
<li><code>save_pretrained()</code> lets you save a model/configuration/tokenizer locally so that it can be reloaded using
<code>from_pretrained()</code>.</li></ul>


		<script type="module" data-hydrate="1r102k">
		import { start } from "/docs/transformers/pr_17170/en/_app/start-7b834ccc.js";
		start({
			target: document.querySelector('[data-hydrate="1r102k"]').parentNode,
			paths: {"base":"/docs/transformers/pr_17170/en","assets":"/docs/transformers/pr_17170/en"},
			session: {},
			route: false,
			spa: false,
			trailing_slash: "never",
			hydrate: {
				status: 200,
				error: null,
				nodes: [
					import("/docs/transformers/pr_17170/en/_app/pages/__layout.svelte-3f22c1e5.js"),
						import("/docs/transformers/pr_17170/en/_app/pages/philosophy.mdx-1f9ff3b4.js")
				],
				params: {}
			}
		});
	</script>
