<meta charset="utf-8" /><meta http-equiv="content-security-policy" content=""><meta name="hf:doc:metadata" content="{&quot;local&quot;:&quot;quantization&quot;,&quot;title&quot;:&quot;Quantization&quot;}" data-svelte="svelte-1phssyn">
	<link rel="modulepreload" href="/docs/optimum/pr_450/en/_app/assets/pages/__layout.svelte-hf-doc-builder.css">
	<link rel="modulepreload" href="/docs/optimum/pr_450/en/_app/start-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/optimum/pr_450/en/_app/chunks/vendor-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/optimum/pr_450/en/_app/chunks/paths-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/optimum/pr_450/en/_app/pages/__layout.svelte-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/optimum/pr_450/en/_app/pages/concept_guides/quantization.mdx-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/optimum/pr_450/en/_app/chunks/IconCopyLink-hf-doc-builder.js"> 





<h1 class="relative group"><a id="quantization" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#quantization"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>Quantization
	</span></h1>

<p>Quantization is a technique to reduce the computational and memory cost of running inference by representing the weights and activations with low-precision data types like 8-bit integer (INT8) instead of the usual 32-bit floating point (FP32). Reducing the number of bits means the resulting model requires less memory storage, and operations like matrix multiplication can be performed much faster with integer arithmetic. Remarkably, these performance gains can be realized with little to no loss in accuracy!</p>
<p>The basic idea behind quantization is that we can “discretize” the floating-point values in each tensor by mapping their range into a smaller one of fixed-point numbers, and linearly distributing all values in between.</p>


		<script type="module" data-hydrate="18z15xn">
		import { start } from "/docs/optimum/pr_450/en/_app/start-hf-doc-builder.js";
		start({
			target: document.querySelector('[data-hydrate="18z15xn"]').parentNode,
			paths: {"base":"/docs/optimum/pr_450/en","assets":"/docs/optimum/pr_450/en"},
			session: {},
			route: false,
			spa: false,
			trailing_slash: "never",
			hydrate: {
				status: 200,
				error: null,
				nodes: [
					import("/docs/optimum/pr_450/en/_app/pages/__layout.svelte-hf-doc-builder.js"),
						import("/docs/optimum/pr_450/en/_app/pages/concept_guides/quantization.mdx-hf-doc-builder.js")
				],
				params: {}
			}
		});
	</script>
