<meta charset="utf-8" /><meta http-equiv="content-security-policy" content=""><meta name="hf:doc:metadata" content="{&quot;local&quot;:&quot;&quot;,&quot;title&quot;:&quot;章末小测验&quot;}" data-svelte="svelte-1phssyn">
	<link rel="modulepreload" href="/docs/course/pr_362/zh-CN/_app/assets/pages/__layout.svelte-hf-doc-builder.css">
	<link rel="modulepreload" href="/docs/course/pr_362/zh-CN/_app/start-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/course/pr_362/zh-CN/_app/chunks/vendor-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/course/pr_362/zh-CN/_app/chunks/paths-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/course/pr_362/zh-CN/_app/pages/__layout.svelte-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/course/pr_362/zh-CN/_app/pages/chapter6/10.mdx-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/course/pr_362/zh-CN/_app/chunks/IconCopyLink-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/course/pr_362/zh-CN/_app/chunks/CourseFloatingBanner-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/course/pr_362/zh-CN/_app/chunks/DocNotebookDropdown-hf-doc-builder.js">
	<link rel="modulepreload" href="/docs/course/pr_362/zh-CN/_app/chunks/Question-hf-doc-builder.js"> 






<h1 id="">章末小测验</h1>


<div class="flex space-x-1 absolute z-10 right-0 top-0"><a href="https://discuss.huggingface.co/t/chapter-6-questions" target="_blank"><img alt="Ask a Question" class="!m-0" src="https://img.shields.io/badge/Ask%20a%20question-ffcb4c.svg?logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgLTEgMTA0IDEwNiI+PGRlZnM+PHN0eWxlPi5jbHMtMXtmaWxsOiMyMzFmMjA7fS5jbHMtMntmaWxsOiNmZmY5YWU7fS5jbHMtM3tmaWxsOiMwMGFlZWY7fS5jbHMtNHtmaWxsOiMwMGE5NGY7fS5jbHMtNXtmaWxsOiNmMTVkMjI7fS5jbHMtNntmaWxsOiNlMzFiMjM7fTwvc3R5bGU+PC9kZWZzPjx0aXRsZT5EaXNjb3Vyc2VfbG9nbzwvdGl0bGU+PGcgaWQ9IkxheWVyXzIiPjxnIGlkPSJMYXllcl8zIj48cGF0aCBjbGFzcz0iY2xzLTEiIGQ9Ik01MS44NywwQzIzLjcxLDAsMCwyMi44MywwLDUxYzAsLjkxLDAsNTIuODEsMCw1Mi44MWw1MS44Ni0uMDVjMjguMTYsMCw1MS0yMy43MSw1MS01MS44N1M4MCwwLDUxLjg3LDBaIi8+PHBhdGggY2xhc3M9ImNscy0yIiBkPSJNNTIuMzcsMTkuNzRBMzEuNjIsMzEuNjIsMCwwLDAsMjQuNTgsNjYuNDFsLTUuNzIsMTguNEwzOS40LDgwLjE3YTMxLjYxLDMxLjYxLDAsMSwwLDEzLTYwLjQzWiIvPjxwYXRoIGNsYXNzPSJjbHMtMyIgZD0iTTc3LjQ1LDMyLjEyYTMxLjYsMzEuNiwwLDAsMS0zOC4wNSw0OEwxOC44Niw4NC44MmwyMC45MS0yLjQ3QTMxLjYsMzEuNiwwLDAsMCw3Ny40NSwzMi4xMloiLz48cGF0aCBjbGFzcz0iY2xzLTQiIGQ9Ik03MS42MywyNi4yOUEzMS42LDMxLjYsMCwwLDEsMzguOCw3OEwxOC44Niw4NC44MiwzOS40LDgwLjE3QTMxLjYsMzEuNiwwLDAsMCw3MS42MywyNi4yOVoiLz48cGF0aCBjbGFzcz0iY2xzLTUiIGQ9Ik0yNi40Nyw2Ny4xMWEzMS42MSwzMS42MSwwLDAsMSw1MS0zNUEzMS42MSwzMS42MSwwLDAsMCwyNC41OCw2Ni40MWwtNS43MiwxOC40WiIvPjxwYXRoIGNsYXNzPSJjbHMtNiIgZD0iTTI0LjU4LDY2LjQxQTMxLjYxLDMxLjYxLDAsMCwxLDcxLjYzLDI2LjI5YTMxLjYxLDMxLjYxLDAsMCwwLTQ5LDM5LjYzbC0zLjc2LDE4LjlaIi8+PC9nPjwvZz48L3N2Zz4="></a>
	
	</div>
<p>让我们测试一下您在本章中学到了什么！</p>
<h3 class="relative group"><a id="1.你应该什么时候训练一个新的标记器？" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#1.你应该什么时候训练一个新的标记器？"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>1.你应该什么时候训练一个新的标记器？
	</span></h3>

<div><form><label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="0">
				<!-- HTML_TAG_START -->当您的数据集与现有的预训练模型所使用的数据集相似时，并且您希望预训练一个新模型<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="1">
				<!-- HTML_TAG_START -->当您的数据集与现有的预训练模型所使用的数据集相似时，并且您希望使用此预训练模型对新模型进行微调<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="2">
				<!-- HTML_TAG_START -->当您的数据集与现有预训练模型所使用的数据集不同时，您希望预训练一个新模型<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="3">
				<!-- HTML_TAG_START -->当您的数据集与现有预训练模型所使用的数据集不同时，但是您希望使用此预训练模型对新模型进行微调<!-- HTML_TAG_END --></label>
			
		<div class="flex flex-row items-center mt-3"><button class="btn px-4 mr-4" type="submit" disabled>Submit </button>

			</div></form></div>
<h3 class="relative group"><a id="2.当使用“-train_new_from_iterator()”时，使用文本列表生成器与文本列表相比有什么优点？" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#2.当使用“-train_new_from_iterator()”时，使用文本列表生成器与文本列表相比有什么优点？"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>2.当使用“ train_new_from_iterator()”时，使用文本列表生成器与文本列表相比有什么优点？
	</span></h3>

<div><form><label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="0">
				<!-- HTML_TAG_START -->这是方法 &amp;lt; code > train_new_from_iterator() &amp;lt;/code > 接受的唯一类型。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="1">
				<!-- HTML_TAG_START -->您将避免立即将整个数据集载入内存中。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="2">
				<!-- HTML_TAG_START -->这将允许 Tokenizers 库使用并行处理。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="3">
				<!-- HTML_TAG_START -->你训练的标记器将产生更好的文本。<!-- HTML_TAG_END --></label>
			
		<div class="flex flex-row items-center mt-3"><button class="btn px-4 mr-4" type="submit" disabled>Submit </button>

			</div></form></div>
<h3 class="relative group"><a id="3.使用“快速”标记器的优点是什么？" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#3.使用“快速”标记器的优点是什么？"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>3.使用“快速”标记器的优点是什么？
	</span></h3>

<div><form><label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="0">
				<!-- HTML_TAG_START -->当你批处理大量的输入时，它可以比一个满速的标记器更快地处理输入。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="1">
				<!-- HTML_TAG_START -->快速的标记法总是比慢速的标记法快。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="2">
				<!-- HTML_TAG_START -->它可以填充和截断文本。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="3">
				<!-- HTML_TAG_START -->它有一些额外的功能，允许你将标记映射到创建它们的文本范围。<!-- HTML_TAG_END --></label>
			
		<div class="flex flex-row items-center mt-3"><button class="btn px-4 mr-4" type="submit" disabled>Submit </button>

			</div></form></div>
<h3 class="relative group"><a id="4.“token-classification”管道如何处理跨多个标记的实体？" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#4.“token-classification”管道如何处理跨多个标记的实体？"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>4.“token-classification”管道如何处理跨多个标记的实体？
	</span></h3>

<div><form><label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="0">
				<!-- HTML_TAG_START -->具有相同标签的实体合并为一个实体。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="1">
				<!-- HTML_TAG_START -->实体的开始有一个标签，实体的持续有一个标签。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="2">
				<!-- HTML_TAG_START -->在给定的单词中，只要第一个标记具有实体的标签，则认为整个单词都带有该实体的标签。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="3">
				<!-- HTML_TAG_START -->当一个标记具有给定实体的标记时，除非标记为新实体的开始，否则具有相同标记的任何其他后续标记都被视为同一实体的一部分。<!-- HTML_TAG_END --></label>
			
		<div class="flex flex-row items-center mt-3"><button class="btn px-4 mr-4" type="submit" disabled>Submit </button>

			</div></form></div>
<h3 class="relative group"><a id="5.“question-answering”流水线如何处理长上下文？" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#5.“question-answering”流水线如何处理长上下文？"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>5.“question-answering”流水线如何处理长上下文？
	</span></h3>

<div><form><label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="0">
				<!-- HTML_TAG_START -->实际上并不是这样，因为它在模型接受的最大长度上截断了长上下文。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="1">
				<!-- HTML_TAG_START -->它将上下文分成若干部分，并对所得结果进行平均。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="2">
				<!-- HTML_TAG_START -->它将上下文拆分为若干部分(有重叠部分) ，并在每个部分中查找一个答案的最大得分。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="3">
				<!-- HTML_TAG_START -->它将上下文分成若干部分(不重叠，以提高效率) ，并在每个部分中找到一个答案的最大得分。<!-- HTML_TAG_END --></label>
			
		<div class="flex flex-row items-center mt-3"><button class="btn px-4 mr-4" type="submit" disabled>Submit </button>

			</div></form></div>
<h3 class="relative group"><a id="6.什么是标准化？" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#6.什么是标准化？"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>6.什么是标准化？
	</span></h3>

<div><form><label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="0">
				<!-- HTML_TAG_START -->这是 tokenizer 在初始阶段对文本执行的任何清理。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="1">
				<!-- HTML_TAG_START -->这是一种数据增强技术，包括通过删除稀有单词使文本更加标准。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="2">
				<!-- HTML_TAG_START -->在最后的后处理步骤中，tokenizer 添加特殊标记。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="3">
				<!-- HTML_TAG_START -->这是当嵌入的平均值为0和标准差为1时，通过减去平均值和除以标准差。<!-- HTML_TAG_END --></label>
			
		<div class="flex flex-row items-center mt-3"><button class="btn px-4 mr-4" type="submit" disabled>Submit </button>

			</div></form></div>
<h3 class="relative group"><a id="7.什么是子词标记化的前标记化？" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#7.什么是子词标记化的前标记化？"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>7.什么是子词标记化的前标记化？
	</span></h3>

<div><form><label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="0">
				<!-- HTML_TAG_START -->这是标记化之前的步骤，应用数据增强(如随机遮罩)。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="1">
				<!-- HTML_TAG_START -->这是标记化之前的步骤，在这个步骤中，所需的清理操作应用于文本。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="2">
				<!-- HTML_TAG_START -->这是应用 tokenizer 模型之前的步骤，将输入拆分为单词。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="3">
				<!-- HTML_TAG_START -->这是应用 tokenizer 模型之前的步骤，将输入拆分为标记。<!-- HTML_TAG_END --></label>
			
		<div class="flex flex-row items-center mt-3"><button class="btn px-4 mr-4" type="submit" disabled>Submit </button>

			</div></form></div>
<h3 class="relative group"><a id="8.选择描述标记化-bpe-模式最准确的句子。" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#8.选择描述标记化-bpe-模式最准确的句子。"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>8.选择描述标记化 BPE 模式最准确的句子。
	</span></h3>

<div><form><label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="0">
				<!-- HTML_TAG_START -->BPE 是一个子词标记算法，从小词汇表开始，学习合并规则。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="1">
				<!-- HTML_TAG_START -->BPE 是一种子词标记算法，从大词汇表开始，逐步从中删除标记。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="2">
				<!-- HTML_TAG_START -->BPE 标记器通过合并最常见的一对标记来学习合并规则。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="3">
				<!-- HTML_TAG_START -->BPE 记号赋予器通过合并一对记号来学习合并规则，该记号最大化了特权频繁对和较少个别部分的分数。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="4">
				<!-- HTML_TAG_START -->BPE 通过将单词分割成字符，然后应用合并规则将单词分解成子单词。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="5">
				<!-- HTML_TAG_START -->BPE 通过从词汇表的开头找到最长的子词，然后在文本的其余部分重复这个过程，将单词转化为子词。<!-- HTML_TAG_END --></label>
			
		<div class="flex flex-row items-center mt-3"><button class="btn px-4 mr-4" type="submit" disabled>Submit </button>

			</div></form></div>
<h3 class="relative group"><a id="9.选择适用于-wordpiece-标记模型的句子。" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#9.选择适用于-wordpiece-标记模型的句子。"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>9.选择适用于 WordPiece 标记模型的句子。
	</span></h3>

<div><form><label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="0">
				<!-- HTML_TAG_START -->WordPiece 是一个子词标记算法，它从一个小词汇表开始，学习合并规则。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="1">
				<!-- HTML_TAG_START -->WordPiece 是一种子词标记算法，从大词汇表开始，逐步从中删除标记。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="2">
				<!-- HTML_TAG_START -->WordPiece 标记器通过合并最常见的两个标记来学习合并规则。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="3">
				<!-- HTML_TAG_START -->WordPiece 标记器通过合并两个标记来学习合并规则，这两个标记最大限度地提高了频繁出现的标记的分数，而这两个标记的独立部分出现频率较低。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="4">
				<!-- HTML_TAG_START -->根据模型，WordPiece 通过找到最有可能的切分符号，将单词标记为子单词。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="5">
				<!-- HTML_TAG_START -->WordPiece 通过从词汇表的开头找到最长的子词，然后在文本的其余部分重复这个过程，将单词转化为子词。<!-- HTML_TAG_END --></label>
			
		<div class="flex flex-row items-center mt-3"><button class="btn px-4 mr-4" type="submit" disabled>Submit </button>

			</div></form></div>
<h3 class="relative group"><a id="10.选择适用于-unigram-标记模式的句子。" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#10.选择适用于-unigram-标记模式的句子。"><span><svg class="" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg></span></a>
	<span>10.选择适用于 Unigram 标记模式的句子。
	</span></h3>

<div><form><label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="0">
				<!-- HTML_TAG_START -->Unigram 是一个子词标记算法，它从一个很小的词汇表开始学习合并规则。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="1">
				<!-- HTML_TAG_START -->Unigram 是一种子词标记算法，它从大词汇表开始，逐步从中删除标记。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="2">
				<!-- HTML_TAG_START -->Unigram 通过最小化在整个语料库中的损失来调整词汇量。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="3">
				<!-- HTML_TAG_START -->Unigram 通过保留最频繁的子词来调整它的词汇量。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="4">
				<!-- HTML_TAG_START -->根据模型，Unigram 通过找到最可能的分割符号来将词转化为子词。<!-- HTML_TAG_END --></label>
			<label class="block"><input autocomplete="off" class="form-input -mt-1.5 mr-2" name="choice" type="checkbox" value="5">
				<!-- HTML_TAG_START -->Unigram 通过将单词分割成字符，然后应用合并规则将其分解成子单词。<!-- HTML_TAG_END --></label>
			
		<div class="flex flex-row items-center mt-3"><button class="btn px-4 mr-4" type="submit" disabled>Submit </button>

			</div></form></div>


		<script type="module" data-hydrate="416x1i">
		import { start } from "/docs/course/pr_362/zh-CN/_app/start-hf-doc-builder.js";
		start({
			target: document.querySelector('[data-hydrate="416x1i"]').parentNode,
			paths: {"base":"/docs/course/pr_362/zh-CN","assets":"/docs/course/pr_362/zh-CN"},
			session: {},
			route: false,
			spa: false,
			trailing_slash: "never",
			hydrate: {
				status: 200,
				error: null,
				nodes: [
					import("/docs/course/pr_362/zh-CN/_app/pages/__layout.svelte-hf-doc-builder.js"),
						import("/docs/course/pr_362/zh-CN/_app/pages/chapter6/10.mdx-hf-doc-builder.js")
				],
				params: {}
			}
		});
	</script>
