Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãORetrievalDocument loadersDocument transformersText splittersPost retrievalText embedding modelsVector storesRetrieversIndexingChainsMemoryAgentsCallbacksModulesSecurityGuidesMoreModulesRetrievalDocument transformersOn this pageDocument transformersinfoHead to Integrations for documentation on built-in document transformer integrations with 3rd-party tools.Once you've loaded documents, you'll often want to transform them to better suit your application. The simplest example
is you may want to split a long document into smaller chunks that can fit into your model's context window. LangChain
has a number of built-in document transformers that make it easy to split, combine, filter, and otherwise manipulate documents.Text splitters‚ÄãWhen you want to deal with long pieces of text, it is necessary to split up that text into chunks.
As simple as this sounds, there is a lot of potential complexity here. Ideally, you want to keep the semantically related pieces of text together. What "semantically related" means could depend on the type of text.
Head to Integrations for documentation on built-in document transformer integrations with 3rd-party tools.
Head to Integrations for documentation on built-in document transformer integrations with 3rd-party tools. ->: Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãORetrievalDocument loadersDocument transformersText splittersPost retrievalText embedding modelsVector storesRetrieversIndexingChainsMemoryAgentsCallbacksModulesSecurityGuidesMoreModulesRetrievalDocument transformersOn this pageDocument transformersinfoHead to Integrations for documentation on built-in document transformer integrations with 3rd-party tools.Once you've loaded documents, you'll often want to transform them to better suit your application. The simplest example
is you may want to split a long document into smaller chunks that can fit into your model's context window. LangChain
has a number of built-in document transformers that make it easy to split, combine, filter, and otherwise manipulate documents.Text splitters‚ÄãWhen you want to deal with long pieces of text, it is necessary to split up that text into chunks.
As simple as this sounds, there is a lot of potential complexity here. Ideally, you want to keep the semantically related pieces of text together. What "semantically related" means could depend on the type of text.