Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKProvidersAnthropicAWSGoogleMicrosoftOpenAIMoreComponentsLLMsChat modelsDocument loadersDocument transformersBeautiful SoupGoogle Cloud Document AIDoctran: extract propertiesDoctran: interrogate documentsDoctran: language translationHTML to textNucliaOpenAI metadata taggerText embedding modelsVector storesRetrieversToolsAgents and toolkitsMemoryCallbacksChat loadersComponentsDocument transformersDoctran: language translationOn this pageDoctran: language translationComparing documents through embeddings has the benefit of working across multiple languages. "Harrison says hello" and "Harrison dice hola" will occupy similar positions in the vector space because they have the same meaning semantically.However, it can still be useful to use an LLM to translate documents into other languages before vectorizing them. This is especially helpful when users are expected to query the knowledge base in different languages, or when state-of-the-art embedding models are not available for a given language.We can accomplish this using the Doctran library, which uses OpenAI's function calling feature to translate documents between languages.pip install doctranfrom langchain.schema import Documentfrom langchain.document_transformers import DoctranTextTranslatorfrom dotenv import load_dotenvload_dotenv()    TrueInput‚ÄãThis is the document we'll translatesample_text = """[Generated with ChatGPT]Confidential Document - For Internal Use OnlyDate: July 1, 2023Subject: Updates and Discussions on Various TopicsDear Team,I hope this email finds you well. In this document, I would like to provide you with some important updates and discuss various topics that require our attention. Please treat the information contained herein as highly confidential.Security and Privacy MeasuresAs part of our ongoing commitment to ensure the security and privacy of our customers' data, we
Comparing documents through embeddings has the benefit of working across multiple languages. "Harrison says hello" and "Harrison dice hola" will occupy similar positions in the vector space because they have the same meaning semantically.
Comparing documents through embeddings has the benefit of working across multiple languages. "Harrison says hello" and "Harrison dice hola" will occupy similar positions in the vector space because they have the same meaning semantically. ->: Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKProvidersAnthropicAWSGoogleMicrosoftOpenAIMoreComponentsLLMsChat modelsDocument loadersDocument transformersBeautiful SoupGoogle Cloud Document AIDoctran: extract propertiesDoctran: interrogate documentsDoctran: language translationHTML to textNucliaOpenAI metadata taggerText embedding modelsVector storesRetrieversToolsAgents and toolkitsMemoryCallbacksChat loadersComponentsDocument transformersDoctran: language translationOn this pageDoctran: language translationComparing documents through embeddings has the benefit of working across multiple languages. "Harrison says hello" and "Harrison dice hola" will occupy similar positions in the vector space because they have the same meaning semantically.However, it can still be useful to use an LLM to translate documents into other languages before vectorizing them. This is especially helpful when users are expected to query the knowledge base in different languages, or when state-of-the-art embedding models are not available for a given language.We can accomplish this using the Doctran library, which uses OpenAI's function calling feature to translate documents between languages.pip install doctranfrom langchain.schema import Documentfrom langchain.document_transformers import DoctranTextTranslatorfrom dotenv import load_dotenvload_dotenv()    TrueInput‚ÄãThis is the document we'll translatesample_text = """[Generated with ChatGPT]Confidential Document - For Internal Use OnlyDate: July 1, 2023Subject: Updates and Discussions on Various TopicsDear Team,I hope this email finds you well. In this document, I would like to provide you with some important updates and discuss various topics that require our attention. Please treat the information contained herein as highly confidential.Security and Privacy MeasuresAs part of our ongoing commitment to ensure the security and privacy of our customers' data, we