Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãOPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersRetrievalChainsMemoryAgentsCallbacksModulesSecurityGuidesMoreModulesModel I/‚ÄãOLanguage modelsLLMsAsync APIAsync APILangChain provides async support for LLMs by leveraging the asyncio library.Async support is particularly useful for calling multiple LLMs concurrently, as these calls are network-bound. Currently, OpenAI, PromptLayerOpenAI, ChatOpenAI, Anthropic and Cohere are supported, but async support for other LLMs is on the roadmap.You can use the agenerate method to call an OpenAI LLM asynchronously.import timeimport asynciofrom langchain.llms import OpenAIdef generate_serially():    llm = OpenAI(temperature=0.9)    for _ in range(10):        resp = llm.generate(["Hello, how are you?"])        print(resp.generations[0][0].text)async def async_generate(llm):    resp = await llm.agenerate(["Hello, how are you?"])    print(resp.generations[0][0].text)async def generate_concurrently():    llm = OpenAI(temperature=0.9)    tasks = [async_generate(llm) for _ in range(10)]    await asyncio.gather(*tasks)s = time.perf_counter()# If running this outside of Jupyter, use asyncio.run(generate_concurrently())await generate_concurrently()elapsed = time.perf_counter() - sprint("\033[1m" + f"Concurrent executed in {elapsed:0.2f} seconds." + "\033[0m")s = time.perf_counter()generate_serially()elapsed = time.perf_counter() - sprint("\033[1m" + f"Serial executed in {elapsed:0.2f} seconds." + "\033[0m")            I'm doing well, thank you. How about you?            I'm doing well, thank you. How about you?            I'm doing well, how about you?
LangChain provides async support for LLMs by leveraging the asyncio library.
LangChain provides async support for LLMs by leveraging the asyncio library. ->: Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãOPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersRetrievalChainsMemoryAgentsCallbacksModulesSecurityGuidesMoreModulesModel I/‚ÄãOLanguage modelsLLMsAsync APIAsync APILangChain provides async support for LLMs by leveraging the asyncio library.Async support is particularly useful for calling multiple LLMs concurrently, as these calls are network-bound. Currently, OpenAI, PromptLayerOpenAI, ChatOpenAI, Anthropic and Cohere are supported, but async support for other LLMs is on the roadmap.You can use the agenerate method to call an OpenAI LLM asynchronously.import timeimport asynciofrom langchain.llms import OpenAIdef generate_serially():    llm = OpenAI(temperature=0.9)    for _ in range(10):        resp = llm.generate(["Hello, how are you?"])        print(resp.generations[0][0].text)async def async_generate(llm):    resp = await llm.agenerate(["Hello, how are you?"])    print(resp.generations[0][0].text)async def generate_concurrently():    llm = OpenAI(temperature=0.9)    tasks = [async_generate(llm) for _ in range(10)]    await asyncio.gather(*tasks)s = time.perf_counter()# If running this outside of Jupyter, use asyncio.run(generate_concurrently())await generate_concurrently()elapsed = time.perf_counter() - sprint("\033[1m" + f"Concurrent executed in {elapsed:0.2f} seconds." + "\033[0m")s = time.perf_counter()generate_serially()elapsed = time.perf_counter() - sprint("\033[1m" + f"Serial executed in {elapsed:0.2f} seconds." + "\033[0m")            I'm doing well, thank you. How about you?            I'm doing well, thank you. How about you?            I'm doing well, how about you?