It can speed up your application by reducing the number of API calls you make to the LLM provider.from langchain.globals import set_llm_cachefrom langchain.chat_models import ChatOpenAIllm = ChatOpenAI()In Memory Cache‚Äãfrom langchain.cache import InMemoryCacheset_llm_cache(InMemoryCache())# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")    CPU times: user 35.9 ms, sys: 28.6 ms, total: 64.6 ms    Wall time: 4.83 s    "\n\nWhy couldn't the bicycle stand up by itself? It was...two tired!"# The second time it is, so it goes fasterllm.predict("Tell me a joke")    CPU times: user 238 ¬µs, sys: 143 ¬µs, total: 381 ¬µs    Wall time: 1.76 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'SQLite Cache‚Äãrm .langchain.db# We can do the same thing with a SQLite cachefrom langchain.cache import SQLiteCacheset_llm_cache(SQLiteCache(database_path=".langchain.db"))# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")    CPU times: user 17 ms, sys: 9.76 ms, total: 26.7 ms    Wall time: 825 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'# The second time it is, so it goes fasterllm.predict("Tell me a joke")    CPU times: user 2.46 ms, sys: 1.23 ms, total: 3.7 ms    Wall time: 2.67 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'PreviousChat modelsNextHuman input chat modelIn Memory CacheSQLite CacheCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.
LangChain provides an optional caching layer for chat models. This is useful for two reasons:
LangChain provides an optional caching layer for chat models. This is useful for two reasons: ->: It can speed up your application by reducing the number of API calls you make to the LLM provider.from langchain.globals import set_llm_cachefrom langchain.chat_models import ChatOpenAIllm = ChatOpenAI()In Memory Cache‚Äãfrom langchain.cache import InMemoryCacheset_llm_cache(InMemoryCache())# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")    CPU times: user 35.9 ms, sys: 28.6 ms, total: 64.6 ms    Wall time: 4.83 s    "\n\nWhy couldn't the bicycle stand up by itself? It was...two tired!"# The second time it is, so it goes fasterllm.predict("Tell me a joke")    CPU times: user 238 ¬µs, sys: 143 ¬µs, total: 381 ¬µs    Wall time: 1.76 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'SQLite Cache‚Äãrm .langchain.db# We can do the same thing with a SQLite cachefrom langchain.cache import SQLiteCacheset_llm_cache(SQLiteCache(database_path=".langchain.db"))# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")    CPU times: user 17 ms, sys: 9.76 ms, total: 26.7 ms    Wall time: 825 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'# The second time it is, so it goes fasterllm.predict("Tell me a joke")    CPU times: user 2.46 ms, sys: 1.23 ms, total: 3.7 ms    Wall time: 2.67 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'PreviousChat modelsNextHuman input chat modelIn Memory CacheSQLite CacheCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.