*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to cache LLM responses

On this page

How to cache LLM responses
==========================

LangChain provides an optional caching layer for LLMs. This is useful for two reasons:

It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times. It can speed up your application by reducing the number of API calls you make to the LLM provider.

    from langchain.globals import set_llm_cachefrom langchain_openai import OpenAI# To make the caching really obvious, lets use a slower model.llm = OpenAI(model_name="gpt-3.5-turbo-instruct", n=2, best_of=2)

**API Reference:**[set\_llm\_cache](https://api.python.langchain.com/en/latest/globals/langchain.globals.set_llm_cache.html) | [OpenAI](https://api.python.langchain.com/en/latest/llms/langchain_openai.llms.base.OpenAI.html)

    %%timefrom langchain.cache import InMemoryCacheset_llm_cache(InMemoryCache())# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")

**API Reference:**[InMemoryCache](https://api.python.langchain.com/en/latest/cache/langchain_community.cache.InMemoryCache.html)

    CPU times: user 13.7 ms, sys: 6.54 ms, total: 20.2 msWall time: 330 ms

    "\n\nWhy couldn't the bicycle stand up by itself? Because it was two-tired!"

    %%time# The second time it is, so it goes fasterllm.predict("Tell me a joke")

    CPU times: user 436 µs, sys: 921 µs, total: 1.36 msWall time: 1.36 ms

    "\n\nWhy couldn't the bicycle stand up by itself? Because it was two-tired!"

SQLite Cache[​](#sqlite-cache "Direct link to SQLite Cache")
------------------------------------------------------------

    !rm .langchain.db

    # We can do the same thing with a SQLite cachefrom langchain_community.cache import SQLiteCacheset_llm_cache(SQLiteCache(database_path=".langchain.db"))

**API Reference:**[SQLiteCache](https://api.python.langchain.com/en/latest/cache/langchain_community.cache.SQLiteCache.html)

    %%time# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")

    CPU times: user 29.3 ms, sys: 17.3 ms, total: 46.7 msWall time: 364 ms

    '\n\nWhy did the tomato turn red?\n\nBecause it saw the salad dressing!'

    %%time# The second time it is, so it goes fasterllm.predict("Tell me a joke")

    CPU times: user 4.58 ms, sys: 2.23 ms, total: 6.8 msWall time: 4.68 ms

    '\n\nWhy did the tomato turn red?\n\nBecause it saw the salad dressing!'

[Edit this page](https://github.com/langchain-ai/langchain/edit/master/docs/docs/how_to/llm_caching.ipynb)

* * *

#### Was this page helpful?

  

#### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchain/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

LangChain Expression Language Cheatsheet

](/v0.2/docs/how_to/lcel_cheatsheet/)[

Next

How to track token usage for LLMs

](/v0.2/docs/how_to/llm_token_usage_tracking/)

*   [SQLite Cache](#sqlite-cache)