did the chicken cross the road?\n\nTo get to the other side!'# The second time it is, so it goes fasterllm("Tell me a joke")    CPU times: user 571 ¬µs, sys: 43 ¬µs, total: 614 ¬µs    Wall time: 635 ¬µs    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side!'Let's now show an example of similarity cachingfrom gptcache import Cachefrom gptcache.adapter.api import init_similar_cachefrom langchain.cache import GPTCacheimport hashlibdef get_hashed_name(name):    return hashlib.sha256(name.encode()).hexdigest()def init_gptcache(cache_obj: Cache, llm: str):    hashed_llm = get_hashed_name(llm)    init_similar_cache(cache_obj=cache_obj, data_dir=f"similar_cache_{hashed_llm}")set_llm_cache(GPTCache(init_gptcache))# The first time, it is not yet in cache, so it should take longerllm("Tell me a joke")    CPU times: user 1.42 s, sys: 279 ms, total: 1.7 s    Wall time: 8.44 s    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'# This is an exact match, so it finds it in the cachellm("Tell me a joke")    CPU times: user 866 ms, sys: 20 ms, total: 886 ms    Wall time: 226 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'# This is not an exact match, but semantically within distance so it hits!llm("Tell me joke")    CPU times: user 853 ms, sys: 14.8 ms, total: 868 ms    Wall time: 224 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'Momento Cache‚ÄãUse Momento to cache prompts and responses.Requires momento to use, uncomment below to install:# !pip install momentoYou'll need to get a Momento auth token to use this class. This can either be passed in to a momento.CacheClient if you'd like to instantiate that directly, as a named parameter auth_token to MomentoChatMessageHistory.from_client_params, or can just be set as an environment variable MOMENTO_AUTH_TOKEN.from datetime import timedeltafrom langchain.cache import MomentoCachecache_name = "langchain"ttl =
This notebook covers how to cache results of individual LLM calls using different caches.
This notebook covers how to cache results of individual LLM calls using different caches. ->: did the chicken cross the road?\n\nTo get to the other side!'# The second time it is, so it goes fasterllm("Tell me a joke")    CPU times: user 571 ¬µs, sys: 43 ¬µs, total: 614 ¬µs    Wall time: 635 ¬µs    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side!'Let's now show an example of similarity cachingfrom gptcache import Cachefrom gptcache.adapter.api import init_similar_cachefrom langchain.cache import GPTCacheimport hashlibdef get_hashed_name(name):    return hashlib.sha256(name.encode()).hexdigest()def init_gptcache(cache_obj: Cache, llm: str):    hashed_llm = get_hashed_name(llm)    init_similar_cache(cache_obj=cache_obj, data_dir=f"similar_cache_{hashed_llm}")set_llm_cache(GPTCache(init_gptcache))# The first time, it is not yet in cache, so it should take longerllm("Tell me a joke")    CPU times: user 1.42 s, sys: 279 ms, total: 1.7 s    Wall time: 8.44 s    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'# This is an exact match, so it finds it in the cachellm("Tell me a joke")    CPU times: user 866 ms, sys: 20 ms, total: 886 ms    Wall time: 226 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'# This is not an exact match, but semantically within distance so it hits!llm("Tell me joke")    CPU times: user 853 ms, sys: 14.8 ms, total: 868 ms    Wall time: 224 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'Momento Cache‚ÄãUse Momento to cache prompts and responses.Requires momento to use, uncomment below to install:# !pip install momentoYou'll need to get a Momento auth token to use this class. This can either be passed in to a momento.CacheClient if you'd like to instantiate that directly, as a named parameter auth_token to MomentoChatMessageHistory.from_client_params, or can just be set as an environment variable MOMENTO_AUTH_TOKEN.from datetime import timedeltafrom langchain.cache import MomentoCachecache_name = "langchain"ttl =