= (        Index("idx_fulltext_prompt_tsv", prompt_tsv, postgresql_using="gin"),    )engine = create_engine("postgresql://postgres:postgres@localhost:5432/postgres")set_llm_cache(SQLAlchemyCache(engine, FulltextLLMCache))Cassandra caches‚ÄãYou can use Cassandra / Astra DB for caching LLM responses, choosing from the exact-match CassandraCache or the (vector-similarity-based) CassandraSemanticCache.Let's see both in action in the following cells.Connect to the DB‚ÄãFirst you need to establish a Session to the DB and to specify a keyspace for the cache table(s). The following gets you started with an Astra DB instance (see e.g. here for more backends and connection options).import getpasskeyspace = input("\nKeyspace name? ")ASTRA_DB_APPLICATION_TOKEN = getpass.getpass('\nAstra DB Token ("AstraCS:...") ')ASTRA_DB_SECURE_BUNDLE_PATH = input("Full path to your Secure Connect Bundle? ")        Keyspace name? my_keyspace        Astra DB Token ("AstraCS:...") ¬∑¬∑¬∑¬∑¬∑¬∑¬∑¬∑    Full path to your Secure Connect Bundle? /path/to/secure-connect-databasename.zipfrom cassandra.cluster import Clusterfrom cassandra.auth import PlainTextAuthProvidercluster = Cluster(    cloud={        "secure_connect_bundle": ASTRA_DB_SECURE_BUNDLE_PATH,    },    auth_provider=PlainTextAuthProvider("token", ASTRA_DB_APPLICATION_TOKEN),)session = cluster.connect()Exact cache‚ÄãThis will avoid invoking the LLM when the supplied prompt is exactly the same as one encountered already:from langchain.globals import set_llm_cachefrom langchain.cache import CassandraCacheset_llm_cache(CassandraCache(session=session, keyspace=keyspace))print(llm("Why is the Moon always showing the same side?"))            The Moon always shows the same side because it is tidally locked to Earth.    CPU times: user 41.7 ms, sys: 153 ¬µs, total: 41.8 ms    Wall time: 1.96 sprint(llm("Why is the Moon always showing the same side?"))            The Moon always shows the same side because it is tidally locked to Earth.    CPU
This notebook covers how to cache results of individual LLM calls using different caches.
This notebook covers how to cache results of individual LLM calls using different caches. ->: = (        Index("idx_fulltext_prompt_tsv", prompt_tsv, postgresql_using="gin"),    )engine = create_engine("postgresql://postgres:postgres@localhost:5432/postgres")set_llm_cache(SQLAlchemyCache(engine, FulltextLLMCache))Cassandra caches‚ÄãYou can use Cassandra / Astra DB for caching LLM responses, choosing from the exact-match CassandraCache or the (vector-similarity-based) CassandraSemanticCache.Let's see both in action in the following cells.Connect to the DB‚ÄãFirst you need to establish a Session to the DB and to specify a keyspace for the cache table(s). The following gets you started with an Astra DB instance (see e.g. here for more backends and connection options).import getpasskeyspace = input("\nKeyspace name? ")ASTRA_DB_APPLICATION_TOKEN = getpass.getpass('\nAstra DB Token ("AstraCS:...") ')ASTRA_DB_SECURE_BUNDLE_PATH = input("Full path to your Secure Connect Bundle? ")        Keyspace name? my_keyspace        Astra DB Token ("AstraCS:...") ¬∑¬∑¬∑¬∑¬∑¬∑¬∑¬∑    Full path to your Secure Connect Bundle? /path/to/secure-connect-databasename.zipfrom cassandra.cluster import Clusterfrom cassandra.auth import PlainTextAuthProvidercluster = Cluster(    cloud={        "secure_connect_bundle": ASTRA_DB_SECURE_BUNDLE_PATH,    },    auth_provider=PlainTextAuthProvider("token", ASTRA_DB_APPLICATION_TOKEN),)session = cluster.connect()Exact cache‚ÄãThis will avoid invoking the LLM when the supplied prompt is exactly the same as one encountered already:from langchain.globals import set_llm_cachefrom langchain.cache import CassandraCacheset_llm_cache(CassandraCache(session=session, keyspace=keyspace))print(llm("Why is the Moon always showing the same side?"))            The Moon always shows the same side because it is tidally locked to Earth.    CPU times: user 41.7 ms, sys: 153 ¬µs, total: 41.8 ms    Wall time: 1.96 sprint(llm("Why is the Moon always showing the same side?"))            The Moon always shows the same side because it is tidally locked to Earth.    CPU