(the classic joke answer!)\n\n- It was trying to find some food or water \n\n- It was trying to find a mate during mating season\n\n- It was fleeing from a predator or perceived threat\n\n- It was disoriented and crossed accidentally \n\n- It was following a herd of other kangaroos who were crossing\n\n- It wanted a change of scenery or environment \n\n- It was trying to reach a new habitat or territory\n\nThe real reason is unknown without more context, but hopefully one of those potential explanations does the joke justice! Let me know if you have any other animal jokes I can try to decipher." additional_kwargs={} example=FalseFallback for Sequences‚ÄãWe can also create fallbacks for sequences, that are sequences themselves. Here we do that with two different models: ChatOpenAI and then normal OpenAI (which does not use a chat model). Because OpenAI is NOT a chat model, you likely want a different prompt.# First let's create a chain with a ChatModel# We add in a string output parser here so the outputs between the two are the same typefrom langchain.schema.output_parser import StrOutputParserchat_prompt = ChatPromptTemplate.from_messages(    [        ("system", "You're a nice assistant who always includes a compliment in your response"),        ("human", "Why did the {animal} cross the road"),    ])# Here we're going to use a bad model name to easily create a chain that will errorchat_model = ChatOpenAI(model_name="gpt-fake")bad_chain = chat_prompt | chat_model | StrOutputParser()# Now lets create a chain with the normal OpenAI modelfrom langchain.llms import OpenAIfrom langchain.prompts import PromptTemplateprompt_template = """Instructions: You should always include a compliment in your response.Question: Why did the {animal} cross the road?"""prompt = PromptTemplate.from_template(prompt_template)llm = OpenAI()good_chain = prompt | llm# We can now create a final chain which combines the twochain = bad_chain.with_fallbacks([good_chain])chain.invoke({"animal":
When working with language models, you may often encounter issues from the underlying APIs, whether these be rate limiting or downtime. Therefore, as you go to move your LLM applications into production it becomes more and more important to safeguard against these. That's why we've introduced the concept of fallbacks.
When working with language models, you may often encounter issues from the underlying APIs, whether these be rate limiting or downtime. Therefore, as you go to move your LLM applications into production it becomes more and more important to safeguard against these. That's why we've introduced the concept of fallbacks. ->: (the classic joke answer!)\n\n- It was trying to find some food or water \n\n- It was trying to find a mate during mating season\n\n- It was fleeing from a predator or perceived threat\n\n- It was disoriented and crossed accidentally \n\n- It was following a herd of other kangaroos who were crossing\n\n- It wanted a change of scenery or environment \n\n- It was trying to reach a new habitat or territory\n\nThe real reason is unknown without more context, but hopefully one of those potential explanations does the joke justice! Let me know if you have any other animal jokes I can try to decipher." additional_kwargs={} example=FalseFallback for Sequences‚ÄãWe can also create fallbacks for sequences, that are sequences themselves. Here we do that with two different models: ChatOpenAI and then normal OpenAI (which does not use a chat model). Because OpenAI is NOT a chat model, you likely want a different prompt.# First let's create a chain with a ChatModel# We add in a string output parser here so the outputs between the two are the same typefrom langchain.schema.output_parser import StrOutputParserchat_prompt = ChatPromptTemplate.from_messages(    [        ("system", "You're a nice assistant who always includes a compliment in your response"),        ("human", "Why did the {animal} cross the road"),    ])# Here we're going to use a bad model name to easily create a chain that will errorchat_model = ChatOpenAI(model_name="gpt-fake")bad_chain = chat_prompt | chat_model | StrOutputParser()# Now lets create a chain with the normal OpenAI modelfrom langchain.llms import OpenAIfrom langchain.prompts import PromptTemplateprompt_template = """Instructions: You should always include a compliment in your response.Question: Why did the {animal} cross the road?"""prompt = PromptTemplate.from_template(prompt_template)llm = OpenAI()good_chain = prompt | llm# We can now create a final chain which combines the twochain = bad_chain.with_fallbacks([good_chain])chain.invoke({"animal":