to handle if we want to be more specific about when the fallback is invoked:llm = openai_llm.with_fallbacks([anthropic_llm], exceptions_to_handle=(KeyboardInterrupt,))chain = prompt | llmwith patch('openai.ChatCompletion.create', side_effect=RateLimitError()):    try:         print(chain.invoke({"animal": "kangaroo"}))    except:        print("Hit error")    Hit errorFallbacks for Sequences‚ÄãWe can also create fallbacks for sequences, that are sequences themselves. Here we do that with two different models: ChatOpenAI and then normal OpenAI (which does not use a chat model). Because OpenAI is NOT a chat model, you likely want a different prompt.# First let's create a chain with a ChatModel# We add in a string output parser here so the outputs between the two are the same typefrom langchain.schema.output_parser import StrOutputParserchat_prompt = ChatPromptTemplate.from_messages(    [        ("system", "You're a nice assistant who always includes a compliment in your response"),        ("human", "Why did the {animal} cross the road"),    ])# Here we're going to use a bad model name to easily create a chain that will errorchat_model = ChatOpenAI(model_name="gpt-fake")bad_chain = chat_prompt | chat_model | StrOutputParser()# Now lets create a chain with the normal OpenAI modelfrom langchain.llms import OpenAIfrom langchain.prompts import PromptTemplateprompt_template = """Instructions: You should always include a compliment in your response.Question: Why did the {animal} cross the road?"""prompt = PromptTemplate.from_template(prompt_template)llm = OpenAI()good_chain = prompt | llm# We can now create a final chain which combines the twochain = bad_chain.with_fallbacks([good_chain])chain.invoke({"animal": "turtle"})    '\n\nAnswer: The turtle crossed the road to get to the other side, and I have to say he had some impressive determination.'PreviousConfigurable AlternativesNextRun arbitrary functionsHandling LLM API ErrorsSpecifying errors to handleFallbacks for
There are many possible points of failure in an LLM application, whether that be issues with LLM API's, poor model outputs, issues with other integrations, etc. Fallbacks help you gracefully handle and isolate these issues.
There are many possible points of failure in an LLM application, whether that be issues with LLM API's, poor model outputs, issues with other integrations, etc. Fallbacks help you gracefully handle and isolate these issues. ->: to handle if we want to be more specific about when the fallback is invoked:llm = openai_llm.with_fallbacks([anthropic_llm], exceptions_to_handle=(KeyboardInterrupt,))chain = prompt | llmwith patch('openai.ChatCompletion.create', side_effect=RateLimitError()):    try:         print(chain.invoke({"animal": "kangaroo"}))    except:        print("Hit error")    Hit errorFallbacks for Sequences‚ÄãWe can also create fallbacks for sequences, that are sequences themselves. Here we do that with two different models: ChatOpenAI and then normal OpenAI (which does not use a chat model). Because OpenAI is NOT a chat model, you likely want a different prompt.# First let's create a chain with a ChatModel# We add in a string output parser here so the outputs between the two are the same typefrom langchain.schema.output_parser import StrOutputParserchat_prompt = ChatPromptTemplate.from_messages(    [        ("system", "You're a nice assistant who always includes a compliment in your response"),        ("human", "Why did the {animal} cross the road"),    ])# Here we're going to use a bad model name to easily create a chain that will errorchat_model = ChatOpenAI(model_name="gpt-fake")bad_chain = chat_prompt | chat_model | StrOutputParser()# Now lets create a chain with the normal OpenAI modelfrom langchain.llms import OpenAIfrom langchain.prompts import PromptTemplateprompt_template = """Instructions: You should always include a compliment in your response.Question: Why did the {animal} cross the road?"""prompt = PromptTemplate.from_template(prompt_template)llm = OpenAI()good_chain = prompt | llm# We can now create a final chain which combines the twochain = bad_chain.with_fallbacks([good_chain])chain.invoke({"animal": "turtle"})    '\n\nAnswer: The turtle crossed the road to get to the other side, and I have to say he had some impressive determination.'PreviousConfigurable AlternativesNextRun arbitrary functionsHandling LLM API ErrorsSpecifying errors to handleFallbacks for