"turtle"})    '\n\nAnswer: The turtle crossed the road to get to the other side, and I have to say he had some impressive determination.'Fallback for Long Inputs‚ÄãOne of the big limiting factors of LLMs is their context window. Usually, you can count and track the length of prompts before sending them to an LLM, but in situations where that is hard/complicated, you can fallback to a model with a longer context length.short_llm = ChatOpenAI()long_llm = ChatOpenAI(model="gpt-3.5-turbo-16k")llm = short_llm.with_fallbacks([long_llm])inputs = "What is the next number: " + ", ".join(["one", "two"] * 3000)try:    print(short_llm.invoke(inputs))except Exception as e:    print(e)    This model's maximum context length is 4097 tokens. However, your messages resulted in 12012 tokens. Please reduce the length of the messages.try:    print(llm.invoke(inputs))except Exception as e:    print(e)    content='The next number in the sequence is two.' additional_kwargs={} example=FalseFallback to Better Model‚ÄãOften times we ask models to output format in a specific format (like JSON). Models like GPT-3.5 can do this okay, but sometimes struggle. This naturally points to fallbacks - we can try with GPT-3.5 (faster, cheaper), but then if parsing fails we can use GPT-4.from langchain.output_parsers import DatetimeOutputParserprompt = ChatPromptTemplate.from_template(    "what time was {event} (in %Y-%m-%dT%H:%M:%S.%fZ format - only return this value)")# In this case we are going to do the fallbacks on the LLM + output parser level# Because the error will get raised in the OutputParseropenai_35 = ChatOpenAI() | DatetimeOutputParser()openai_4 = ChatOpenAI(model="gpt-4")| DatetimeOutputParser()only_35 = prompt | openai_35 fallback_4 = prompt | openai_35.with_fallbacks([openai_4])try:    print(only_35.invoke({"event": "the superbowl in 1994"}))except Exception as e:    print(f"Error: {e}")    Error: Could not parse datetime string: The Super Bowl in 1994 took place on January 30th at 3:30
When working with language models, you may often encounter issues from the underlying APIs, whether these be rate limiting or downtime. Therefore, as you go to move your LLM applications into production it becomes more and more important to safeguard against these. That's why we've introduced the concept of fallbacks.
When working with language models, you may often encounter issues from the underlying APIs, whether these be rate limiting or downtime. Therefore, as you go to move your LLM applications into production it becomes more and more important to safeguard against these. That's why we've introduced the concept of fallbacks. ->: "turtle"})    '\n\nAnswer: The turtle crossed the road to get to the other side, and I have to say he had some impressive determination.'Fallback for Long Inputs‚ÄãOne of the big limiting factors of LLMs is their context window. Usually, you can count and track the length of prompts before sending them to an LLM, but in situations where that is hard/complicated, you can fallback to a model with a longer context length.short_llm = ChatOpenAI()long_llm = ChatOpenAI(model="gpt-3.5-turbo-16k")llm = short_llm.with_fallbacks([long_llm])inputs = "What is the next number: " + ", ".join(["one", "two"] * 3000)try:    print(short_llm.invoke(inputs))except Exception as e:    print(e)    This model's maximum context length is 4097 tokens. However, your messages resulted in 12012 tokens. Please reduce the length of the messages.try:    print(llm.invoke(inputs))except Exception as e:    print(e)    content='The next number in the sequence is two.' additional_kwargs={} example=FalseFallback to Better Model‚ÄãOften times we ask models to output format in a specific format (like JSON). Models like GPT-3.5 can do this okay, but sometimes struggle. This naturally points to fallbacks - we can try with GPT-3.5 (faster, cheaper), but then if parsing fails we can use GPT-4.from langchain.output_parsers import DatetimeOutputParserprompt = ChatPromptTemplate.from_template(    "what time was {event} (in %Y-%m-%dT%H:%M:%S.%fZ format - only return this value)")# In this case we are going to do the fallbacks on the LLM + output parser level# Because the error will get raised in the OutputParseropenai_35 = ChatOpenAI() | DatetimeOutputParser()openai_4 = ChatOpenAI(model="gpt-4")| DatetimeOutputParser()only_35 = prompt | openai_35 fallback_4 = prompt | openai_35.with_fallbacks([openai_4])try:    print(only_35.invoke({"event": "the superbowl in 1994"}))except Exception as e:    print(f"Error: {e}")    Error: Could not parse datetime string: The Super Bowl in 1994 took place on January 30th at 3:30