langchain_core.runnables.fallbacks.RunnableWithFallbacks¶
class langchain_core.runnables.fallbacks.RunnableWithFallbacks[source]¶
Bases: RunnableSerializable[Input, Output]
A Runnable that can fallback to other Runnables if it fails.
External APIs (e.g., APIs for a language model) may at times experience
degraded performance or even downtime.
In these cases, it can be useful to have a fallback runnable that can be
used in place of the original runnable (e.g., fallback to another LLM provider).
Fallbacks can be defined at the level of a single runnable, or at the level
of a chain of runnables. Fallbacks are tried in order until one succeeds or
all fail.
While you can instantiate a RunnableWithFallbacks directly, it is usually
more convenient to use the with_fallbacks method on a runnable.
Example
from langchain_core.chat_models.openai import ChatOpenAI
from langchain_core.chat_models.anthropic import ChatAnthropic
model = ChatAnthropic().with_fallbacks([ChatOpenAI()])
# Will usually use ChatAnthropic, but fallback to ChatOpenAI
# if ChatAnthropic fails.
model.invoke('hello')
# And you can also use fallbacks at the level of a chain.
# Here if both LLM providers fail, we'll fallback to a good hardcoded
# response.
from langchain_core.prompts import PromptTemplate
from langchain_core.output_parser import StrOutputParser
from langchain_core.runnables import RunnableLambda
def when_all_is_lost(inputs):
    return ("Looks like our LLM providers are down. "
            "Here's a nice 🦜️ emoji for you instead.")
chain_with_fallback = (
    PromptTemplate.from_template('Tell me a joke about {topic}')