Hey folks, it's Sydney from the Python

open source team at Langchain, and I'm

super excited to bring you our next

edition of our middleware series. So

model outages can be incredibly

unpredictable, but luckily with

Langchain's new model fallback

middleware, the reliability of your

application doesn't have to be. Whatever

the reason may be that your model calls

are no longer working, that could be a

provider outage or an API quota

exhausted, lingchain's model fallback

middleware can help you fall back to a

safe and functional model to keep your

application running. Let's take a look

at our new docs to check this out.

All right, so we're looking at our new

middleware docs. We have an overview

page, a list of all of our built-in

middlewares, both provider agnostic and

provider specific middlewares, and then

also a whole page on how you can create

your own custom middlewares. So, we're

looking at the model fallback

middleware, which as mentioned can help

you build more resilient agents that can

handle model outages, falling back when

you run out of API credits for a given

model. And then specifically one benefit

of using lingchain here is that you can

fall back to models that have different

providers. So you can have a primary

open AI model and then fall back to

anthropic model etc. So specifying a

fallback model is as simple as providing

your primary model to create agent and

then adding this model fallback

middleware with any [snorts] number of

fallbacks that you'd like. Let's take a

look at this in action. So here I have a

super simple script. I'm using lang

chain's create agent primitive to

simulate model failure. I'm just using

anthropic model that doesn't exist and

then I also have a fallback model that

doesn't exist and then our final model

that we expect to work which is GPT4

mini from openAI. So let's demo this in

studio. So we have a super simple agent

graph and I'm just going to ask the

model what's 15 * 23.

We can look at our linksmith trace view

here to get a good idea of what's

happening. So we see the model is called

with our human message input. What's 15*

23? We see that initial call to our

chatanthropic model fails. It's because

we have that typo. But this could be in

real life like an outage or an API

credit limit. Then we see our next

fallback model also simulated with a

typo here fails. And then finally, we

fall back to our last GPT 40 mini model

and get the expected result. Great. So

this has been a quick demo of

Lingchain's model fallback middleware.

We're super excited to see the

resilience that you build into your

applications with this.