Once we have this, we can run it. And as

each node runs, it emits events um or

traces. And as this sort of events come

out, you can sort of look at them here.

And essentially, you can also see that

visualization that sort of shows you as

a developer what's happening at each

time as your entire workflow gets

executed.

Hello everyone, welcome back to Open at

Microsoft show. We have Evan and Victor

back here to talk about Microsoft agent

framework.

Hey guys. Uh yeah, what are you going to

talk tonight? Um on this episode, Evan,

it will be workflows.

Tell me.

>> Yes. Yes, we are going to bring you

another series here about agent

framework. We are super excited to

introduce it uh at the beginning of

October. I have some wonderful feedback

so far as we work through public

preview. Um, but today we are going to

dive into agent framework workflows. So,

Victor and I are going to present these

to you today, explain why you need

workflows, the benefits of workflows,

the abstractions and patterns that we

support. And so, we are going to dive

into that. I guess you know first my

name is Evan Matson. I'm a principal

software engineer on the engine

framework team.

We have Victor with us. Victor, do you

want to say hello?

>> Hi everyone. Uh my name is Victor Dvia.

I'm a principal research software

engineer. Um I used to be on the autogen

team and now I'm on the core team and

we're all sort of working together to

make agent framework um a really really

strong compelling experience for

everyone to build the AI agents

>> and then Evan that's a nice thing Victor

and Evan like people to understand that

those teams now Sam Kano and Autogen

they're working together so we have a

stronger team working on Microsoft agent

framework

And that's a big, you know, advantage I

think going forward.

>> Well, we are all excited to see what's

coming and understand a little bit more

about workflows.

>> Yes. Yes. And I just have to say I'm

super excited that our teams are now

together and uh we get to use the

experience of both sides and bring this

amazing framework uh to the community.

So let's let's get started. Uh let's

talk about workflows.

you know for for those coming from

semantic kernel to agent framework

workflows they're going to be some of

these abstractions that look familiar

but I do want to say it's workflows is a

complete uh rewrite of process framework

you know we received amazing feedback

from it uh but also knew that there were

some shortcomings and so we wanted to

provide a better developer experience um

and so we're just going to talk about

some highle some I I guess high level

but core abstractions Um, and so just

want to communicate that a a workflow is

a deterministic orchestration of of

executors or nodes through uh edges that

you can configure. And so they're

they're really they they excel when you

need more than just prompting. you know,

agents can work well together and go off

and and figure out, you know, uh work on

a task for you, but sometimes you need

explicit uh, you know, deterministics

stepping through a DAG or or a graph.

You want the step A to lead to step B to

lead to step C. You want to be ability

to fan out. You want the ability to fan

in. You have these more complex

scenarios you need to represent. Your

business process isn't potentially just

linear. Um, and so you have these these

abstractions that let you put these

pieces together and build this really

complex workflow that is audible,

stateful, and and things like that. And

so we're we're introducing these

executors. We're using this executor.

It's a base class that uh can consume

these typed inputs and expose these

these annotated handlers. We're really

putting this type kind of type safety

typing you know heavily into the API

because we want it to be robust. We want

there to be upfront validation which

we'll talk about soon and we want really

make the experience so much better uh

and than what we at least had in

semantic kernel side and so that if you

don't wire things up properly uh you'll

know up front. And another component is

the workflow context. And so this is a

allowing you to send messages between

executors, yield output,

uh provide results, and and we'll dive

into more of that in in a couple slides.

And one thing I I really like about

these these new APIs is the fluent

builder notion. And so you're building,

you have your workflow builder, you're

be a able to add these edges. You can

set the start executor and build it all

in this really nice concise code. Really

easy to understand as you're putting it

together. It's not extremely verbose,

but it it allows you just to understand

how you're plugging these, you know,

these different methods together to

build that that graph. And calling out

too that part of the core abstractions,

we're supporting conditional edges.

We're allowing easy fan in, fan out.

That's what I mentioned earlier about

representing your business process now

as this AI workflow. And importantly,

it's supported today in Python and .NET.

So for those developers who are working

in those languages, you can get started

really easily with those two languages.

>> And that's what when people asking about

other languages, some people about

asking about Java and other things. I

think it's a it's a open source project.

We need more people to contribute and

see how that you know can grow in the

future. I think

>> yes. So let's let's continue from these

these core abstractions and actually

look at what an executor and an edge uh

you know look like in Python. Uh as you

can see on the screen so the the

uppercase uh is a subclass of an

exeutor. So this is something you write

your b you bring your business logic you

have some need this is your node and you

can have message uh you know message

handlers here and you can see we're

decorating the two uppercase method with

a handler so we know that this is going

to be part of our graph uh it's taking

in a text type string and so we have

that text type uh string you know

annotated here and then we have a

workflow context and that's also

explaining that we have it typed as a

string. That means that it's going to

send a message that is of a string type.

And as we wire these together, again,

we're really putting emphasis on this

typed message handling. This allows us

to figure out, can this node send

information to the next node? Can it

receive this type? If it can't, we can

validate that up front and we can

provide the user with a useful log

message or error message if things

aren't connecting properly. So that

that's an example of a of a subclass uh

executor. You can also just bring a pure

function style executor and decorate it

with the executor um decorator as you

can see here. So you're giving an ID.

You're still annotating the the method

signature, the function signature the

same text string and then workflow

context. But in this sense we're

actually yielding output. So unlike the

previous uh two uppercase executor, this

one is it can be a terminal node and

we're handling you know whatever logic

is inside of it and then we're yielding

output on the context. So you can see

it's annotated with never because it's

not sending a message to another node

but it is producing the the second type

string and so again allows us to do the

validation and things like that. And

then we we mentioned in the previous

slide that the there's a fluent builder.

So we can see the workflow builder

adding an edge. So we're adding

uppercase is has an edge between

uppercase and reverse text. Then we're

just telling the workflow that the start

exeutor is our uppercase executor. And

then we're building it. We get a

workflow uh object type back after we

call build. And so, you know, adding

edges, we we'll get into this in another

slide, but it can be an executor edge.

It can also be an agent edge. And so,

one thing as we built out these

workflows, we wanted to integrate part

of agent framework the ability to easily

add agents as nodes. And now you're able

to wire up agents with other function

style executors or whatever business

logic you need between those two in a

really really easy manner. That's that's

a you know really first class support

for agents is is at top of mind for us

>> and so we do want to call out

>> just go ahead

>> like that become very powerful when you

have agents on those you know on those

nodes

>> exactly

>> people don't understand like the you

know the power if you have in your hands

now with a simple workflow and and

agents be on the those nodes and amazing

>> exactly and that that's one one reason

that We wanted to make it so easy. We,

you know, in the process framework, all

you had the ability to do was create

these executor style things. They were

kernel functions and you had to wrap

your agent instances inside of them and

handle these different dependencies. It

was so verbose and and I would say

clunky. So now we're just giving the

ability to create these agents, add them

as nodes. We handle all the message

brokering uh between those. And so it's

it's using uh you know underlying

abstractions that that we built out and

it it really makes the developer

experience I think much richer and and

more valuable.

And so speaking you know we kind of

touched upon this graph validation a

little bit. It's the type messaging that

that the typing that gives us the

ability to perform this graph validation

up front.

uh we really want to make sure that

while you're building these workflows

that you're are you're knowing that

you're wiring up these edges and nodes

properly and not you know allowing you

spending all the tokens and usage to

later figure out oh I can't actually

send message to that node you know we're

saving cost by doing this validation up

front I know that in Python not

everything needs to be typed you know

we're an enterprise uh SDK and so we

want to make sure that that things are

are typed as much as possible. It's a

little extra work to figure out the

types on things. We do these checks up

front. Even if you're not typing the

workflow context properly, letting you

know, hey, please give the type of what

you expect. It can be a union. It can be

complex types like data classes or such.

So, you're not limited to just

primitives, but you know, we really want

to make the experience uh you know, as

as robust as possible. And so along with

just graph validation, we're doing edge

checks. Uh you know, making sure that

the the particular edges can send that

information to each and raising these

these failures for mismatches. And then

also if you've wired up, you know, a

graph in an potentially an infinite

loop, we also want to help you

understand that that could be an issue.

That's okay to have cycles in graphs.

It's okay to have termination logic

inside of your executor. We're just

going to let you know, hey, double check

that everything's okay, that you are

exiting, you know, and providing that

that yielding output as you expect so

that you don't get into some situation

again where you're just wasting tokens

and you're never going to exit. So, it's

kind of a, you know, not something you

want to get through if if that happens.

And

the other thing is that you know as I

think from the semantic kernel

perspective I I talking a lot about

semantic kernel. I was previously on you

know semantic kernel team. Uh and so one

thing that there was a disconnect with

is that there were different APIs

between orchestrating agents and running

process framework. Um and so we wanted

to bring in you know workflows as the

base orchestration layer and build APIs

on top of that to give you really easy

ability to configure sequential

orchestration, concurrent orchestration,

magentic and group chat and handoff

which is coming soon. So we provide

these sequential builders, concurrent

builders and so on out of the box the

you're able to add agents to it. You

don't have to worry about what that

means to add the edges underneath. We do

that for you. So you're able just to

give the participants of these builders

configure you know for example for

concurrent uh a particular aggregator

custom aggregator if you need to

otherwise you can use ours out of the

box or even with magentic now you're

you're able to specify the participants

you can have human in the loop to help

uh you know uh move the plan forward.

It's not just kind of a onedirectional

piece now. And you get all these other

things too like checkpointing which is

is built in as a first class you know

pattern. So these are really really

wonderful wonderful out- of- the box

patterns are getting some great feedback

still looking at how we can improve

these simplify these. So the dev

experience is really good. Um but these

are present and so we want you to play

around with them use them build with

them and let us know you know what else

you'd like to see.

And

want to also touch on what the execution

modes are for workflows.

So we provide two execution modes today.

There's a non-streaming which you're

going to call run method on a workflow

and that's going to run through the

entire graph and then it's going to

allow you to get the final state and get

the outputs and everything underneath is

actually event driven. And so we're

producing events through you know the

executors. So your business logic can

raise custom events. You can handle

internal events. You can see the state

and things running uh between what your

executor is doing and also what internal

states are. And so you kind of you get

that differentiation.

And then on the other side if you

actually want to run it as a stream you

have that ability. So now you're able

to, you know, get an an async generator

of the particular events, handle those

really useful for UI components, um,

showing what you want to show as as a

developer to your to your customers and

getting that that final result, too. And

so, as you can see on the screen here,

we're we're able to get that that final

output event based on the workflow

output event type. Um, and and Victor

will show this soon as as we get into

some more tangible demos.

Um, but let's just let's just touch base

quickly. Uh, I don't think we'll have

time to go into all of this today, but I

just wanted to to show that there are

some advanced features, really exciting

things that we didn't have present in

the process framework. You can actually

just call as agent, the as agent method

on any workflow, and you just turn that

workflow into an AI agent. You can now

interact with it in the same with the

same uh you know interface as an agent

has. It accepts chat messages. It

produces chat messages. You can get run

events every you know using the single

agent experience translates directly to

this and it's a super powerful

experience now to give you know create

this complex workflow and just interact

with it as if it's an agent. So that's

that's available today. That's something

that's

>> that's a really exciting really exciting

thing that that you know powerful thing

that we're getting some awesome feedback

on and uh you know loving seeing what

people are building with and and there's

going to be more coming soon uh around

using an agent a workflow as an agent

and so we'll we'll say kind of a teaser

stay tuned for that. Um the other thing

that we're providing right out of the

box is as I alluded to earlier or talked

about earlier is checkpointing.

Checkpointing is such an important

concept when you're getting into these

complex scenarios. Not everything is

running through the happy path. There

are issues, there are timeouts, there

are, you know, errors from from agents

running underneath. You want the ability

to rehydrate your workflow from a

particular state.

Victor, I saw it. You want you want to

>> Yeah.

>> Yeah, please.

>> Yeah. I was going to say it's a it's a

it's an extremely useful pattern. um

especially if you were going to build

stateless applications. Um so you can

think of a scenario where um you have an

endpoint and you have this thing

integrated over let's say an interface

like teams or slack or or WhatsApp and

there's always a chance that like from

those interfaces your application

endpoint gets like a trigger starts up a

workflow and maybe it needs user input

at some point and so a checkpoint using

the checkpoint feature is a good way to

say hey you know I need user input I

serialize current state to a checkpoint,

send some sort of information to the

user, the user might respond to that at

any arbitrary time, but in the meantime,

you know, you have a checkpoint, you can

stop that service, sort of spin it down,

and then when some new input comes back,

you can sort of rehydrate that service,

rehydrate your checkpoint, and sort of

continue with um with execution. So um I

just wanted to call that out because a

lot of times people run into this

situation where they need to build this

sort of stateless scalable applications

and having you know thinking in terms of

checkpoints um it's a useful thing to do

>> definely

>> that's essential that will be very

important for the enterprise and you

know most of the workflows probably

going to need some kind of checkpoint

>> definitely

>> and and today for checkpointing at least

in Python we support an in-memory

checkpoint. We support a file

checkpoint. We're going to have more

checkpoint stoages soon. You know, a

Azure Cosmos DB,

>> other providers. Uh but that, you know,

just the fact that we already have this

in public preview and are able to

receive feedback and iterate on it is is

really important because it's something

we always wanted in process framework

and it wasn't to the same level it is

now.

>> Oh, Evan. And also if you're watching

this, you want to help on those new kind

of storage, I think that's your

opportunity, your partners.

>> Definitely.

>> Yeah, that's your time to to have your

storage available there as well.

>> Exactly. Right. And that's that's the

beauty of being the open source project.

We're open about all this to the

community. Nothing we're not hiding

anything. You know, let us know if

there's a particular provider you need

and as as George said, implement it if

you really want to and and help us out.

We'd love the contribution.

Um, okay. So, all right. Checkpointing.

We talked about checkpointing. Another

super powerful concept and one that we

provide out of the box is nested

workflows. So, let's say you have a

particular process you you've already

modeled as a workflow. You can now run

that workflow as a subworkflow as a

node. you can have human in the loop

even in that subworkflow and we will

handle all all the message routing to

get that up back to the user and so I

think it's an extremely powerful concept

you can package that workflow as a

subworkflow and bring it in and you know

elevate your your other workflow with

that

and then yeah we we've touched on this

uh already but human in the loop is is

you know first class component here and

so fact that we already are providing

human in the loop it was an

really asked for from semantic kernel

side. Um I know autogen had some some

really good way of handling human in the

loop from you know agent proxy and and

things like that but now bring it into

this workflow that that is typed message

driven it is stateful has checkpointing

and the ability to allow the user to

more easily integrate into that because

not everything needs to be AIdriven you

know you're you're creating a bank

application and you still want a human

to to give thumbs up thumbs down on some

things that's totally fine we all

envision these AI agents running off and

doing things for us. And although we may

be there in some regard, other times

it's perfectly okay to have human in the

loop, right? And so we're we're giving

this this uh you know first class

construct here allowing that to be

really easily integrated. We have

samples around that, but we we'll be

able to dive into more human in the loop

probably in the next segment. I

>> I think those features here will be very

important. I think everybody should be

using most of them. You know, run this

as a agent. I think will be helping to

integrate with other you know platforms

um checkpoints you know to keep state

sub flows and human view

that's what's most you know real life

workflows will be be asking for

>> congrats on the team very nice

>> want to go for Victor now show this run

>> let's let's get over we've been talking

enough of showing all this stuff let's

dive into I need to share your Victor.

>> Yeah, I would love to see how we get to

use workflows with Dev UI.

>> Yeah, absolutely. Um,

>> one second, one second that we're here.

Just before you start, make sure that if

you're following, you know, this show

that's part of Microsoft Developer

YouTube channel. There's also the first

video that we did on the Microsoft

Nation Framework two weeks ago is on the

video description. We get a lot of

attention, more than 30,000 views. Now,

make sure that you watch and follow the

project here on GitHub for the link here

at the bottom. So, make sure that you

leave a star for the project. I know

that we have a lot of stars already.

Congrats for the team because it's a big

achievement to get like 4,000 stars in

two weeks. And yeah, let's see now that

running. All yours, Victor. Go ahead.

>> Yeah. So let's look at some code just to

make what um Evan has talked about just

a little bit more concrete. And so in

this very simple code snippet we're

looking at um we have a workflow builder

has a name and a description. The first

thing we want to do is we want to set

that executor. And so what what are all

this executors? So if you scroll up here

we'll just see that um we have things

like a final processor and it inherits

from the base executor class that Ian

sort of referred to.

So the top part of all of this is that

we have defined all of these executors

and now at this section of the code we

will sort of build it into a workflow

and the API is actually quite

comfortable. So we set a start executor.

We construct an edge between these two

executors given yet another edge. And in

this case what we have is some sort of

conditional edge. And in this case we're

saying um if we have a specific case we

want to enable a transition um from the

spam detector to a specific handler. And

then similarly we might add additional

edges. And once we're done we sort of

build this and we get a workflow object.

Now, um you probably saw in the previous

video, we have like a UI application um

that ships with Dev UI, a sample UI app.

And if we in if we sort of import that,

we can wrap our workflow just like we

did with agents in the last video. And

then we get a user interface that lets

us sort of interact um interact with um

with this uh workflow. And so the exact

code I just showed um on the dev UI

endpoint, what happens is that we can

dump that workflow as a JSON and we can

use that information to sort of render

it in a user interface that looks like

this. So we see the exact same things we

just looked at the email processor

content analyzer all of that. And in

order to run this, what Dev UI does is

that it will introspect on the data type

of the first or the start executor and

it will give us a form that lets us

provide the right input to that node or

that executor. Once we have this, we can

run it. And as each node runs, it emits

events um or traces. And as this sort of

events come out, you can sort of look at

them here. And essentially, you can also

see that visualization that sort of

shows you as a developer what's

happening at each time as your entire

workflow gets executed. Then once we

have a final result, we can sort of view

this in a bit more detail. Now, a

workflow can be as simple as this or it

might be something even more complex

with final edges. So edges that execute

in parallel and aggregator uh nodes that

sort of aggregate the outcome of

multiple edges. And similarly the input

to these nodes um might be complex. They

might be complex paid or data class

models and it will run um in in in a

scenario where nodes execute in parallel

that will work. And um in cases where uh

inputs for multiple nodes get ex get

aggregated that will also um work. And

so for this specific uh node for the

data ingestion node it's a fairly

complex input data structure.

Essentially what we do here is that we

introspect on that data structure and we

can construct a form that lets the user

provide the right input. And so before I

close out here the final thing I wanted

to touch on was that um in addition to

executors just being either classes or

just pure Python functions, they can

also be agents. And in this case we have

um agent executor nodes. And if we sort

of run this um we would expect that we

get some type of prompt. Let's say write

a report on AI models and advances.

And once we run this what we expect is

that this agent has an LLM. It might

have tools. It sort of starts to sort of

take the prompt. It system message

starts to sort of generate text that

addresses that. And once that uh

specific node is done, our sort of

control is sort of sent or passed to the

next uh node and as data sort of comes

out it gets thrown to the UI and we get

all those events we show that to the

user. Um if if hotel traces are created

we sort of see that and then finally

once all of that is complete we can sort

of look through all the output that

comes out of all of the executors.

Um I think the important bit here

overall is that if you do have specific

complex business logic um and you do

have a specific order of steps in which

you want the things to um sort of be

executed or explored when uh the task is

run. Um the workflow API is just um the

right place to sort of implement all of

these. If you're coming from the autogen

side um you probably have worked with um

graph flow. we tried to handle something

similar but um just didn't provide the

level of control that we have here. So I

definitely would recommend you sort of

check this out and definitely leave us

um a bunch of feedback.

>> Um

>> so nice. Yes, thank you so much for

showing us this. I love it. Always

dreamt of having a visualizer for my

workflows in in such a nice clean way

and so we glad that we're able to ship

this now.

Yeah, that's that's I love you did show

me that before, Victor, and every time

that you show me like I'm in love with

that UI and how easy is for,

>> you know, just to debug, change your

workflow, try again. And keep in mind

that's all running in your local

environment.

>> Yeah,

>> it can be connected with a model

somewhere. But it's so powerful for for

the developers. What kind of feedback

you getting so far from those workflows?

>> Oh, yeah. Um

so one of the things that like um you

you sort of touched on one of the key

feedback that we've gotten so far. So

there are two two large things that

we've gotten. So the first thing is

people want even more richer data about

the state of each node. So for example

right now we show we tell you about like

the you know we get events that get

streamed out when we do like run stream.

So it tells us things like workflow

events. Um a node is currently running,

a node is completed and that's sort of

what we use to sort of implement the

visualization we just saw. But more

importantly um nodes can mutate state

and so Ivan mentioned that there's a

concept of workflow context and each of

these nodes have access to that and they

can mutate updates the workflow context.

Um, and so it might be useful to sort of

see, you know, as each node executes,

how does the workflow context change?

And so we're sort of thinking of nice

useful ways to represent that to the

user in a debugging interface like this.

And another useful thing would be to

sort of show the input and output for

each nodes um especially during runtime.

Um and so as each node executes um the

the output that it pro produces becomes

the input for the next node. It can be

helpful to sort of inspect what that

looks like just to make sense of the

behavior of the entire workflow.

And then finally there's a concept of

hot reload. Um and so the the the the

happy developer experience is that we go

we make some changes to files on disk

and once we save that then the UI is

aware of that and we can we can be

notified and we can reload um those

files just for a much smoother debug

experience. Um we have a PR ready for

that and all of these things you know

it's all great feedback. We're um we

we're really thankful for all the

engagement so far. So hey, keep them

coming, you know. Yeah.

>> Yeah, sure. It's a sh thing. Thanks a

lot. You know, have anything you want to

say for the community?

>> No, we just we love the the feedback

we've been receiving. We love for you to

try it out if you haven't yet. Um, you

know, we we have weekly office hours.

You can join. It's still under the same

semantic kernel autogen series. um you

know file issues,

open a discussion. The team is actively

triaging these things and going through

them which you know really appreciate

the the support we've received so far

and and really excited for you to try

all this out. Again, so much so much

more to talk about. We probably have to

do another series or another video on

this series about workflows because we

have those advanced topics. Um but if

you're eager to to look into those now,

we have a lot of getting started samples

in the repo. So, please dive into those,

ask questions, you know, let us know

what you're building.

>> Yeah. And uh make sure that you follow

I'm going to put also Victor is doing

like on Linkadini like a blog series,

blog post series and that's amazing. So,

I'm going to put the on the video

description that as well and give us

feedback. Thanks a lot you two again and

see you on the next one. For sure you're

going to on agent box. Thank you.