So it's a code first approach. So you

you need to revise your agents and your

workflow in code and to be able to use

by the Wii. The Wii itself doesn't

create agent. It only um uh run those

agents and obser observe these agents.

Hello everyone. Welcome back to OP at

Microsoft show. Elijah, Eric, hello.

>> How's it going folks?

As you can see, we are back with

Microsoft agent framework. Now talking

about migration from autogen and have

Eric here, the expert on autogen and

also now on the Microsoft agent

framework. Tell me Eric, are you excited

to talk about migration?

>> Yeah. Yeah. Totally. I would love um

people who already using autogen uh to

come forward to using the Microsoft

agent framework. Um we learned so much

from building autogen. So we apply a lot

of lessons to to the new framework. You

can think of it as the new version. Uh

and you will find much similarities

between the two.

>> Yeah, that that's the that's the deal.

And Elijah, tell me tell me about the

excitement about Microsoft Agent

Framework. Everything that's happened

last last three weeks. I think three

weeks.

>> Three weeks. No Jorge, we've been live

for three weeks now and the excitement

has been incredible. We've gotten so

much great feedback from the community,

thoughts, um critical feedback, all

kinds of stuff. Um and so it's been

great to to hear what people have to say

and um we're really excited and and if

you have a something that you wanted to

add, a comment, etc., please go to our

repo ak.msback/agent

framework and leave us a um a comment,

GitHub issue, a discussion. We'd love to

hear from you.

Yeah, it is nice to see how many stars

you have. I don't know more than four,

five going to 5,000 soon. So, yeah, keep

keep living your style of the project

and leave the feedback. Open shoes and

let us know. And now you're going to see

I'm going to share your screen, Eric. Um

and yeah go ahead and start telling us

how those autogen people can migrate

those uh you know applications or those

agents

>> or their codes to to Microsoft agents or

>> yeah totally and so I think first of all

what is uh the Microsoft agent framework

right uh if you already watched the

previous series in this um you probably

have already have a good idea. So I'm

just going to quickly cover what it is.

So autogen and semantic kernel um

they're both great at separate kind of

purposes. And what we have done is take

the best of both and combine into a

single framework. So you don't have to

choose or think about oh which one

should I use right? So in the future go

with Microsoft in your framework. And

um I think before I start I want to

quickly give uh level set you know what

uh you you're going to expect when it

comes to

uh migration. So if you're already using

autogen

and you are mostly using the agent chat

library then the new Microsoft agent

framework is going to be very uh uh uh

familiar to you because you will see the

interface almost the same uh with a

slight differences when it comes to how

you know conversation threat managed and

we will talk about that later but more

or less the migration experience will be

quite smooth. Um so I'm going to um uh

start with the most basic component uh

in in the framework which is the

>> Eric Before you jump into that I just

want to let the folks who are watching

here know Eric is a very humble guy but

he is one of the most talented engineers

I've met and he came from the autogen

team. So we've had a lot of great input

from these folks that were working on

autogen improving it directly baked into

the agent framework. So, if you're

familiar with Autogen, you're going to

see here in these slides that are coming

up, it's going to look very similar. We

try to, as Eric said, bring the best of

both worlds, focus on Autogen and now

trying to make this um as migration as

seamless as possible.

>> Yeah. Yeah. Yeah. Thanks. Thanks,

Elijah. Um, yeah. So, like if we're

looking at the most fundamental B

building block of of your frameworks, uh

it's going to be the uh client that you

access the model API, right? So in

autogen and in the new agent framework

it's basically the same right. So you

you you are going to configure your

client you know your endpoints API key

and so on. Uh the new a framework going

to have support uh for responses API

that autogen does not have. So if you

are looking to use built-in tools,

serverside tools like web browsing, you

know, code code interpreter, make sure

you check out the new framework for

those capabilities.

Um so uh we well the main um uh content

of this talk is going to start from

single agent. So in autogen you have

many different ways to create a single

agent. there are some built-in agents

but for 95% of the case I see people

using uses the assistant agent so

assistant agent is a specific class of

agent in autogen and um it's important

to to note that assist there are other

agent in autogen that have different

constructure and interfaces the

assistant agent um so for the purpose

for of the talk most people are using

you know assistant agent so let's just

look at that um

a key thing here is if you look at the

um constructors constructors uh

parameters and it's pretty much the

same. If you look at assistant agent and

then you go to the right side look at

the chat agent the direct creation

option is pretty much one to one right.

Um, one thing to note is uh the

assistant agent implementation is single

term by default which means that if you

are uh if you have a tool and if you

don't set the max tool iteration

parameters in autogen the agent is going

to only execute the tool once and return

the result. On the other hand, uh the

chat agent in Microsoft agent framework,

it will um execute a tool until the uh

execute the tool and call the model and

execute a tool again. If the model

produce a tool call, it's going to keep

doing that until the model stop

producing a two call. So it's a

multi-term by default. So this means

that you don't need to uh set the max to

iteration parameters here. You just it's

it's going to keep executed to until it

stops by default. Of course you it it

has built-in mechanism like approval to

make ensure that like the application

has uh have access to

aation can stay in control whether that

tools get executed or not. Um there is a

separate uh a new way to create agent

instances. It's to use the factory

method associated with the client. So if

you use that further simplifies the

process of agent creation. So you can

just call client.create agent uh to get

your agent instance without having to

specify the client because the client is

already created.

And you can also I think this is a new

API that I really like is that in

autogen you cannot configure how agent

runs once it is created. Um but in the

new engine framework you can pass in the

parameters including tools the choices

tools and many many other parameters

that you would configure the model with

inside the run method so that for each

time you call the agent you can have a

different um set of parameters. I think

this is extremely useful uh as in often

time you want to switch you want to use

a different context you want to use a

different set of tools when you invoke

the agent but you don't want to change

the other configurations so yeah so this

is the kind of a slight difference but

it's mostly an enhancement on how you

can use the agent um uh Eric the agent

run sounds like very interesting have

like any like a a specific use case that

you remember that people are using that

Mhm. Yeah. So um a single agent when in

autogen typically used for like when you

when you have to kind of um

automate the choice of tool calls like

in often time in autogen you use a

single agent as a part of like a multi-

aent team right and in a multi- aent

team some agents are responsible for uh

generating content some agent

responsible for executing the tool. So

often time the reason why there's a

maximum two iteration forevers and the

default is a single turn is because a

lot of agents all it does is to just run

a tool and and so um yeah so but but if

you don't give the tool to agent then

the agent is going to just generate the

content based on the model. So, so if

and then you chain these agent together

using a team concept. Um, so that you

can have different agents responsible

different things. Um, yeah. So that's

that's what most people using autogen um

they use a single agent abstraction for

and yeah so

>> sounds good. Looks like was a feedback

from the community to improve that as

well.

Yeah, you can do both in in the new

engine framework as well. U and we'll

talk about workflow uh later which give

you a much more um uh much more control

over how you want to or the agents and I

think yeah and

>> there's a video on workflow coming will

be will be published when you watch

this. So we have the introduction video

then you have a workflow and that's

that's the next one. Yeah, let's go

forward. That sounds great,

>> right? And then the next one, um, just

quickly cover where all the things that

you are familiar with with if you're

coming from model gen. Um, so the tool

in autogen is a function.

Uh, same with the agent framework. So

you can you can write a simple function

and convert it into a tool. Right? So

this is identical in both a framework

and autogen

and same thing with streaming. So uh

this the the to if you are streaming

content to your UI for no more

interactive experience for the user. Uh

in autogen there's a type of message

called model client streaming chunk

event. So this message type would carry

the chunks which is string and then you

just uh kind of uh stream that chunk to

your UI. The difference is in the new

agent framework is that you will call um

run stream again but it's going to be

streaming by default. So there is a um

I there there's no by the way I think I

in this slide I did not include the

setting on the agent itself. So in

autogen uh for the assistant agent you

also have to configure the uh agent to

stream uh chunks in the constructor by

setting a model client stream equal to

true. And so that is no longer needed in

the Microsoft agent framework. Uh

because what we realize is that almost

always if you you call run stream, you

always want streaming chunks and and so

when you call agent.r runstream without

having to configure the agent

beforehand, it's going to give you the

chunks and you can access the content of

the chunks through the text field of the

agent. So that way um you know the

process to set up the streaming is much

more simple. Um yeah so you can do both

with either model client you can stream

from there and you can also stream from

agent the result is it uh it's the same

right so

and of course we we support MCP and we

have even more uh features in our MCP

support in the Microsoft agent

framework. So not only you can do uh

studio

um studio IO uh stand

uh standard IO um MCP servers. You can

also have you know remote MCP through

HTTP streaming as well as websocket.

So

I think one multi- aent pattern that

when you first start uh to use autogen

is often time the agent as a tool and

what this pattern does is you are

wrapping each of the agent your sub

agent as a tool uh using the agent tool

uh constructor

and then you put that agent uh in uh

your parent agent as an as a tool. So

this allows you to essentially let the

model decide when to call a sub agent

like you don't need to provide your

oration logic at all because uh it's all

up to the model to decide like the

sequence of sub aents uh to call. Now

there's uh the way you would do it in

the new agent framework is also

identical. So instead of using a

constructor called agent tool, uh each

agent has a method called as tool. So

it's going to return you a a tool

instance that you can put inside another

agent to be used. So the the way it

works is the same and the effect on the

model itself is the same as well. Um so

this is if you are thinking about multi-

aent multi- aent or fishisioning before

you go to used workflows

first try this uh maybe this is already

sufficient

>> quick question for you because I know we

talked about both MCP and now agent as a

tool a lot of questions we get asked a

lot is around aa do we also support aa

on agent framework

>> yes so that is another thing that

autogen doesn't have and the new engine

framework has to u another reason if you

want to migrate. Um so yeah so the um

again I think the this covers so far the

similarities between autogen and the new

agent framework as you can see like you

you don't have to change how you think

about an agent uh very much it's it's

pretty much the same. Yeah, I think

Elijah right on on the MCP thing. Just

remember that those frameworks came

before MCP. MCPs are very new still very

new. So tool was something even before

MCP like MCP just you know probably they

turned away now to to do the tools and

you know but we had the word before MCP

as well. So

>> yeah, I I think I can I can also mention

a few points on on this is what MCP

really uh or A2A uh this protocol really

helps is to standardize

uh the ecosystem of of of tools and then

before yes you can always you know

import a tool library but then each time

you integrate it with the tool you have

to wire it up manually right but MCP

make that process like they use a

standardized protocol so that the model

can directly access without having to

you wire up through some your custom

Google code. Uh so that is great but I

think whenever you use these type of

standardized protocol always keeping in

mind is um the the server is agnostic to

the client meaning that it doesn't know

that your client has certain property

doesn't really know what the model your

client is using. So uh it may not have

the optimal sort of performance because

like the tool description for example

right the number of tools on the MCP

servers all those things are um not

optimized for your use case right so

sometime you have to think about those

as well like so maybe you eventually

have to go back to using tools or wpping

the MCP server as a tool at some point

just to make sure that your uh your

application is fully optimized, right?

Because you need to modify the tool

descriptions or number of tool use uh

for each agent to make sure the agent

doesn't go off the rail um or get

confused. So,

>> and I I would say adding to the system

prompt as well, not only just the tool

description, I found really helpful in

my own MCP use like, hey, making sure

the agent giving it a highle summary of

what that MCP is doing and when are some

appropriate use cases goes a long way.

And especially when you're dealing with

agent framework and you got a lot of

different moving pieces with multiple

agents, making sure it's clear what the

job and function is each one for the

agent themselves and then for you as the

developer to keep it all straight.

>> Yeah. Yeah. Totally.

Right. Um so now we're entering the

realm of differences and the one major

differences between autogen and the new

framework is how conversation state of

the agent is managed. Um so in autogen

uh it don't have the code showing here.

This is all agent framework code. In

autogen when you when you call an agent

you actually modify the internal state

of the agent and the state of the agent

is stored within the agent instance. So

what does that mean is that uh the next

time you call the agent it's going to

remember last time the the history from

previous calls and um you can export of

course you can load and serialize

dialize the internal state but that

instance itself is default. The

difference in the Microsoft agent

framework here is that each agent itself

the instance itself is stateless. Uh

where the conversation state is actually

stored inside a separate thing called a

thread. So you can see the code here. Um

agent you can you can obtain a thread

from the agent and then use that thread

to to trigger the agent. If you don't

again if you don't create a a thread and

use the thread as part of the run method

uh the thread is actually created

automatically behind the scene. So and

that that will be done each time you

call the run. So the next time you call

to run, it's not going to remember

you know what what what you talked about

before. But you then then you have to

actually pass the thread as part of the

next invocation to the run for in order

for the agent to remember the previous

conversations.

So it feels like hey why it's a little

bit more complex you now have to write a

few more lines of code. Why do you even

do that? Um so what we found is that

like often time a

in a in a multi- multi u so for example

especially when a multi-session or

multi-user multi-tenant applications

each time invoke agent you're actually

coming from different user or different

uh even with the same user you might

might be triggering the conversation

from a different session a previous

conversation. So uh decoupling the logic

of agent execution from the state allow

you to reuse the agent for different

states uh for different conversations.

So so this is actually really a nice

feature if you're building you know your

your um your your chatbot that interact

with different users or different users

uh sessions. The other thing is it make

making the state explicit allows you to

have a uh allow us to have a standard

way to uh to persist that state. So, so

now what we h what we have is if you

have a let's say a responsive API agent

an agent build with your responses API

you you can store your state on the

server side then you can have a you know

a thread that keep track of the server

side state and the same interface

applies if your thread is stored as

inmemory you know chat messages as well

as if your thread is stored as uh uh on

the disk in the SQL database.

So the same interface will apply to all

of those scenarios. So to make it you

know your code really modular and

reusable. So this is the kind of the

rational why we go from you know

stateful instance as a whole to state

stateless instance and stateful state

sorry state for thread um and decouple

them

that

>> the new checkpoint the new checkpoint

Eric I think is going to

>> rely a lot on that. Yeah.

>> Yeah. Yeah. So, uh the agent single

agent itself doesn't explicitly expose

the concept of checkpoint. So, because

you you you have possession as the

application you have possession of the

thread. So uh it's the application's

responsibility to to you know produce

the thread and the thread itself has

serializ D serialize method and in it

itself is um uh you can also implement a

custom implementation of it and so that

it will always you know incrementally

persist as as as go as you go. So um

yeah so this is a you know I I think

this is actually a really nice concept

to to have.

Okay. Right. And um

I think this um diagram show how you can

think about thread. So there's a

separate concept of each time you invoke

the agent is we call it a run. So it run

is like a run a call to the method run

and you can see the user message coming

in and then the agent message come back

is update a thread with the latest

message and the next user matches on the

same thread

uh with a next run will be up will have

the agent update the same thread with a

new agent message. Um so there's a

separate concept of memory which is uh

also exists in the autogen. Um so it's

just the concept is the same. The memory

is where you store the long-term um data

across different uh user sessions or uh

in certain in some cases different

users. So the memory can be associated

with a thread in the new agent framework

and that allow you to automatically

update some of the um you can think of

it as a user profiles or user

preferences that that you want to

produce across different tractions

between the model uh a uh between the

user and agents. So yeah, I think in the

future we're probably going to have a

another new session to go to this memory

concept more in details, but for this um

let's focus on you know just the

differences and similarity between

autogen and the new framework.

All right. So um come to message types.

The message type this is more of a

syntax differences. So for autogen we

have a set of messages like for text for

multimodel

and uh there's more messages that more

like life cycle events and for mic

Microsoft agent framework it's also

similar we we we have the same thing

with different names but a lot more

granular so we we also introduce you

know data messages URL content so that

you can you can embed things like or PDF

files as a data message data content

Uh you can also embed your images at

data content. Uh so the data content

will be different by their you know

content types

media types. Um

but I I think there's a very important

concept to talk about which is the the

content of the message can also be um

used to we're also using the content

message to to you know let the

application approve and disapprove some

actions that the agent wants to take

because agent you know communicate with

the application via these message

content types. So in this example um we

talk about the new content type in the

agent framework called user approval

content types and here's an example how

it works for function execution

approval. So let's say you send a

message to the agent on the top you can

see the I send a text content inside my

chat message and the agent will use the

model to make an inference and it's

decide to invoke tool and before

invoking the tool the tool is going to

send back a request to the agent which

will translate to a uh function approval

request content. This is another content

type as part of the chat messages. And

when the application see this content

type come back by the way by this time

the agent has not executed the tool. It

simply see that it needs to run the tool

but stop and return the content back to

the user or the application so that the

application can decide whether this

content uh this tool should be invoked

and I then decided say okay yes you can

invoke this tool send back a function

approval response content type back to

the chat agent and the tool will run and

the result will come back and go through

the model which will make another

inference and returns a text content

based on the result of the content uh

result of the tool execution. So it's

the application is going to see the

function without content and as well as

a inference based on that result.

>> And Eric, can you outline for us like

what are some instances where you would

use this for tool invocation, agent

invocation, etc. Like what are some good

use cases for this user approval?

>> Yeah, there's many many cases in

autogen. We we we have people asking

um I want to have the application decide

whether

uh we want to visit a specific URL if

I'm like as a web browsing agent uh or I

want to you know download a file or I

want to run this piece of code. Um so a

lot of these uh tool they have side

effects that are either irreversible

or could have you know uh security

implications.

So, so for those tool execution

um it's always recommended to you know

stop like somehow we can we can suspend

the agent and keep the state

um which it's part of a thread by the

way and then give the control back to

the application so that the computer can

decide what to do with it right yeah so

this is um I would say really nice

because it allow you to you know put put

the agent to sleep before it wake you

wake it up and handle the the two call

uh just where you left it off. So that's

a

many use case for for this type of um

approval request

um workloads. Yeah.

>> Yeah. I I love that diagram because

there's so many people asking and so

many people that never understood this

whole you know flow of things and with

MCP for me what's the main difference is

that now the you know the model is kind

of figure out how to call those tools

without any you know binding it's not

like a API is something that's be

dynamically done by by the module to say

what kind of you know parameters I have

to or I have to have to call this these

two and the things become so flexible

and so dynamic that

sometime people don't forget that they

running completely independent.

>> Yeah. Yeah. I I think it's especially

now you know if you want to implement an

agent that handles many different tasks

like a general purpose agent like we we

want to help you uh you know to to

ensure the safety and security right of

of the agent. I think the if you already

you know understand this approval

content you may want to move on to even

more advanced concept here that's new in

agent framework uh it's called

middleware so the this is not in autogen

so middleware we can think of it as a

general purpose way of handling um

like message interception so as we've

seen before right the tool invocation

you Think of it as a tool middleware and

there are also other type of middleares

including agent middleware and not shown

here but also model middleware. So if

you are familiar with you know web

application design or you know backend

servers you you you know that middleware

is something sitting in between the user

request or the you know HTTP request

coming in and the actual handling of the

HTTP request. You can stack different

middleares for authentication, for

logging, for for other kind of uh you

know um uh content interception or uh

content modification in flight that this

middleware pattern is uh is been used by

many many web servers. So same same

concept uh for agent. So if you think

about agent as message handlers, the

data the messages that goes from

application will first pass through a

set of agent middleware and agent

middleware can intercept the intercept

the messages and look into the message

you see is is this messages um uh

contain let's say jailbreaking

uh instructions and if it does then

immediately short circuit it return back

to the application and say that we don't

handle this type of message right or you

can have a tool middleware like what I

described earlier right you can you can

uh have your own way of handling the

approval process

so before the agent involves the tool

it's going to first go through the

middleware and middleware can look at

the tool call and decide whether we

should intercept this tool call um based

on some uh predefined rules and then uh

decide whether to short circuit it and

send it back to the application and say

like you need approval for this to call

or when you have data going back to the

application, right? You can have uh a

tool middleware or an agent middleware

sitting there and detect for PI, you

know, data privacy uh breach, potential

data privacy breach where you can match

your, you know, data with some maybe

certain databases or based on their

formats or predefined rules. So that

ensure that there's no you know PII

information or security sensitive data

that come out of the agent because agent

have access to many different tools

right some of the tools might have

access to data that the user of the

agent should not have access to. So so

this middleware um designer let you to

do that

>> and I think this is critical too Eric

just to add here because as we continue

to see more and more agents in

production in the real world etc.

Unfortunately, we also have a lot more

security concerns, jailbreaks as you

mentioned, prompt injection and tax, all

kinds of stuff. And so this middleware

and and then also toolware is is tool

middleware is great to to have. And

actually with agent framework, we have

an an awesome integration with our

friends at the on the Microsoft Purview

team that allow you to add Purview kind

of as a middleware layer to be able to

track what's coming in and out of like

not even just your agents, but also the

workflows as as you may have seen in the

previous episode. We'll talk here in a

little bit. um to make sure none of that

stuff is is being exfiltrated or

ingressed incorrectly,

>> right? Yeah. Um so here is a code

example of creating

um a a middleware. So there the first

one is just a login. So you can see the

it's very similar to how if you already

familiar with middleares and web

servers, you can see it should be at

home and it's simply you call the next

to pass the data along. If you don't

call next and return directly, you short

circuit. Um so this is adding the

middleares to the agent. Agent has a you

know field here called um middleware.

You can also pass the middleware as part

of the run method as just like uh the

other kind of uh run options.

Right? So uh this is uh we we we have

finished the first half of uh migration

which is focusing on the single agent

and concept related to single agent and

so the next half we're going to focus on

uh multi- aent right so in autogen

autogen is well known for the multi-

aent orientation patterns in fact I

think I I would say that autogen

pioneered many of these um multi-agent

orientation patterns and many many other

frameworks

um you know uh learned a lot from

autogen in this particular aspect

>> because for folks for folks that don't

know Eric the original autogen paper

came out in like early 20 or earlier mid

2023 way before agents were cool way

before orchestration and whatnot was

cool. So I think it's safe to say that

autogen pioneered and and kind of

revolutionized a lot of this stuff that

that we see here today.

>> Go check out the white paper the

original research paper. Would highly

recommend. It's a great read.

>> Right. Yeah. Yeah. It was very it was so

much uh excitement back then also like

when we first published the paper as

well as the autogen um autogen project

open source um yeah people was like wow

you can naturally just have a group chat

between the agent that seem to help in

improved the performance and anyway so

long story short um I think the concept

of multi-agent ostations now is a

standard concept in it already exists in

many different frameworks And um yeah,

let's take a look at how you know the

multi- aent sorry the Microsoft agent

framework um work with these concepts.

And

so I think this is where the most of the

difference will come from. If you are

thinking about you have a multi- aenting

workflow in autogen um or multi- aent

team for example um you want to go from

there to the new agent framework you're

going to have to you know follow this

section carefully.

So, so first of all, if you're using

autogen if and you're not using the

agent chat, you're using the lowle agent

core, sorry, autogen core, um that is I

would say um we currently don't have a

migration path for that. So which means

that you you're going to have to look

into the um your low your implementation

of event that that event driven you know

message handlers and then understand the

workflow API in the new agent framework

and then do the mapping. Um I I wouldn't

say it's not a no migration path. I

would say it's more of a much more

involved migration path than if you were

simply using the team concept in autogen

because the team concept is at very high

level and we have a matching API for

each of the team in autogen.

So maybe not too much detail up front.

Uh that's that's for for this um I I

want to kind of just go over each of the

concepts when it comes to multi-agent

and workflows so that once you have a

very good understanding of the uh the

differences and similarities then I

think you will be equipped to make to to

to do the migration. Um

why don't we start with the uh the basic

you know two agent interactions. So in

autogen there's uh a concept called

roundroin group chat right. So the

roundroin group chat essentially you put

two agent and then they will kind of

talk to each other right through sending

message back and forth and um you can

run the team using team.run run method.

And so this method is similar to how

agent runs, right? And you pass in a

task and so the task will be first

passed to the first agent and then agent

will produce a result and that will have

the control pass into the second agent

and then go back to the first agent and

it loop until after five messages. So in

Microsoft agent framework the concept is

going to be different in terms of how

you write how do you express uh this

this um uh order of um execution. So in

Microsoft engine framework we have a

concept called workflow and the workflow

uh is consists of executors and edges

and ed executors is where computation

happens right this is where your logic

your business logic are implemented and

edges represents

uh the flow of data

so that's different from in where like

your transitions between the two agents

is more about control uh passing of the

control from one agent to the other. Um

so in the new agent framework the

transition is actually a passing of data

and the executors are activated as soon

as it's dependent data are available

meaning the message has arrived. So, so

that is the key difference between the

two programming model is that autogen is

a control flow when it comes to multi-

aent whereas a Microsoft agent framework

is a data flow. So um the in this

example right so you can have two agent

uh so of course uh it's not a cycle here

um it has uh single direct one direction

uh one direction edges here. So the

first agent is wrapped around inside a

function and is uses the executor

decorator to convert it into an executor

node and the same with the second agent.

And so the um in this direction direct

edraph

you you have this first agents uh

produce a content and that get passed to

the second agent which will produce

content based on that result and then uh

the workflow will conclude. But if you

can add another edge go from the second

edge back to the agent one executor then

you have the cycle as you can have the

same uh behaviors around wing group chat

and then within the is aging executor

you can you can um you can configure

like you can write the termination logic

for example um if the message contains

certain object then instead of send the

message I just return when you return

then there's no data coming out of one

agent to the annex agent uh the waffle

automatically stops. So so that is the

uh the basic examples in in the uh

workflow API.

>> Yeah. So

for me like looking as a developer I see

that now the workflow is the controller

and you just passing the data. That's

the main difference for me that the the

workflow will be controlling who is

executing

each step will be executed at the time

and the data is passing

the next one. Is that the main

difference?

>> Yeah. So in yeah I would say like you

can think of it as that the workflow is

keep running as long as there are data

being passed. So the uh as long as there

is message data messages in flight

between the executors uh the workflow is

always is is running in the background.

Um as soon as no executor produce any

data then there's no data to handle then

the workflow will just stop running and

it will return uh the control back to

the application. Yeah. So in autogen I

think it's more about the orchestrator.

So in origen there's a concept of like

an orchestrator decide whether there

should be more work to be done right. So

the the control is determined by the

orator and if decide there's no work to

be done based on termination conditions

or based on some other logic and it stop

triggering other agent then the that in

autogen that um the team I would say

like the cont will will stop and return

the result back to the user. So, so

yeah, so that's uh it it it may seem

like the same from a very high level

like if you just interact with the

result directly. Um

you may not notice difference but as

soon as you start building the workflow

you will notice the difference. Um yeah

so I

I'll I'll go into more of this uh just

in a bit.

>> Yeah, let's try to move otherwise it

will be so long video. Let's try to move

on and and uh get the maximum that you

can about migration. That's so

important,

>> right? Um okay. So we can like so when

it comes to migration, right? So there's

a um if you're writing let's say a

roundrobin group chat you're writing

selector group chat uh swarm or magentic

you have a an like one to one map to a

another API in the Microsoft a framework

you can basically just migrate the code

that what you have and get go to the new

uh API and get the same behavior. So if

you that that's given that you're using

the agent chat um agent chat uh library

from autom

so so here in this um session I I'm

going through each of the um the oration

pattern in the new agent framework and

give you a flavor of what it feels like

to use it and what what it correspond to

in the um

um in autogen. So it give you some idea.

So uh first one is the most basic one.

This is sequential. So this maps

directly to uh round robin. The in robin

there's a cycle of course but there's a

termination condition. So the in

sequential if you instead you kind of

flatten it out into a a chain of um

agent that you need to run

that that is what roundroin with a

termination condition after a certain

number of speakers. So that is

equivalent. And so in the sequential um

orchestration

the new API provide you with the builder

like a fluent uh builder API and you can

essentially set each of the

configuration as a um a separate method.

So you can run fluent uh sequential

builder.participant and give you a list

of agent and build and that would give

you a workflow.

Uh that workflow looks like the one on

the right. uh it will build for you um

and then it start the chain of

conversation between these agent and

they give you the same behavior from

Robin.

>> Um so

in the second

>> quick question though on on sequential

when would I when is a good use case to

use a sequential as opposed to just

creating a workflow with a bunch of

agents in sequence.

Um right. So I I think you can achieve

you achieve the same effect if you do uh

do both. But the sequential builder uh

essentially give you sort of a a quick

start or like the um

uh more

uh pre-built

uh experience

>> out of box

>> right out of box experience. Right. So

>> got it. You don't like if you look at

the the what constructed on the right

hand side

like there there are nodes executor

nodes there that you didn't explicitly

build right so you got you you see the

agent one agent two and three that that

you are familiar with

>> but there's also two conversation one to

conversation two and so on. So these

nodes are there for for the for the

purpose of like transforming the data

between the agent one agent two and also

making sure the full conversation

contacts get passed over.

>> I see.

>> So yeah so these are you can also create

this explicitly by like what we did here

and you would have to create separate

executor for each of the data

transformation nodes. Um it's just that

like if you use the sequential builder

it's just simpler if you

>> consider one line of code versus having

to build all of this myself. Okay. Got

it. Love it. Love it.

>> Right. Right. Right. Exactly. Yeah.

Exactly. Yeah. Yeah. Same with the

concurrent builder. So concurrent

builder if you are using the graph flow

features in autogen which has some

concurrent execution and so this maps to

that specific feature. Um so uh in in

the new aging framework if you want to

have a simple kind of fan in a fan out

fan in behavior you can use uh

concurrent alteration and so again you

don't have to worry about dispatcher or

the aggregator. So these articulars are

created for you when you use the builder

API. You of course you can also create

them create create them yourself with

the low-level added uh u methods of the

workflow builder API. It's just that

there's going to be more work but it's

of course it's more customizable more

powerful.

>> Yeah.

>> Yeah. I like I like how you you

simplifying but also leaving the option

for them to customize if they need it

and to extend like that.

>> Yeah.

>> And that's like what what Eric said at

the beginning like we really tried to

take the best of both worlds of semantic

kernel and autogen of like leave the the

simplicity of autogen and bring the

customization and not that autogen

doesn't have customization but semantic

kernel is very customizationheavy. Bring

those two experiences together to give

people what they what they need.

>> Yeah.

And then so here like the you can see

the magentic which is mapped to the

magenta one orchestration in autogen and

then you have handoff orchestration

which maps to the swarm in autogen. So

like if you already using the go ahead

and use the one in Microsoft agent

framework that will you give you same

experience. Now for the magenta we also

provide magentic and handoff will

provide a number of enhanced features.

So for Magentic a lot of people have you

know voiced their opinion saying hey you

know we want to be able to customize how

uh the planning step happens and how

a lot of people wants to you know allow

user to review the plan before the agent

go ahead and execute the plan. So so we

make it possible for uh in inside the

agent framework in the Microsoft engine

framework. So um it's it's hard to show

with the just the picture here but if

you look at the API there's a separate

API called um no set planner so or set

human in loop so that which which allow

you to to stop the magentic workflow um

and returns a review request for the

application to review and then you can

pass in a review response to you know

either re plan or continue with the

plan. So this make it much much easier

to um customize the behavior of

magentic. Um same thing with handoff. So

handoff is also a number of

customization now is possible and we

also find that um in autogen which is

called a swam pattern. In autogen it

always runs kind of in an infinite loop

because the model doesn't recognize that

it needs to transit to the next agent.

Um

to address that we have um added kind of

humane in the loop every time the uh

agent produce a response. So by default

the workflow the handout workflow is

going to suspend and return a request

for info like basically request a

response from the user or the you know

the application uh before it's going to

move on to the next agent. So we'll

continue with the same agent. So this

make sure that application is always in

control of the this workflow. Uh this is

to address that infinite loop problem

saying in the swarm.

So yeah there was more customization

more um more way that you can you know

um optimize optimize or motivate your

workflow in the new new framework.

All right. So um so we're gonna go deep

into the advanced territory here. So if

you are you can probably close this

videos now if you are only you have only

been using uh you know round robin group

chat and magentic or selector group

chat. Um

so in autogen the most advanced feature

is called uh graph flow and the graph

flow um give you a control flow API. It

lets you to specify the sequence um of

invocation of these agents in a in a

group chat. Um so this I want to make

sure that you guys are not confused

graph flow with the new workflow API in

the Microsoft agent framework. So um in

graph flow fundamentally a graph flow is

a control flow again. So because in

autogen all the workflows are uh

essentially consists of agents right and

the orator will decide which agent to

invoke next. So it's a control flow

based on the state of the conversations

and then based on the state the or

decide uh what is the next agent to

invoke. The difference between this and

the workflow is that the workflow

fundamentally is a data flow again. So

as long as there is a message in flight

the workflow will run and the message

determines

uh which executor get execute uh which

executor or agent as part of the

executor get invoked. So you can imagine

uh the workflow can have multiple agents

working at the same time in parallel

whereas a control flow often time you

only have one agent or have very limited

parallelism. um when you the offer is

like giving control to two agent and

then wait for both of the agent to

finish. So but uh in in in the in the

new workflow API as the each of the

executor can decide to trigger another

executor without any kind of centralized

u work.

So so that's a kind of fundamental

difference in a very technical like very

theoretical level. Um let's look at some

of the examples. Okay.

So in graph flow if you were to simply

have a sequence right sequence of um of

of agents beacon. So in graph flow you

would use add the agents as nodes and

add the edges and this is basically

edges of a linear sequence. Whereas in

Microsoft agent framework because no

graph flow is a you can think of it

advanced features in autogen

we don't we map it to the low-level

workflow API uh in the new reagent

framework. So in the new ed framework

again you create the executors each of

them is a node in the uh workflow and

then you also add the edges between this

node and create a chain and so if you

look at this and this pretty much the

same uh from from the you know authoring

experience but um the way they are

executors are different okay and so this

is an example of fan out and fan in So

assuming you have these agents in

autogen's graph flow uh uh in in

building autogen you can this is for

when you want to trigger the fan uh in

node with a join all the messages coming

from um from all the fan out agents. Um

so you want to wait until

all the agents before

has finished execution then you invoke

the join agent.

Um in this case the second example is

this this code example is to show that

you want the um the join agent to be

triggered whenever there is an uh

there's an agent has finished execution.

So, so this is two different scenarios

and in in the Microsoft agent framework

you can achieve the same thing right. So

again you you can use join any in this

example here it will whenever there is a

new message coming in it will just

immediately produce an output.

Uh in this case we we keep a state as

part of the workflow context that will

check if all the previous agents have

finished emitting that message and

finishing you know their work and only

produce an output when you know there

are certain uh number of messages has

arrived. So the you can see that these

implementations um is is going to be

different. Yeah.

>> So that's uh fan in fan out and how

states are maintained. And then also I

think it's important to point out

>> I just put here the link for the for the

workflows that's the Python one if you

want to get examples and uh from the

source code just make sure that people

know where he is and also for for the

net

python.net. So there are a lot of you

know examples you can play there.

Right.

Yeah. And then I think we also need to

uh point out another difference. I think

maybe I already mentioned it earlier is

that um in autogen all the all the teams

uh including graph flows their their

property is that any agent they publish

a message is going to be seen by all the

other agent by default. There's some way

to configure which agents can see which

message but that's by default all agents

see all the messages public by all the

agents. So again because it's not a data

flow like so it's the the data is being

passed to all the agents and the control

is been assigned to the agent by the

offer whereas in the workflow in

Microsoft agent framework um the the

data is not by default seen by all the

agents right the whenever you send a

message like this uh from educator B to

executor C educator A cannot see the

edge uh because you only have an edge

from um from B to C

again. So like same with A to B, B to C

and so on. So the these these

constrained um uh because in the this is

constrained because in the agent

framework Microsoft agent framework the

executor will be in it will be triggered

as soon as it sees uh messages. Right.

Right. So um uh

yeah so that that again like there's no

centralized executor centralized

orchestrator determine um when to invoke

a specific executor.

Okay. Um this is the uh targeted

routing. You can see this is not going

to be possible with um with autogen as

like as this is uh you are deciding what

is the next recipient of your message

because in autogen you know when you

send a message from an agent all the

other agent sees the message whereas in

here you can see like you can determine

you can decide whether my target should

be the vision or the writer executor.

based on whether the message start with

image. So this is the new feature that's

only available in the Microsoft page

framework.

Okay, so we're almost at the end of the

advanced scenarios. Um so just to

summarize the graph flow versus

workflow. So in in autogen's graph flows

is our control flows and edges represent

transitions and in Microsoft agent

framework data flow repres uh the the

the graph represent data flows edges are

messages and the node types in graph

flow you can only have agents whereas in

web flows you can have agents or

functions you can also have nested

subflows it's not part of this um

session but in a future session about on

a web flow we're going to talk about

about that

and about activation. So the um the

messages have been broadcast to all the

other agents and activation is done by

the orchestrator and uh in workflow API

edges are represent how you know edges

are being activated. So I think the

remaining uh um differences is that um

the workflow you can nest you can create

a recursive structure of nested

workflows whereas in autogen is much

more limited. You have to customize by

implementing a custom agent whereas in

type safety point of view like the

autogen of course because it has it only

work with a set of types coming as part

of the library. you cannot introduce

your custom type whereas in a Microsoft

agent framework you are actually using

your own types uh when you when you um

when you create your own executors. So

it has a very strong um validation

um feature which allow you to which

allow you to you know validate the

workflow before you actually run it to

ensure that workflow will run and the

messages um can go through the executor

making sure the types matches between

the input and output. So yeah, that's

the end of the the

>> I yeah was a was

I want to see the UI at the end here we

have a couple of slides but before that

about the checkpoints what happened on

those workflows now when you have a

checkpoint

and what's a checkpoint if people

haven't watch it by the way there are

two videos that already published that's

the introduction that we did at house

went three weeks ago and there is also a

workflow video that was done with uh

Victor and Evan that's also available. I

can see there's a lot of hundreds views

in minutes here. So people are excited

about that and we just published doing

that recording. So um and that one's

coming. So make sure that you watch both

videos on the video description here.

But tell about the checkpoint and how

that works.

>> Right. Right. Yeah. Oh for sure. So well

of course we're going to cover

checkpoint in much more details in a

future section about the workflow. So

here we're going to give you a quick

overview. So in autogen there isn't a

concept of checkpoint. Uh whereas

there's just a concept of state

associated with team with agent and you

need to manually manage those states. Um

in in the new Microsoft agent framework

the difference is that the workflow

automatically have uh a checkpoint

created for you. So it's checkpoints are

are you can think of it as a snapshot of

the execution state of a workflow. So

each time an executor or um each time

some executor finish execution

um the workflow will create a checkpoint

and save it into a checkpoint storage.

And so you can think of it a chain of

executor one after another. Each time

one execute finish, it's going to

snapshot the state of the current

execution and execute finish and

snapshot the state and as a result uh

anytime your workflow fails halfway

because there might be a exception flow

in one of the executor right it happens

all the time in in and if you have tools

that you know cause errors or if you

have agent that that encounters some

recoverable failures. Um so it gonna

throw right so then your workflow will

terminate because you know um you have

an error and the workflow run will raise

an exception. But the good thing is you

can recover the workflow. You don't have

to start from the beginning. You can

start from a a a checkpoint just before

the the executor failed. Assuming that

you have resolved the issue that would

allow the executor to run,

you can you don't have to rerun the

previous executors. You can start from

the one just before the failed one. So

so this allow much more fault tolerant

workflow and uh give you the uh also in

in a in a different scenario you might

want to experiment. you want to you know

trial and error and see uh can I modify

some of the configuration that's part of

this executor and and but I don't want

to run all the previous I want to start

from this one that's part of this

workflow so you can also use a

checkpoint for that as well so so yeah

so that yeah

>> that's that's exactly why I asked the

question because that's probably the a

good reason to migrate just to have

that. What do you think, Elijah? I would

just migrate just to have that.

>> No, it's great. And it's a feature that

we get asked about a lot, too. So, no,

it's it's great that that we have it

now.

>> Yeah.

>> Yeah, that's great. And let's see the

UI. What's what's the difference? A lot

of people asking, I have this drag and

drop. Do we need that? We don't have the

drag and drop on the new UI. Tell me

about this.

>> Right. Yeah. So, dev UI again. So the

the in autogen we have some we have very

popular um autogen studio and it's it's

it's the purpose of autogen studio again

is a developer centric um debugging UI

and it's not meant to be a production

ready

um application for you to create and

manage um multi- agents and and teams.

So that need to be uh clear again. So in

in the uh new agent framework we also

have something similar we call dev UI

and so the dev UI give you a much lower

level detail about your agents and

workflow like you can see the uh

individual agent messages that coming

out that you know you can see the logs

you can also see the traces um you can

see like what model call the agent has

made, what tool calls the agent has

made, how many of the tokens have been

used like you can see like it's almost

like a you know a monitor for like a

activity monitor for your agents and

workflows. So it's much more loable and

also it allow you to go from your code

agent meaning that that you can write

your agents in in Python code or inn net

code is coming and uh you don't have to

use the drag and drop UI with um so you

are not limited by the uh the UI uh

authoring um uh you're not limited by

the drag and drop interface you can you

can create a agent from code directly ly

and then you can pass that agent

directly to the dev UI and it observe

and run those agents.

So so I would say like it's much more

powerful. Um so again yes so it's a code

first approach. So you you need to

revise your agents and your workflow in

code and to be able to use by the devi

itself doesn't create agent. it only um

uh run those agents and observe observe

these agents and yeah so that's uh um

that UI

yeah that's the we getting here today

and I make sure that people you know

follow the project leave a star we we

loving that excitement that Elijah was

talking at the beginning on the project

there are a couple of videos on the

video description that they can watch do

watch on the workflow and the

introduction as well as more coming you

know ask what do you want to see from us

that are also the community create a lot

of content blogs and you know people

talking at conference if you want to

talk at the conference you need our

support Elijah you know even as slide

deck or anything just ask we happy to

support you know uh people presenting

talking about agent framework and also

let us know because we want to promote

that as well and we want to let the

community know about all these amazing

content that have been created.

>> No, Elijah

message for the community

>> to to that point too like we really this

is the time to get community feedback.

We're still in public preview and we uh

in the spirit of open source really want

the community to drive um what we're

building and give us feedback on hey

this is important, this is less

important, this is what we need, this is

what we don't need. Um, and especially

that since while we're in public preview

and we're that we can make a lot of

these changes easily, um, now is the

time. So, uh, Jorge, if you wouldn't

mind putting the link on the screen

again for people just to remind them

where to go, leave us a star for PR, all

the good stuff and and we'd love to see

you there.

>> Yeah, thank you so much again, Yu, and

uh, hope to see everybody soon. Thanks a

lot. Don't forget to for to follow

Microsoft developer channel and follow

the show as well, open at Microsoft. See

you next time.

[Music]