Hello everyone. Thanks for joining us

for our next live session. I'll be your

producer for our session. But before we

start, I do have some quick

housekeeping.

Please take a moment to read our code of

conduct.

We seek to provide a respectful

environment for both our audience and

presenters.

While we absolutely encourage engagement

in the chat, we ask that you please be

mindful of your commentary. Remain

professional and on topic.

Keep an eye on that chat. We'll be

dropping helpful links and checking for

questions for our presenters to answer.

Our session is being recorded. It will

be available to view on demand right

here on the Microsoft Reactor channel.

With that, I'll turn it over to our

speaker for today. Thanks for joining

us.

Hello. Hello everyone. Uh welcome uh to

Microsoft Reactor.

I am Costadinos and I'm going to be your

host uh today to continue on our third

uh meeting around MCP and Azure

integration.

Today we're looking into uh quite an

interesting case that involves a lot of

moving parts.

So we're going to explore uh how we can

create an engine that can support

different uh connectivity

uh for example

plane API calls from AI agents or AI uh

chat web apps and know agents that

utilize the MCP server or even a desktop

uh agent like CLO code that can connect

to our MCP and utilize uh whatever we

have built there. There is a twist in

this case that uh the main framework

that we are using is streamlit and uh

the models that we are uh utilizing is

chip4 and model router from Azure Open

AI.

a very smart uh addition in Azure AI

foundry that uh helps us uh utilize the

best when we we ask for information when

we interact with our uh SLM or LLM.

So I do have my presentation that I'm

going to explain uh the idea and uh what

I was looking for and how I came up with

this and then we're going to see the

application uh locally a part of it in

Azure and a part of it in the globe

desktop. So

let's see let's see and explore uh

what's going on here.

The idea was coming from a place where I

was thinking that the all the changes

that came up the last months

could produce a lot of complexity and a

little bit of a chaos for people that

wanted to you know be on the on the edge

on the cutting edge let's say better

utilizing the latest in order to test it

in in order to go and uh create

something for production reasons because

it was about time to have that uh you

know functionality or whatever and uh

uh different tools that were available.

So the the choice was not quite clear.

So this is like constantly switching AI

personas and friction and this creates

friction for developers. We have to

manage different types of applications,

different types of frameworks.

uh MCP now came into uh the game and it

has produced a lot of uh waves in the

developer community

and everyone is building and uh we are

building and we are evolving and we are

building but if we if we take a step

back and say hey what's going on here we

will realize that the traditional way of

interacting with our application will

never go away. there will still be APIs

and in fact what we will uh be left with

is a number of choices.

So how how would be to have a chat

playground system that use a centralized

orchestrator that enables this seamless

scenario where for our AI chat sessions

also have the ability

for independent chat sessions to enhance

workflow and reduce context switching

challenges. So

a number of people, developers, agents,

systems, applications can connect to

this centralized orchestrator and

utilize whatever is built there. A

simple chat playground in our case, but

you know, we can take it a lot further

and also have this fast iteration cycles

that can drive experimentation and

future development efficiency.

The key capabilities here is that we

have scenario awareness. We have uh we

can and we have the capability for

tailored conversation based on user

context. Each session is on its own

let's say uh sandbox in in quote sandbox

and it doesn't have to do anything with

other user sessions.

The session management itself is very

easy. You know, we have already in the

same framework in the same tooling

system the management uh tools and the

management interface. Let's say

the access can be dual. So we have REST

API on one hand and MCP interfaces on

the other hand or MCP wrapper. uh if we

want to wrap our MCP into HTTP and uh

publish it for uh consuming it through

web applications

and there is here

the the you know the idea moment the the

light bulb moment. I was experimenting

also with Streamlit. So

I had this you know

in in in the start I thought it was you

know a silly idea but then I realized

that it was a good idea to integrate

streamlit for this seamless workflows

and for people that you don't know

streamllet is an AI machine learning

framework for developers uh very easy to

manage deploy and uh and build on it. uh

it provides a lot of uh a lot of layers

of um connectivity of uh visualization

and interface. So it makes it easy uh

for developers and newcomers to create

uh quickly uh applications and u work on

their uh AI cases.

So here we have our let's say

architecture. Okay, in the middle we

have this this engine constructed with

streamllet and fast API that drives web

that drives web applications and MCP

seamlessly the great AI tools and this

is what provides us this unified

interface for enhanced user experience

and I think I do have this architecture

here I'm I try to depict it the the best

possible way. I think this is the one

that makes sense for everyone and for us

to understand what's going on. So we

have streamlit uh being the the center

in the center of this engine uh fast API

provides access to API and web clients.

MCP server wrapped in HTTP and left

alone let's say for MCP clients to

consume it and streamllet is the gateway

for endpoints and tools. So for for

making this um this idea uh have this

how how we can say it um represent the

actual case that I'm explaining. I have

created four uh tools let's say. So one

tool should be document processing

another tool would be code review

another tool would be technical support

and the final tool could be creative

writing. Uh in reality you can have

anything you want and uh also uh streaml

provides us the capability to

switch the model. So we can use GPT40 or

go for model router and uh that provides

us a an another layer of flexibility

regarding the interaction with our LLM.

There are challenges. Uh this uh this

specific uh deployment uh idea slash

deployment uh took me a lot of time. Um

there are a lot of protocols involved

here. We need to kind of bridge the

connectivity,

you know, make different aspects work

together. Uh make the user experience

seamless, you know. So uh after a lot of

effort we are able to say that yes this

is possible we can do it and of course

uh this demo that we're going to see in

a few uh it's it's a very small

representation of the greater idea and

the actual uh capabilities that are

there are quite enormous.

What we are achieving here is enhancing

productivity and collaboration efforts.

We have a faster iteration cycles

significantly significantly improved

team collaboration.

So there is a more efficient workflow.

The processes are streamlined as we said

earlier through this uh separation of

context and user sessions under the same

hood. There is central management but uh

independent developers and thinkers and

builders uh utilizing this engine.

There could be next steps. So for

example implementing support for

multiple users sharing resources at the

same time persistence introducing data

storage solution. This is something that

I really wanted to implement but I was

out of time. Um you know provide data

storage solution

for everyone under again the same

engine. So manage the other one uh

interface let's say but providing this

uh data sharing or uh data access or

file access capability to the users

integration really really important here

I think this is one of the most

important steps into this idea to

connect external systems and broadener

functionality data data access for

example let's say that we could have um

a CRM or an ERP system that could in

could um interact independently with

each user. So imagine the same system

under one connectivity

uh route but each user through their own

context and session can have their own

uh interactivity with this external

integration. I know it sounds very

difficult. It is difficult but it is

possible and we can do it.

And finally work workflow enhancements

for improving the orchestration of tasks

for increased efficiency and flexibility

and uh choosing the correct platform.

This is also something that uh is really

really challenging. What I what I'm

understanding right right now that we

are also uh into this and I'm presenting

it is that and this is something that

also came to me while I was building

this that AKS is probably the best for

this type of uh solution. So I'm using

container apps [clears throat] and like

90% of the system works seamlessly but

there are things that I think only

Kubernetes can solve for us and that's

why I believe that AKS is the best uh

hosting uh solution for this type of uh

deployment.

So uh when we uh come to our end we can

have uh questions. I'm Gadinos and let's

have a look on our demo. Allow me to

stop the presentation and switch to our

uh terminal. I'm going to zoom in.

So let me zoom in here.

Don't worry about

um

uh

sensitive data. No sensitive data here.

Everything is uh temporary. So

there is no issue. So we're using VS

Code.

We're using Cloud Desktop and we're

using Azure. Okay. So don't be scared

with the the number of files and folders

that are here. uh this is very heavy uh

experimented uh situation. I I

experimented a lot here until I reach

the final state.

So let's have a quick look

on how this works locally and then we

will have a quick uh taste of how this

could work in the cloud. Okay.

So

let me show you

>> [clears throat]

>> Let me close this.

Let me also close this. This is the

management interface. So we have as we

said fast API stream, Python is the

master here. Okay, it's our framework.

There is nothing else involved. Uh it is

fast API, Python, streamllet. The this

is our uh collection of tools.

The first thing that we are able to see

is this management interface and also

this gateway interface. So

a user can select

one of the four different tools to work

with. So it can be code review, creative

writing, technical support or document

analysis. Let's say that we want to go

with code review. When the users click

on a specific tool, they are taken to

this interface that is designated to

work for this specific type of

requirement.

Of course, I'm sorry.

Ideas can flow here. Okay, we can we can

provide users dedicated URLs for

example. So, let's say we have a team of

developers that are working on a

specific case. We can provide a

dedicated URL so they can have their own

context without having to select for

example but this is these are minor

cases for this for the bigger picture

right this is a very raw project

remember this is very raw project

so uh the user is presented with this

user interface you can make some toggle

here I don't think that I have

implemented the actual uh dark light

theme you can choose your uh model GBD40

or model router. Of course, you can play

around with temperature with maximum

tokens and the configuration parameters

for your AI model.

You can change

the preset for your system prompt. There

are a lot of uh prompts that you can

implement here. Creative writer code

reviewer

for for our example it should be code

reviewer and save it as a preset and you

know keep it like my preset or provide

the name for uh your specific

[clears throat]

scenario.

And now you're you're here with your

main uh interface.

So we can say

uh very simple questions. What is

streamllet?

Let's send it to our

uh

model router.

Give it a moment and you can see that

the assistant

now what happened in the background.

Let's have a quick look here. Since we

have selected model routter, Azure AI

foundally selected the best model for

this specific type of question and after

it made the selection for us, it is

returning the answer.

And you can see here that this it is

quite an enriched answer. Okay, very

nice, very well constructed with a lot

of information and details. Everything

that we are doing into this interface

provides us with information of what's

going on. For example, we can see the

number of tokens that we consumed.

We can clear the chat, clear the input

and also we can have some quick actions

here for our model. So for example, we

can say give me examples and uh

that's it. the the the model will need

to uh find us examples of this.

I need to present I know. So yes, giving

examples is the pre is a uh quick

action. So it's a follow-up question for

our uh model.

Uh let's wait

also some uh summarization

uh we can have with uh

are we waiting? Yes. Uh let me refresh

here.

We can also have summarization. So uh

quick quick questions

with uh specific intent summarize the

key points of the agile methodology.

That's it.

Another uh question implemented with the

system prompt act as a career course and

help improve my uh resume. Okay.

Review this code for potential

improvements and paste your code there.

You can have also this advanced features

so you can search within the messages

and you can see how what's the average

tokens that the assistant consumed and

also quite important you can get the raw

chat history in this JSON format.

Also you can just

select another model and of course we we

can uh start adding here GP5 the mini uh

family of models fi or whatever you want

from Azure AI foundry.

At the same time that this user is

working another user can come up

and say okay I want some creative

writing.

So

what's going on? The user is provided

with this session.

That's it.

They they have their own chat session

creative writing assistant.

Let's select the model rather and uh

let's say

um create

a good

epilogue

for a drama

book

sle to make it more clear because it's

very uh vague to say

uh just you know an epilogue for a drama

book.

So

here it is our dedicated assistant with

this specific instruction is able to uh

provide us with our uh with what we

asked for. At the same time another user

is working here. You can see that the

user is

identified as web user and I will

explain in a bit why we can see it like

that and there is another web user as

well here.

At the end of the day more and more

users are connecting into our

application and the

the admin interface let's say remains

active. you have the capability to

connect to the session that someone may

have left open or provide this

capability to your users. So a user

starts a session then leaves their their

position their office they they take a

break and then they can return back and

just hit open in their session and

everything will be exactly as they left

it. So it is their own context as we

said their own session independent from

each other. Of course you need different

ports for each uh session. We're serving

this through our uh fast API.

You can also close the session. So this

is something that for example is a good

idea to have separate administrative

interface and user interfaces.

Now there is also

another uh type of user that can be

implemented here. Give a moment to uh

quickly uh look around and I think I

know where is this.

I think it's this one.

It's using MCP to connect via CLA. So

let me

give it a shot here. Okay,

it should be

come on. It should be this MCPLI.

There are a lot of files. Again, this is

one of the most complex things that I

have ever created. I was very worried

that I was creating a monster,

a Frankenstein as we call it. But you

know, sometimes if you don't uh push

your uh your limits, sometimes you need

to

go and walk the road as we say and there

is light at the other end of the tunnel.

So yeah, you can see here that uh the

MCP CLI

connected to our MCP

portion of our solution discovered 1 2 3

4 5 6 7 8 tools. I will explain the

tools and the and starting to make some

some tests. So ping agent is just

checking the connectivity. It will list

the sessions and it f just found two

active sessions. It will launch a custom

session.

It will test it will test this uh custom

uh session with one of the scenarios

available.

And that's it.

If we return to our

management let's say or to our main

interface you will see here that more

users have been added and now in this

case this is the demo support user the

demo writing user and so on.

I don't know what will happen if I open

this session since it has been closed.

So yeah,

I can also take over this session that

was created by the demo writing user.

Very interesting, right? You can see

here more information about the session

context, the session ID, the user ID, uh

time stab,

the max tokens configured, and there is

no additional context since we did this

through uh the CLI.

So we have already seen

let me close everything.

So we have already seen

uh

web users being able to connect and

utilize it through uh a very simple web

interface but also MCP

CLI uh prompt I'm sorry MCP CLI engine

with preconfigured prompt within the

program utilizing through MCP

multi-tool tool capability.

Imagine also having developers working

on CLI and instead of having to open

another interface, open another window

or uh trying to find you know uh another

tool to open to get some assistance

having their assistance directly on the

CLI.

This is also something that's I think

it's a very nice idea.

Now for the tools that I explained

earlier, there is

there is this uh

separate uh interface

in another port.

Yeah, I think it's this one

or we have stopped it in a moment.

So here we have

this one. It's running.

It's running.

Yes, this is good.

I think we stopped. Yes, I think the

CLI. Okay.

CLI didn't stop it. I hope

it is another portal. Let me find the

correct port.

Maybe I will not use the correct port

and I I'm suspecting that it's not the

the correct port right now. So, give me

a moment to hit the correct port.

It's not here. Give me a moment.

this one.

See if it is running.

Okay, let let me stop it. I think it it

there the CLA session. I think it stops

the MCP server. So, let me stop it and

restart it.

While this is trying to be killed, let

me explain something really really nice.

Now I I explained earlier that it would

be beneficial and that the most

appropriate hosting tool for such an

engine would be AKS

and yes I agree the first experiment

with Azure container apps is very very

uh promising. So what we have here, let

me clear up a little bit since you have

seen already what's what's going on and

let me explain what we have here. So we

have deployed the the core engine as a

new container app and the web engine as

another container app. So you can see

here I can have my web engine with the

URL clickable here and here is the MCP

chart pre chart playground that I'm I

was looking for in the local host but

anyway it is available through our web.

It's it's a nice uh transition to the

cloud. So from here you can see that you

have the available tools

that provide management capabilities

through MCP for your sessions. So you

can launch a new chat session, you can

list your session, you can close a

session and quick launch the other

available predefined sessions created uh

via our server. the ping agent that just

makes tests and you can also have quick

launch uh through the MCP wrapper. This

time

also we have the main

uh streamllet fast API

core web app using this MCP core. Let me

show you

what we saw just earlier. Okay. And I

can close this and I can close this. And

maybe we can start, you know, a

technical support agent or creative

writing. And you can see here that it

will try to open uh the the appropriate

tools.

But this this part is not yet uh

completed. So let's return to the

tooling.

So container apps very promising for

testing. Again I believe the correct and

more appropriate tool will be AKS. So

you will see uh very soon in my GitHub

repo

uh the finalized AKS version of it.

Probably I will share it with uh ASB if

everything checks out as I'm um

aspiring. And finally,

cloud. Let me stop everything and

restart it so our engine doesn't

fail us.

So you need to start it in a specific

order. You can see here that you have a

lot of uh messaging and logging

for each and every session and

everything that is happening in the

background. So

let me also close everything here.

Very nice. So uh let's start it again.

And this time we're going to utilize

code.

I need to start it. So from this uh

console we need Python start pi

and we need the MCP web client from

another

CLI.

Let me bring it close to the one that we

are using

so we can have better control in our

console.

a lot of experimentation. I will show

you once more the final files that I

have there. Yeah, you will understand

it's our MCP web client.

Yes,

perfect.

Yes, this is the port that I was trying

to show you, but it was a good idea to

show you uh the cloud version of it.

Here is also the on-prem version of it,

right? So, list sessions

executed successfully, the ping agent,

quick launch, document analysis, and so

on. Now, let's go to our locally running

cloud tool. We are aware that there is a

specific way to uh integrate your own

MCP server with this type of um tools,

this type of software that they can

seamlessly uh connect to MCP servers

without the need of wrappers or without

the need of any bridging or proxying

mechanisms

for cloud. All you have to do is

create this type of connector. If you

don't know how this is uh done, let me

show you real quick. I think I remember

how it is done. You need to go to your

user profile

from uh your users

and here you need to find

I think it is application data.

I think I'm on the correct path.

Clo

update

says

no, it's not this one.

Yeah, I think I don't remember the

correct way to do this. Uh cloud code.

There is a specific file that you need

to edit in order to provide this

connector to your cloud um application.

If I remember it, I will tell you.

Otherwise, we can also search it on the

internet. So, you just need to make that

uh addition to the file. Once you make

it, you will be presented with this uh

addition, this switch, let's say, that

provides you access to your MCP server,

to your MCP engine using uh the uh the

data that we just created. And the

the tools that we have available through

our web interface, through our first

API, through our MCP wrapper are also

available as tools for our cloud

engine. So let's say we have this one.

Let me also start a new session.

Here is the session. You can see that it

appeared on the right. This is a little

bit better from the um user interface

perspective.

Nicer graphics, but it's a little bit

heavier, so it may hang

creative assistant. We just started a

new session. Okay. So, uh, let's get the

list sessions tool

and

I think

it will be able to use it. Let's see if

it doesn't I haven't make the

Oh, it's you know it's starting to using

it. And here is the results. Yes, I have

made the correct configuration. I'm

sorry I didn't found it. But if we have

time at the end I can have another look.

It's all it's it's there. It's in the

internet. You can see how you can add

connector to your cloud uh desktop

for connecting with your MCP. So I'm

sorry. So you see here list session is

the tool that we are exposing to cloud.

Cloud is able to understand easily

what's going on because we have done

whatever is needed for cloud to be

seamlessly integrated into this MCP

server and each tool that we're going to

ask information from it will be

uh it will respond. So let me try

something.

I think there are some limitations on

the tools, but why not experiment,

right? I'm

I'm advocating for people that like to

experiment.

I think the quick launch is not working,

but

okay.

Allow

I don't remember this part. So, let's

experiment. Yeah, success.

The session is active and you can also

get it through your URL. Just click on

it

and here you are. You will have a new

what type of session was that? The

technical support chat session agent for

your user.

Now it it doesn't have any meaning to

start interacting here. The point is to

see how the engine works, right?

So here is our code.

Here is our

I'm sorry.

Here is our code. Here is our uh

sessions,

our MCP wrapper and our uh our

streamllet fast API engine. Everything

can be deployed in a container apps. Uh

let me show you now the real deal.

So let's see.

I was able to gather everything and put

it under one directory. This multimmcp.

Let me make it a little bit bigger since

I'm going to stay a little bit here.

It's all about the challenge. This is

what I'm trying to explain to you.

First of all, you need your uh Python

environment. Don't forget that when you

you are working with Python, create your

Python environment beforehand.

You need one environment variable. So

that would be your Azure OpenAI

uh details. I'm using only one key, but

uh that's not a good practice. You need

every sensitive information to be uh uh

prepared as uh environment variable and

not expose it

uh through any means.

And we have here the MCP web engine

that's using this MCP server enhanced PI

and the MCP web client.

So

each one is uh completing the other one.

Okay. So one hand on one hand we have

this stylish web web user interface for

the MCP child playground. And here is

the actual MCP server. It's the same one

that we're using on prem with a couple

of corrections for the cloud.

And here we have the MCP server enhanced

for cloud to be able to run through a

server in the cloud. That part it hasn't

been done yet, but it will uh it will be

completed eventually. So, uh our cloud

desktop will be able to connect to a

cloud instance of our MCP server. And

finally

the core engine where our streamllet

lives

our stream in reality is a very long

Python application

and our uh

our agent let's say or our our uh how

can I say uh yes our agent that provides

connectivity through uh through the

streaml main engine for our MCP and fast

API endpoint.

So you can understand here

how complex this thing became from a

very simple idea provide through the

same engine

through the same engine different uh

contexts and different users capability

to utilize this engine gateway to LLM

and tools

and it is something that can be done as

You can see

you need to be patient if you want to uh

work on this. I'm open to contributions

but it is not yet published in GitHub.

Okay. There is uh uh we have this uh I

have this blog post that explain a

little bit how this thing works.

You can search it through my

cloudblogger.eu. EU

uh blog how you can build this, but you

will need help definitely. So don't

hesitate to contact me so I can help you

uh build it. And at the time that you're

going to ping me, since I'm already

working on this, you will get cleaner

and more efficient uh code for your MCP.

I think I have said enough.

I hope that you like this session. It's

very different than the usual sessions

that uh we have presented so far.

But uh you know what in the era of AI

and uh in the era of innovation we are

allowed to experiment as much as

possible. And you know we're not only

allowed we are I mean we are obligated

to experiment. you know, if we're not

experiment now, uh, we're going to lose

the train and maybe some fine ideas

maybe get lost in time and, you know,

innovation doesn't wait. So, do it. Do

it for yourself. Do it for your friends.

Do it for your company. Just go ahead

and experiment yourself with the best

tools that are out there. uh for me is

the best hosting platform and AKS right

now is the best option for this type of

implementations. We're going to see more

remember my words of this and uh

container apps is uh uh excellent for uh

a prototyping and maybe something a

little bit similar but this type of

complexity requires AKS.

I don't know if we have any questions.

Uh I I would like to thank you so much

for being here with me and uh being

patient with me on this um

a little bit odd uh mechanism.

I think it is interesting and I think uh

that anyone that would like to take it a

little bit further will be able to do

it.

I will be there for you if you need any

help. I'm Constantinos. This is

Microsoft Reactor and we are closing our

third uh appointment of our 4 days event

series. Tomorrow we have our final

session with one of the most interesting

themes that I could think of. Imagine

having your own

uh MCP server in a web app and uh

providing your users in tune deployed

tools to utilize this web uh this MCP

server

chat interface file search or whatever

tool you want to build and everything

managed under your organization's hat

with in tune and you know everything

another very interesting idea

and uh yeah I think that's it for today

I would like to thank also our host for

having us for having me also today

our appointment goes for tomorrow thank

you so much

>> thank you everyone for joining us today

on this session like Constantino said we

do have one last session tomorrow. Uh,

and if I do say so myself, I think it's

going to be a very fun and interesting

one. So, we do look forward to seeing

you again same time right here at the

Microsoft Reactor. As always, guys, we

strive to create excellent content for

all of you and we would appreciate if

you took some time to scan the QR code

on the screen and shared your thoughts

and comments about today's session with

us. This is going to help us continue uh

creating more sessions, improving them

for all of you based on whatever is it

that you share with us. And as always,

we appreciate your time and we look

forward to seeing you here at the

Microsoft Reactor. We hope you have a

wonderful rest of your days.