Loading video player...
Hello everyone. Thanks for joining us
for our next live session. I'll be your
producer for our session. But before we
start, I do have some quick
housekeeping.
Please take a moment to read our code of
conduct.
We seek to provide a respectful
environment for both our audience and
presenters.
While we absolutely encourage engagement
in the chat, we ask that you please be
mindful of your commentary. Remain
professional and on topic.
Keep an eye on that chat. We'll be
dropping helpful links and checking for
questions for our presenters to answer.
Our session is being recorded. It will
be available to view on demand right
here on the Microsoft Reactor channel.
With that, I'll turn it over to our
speaker for today. Thanks for joining
us.
Hello. Hello everyone. Uh welcome uh to
Microsoft Reactor.
I am Costadinos and I'm going to be your
host uh today to continue on our third
uh meeting around MCP and Azure
integration.
Today we're looking into uh quite an
interesting case that involves a lot of
moving parts.
So we're going to explore uh how we can
create an engine that can support
different uh connectivity
uh for example
plane API calls from AI agents or AI uh
chat web apps and know agents that
utilize the MCP server or even a desktop
uh agent like CLO code that can connect
to our MCP and utilize uh whatever we
have built there. There is a twist in
this case that uh the main framework
that we are using is streamlit and uh
the models that we are uh utilizing is
chip4 and model router from Azure Open
AI.
a very smart uh addition in Azure AI
foundry that uh helps us uh utilize the
best when we we ask for information when
we interact with our uh SLM or LLM.
So I do have my presentation that I'm
going to explain uh the idea and uh what
I was looking for and how I came up with
this and then we're going to see the
application uh locally a part of it in
Azure and a part of it in the globe
desktop. So
let's see let's see and explore uh
what's going on here.
The idea was coming from a place where I
was thinking that the all the changes
that came up the last months
could produce a lot of complexity and a
little bit of a chaos for people that
wanted to you know be on the on the edge
on the cutting edge let's say better
utilizing the latest in order to test it
in in order to go and uh create
something for production reasons because
it was about time to have that uh you
know functionality or whatever and uh
uh different tools that were available.
So the the choice was not quite clear.
So this is like constantly switching AI
personas and friction and this creates
friction for developers. We have to
manage different types of applications,
different types of frameworks.
uh MCP now came into uh the game and it
has produced a lot of uh waves in the
developer community
and everyone is building and uh we are
building and we are evolving and we are
building but if we if we take a step
back and say hey what's going on here we
will realize that the traditional way of
interacting with our application will
never go away. there will still be APIs
and in fact what we will uh be left with
is a number of choices.
So how how would be to have a chat
playground system that use a centralized
orchestrator that enables this seamless
scenario where for our AI chat sessions
also have the ability
for independent chat sessions to enhance
workflow and reduce context switching
challenges. So
a number of people, developers, agents,
systems, applications can connect to
this centralized orchestrator and
utilize whatever is built there. A
simple chat playground in our case, but
you know, we can take it a lot further
and also have this fast iteration cycles
that can drive experimentation and
future development efficiency.
The key capabilities here is that we
have scenario awareness. We have uh we
can and we have the capability for
tailored conversation based on user
context. Each session is on its own
let's say uh sandbox in in quote sandbox
and it doesn't have to do anything with
other user sessions.
The session management itself is very
easy. You know, we have already in the
same framework in the same tooling
system the management uh tools and the
management interface. Let's say
the access can be dual. So we have REST
API on one hand and MCP interfaces on
the other hand or MCP wrapper. uh if we
want to wrap our MCP into HTTP and uh
publish it for uh consuming it through
web applications
and there is here
the the you know the idea moment the the
light bulb moment. I was experimenting
also with Streamlit. So
I had this you know
in in in the start I thought it was you
know a silly idea but then I realized
that it was a good idea to integrate
streamlit for this seamless workflows
and for people that you don't know
streamllet is an AI machine learning
framework for developers uh very easy to
manage deploy and uh and build on it. uh
it provides a lot of uh a lot of layers
of um connectivity of uh visualization
and interface. So it makes it easy uh
for developers and newcomers to create
uh quickly uh applications and u work on
their uh AI cases.
So here we have our let's say
architecture. Okay, in the middle we
have this this engine constructed with
streamllet and fast API that drives web
that drives web applications and MCP
seamlessly the great AI tools and this
is what provides us this unified
interface for enhanced user experience
and I think I do have this architecture
here I'm I try to depict it the the best
possible way. I think this is the one
that makes sense for everyone and for us
to understand what's going on. So we
have streamlit uh being the the center
in the center of this engine uh fast API
provides access to API and web clients.
MCP server wrapped in HTTP and left
alone let's say for MCP clients to
consume it and streamllet is the gateway
for endpoints and tools. So for for
making this um this idea uh have this
how how we can say it um represent the
actual case that I'm explaining. I have
created four uh tools let's say. So one
tool should be document processing
another tool would be code review
another tool would be technical support
and the final tool could be creative
writing. Uh in reality you can have
anything you want and uh also uh streaml
provides us the capability to
switch the model. So we can use GPT40 or
go for model router and uh that provides
us a an another layer of flexibility
regarding the interaction with our LLM.
There are challenges. Uh this uh this
specific uh deployment uh idea slash
deployment uh took me a lot of time. Um
there are a lot of protocols involved
here. We need to kind of bridge the
connectivity,
you know, make different aspects work
together. Uh make the user experience
seamless, you know. So uh after a lot of
effort we are able to say that yes this
is possible we can do it and of course
uh this demo that we're going to see in
a few uh it's it's a very small
representation of the greater idea and
the actual uh capabilities that are
there are quite enormous.
What we are achieving here is enhancing
productivity and collaboration efforts.
We have a faster iteration cycles
significantly significantly improved
team collaboration.
So there is a more efficient workflow.
The processes are streamlined as we said
earlier through this uh separation of
context and user sessions under the same
hood. There is central management but uh
independent developers and thinkers and
builders uh utilizing this engine.
There could be next steps. So for
example implementing support for
multiple users sharing resources at the
same time persistence introducing data
storage solution. This is something that
I really wanted to implement but I was
out of time. Um you know provide data
storage solution
for everyone under again the same
engine. So manage the other one uh
interface let's say but providing this
uh data sharing or uh data access or
file access capability to the users
integration really really important here
I think this is one of the most
important steps into this idea to
connect external systems and broadener
functionality data data access for
example let's say that we could have um
a CRM or an ERP system that could in
could um interact independently with
each user. So imagine the same system
under one connectivity
uh route but each user through their own
context and session can have their own
uh interactivity with this external
integration. I know it sounds very
difficult. It is difficult but it is
possible and we can do it.
And finally work workflow enhancements
for improving the orchestration of tasks
for increased efficiency and flexibility
and uh choosing the correct platform.
This is also something that uh is really
really challenging. What I what I'm
understanding right right now that we
are also uh into this and I'm presenting
it is that and this is something that
also came to me while I was building
this that AKS is probably the best for
this type of uh solution. So I'm using
container apps [clears throat] and like
90% of the system works seamlessly but
there are things that I think only
Kubernetes can solve for us and that's
why I believe that AKS is the best uh
hosting uh solution for this type of uh
deployment.
So uh when we uh come to our end we can
have uh questions. I'm Gadinos and let's
have a look on our demo. Allow me to
stop the presentation and switch to our
uh terminal. I'm going to zoom in.
So let me zoom in here.
Don't worry about
um
uh
sensitive data. No sensitive data here.
Everything is uh temporary. So
there is no issue. So we're using VS
Code.
We're using Cloud Desktop and we're
using Azure. Okay. So don't be scared
with the the number of files and folders
that are here. uh this is very heavy uh
experimented uh situation. I I
experimented a lot here until I reach
the final state.
So let's have a quick look
on how this works locally and then we
will have a quick uh taste of how this
could work in the cloud. Okay.
So
let me show you
>> [clears throat]
>> Let me close this.
Let me also close this. This is the
management interface. So we have as we
said fast API stream, Python is the
master here. Okay, it's our framework.
There is nothing else involved. Uh it is
fast API, Python, streamllet. The this
is our uh collection of tools.
The first thing that we are able to see
is this management interface and also
this gateway interface. So
a user can select
one of the four different tools to work
with. So it can be code review, creative
writing, technical support or document
analysis. Let's say that we want to go
with code review. When the users click
on a specific tool, they are taken to
this interface that is designated to
work for this specific type of
requirement.
Of course, I'm sorry.
Ideas can flow here. Okay, we can we can
provide users dedicated URLs for
example. So, let's say we have a team of
developers that are working on a
specific case. We can provide a
dedicated URL so they can have their own
context without having to select for
example but this is these are minor
cases for this for the bigger picture
right this is a very raw project
remember this is very raw project
so uh the user is presented with this
user interface you can make some toggle
here I don't think that I have
implemented the actual uh dark light
theme you can choose your uh model GBD40
or model router. Of course, you can play
around with temperature with maximum
tokens and the configuration parameters
for your AI model.
You can change
the preset for your system prompt. There
are a lot of uh prompts that you can
implement here. Creative writer code
reviewer
for for our example it should be code
reviewer and save it as a preset and you
know keep it like my preset or provide
the name for uh your specific
[clears throat]
scenario.
And now you're you're here with your
main uh interface.
So we can say
uh very simple questions. What is
streamllet?
Let's send it to our
uh
model router.
Give it a moment and you can see that
the assistant
now what happened in the background.
Let's have a quick look here. Since we
have selected model routter, Azure AI
foundally selected the best model for
this specific type of question and after
it made the selection for us, it is
returning the answer.
And you can see here that this it is
quite an enriched answer. Okay, very
nice, very well constructed with a lot
of information and details. Everything
that we are doing into this interface
provides us with information of what's
going on. For example, we can see the
number of tokens that we consumed.
We can clear the chat, clear the input
and also we can have some quick actions
here for our model. So for example, we
can say give me examples and uh
that's it. the the the model will need
to uh find us examples of this.
I need to present I know. So yes, giving
examples is the pre is a uh quick
action. So it's a follow-up question for
our uh model.
Uh let's wait
also some uh summarization
uh we can have with uh
are we waiting? Yes. Uh let me refresh
here.
We can also have summarization. So uh
quick quick questions
with uh specific intent summarize the
key points of the agile methodology.
That's it.
Another uh question implemented with the
system prompt act as a career course and
help improve my uh resume. Okay.
Review this code for potential
improvements and paste your code there.
You can have also this advanced features
so you can search within the messages
and you can see how what's the average
tokens that the assistant consumed and
also quite important you can get the raw
chat history in this JSON format.
Also you can just
select another model and of course we we
can uh start adding here GP5 the mini uh
family of models fi or whatever you want
from Azure AI foundry.
At the same time that this user is
working another user can come up
and say okay I want some creative
writing.
So
what's going on? The user is provided
with this session.
That's it.
They they have their own chat session
creative writing assistant.
Let's select the model rather and uh
let's say
um create
a good
epilogue
for a drama
book
sle to make it more clear because it's
very uh vague to say
uh just you know an epilogue for a drama
book.
So
here it is our dedicated assistant with
this specific instruction is able to uh
provide us with our uh with what we
asked for. At the same time another user
is working here. You can see that the
user is
identified as web user and I will
explain in a bit why we can see it like
that and there is another web user as
well here.
At the end of the day more and more
users are connecting into our
application and the
the admin interface let's say remains
active. you have the capability to
connect to the session that someone may
have left open or provide this
capability to your users. So a user
starts a session then leaves their their
position their office they they take a
break and then they can return back and
just hit open in their session and
everything will be exactly as they left
it. So it is their own context as we
said their own session independent from
each other. Of course you need different
ports for each uh session. We're serving
this through our uh fast API.
You can also close the session. So this
is something that for example is a good
idea to have separate administrative
interface and user interfaces.
Now there is also
another uh type of user that can be
implemented here. Give a moment to uh
quickly uh look around and I think I
know where is this.
I think it's this one.
It's using MCP to connect via CLA. So
let me
give it a shot here. Okay,
it should be
come on. It should be this MCPLI.
There are a lot of files. Again, this is
one of the most complex things that I
have ever created. I was very worried
that I was creating a monster,
a Frankenstein as we call it. But you
know, sometimes if you don't uh push
your uh your limits, sometimes you need
to
go and walk the road as we say and there
is light at the other end of the tunnel.
So yeah, you can see here that uh the
MCP CLI
connected to our MCP
portion of our solution discovered 1 2 3
4 5 6 7 8 tools. I will explain the
tools and the and starting to make some
some tests. So ping agent is just
checking the connectivity. It will list
the sessions and it f just found two
active sessions. It will launch a custom
session.
It will test it will test this uh custom
uh session with one of the scenarios
available.
And that's it.
If we return to our
management let's say or to our main
interface you will see here that more
users have been added and now in this
case this is the demo support user the
demo writing user and so on.
I don't know what will happen if I open
this session since it has been closed.
So yeah,
I can also take over this session that
was created by the demo writing user.
Very interesting, right? You can see
here more information about the session
context, the session ID, the user ID, uh
time stab,
the max tokens configured, and there is
no additional context since we did this
through uh the CLI.
So we have already seen
let me close everything.
So we have already seen
uh
web users being able to connect and
utilize it through uh a very simple web
interface but also MCP
CLI uh prompt I'm sorry MCP CLI engine
with preconfigured prompt within the
program utilizing through MCP
multi-tool tool capability.
Imagine also having developers working
on CLI and instead of having to open
another interface, open another window
or uh trying to find you know uh another
tool to open to get some assistance
having their assistance directly on the
CLI.
This is also something that's I think
it's a very nice idea.
Now for the tools that I explained
earlier, there is
there is this uh
separate uh interface
in another port.
Yeah, I think it's this one
or we have stopped it in a moment.
So here we have
this one. It's running.
It's running.
Yes, this is good.
I think we stopped. Yes, I think the
CLI. Okay.
CLI didn't stop it. I hope
it is another portal. Let me find the
correct port.
Maybe I will not use the correct port
and I I'm suspecting that it's not the
the correct port right now. So, give me
a moment to hit the correct port.
It's not here. Give me a moment.
this one.
See if it is running.
Okay, let let me stop it. I think it it
there the CLA session. I think it stops
the MCP server. So, let me stop it and
restart it.
While this is trying to be killed, let
me explain something really really nice.
Now I I explained earlier that it would
be beneficial and that the most
appropriate hosting tool for such an
engine would be AKS
and yes I agree the first experiment
with Azure container apps is very very
uh promising. So what we have here, let
me clear up a little bit since you have
seen already what's what's going on and
let me explain what we have here. So we
have deployed the the core engine as a
new container app and the web engine as
another container app. So you can see
here I can have my web engine with the
URL clickable here and here is the MCP
chart pre chart playground that I'm I
was looking for in the local host but
anyway it is available through our web.
It's it's a nice uh transition to the
cloud. So from here you can see that you
have the available tools
that provide management capabilities
through MCP for your sessions. So you
can launch a new chat session, you can
list your session, you can close a
session and quick launch the other
available predefined sessions created uh
via our server. the ping agent that just
makes tests and you can also have quick
launch uh through the MCP wrapper. This
time
also we have the main
uh streamllet fast API
core web app using this MCP core. Let me
show you
what we saw just earlier. Okay. And I
can close this and I can close this. And
maybe we can start, you know, a
technical support agent or creative
writing. And you can see here that it
will try to open uh the the appropriate
tools.
But this this part is not yet uh
completed. So let's return to the
tooling.
So container apps very promising for
testing. Again I believe the correct and
more appropriate tool will be AKS. So
you will see uh very soon in my GitHub
repo
uh the finalized AKS version of it.
Probably I will share it with uh ASB if
everything checks out as I'm um
aspiring. And finally,
cloud. Let me stop everything and
restart it so our engine doesn't
fail us.
So you need to start it in a specific
order. You can see here that you have a
lot of uh messaging and logging
for each and every session and
everything that is happening in the
background. So
let me also close everything here.
Very nice. So uh let's start it again.
And this time we're going to utilize
code.
I need to start it. So from this uh
console we need Python start pi
and we need the MCP web client from
another
CLI.
Let me bring it close to the one that we
are using
so we can have better control in our
console.
a lot of experimentation. I will show
you once more the final files that I
have there. Yeah, you will understand
it's our MCP web client.
Yes,
perfect.
Yes, this is the port that I was trying
to show you, but it was a good idea to
show you uh the cloud version of it.
Here is also the on-prem version of it,
right? So, list sessions
executed successfully, the ping agent,
quick launch, document analysis, and so
on. Now, let's go to our locally running
cloud tool. We are aware that there is a
specific way to uh integrate your own
MCP server with this type of um tools,
this type of software that they can
seamlessly uh connect to MCP servers
without the need of wrappers or without
the need of any bridging or proxying
mechanisms
for cloud. All you have to do is
create this type of connector. If you
don't know how this is uh done, let me
show you real quick. I think I remember
how it is done. You need to go to your
user profile
from uh your users
and here you need to find
I think it is application data.
I think I'm on the correct path.
Clo
update
says
no, it's not this one.
Yeah, I think I don't remember the
correct way to do this. Uh cloud code.
There is a specific file that you need
to edit in order to provide this
connector to your cloud um application.
If I remember it, I will tell you.
Otherwise, we can also search it on the
internet. So, you just need to make that
uh addition to the file. Once you make
it, you will be presented with this uh
addition, this switch, let's say, that
provides you access to your MCP server,
to your MCP engine using uh the uh the
data that we just created. And the
the tools that we have available through
our web interface, through our first
API, through our MCP wrapper are also
available as tools for our cloud
engine. So let's say we have this one.
Let me also start a new session.
Here is the session. You can see that it
appeared on the right. This is a little
bit better from the um user interface
perspective.
Nicer graphics, but it's a little bit
heavier, so it may hang
creative assistant. We just started a
new session. Okay. So, uh, let's get the
list sessions tool
and
I think
it will be able to use it. Let's see if
it doesn't I haven't make the
Oh, it's you know it's starting to using
it. And here is the results. Yes, I have
made the correct configuration. I'm
sorry I didn't found it. But if we have
time at the end I can have another look.
It's all it's it's there. It's in the
internet. You can see how you can add
connector to your cloud uh desktop
for connecting with your MCP. So I'm
sorry. So you see here list session is
the tool that we are exposing to cloud.
Cloud is able to understand easily
what's going on because we have done
whatever is needed for cloud to be
seamlessly integrated into this MCP
server and each tool that we're going to
ask information from it will be
uh it will respond. So let me try
something.
I think there are some limitations on
the tools, but why not experiment,
right? I'm
I'm advocating for people that like to
experiment.
I think the quick launch is not working,
but
okay.
Allow
I don't remember this part. So, let's
experiment. Yeah, success.
The session is active and you can also
get it through your URL. Just click on
it
and here you are. You will have a new
what type of session was that? The
technical support chat session agent for
your user.
Now it it doesn't have any meaning to
start interacting here. The point is to
see how the engine works, right?
So here is our code.
Here is our
I'm sorry.
Here is our code. Here is our uh
sessions,
our MCP wrapper and our uh our
streamllet fast API engine. Everything
can be deployed in a container apps. Uh
let me show you now the real deal.
So let's see.
I was able to gather everything and put
it under one directory. This multimmcp.
Let me make it a little bit bigger since
I'm going to stay a little bit here.
It's all about the challenge. This is
what I'm trying to explain to you.
First of all, you need your uh Python
environment. Don't forget that when you
you are working with Python, create your
Python environment beforehand.
You need one environment variable. So
that would be your Azure OpenAI
uh details. I'm using only one key, but
uh that's not a good practice. You need
every sensitive information to be uh uh
prepared as uh environment variable and
not expose it
uh through any means.
And we have here the MCP web engine
that's using this MCP server enhanced PI
and the MCP web client.
So
each one is uh completing the other one.
Okay. So one hand on one hand we have
this stylish web web user interface for
the MCP child playground. And here is
the actual MCP server. It's the same one
that we're using on prem with a couple
of corrections for the cloud.
And here we have the MCP server enhanced
for cloud to be able to run through a
server in the cloud. That part it hasn't
been done yet, but it will uh it will be
completed eventually. So, uh our cloud
desktop will be able to connect to a
cloud instance of our MCP server. And
finally
the core engine where our streamllet
lives
our stream in reality is a very long
Python application
and our uh
our agent let's say or our our uh how
can I say uh yes our agent that provides
connectivity through uh through the
streaml main engine for our MCP and fast
API endpoint.
So you can understand here
how complex this thing became from a
very simple idea provide through the
same engine
through the same engine different uh
contexts and different users capability
to utilize this engine gateway to LLM
and tools
and it is something that can be done as
You can see
you need to be patient if you want to uh
work on this. I'm open to contributions
but it is not yet published in GitHub.
Okay. There is uh uh we have this uh I
have this blog post that explain a
little bit how this thing works.
You can search it through my
cloudblogger.eu. EU
uh blog how you can build this, but you
will need help definitely. So don't
hesitate to contact me so I can help you
uh build it. And at the time that you're
going to ping me, since I'm already
working on this, you will get cleaner
and more efficient uh code for your MCP.
I think I have said enough.
I hope that you like this session. It's
very different than the usual sessions
that uh we have presented so far.
But uh you know what in the era of AI
and uh in the era of innovation we are
allowed to experiment as much as
possible. And you know we're not only
allowed we are I mean we are obligated
to experiment. you know, if we're not
experiment now, uh, we're going to lose
the train and maybe some fine ideas
maybe get lost in time and, you know,
innovation doesn't wait. So, do it. Do
it for yourself. Do it for your friends.
Do it for your company. Just go ahead
and experiment yourself with the best
tools that are out there. uh for me is
the best hosting platform and AKS right
now is the best option for this type of
implementations. We're going to see more
remember my words of this and uh
container apps is uh uh excellent for uh
a prototyping and maybe something a
little bit similar but this type of
complexity requires AKS.
I don't know if we have any questions.
Uh I I would like to thank you so much
for being here with me and uh being
patient with me on this um
a little bit odd uh mechanism.
I think it is interesting and I think uh
that anyone that would like to take it a
little bit further will be able to do
it.
I will be there for you if you need any
help. I'm Constantinos. This is
Microsoft Reactor and we are closing our
third uh appointment of our 4 days event
series. Tomorrow we have our final
session with one of the most interesting
themes that I could think of. Imagine
having your own
uh MCP server in a web app and uh
providing your users in tune deployed
tools to utilize this web uh this MCP
server
chat interface file search or whatever
tool you want to build and everything
managed under your organization's hat
with in tune and you know everything
another very interesting idea
and uh yeah I think that's it for today
I would like to thank also our host for
having us for having me also today
our appointment goes for tomorrow thank
you so much
>> thank you everyone for joining us today
on this session like Constantino said we
do have one last session tomorrow. Uh,
and if I do say so myself, I think it's
going to be a very fun and interesting
one. So, we do look forward to seeing
you again same time right here at the
Microsoft Reactor. As always, guys, we
strive to create excellent content for
all of you and we would appreciate if
you took some time to scan the QR code
on the screen and shared your thoughts
and comments about today's session with
us. This is going to help us continue uh
creating more sessions, improving them
for all of you based on whatever is it
that you share with us. And as always,
we appreciate your time and we look
forward to seeing you here at the
Microsoft Reactor. We hope you have a
wonderful rest of your days.
Enterprise needs are rarely static—they evolve fast and unpredictably. In this session, we demonstrate how to build and deploy a Streamlit-powered AI application tailored for dynamic use cases, using scenario-based templates like Coding Assistant, Document Summarizer, and more. The solution serves both Web Clients via an MCP Wrapper and MCP-native clients like Claude.ai, offering seamless integration across platforms. The user interface is intuitive and highly configurable—users can select models, modify system prompts, download JSON logs, and manage multiple sessions effortlessly. Session orchestration is powered by FastAPI, exposing a rich set of tools to start, monitor, and control interactions—whether initiated from the web or directly through the MCP Client. Under the hood, Azure Container Apps and DAPR deliver scalable hosting and resilient performance, making this architecture ideal for demanding enterprise environments. [eventID:26505]