Let's talk about building private agentic flows. Now, these aren't just chatbots. These are agents

that can reason, take action and still keep your data completely private. Let's explore what

private actually means and why it's incredibly important for building with AI. So let's imagine

we are healthcare developers, right? We wanna use AI to summarize medical docs or craft an email

based on a patient's recent visit. We can take those docs and we can take our notes, and we can

send them to a consumer-facing web application that fronts one of the big gen AI platforms and

get back really decent results. Now, obviously do not do this, but there is

another huge problem besides the obvious. You'd be sending protected healthcare info to a third

party without proper safeguards, violating HIPAA. And you really don't want to do that.

And if you're in finance, legal or defense, just like healthcare, these public API endpoints are

usually off the table. Agentic AI is incredible, but if you're working with sensitive data,

consumer-facing LLMs and public APIs are a deal breaker due to the privacy concerns. So, how

exactly do we use the power of agentic AI in these applications? The answer is private agentic

flows where the agents that act on your behalf are keeping your data behind your firewall. In the

next few minutes, I will tell you what makes an agent private. Some of the architecture behind

these systems, and maybe a few practical steps to start building your own private agents. Let's

first quickly go over what we mean by agentic AI. The LLMs of the past, and realistically, that's

like last year, were fundamentally reactive and relatively simple. You ask a question and they

respond. But now we have agents and they can reason. They can act on your

behalf. They can do all these complex tasks. But if you're still using

LLMs that are connected to the public cloud, you cannot work with

sensitive data. So what is a private agentic flow? Let's think of it in three layers. Our

first layer is gonna be the foundation layer. And this is where our LLMs would run, right? It

doesn't matter if it's closed source or open source. It just matters where they run. Your model

must run entirely on your infrastructure, whether on prem or in your private cloud environment. Then

there's the augmentation layer. And this is where we can have our RAG or we could have our VectorDB

or we can have our fine-tuned adapters. Your agent retrieves info from your private knowledge

bases or your document repositories to ground its responses with your specific and private data.

And finally, we have our action layer, right? Now this is where the tools and the APIs that your

agent needs to use in order to function will live. So, when it makes a call to your DB or when it

does anything or creates any kind of information that it needs for you, this is where it's gonna

live. But just because we're behind a firewall does not remove all potential risks.

Right? Think about when you fine tune an LLM and you're using private data. That information is

embedded into your model. So even though your model is private and behind your firewall, that

data that's inside there, if someone gets access, they could potentially extract it. Next, and this

is a big one, is regulations, right? GDPR, HIPAA they each have different requirements.

Compliance with requests to remove personal data is a pretty complex task when data is embedded in

a model. There are some techniques that can extract it and remove it, but they are imperfect

and they are still evolving. And finally, we have insider threats, right? And it's not

necessarily that someone's trying to do something bad, but they might misuse the system or

accidentally expose data. So how can we resolve this? First,

the data that we're training the model on, you can anonymize it, right? That's big. You just scrub any

PII before it ever touches your LLM. Replace names with tokens or hashes. Remove identifiers. Strip

out anything that could trace back to a real person. Make your data anonymous from the get go.

Second, access control. Implement strong access controls. Not every single

person needs ta to touch your system, right? Log every prompt. Log every

interaction, every query, every retrieval. This will create compliance trails and ensure only those

who absolutely need to touch the data can touch the data. And finally, data minimization. Only give

your agents access to the minimum data they need for their specific tasks. Don't give your

appointment scheduling agent access to full medical histories when it only needs the patients

names and when they're when they're available. So this isn't just theoretical. Devs are building

these private agents in the most regulated industries on earth. Like we mentioned before,

healthcare, right? Developers in healthcare are building agents to help doctors summarize patient

histories, craft emails, track statuses. The agent retrieves data from electronic health records and

references medical research. But the data never leaves the network. And then we have

financial services. Teams at banks are building their agents for fraud detection customer service

where they analyze transactions, flag anomalies, use customer data, and that all must stay within

their secure infrastructure. And then we have legal.

Developers at law firms are building agents to search case law, draft contracts and identify

relevant precedents from private case databases. And once again, it stays within their network.

And finally, and probably the most regulated, is gov and defense, right? If you're

working with intelligence or defense, you need these agents to analyze classified documents or

connect dots across data sources. And consumer LLM services aren't even considered. Private agentic

systems are the only option here. Now, these developers and teams know the value of agents, and

they had to find a way to implement them. Private agentic flows are a necessity if you're

building applications that handle sensitive data. And as AI becomes more integrated into sensitive

and critical workflows, the questions will not be should we go private? But rather, how quickly can

we get there?