Whether you're building an experimental prototype for your own personal use, or creating an

application to power an entire organization, there are key components of the AI technology stack

that you must get right to build AI systems that can do more than just generate answers but solve

real, meaningful problems. Say, for instance, I'm building an AI-powered application to help drug

discovery researchers understand and analyze the latest scientific papers in their domain. Maybe it

starts with a model that I recently heard about that is supposed to be better

at highly complex tasks like that of a PhD researcher. Model is an important layer of the

stack, but it's just one piece of the puzzle. There's also the infrastructure that that model

will run on, because not all LLMs, large language models, can run on

standard enterprise CPU-based servers, and not all are small enough to run on a laptop. So it

matters what infrastructure you have access to and how you choose to deploy it. Next is data

because in this example, the whole point is to help scientists understand the latest papers in

their field. And models typically have a knowledge cutoff date. So if we want to talk about papers

from, say, the past three months, that means we have to provide the AI system with extra data.

That will be the data layer. Next would be the orchestration layer. Because to do

a complex task like this probably is going to require more than simply providing a large prompt

into the AI system and getting an output, a single output, out. Instead, we'll want to break that user

query up into different parts. Help um, plan how the AI solution is going to actually

tackle this problem, what data it needs, and then do the summarization and creating an answer and

maybe even review that answer. Finally is the application layer. And this is because at the

end of the day, there's a user using this tool. So there will have to be an interface that defines

what the inputs will be and what the outputs will be. It might not be as simple as text in and text

out. And there's also the issue of integrations. So, will the actual results of this be something

that's integrated into other tools that this user uses? It's important to understand

all the layers of the AI stack, whether you're building a solution from scratch or using

solutions which might manage several of these layers for you as a service. This is because

across the stack, from the hardware, all the way up to the user interface level, the choices you make

will have important implications on your solution's quality, itsspeed, its cost and its

safety. When it comes to infrastructure, LLMs generally require AI-specific hardware,

specifically GPUs, and these can be deployed in one of three ways. The first would be on premise,

that is, assuming you have the means and resource to buy this kind of infrastructure yourself.

Second option would be cloud, and that would allow you to rent this capacity and be able to scale it

up or down as needed. Finally would be local, which usually means on your

laptop. Not all lap ... laptops can support LLMs of different sizes, but there are certainly LLMs on

the smaller end of the range that can be run on the kind of GPUs available in a standard laptop.

The next layer is models. So AI builders have plenty of choice when it comes to what model they

can use. One dimension to consider is whether the model is open

versus proprietary. Another dimension is the model

size. So we have large language models; we also have small language models that might

be lighter weight and able to fit on more lightweight hardware,uh, but might not have exactly

the same thinking capacity as a large language model and instead be specialized for more

specific things. Finally is specialization.

Which sometimes goes hand in hand with size. Some models might perform better on things like

reasoning or tool calling or generating code. Others might have different language strengths

than others. There are plenty of new models over 2 million already in model

catalogs, like Hugging Face that can serve any mix of these different needs that an AI builder might

have. The next layer of the stack is data. This breaks up into a few different components,

so the first would be data sources themselves to supplement the model's knowledge.

This could also include the pipelines to do any processing,

pre-processing, post-processing of that data, as well as vector

databases you may use. Or retrieval systems,

also known as RAG. Vector databases is the step where that external data is actually vectorized

into embeddings that are saved so your model can retrieve that context more quickly and augment it

with this additional knowledge that the base model does not have. That's important because base

models are usually trained on publicly available information, which might not always be complete to

accomplish the task that you have. You might need to supplement with additional data. The next layer

is orchestration, because building an AI system that does something more complex than

just generating text or answering questions requires breaking the initial user input down

into smaller tasks. Those can start with things like thinking,

using the model's reasoning ability to plan out how it will tackle the problem. That

can also include things like execution, where the model does tool calling

or function calling, as well as steps like

reviewing, where an LLM can actually provide its own critique of the initial generated

responses and initiate feedback loops to even improve those responses. This layer

is very quickly evolving, with new protocols like MCP and new architectures for how to best

orchestrate increasingly complex tasks.

Next is the application layer, so the most widely used AI systems do follow a pretty simple design of text in and text

out. But as we use these tools in our work in life, there are important features that become critical

for the actual usability of AI and these factors make up the application layer.

First factor is interfaces.

The most classic interface is text in and text out,

but there are other modalities that can be very valuable for certain tasks too, like image, audio,

numerical data sets, and plenty of other custom data formats.

Also, in the interface,

it's really important to keep in mind the ability to do things like revisions or

citations so that when the user sees what the model comes up with, they have the ability to edit

that or inquire on it further. The second consideration is integrations, and that

comes both in the form of integrations, of allowing other tools that the user uses to

actually send inputs to the AI system, or to take the

model outputs and automate how that gets integrated into some of the tools that they use

in their day-to-day work. All together, these layers of the AI stack, from the

hardware to the models, the data you use, how you orchestrate it, and the application and the

usability of it, matter because when we have a clear understanding of how they fit together, we

can see what's truly possible and make practical choices to design AI systems that are reliable,

effective and aligned to our real-world needs.