Welcome everyone. Uh today is going to

be a fun workshop on building an

internal developer platform

uh based off of the backstack. So uh

let's introduce who's here with us

today. Anna.

>> Hey y'all. I'm Margarita Medina. I'm a

staff developer advocate at Upbound and

I also sit on the Kubernetes code

committee.

>> Awesome. And Christian?

>> Yeah. So Christian Hernandez uh from

Cisco is valent uh technical marketing.

I'm also a Argo CD contributor

maintainer.

>> Awesome, Courtney.

>> And my name is Courtney Nickerson. I'm

head of community at Nermatada, the

creators of the Coerno project, and I'm

also a CNCF ambassador.

>> Awesome. And I'm uh Scott Rosenberg, uh

lead architect in the CT office at Teras

Sky, uh and a platform engineering

ambassador. Um so awesome. Um, and today

we're basically going to go over a bit

of slides at the beginning, intro to

platform engineering, intro to the

projects we're dealing with. Um, then

we're going to do a hands-on guided

workshop. Um, where we'll go through

really building out the platform and

going through the different use cases.

Uh, there's some free time as well.

There's guides with documentation in the

labs for everyone to play around. We'll

be walking around if there are questions

and everything. Um, and then a wrap-up

at the end.

Um and yeah really the projects we're

talking about today is the backstack

right backstage Argo Crossplane and

Chaivero. Um and this is basically going

to be our cornerstone for building this

internal developer platform. Um, but

simply because of conference Wi-Fi and

everything, um, we're going to start

with something before we talk about the

projects, which is if people go to this

GitHub repo here, um, scan the QR code

with your computer if you want or just

go to the URL or the shorten URL. Um,

what you're going to do is you're going

to fork that repo. Um,

and then basically you want to start a

code space. So in order to do this just

fork the repo um and then open a code

space. I'll do that on the screen as

well just to

uh how do I do that? Second

duplicate screen.

Switch to the right user here.

Um yeah. And you're basically just going

to want to fork it.

And then just on the code button, you

can code spaces open in code spaces. Do

not just create the code space. Actually

do the three dots with new with options

and select the four core option. Um

everything we're doing here will fall

into the free tier of code spaces. So no

need to worry. I do recommend people

that have a credit card set up on their

GitHub account to shut this off to tear

down the code space at the end of the

workshop otherwise your credit card may

be build heavily. Um so just change this

to the four core model uh in US East and

create that code space. Um this is just

going to take about 15 minutes in the

background to do the download and setup.

So once we get this set up for everyone

um you know this will happen in the

background as we go through the slides

here. Um so people can do that and let's

go back to

>> You want to put the code up again?

>> Yeah. He's going to

>> Exactly.

>> There we go. Yeah.

>> So that's the URL again. I'll give

everyone just like two minutes to do

this and then we're going to continue on

to the projects.

should have had music or something

playing while

some like background

>> some lowfi beats.

>> Yeah. Yes. And some lowfi. Yeah.

Something to get them concentrated.

>> Exactly.

>> Yeah. Or some like Jeopardy music.

>> Yeah. Yeah. Jeopardy music or

options.

>> Yeah. So, when you do the new with

options, what you want to change is by

default it's going to be two cores. You

want to change that to four core. Um,

spinning up the entire backstack does

take about four cores to make it

actually perform normally in a

constrained environment. Um, spinning up

an entire Kubernetes cluster and a bunch

of stuff.

>> It it takes it takes it a while. So,

we're having you do this in the

beginning and then while you're waiting,

we'll uh we'll go through what what uh

what every every component does.

>> Exactly.

>> All right.

Um and GitHub Codespaces does have a

desktop version as well as web. We

recommend using the web version here. um

it's much easier uh to get working and

all of that and there's some less quirky

behavior when you use GitHub code spaces

in the web browser.

>> Um so that is the strong recommendation

I would say.

>> Um

but awesome.

>> Yep.

>> Yep.

>> Awesome. So let's start and we're going

to start this time in the order of the

backstack. So we're going to start with

backstage. Uh so let's talk for a moment

on what backstage is right backstage

there is a common misconception and lie

told that backstage is an internal

developer portal that's an absolute lie

it is a framework for building an

internal developer portal and that's

very important to know because it's not

a product there are products that exist

out there some of them have booths

downstairs that deal with these things

for us that build internal developer

portals as a product some of them are

even based off of backstage but

backstage is a framework that allow

allows us to add our own plugins and to

really customize it to what we want our

portal to look like. Um, it is powered

by a central software catalog and this

really is about bringing back order and

really trying to bring order to the

chaos of the IT landscape within all of

our organizations these days. Right? If

we really try to sum it up, a developer

portal is one front end for your entire

infrastructure. And that's really the

idea here. Now, if we take a look at

backstage, right, what is it really all

about? Backstage is about letting any

developer do a few things. Number one,

create new software in seconds aligned

to your best practices using a tool

called the software templates or the

scaffolder. Um, to manage all the

software that we own in one centralized

location as well. Um, also to explore

our entire software ecosystem, right? To

enable really that collaboration across

our organization. One of the biggest

challenges is, oh yeah, I created

another module in Terraform of a VPC and

now we have 37 because I didn't know

about the other 36 that already existed.

Um, and having this exploration and

knowing what exists is really important,

right? Backstage has a very customizable

and extensible plug-in architecture with

both frontend and backend plugins. Um, I

will say it is not for the faint of

heart if you do not know JavaScript and

TypeScript and React and all of that.

But the good thing being based off of

those frameworks is that they are common

knowledge within there are a lot of

people that have them. This isn't using

some esoteric language that is difficult

to deal with. Um but it does require

TypeScript and React uh knowledge to

really get running. When we look at

really the key components, it really

starts with this software catalog which

is where we can see all of our different

components, who owns it. We can create

hierarchies of which system a component

belongs to, which resources,

dependencies, APIs, things like that. We

can go into our components themselves

and get this visualization where we get

resource graphs of dependencies of what

relies on what, things like that. Links

out to technical documentation to the

source code. Um, we also have software

templates, right? This is where we're

able to click on a template, fill out

forms of what we want, and have some

type of action run in the back end.

Usually scaffolding of a new GitHub

repo, something like that with things.

Um, but completely customizable, and

we're going to be going through that in

the workshop as well. Um, we also have

API documentation, right? So, we have

the ability to get all of the APIs of

our services documented here and linked

up to our components, making this really

discoverable and easy, as well as

technical documentation. Right here you

can see just an example the entire

backstage documentation hosted within

backstage um right which is a really

powerful thing that we can have our

documentation living alongside our

components and everything visible from

one place um it's used at a lot of

companies um it's used across every

industry it's the if I remember

correctly the 11th most contributed to

project in the Linux Foundation uh let

alone the CNCF um it is a hugely widely

adopted project. Um,

and really when we look at the

ecosystem, there's over 250 plugins in

the official marketplace. Uh, there's

100 plus plugins in the community

repository, which is like a community

repo owned kind of by the backstage

community itself. Um, obviously we have

well there's no backstage plugin because

it's backstage, but the rest of the

backstack has plugins. Um, so the axe

stack. Um, and here's just a few of the

other common ones that exist, right? And

common tools in the Kubernetes space, in

the DevOps space, things that we have in

our environments, be it the cloud, CI/CD

tools, source controls, secret managers,

and security, uh, cost optimization

tools, monitoring, right? Really, the

whole plethora of things um that make it

an extremely powerful thing. And you can

always build your own plugins as well.

Um

just a few key things that have changed.

Um we have the new front-end system is

now G. We're going to be using that

here. What this means and the reason

this is important is that people have

looked at backstage as oh it is so

difficult just to maintain it. I need to

know React. I need to know all of these

things to integrate a plugin that you

didn't write was knowing a lot of React

to hook things up. That's not true

anymore. Yarn install. You literally add

a dependency and boom, it's in backstage

because we have the ability now with

this new front-end system making

basically a no code integration of

plugins possible which is extremely

powerful. Backstage can now be used as

an MCP server. Obviously, we need to

talk about AI because it's 2025. So,

there will be AI in the workshop as well

because you know otherwise there's no

money in anything today. Um

we also moved uh from Webpack to RSpack

um for packaging up uh backstage. This

is a major change. Um it make the build

times much faster. Um so really um it's

a great thing with basically no uh bad

sides to it. Um and the documentation

site is being revamped to make it easier

to get started and to find what you're

looking for. I apologize for anyone that

has tried to figure out what they're

doing in the backstage documentation.

Um, but I'm it's getting a lot better.

Um, so I'm going to pass it off now to

Christian to uh talk about Argo.

>> Awesome. Thank you. Appreciate it. All

right. Uh, I'm going to talk about Argo.

Brief introduction to Argo. Um, you

know, what what Argo is, right? So, um,

Argo in general, right, is a suite of

DevOps tools for Kubernetes, right? So a

lot of people when when when you say

Argo when you mention Argo the first

thing they think about is is Argo CD

right but it's actually the Argo project

is made up of four uh tools right it was

originally developed at into it um in it

uh made the strategic d uh decision to

go all in on Kubernetes so no more VMs

nowhere cloud instances everything on

Kubernetes um everything cloudnative but

they had no way to um onramp their

applications onto Kubernetes, right?

There was at at that time there were

very very early adopters. So they

created these tools and then um to help

accelerate their adoption of Kubernetes

and so uh they later they created these

tools and they later donated it to the

CNCF. Right? So the four tools are uh

Argo workflows. We we call them the big

four right the Argo workflows, Argo CDs,

Argo rollouts, um uh Argo events and I

said four but there's a secret fifth one

that we say we call Argo Helm, right? So

um and the and the reason why um uh we

we don't really call it a uh um uh one

of the big four but it is an official

project is because that is the official

way to install Argo is using the um any

one of these tools is to use the

Helmchart. It is managed and maintained

by the entire community people from

Intuit Red Hat um uh Acuity you know all

the other major players they actually

actively maintain that right and so

anytime someone asked me what's the

official way to install Argo is Argo

help so too long didn't read right for

the background of Argo is that into it

went in all in on on Kubernetes and just

needed tools to manage their production

workflows right but you're not here to

hear uh about entirely about the whole

ecosystem We're going to talk about um

Argo CD and like I like how that

rendered not very well. It's looks like

I'm going to talk about Argo rollouts.

So um the conversion right between uh

slide.

So uh Argo CD at its core it is a GitOps

tool right and so um and for those of

you who don't know there are actually

four GOPS principles right if you go to

openGithops.dev dev you can read all

about it right um Argo CD basically

conforms to the GitOps principles right

something that's declarative um

something um that it it it takes its

configuration from uh uh from its

storage that is version and immutable

right so something like get is version

immutable S3 Helm right all that's good

stuff OCI uh it's pulled automatically

right meaning there's it doesn't rely on

an event right it's not an event-based

system right it basically runs every 3

minutes to check uh if there's a change

in your um in your source of truth,

right? In git, uh it's continuously

reconciled, right? I said that like the

three minutes uh type of u loop, right?

So when I say loop, think about like

control theory, right? Um closed loop

systems. I always just say what is

githops? GOPS is uh a thermostat, right?

So I remember my first apartment, right?

writing going to college had like those

wall mount air conditioners. Uh when I

got too hot, that was an event, right? I

got up, I turned it on. When I was cold

enough, that was another event. I stood

up and I turned it off. Whereas GOPS

acts more like a thermostat where I set

my desired state, right? I set it to 72

or if you're from Europe, 20. Um and the

thermostat continuously monitors the the

ambient temperature and turns off and on

when it needs to. Right? That's githops

in a nutshell. Um, and uh, what Intuitit

used Argo CD for is uh, really an

application stack deployer. And so

really it it looks at Git for changes

and it'll synchronize the uh, the

manifest for your application onto any

number of clusters, right? However many

clusters you attach to it. It's a hub

and spoke design. So um so when you

think about your application in terms of

Kubernetes right your application is

made up of kind of loosely coupled

objects right you have like a service

you have a deployment maybe a stateful

set uh maybe a persistent volume claim

uh maybe a gateway API you know HTTP

route uh they're all kind of loosely

coupled with labels right so Argo CD

allows you to manage your entire stack

right because your application is made

up of you know a uh a mountain of YAML,

right? That that you have some sort of

so but you can manage it as like kind of

a single entity and uh it's not just

Git, right? We we we we say get ops um

because it's, you know, it was catchy.

It's kind of earwormy uh but it actually

supports uh things like um like Helm,

right? Like it'll go and pull things

from Helm uh from Git obviously uh from

OCI or like from customize, right?

Right? Like if you have like a

customize, it'll automatically detect

you're using customize and it'll render

those out. Um OCI, I actually did a talk

yesterday about OCI and Argo CD. So if

you're going to look at the recording

from Argokon, um I did a I did a talk

about that. And uh while uh while

Scott's standing here um a a flux

person, right? So I do want to um

address the elephant in the room. Like

one of the one of the things that I get

asked a lot um I've been in the the

GitOps ecosystem for a long time now. I

was like, "Oh, what's the difference

between Argo City and Flux, right?" Um,

and uh really like the it's it's it's

just nuance, right? Because the both

projects approached the same problem

from a different angle, right? Um the uh

We Works developed Flux internally and

they wanted to keep the lights on. They

wanted something for their DevOps

engineers to be able to to deploy and

manage their Kubernetes clusters.

on the Intuit side, they wanted

something that's developer friendly to

be able to deploy applications and adopt

Kubernetes, right? So they they saw the

same problem. They kind of uh solved it

in a different way, but at the core,

it's it's they're actually very very

similar. They're both GOPS tools.

They're both GOPS compliant. Uh they

both do a lot of the same thing. Uh but

kind of just some of the things here. Um

Argo CD u they started from the UI down,

right? So they created a UI uh because

it was for their developers. Um, uh,

Flux, you need to get a UI from like

third parties, right? Um, Argo City is

actually built as a monolith, right? For

better or worse, it's built as one big

app, you know, app application, right?

There's different controllers and things

like that, but it's it's built like a

monolith, right? You you can't take

those individual pieces and use them.

Um, whereas, um, Flux was built on the

GitOps toolkit, right? A series of of

of, uh, of of libraries and, um, and

controllers that you can actually use

independently. You can actually use flux

as a library for your, you know,

basically the GitOps toolkit, right?

There's like a toolkit that you can use.

Um, and Flux is just an implementation

of the GitOps toolkit if you can think

about it that way. Um,

uh, Argo City implements its own

arbback. This trips up people a lot. Um,

this is actually one a point of

contention is that Argo they it manages

its own arbback. Um, and Flux just uses

whatever Kubernetes does is basically it

uses the Kubernetes arbback system. Um,

Argo CD for better or worse, uh,

emulates cubectl as much as possible.

Um, we talked to the Intuit guys, that

was kind of like their core philosophy

is like, okay, we want to emulate

cubectl as much as possible, even with

all its uh, um, kind of unexpected uh,

side effects. We even if you know, even

if it doesn't make sense. Um, and it

uses raw YAML. That's another thing uh,

that that Argo CD does, right? If you do

a Helm Helm ls on an Argo CD system, you

won't see anything even if you're using

Helm charts because Argo CD will render

those out and will apply those as raw

YAML. The decision was that for like

there's no it didn't want to write a

controller for each implementation,

right? Like it it wanted to always use

raw YAML. Um and Flux uses the the

standard uh libraries, right? It uses

the Helm uh standard library for example

to deploy Helm charts. um support for

SSO integrations, right? Um SSO for Flux

is really only third party. I think

control plane has has put a lot of work

into that. Um and you can extend it for

other config management tools. Um and

right like if you're using qlang, I get

but I think maybe qlang no one's using

it anymore. But if you have like another

uh config management tool that you would

like to use, Argo has a plug-in system.

um and extending other config

management. It by deep by by the flux

core is the not currently available. So

kind of very very nuanced, right? But at

the core it does it does the same thing.

And so I do want to end my uh my my

portion for a shameless plug. I wrote a

book on Argo CD. Uh so um it's sponsored

by Acuty. So thanks to Auty sponsoring

the book. I'm doing a book signing

tomorrow. So, if anyone wants a hard

copy, uh they have it at the Acuity

booth tomorrow, uh free hard copy, but

if you won't be there or are doing

something else, um or watching this

later on, uh go ahead and scan that QR

code and you get a free um uh PDF copy.

So, and I think with that, I am going to

pass it on over to uh to learn a little

bit more about Crossplane.

>> Send that email to you.

>> What's that?

>> What's that?

>> Send that email to you.

No, you you just scan it. You should be

able to fill out the form and it'll

email it uh email to you.

>> Oh, you have you have your mic.

>> Thank you.

Hey y'all. I'm going to be doing an

introduction to Crossplane.

If y'all don't know, crossplane is a

control plane framework used for

platform engineering. We know how

Kubernetes is based on control planes.

Well, crossplane extends Kubernetes to

manage any resource anywhere. So you can

think of that as like resources that you

have on the cloud, third-party resources

as well as anything that you want to

create custom on-prem resources also

applies. So we get to see that we get to

use a declarative model that Kubernetes

is known for along with that

reconciliation loop and really put like

Kubernetes as a center of a way of

building your platform. Sometimes

crossplane gets grouped together with

infrastructure as code and I like

telling folks that you're building a

platform instead when you're comparing

it to other tools like that uh because

you get to base on the recities.

Okay, this allows for platform teams to

expose infrastructure and application

APIs for developers. So if you have

heard of crossplane in the past, maybe

you've heard of crossplane before 2.0 0

where it was mostly focused on

infrastructure. With crossplane 2.0,

we're really extending that and letting

folks do infrastructure and applications

within their control plane. And we're

also a newly CNCF graduated project as

of last week. So, we've gotten a chance

to have over 100 million downloads. We

have contributors from over 450

companies and we have folks from many um

companies contributing and adopting.

when you want your developers to use

crossplane, they get to use it through a

developer portal like we're going to be

doing today with uh backstage. You can

leverage like your developers can

leverage it via console or just CLI on

its own. So when someone is using your

API, they get to just request a

composite resource and this composite

resource can actually create Kubernetes

objects. They can create managed

resources on the cloud or they can

create any third-party uh resources that

you also want to do.

We see some logos of some crossplane

adopters. There's like

67 I think that have like publicly

talked about their adoption on the

adopters um markdown page if you want to

take a look but being used in production

worldwide across many many industries

and wanted to go over some of the

crossplane concepts that you kind of

will get to see in this workshop today.

First we have managed resources. These

are the Kubernetes representations of

the resources that you are managing with

crossplane. Then behind that we have

composite resource definitions. These

are the blueprints of your APIs.

Basically the schemas for your composite

resources. And then the compositions are

going to be defining which way the

resources are going to be created and

how. And then we also have packages

which that's the extension of crossplane

where we have providers that allow for

you to connect to like AWS, GCP, Azure,

um Kubernetes and other things like it.

Then we also are allowed to extend the

logic of when you're building out your

compositions. You can do this in your

preferred language as like Go, go

templates, KCL, and Python. You'll get

to also see that on the GitHub repo that

we've documented that. And when you're

ready to version the way that you're

offering your platforms, you can

leverage configurations. These are ways

that you can allow folks to use your

reusable resources at scale. And with

Crossbane 2.0, we've also introduced

operations. These things keep things

running smoothly over time, whether it's

rotation of credentials,

uh, uh, rotation of keys and backups and

anything that's like schedule

maintenance. And then the resources that

your developers are going to be using

and like requesting are composite

resources. These are just a single

Kubernetes object based on the

composition that you have written. In

the past before crossplane 2.0 you may

have heard of the term claims. That's no

longer something that we're going to be

using in crossplane 2.0.

So we're enabling self-service through

APIs that allows for platform engineers

to give reusable extractions to folks in

your company to leverage cloud

infrastructure on prem resources,

thirdparty services. Developers are just

going to be requesting those composite

resources through YAML files. You can

maybe abstract the YAML files through

something like backstage and then this

is going to integrate with GitOps with

Argo CD and flux being something you can

do. You can also integrate with portals

such as backstage and do policy with

things like Cerno and Opa. And with

that, I'll pass it over to Courtney to

talk about Kerno.

Thank you.

Um, great. So, let's talk about

Kyiverno. I'll try to be quick because I

see time sticking and I want to make

sure that everyone has time to get

through the workshop. So, Kyiverno is an

open- source policy

for Kubernetes in the beginning, but now

you can also leverage it on JSON

payloads. Um, it leverages declarative

YAML and recently we had a release that

added cell policy based types also. So,

if you're not familiar with common

expression language, it's also been

adopted by Kubernetes as of late. Um, so

very interesting things going on there.

Kybero also helps you manage your full

resource life cycle um as well as

integrate reporting a fine grained

expectations and it was again built

specifically for Kubernetes. So you're

going to see the power of that in our

workshop today. You can see um if I

change this slide there we go. You can

see we have a lot of very big adopters

within our community. Our project has

actually been around for almost 10

years. Um, and so we we have very large

enterprises that are using our policy

engine at scale. Uh, but we also have a

very active community. So again,

tomorrow if you're interested in getting

involved in contribution to a project

like ours, we do have a contrib.

I think some of our other projects might

have them as well. But uh please do join

into those so you can learn more about

uh our project and and meet other users

to get other answer other um questions

answered.

Uh quickly what is policy is code?

Policy is code is a way to leverage

policies specifically in the same way

that you do with code. So it gives you

declarative rule definition. It allows

you to write your policies in a format

that's easily readable by humans. um it

makes it much more accessible for your

developers and your operations teams in

order to be able to use them and

understand the the power that's coming

from them and have actionable ways to

make things better and more compliant.

Um it also integrates version control

which is very important and one of the

reasons that we do function very well

with tools like Argo CD uh because you

can store, review and version your

policies exactly like you would your

application code. Um uh it also

automates enforcement for different

things, not just security, but resource

management, CPU limits, naming

conventions. You don't have to use

policy just for security applications.

You can use it for just about any

anything that you need to set a

guardrail on to help your team become

self-service. Uh and finally, it makes

all of your deployments more consistent

and auditable. So you rech you achieve

um better repro you can reproduce uh

your your applications and your

deployments in a much more efficient

way.

So why do policies matter? They matter

because they allow you to standardize

things at scale very easily. Um they're

going to make things more consistent

which we already said. you can make

things a much more safer development

environment uh for your team and they

can self-service because they now know

what they're allowed to do and what they

shouldn't do. Uh so it reduces context

switching and makes things a lot easier

on on them to know what they can and

can't do. It eliminates bottlenecks

above all because now they don't have to

ask for permission for things. They they

automatically have things in place that

are set up for them. Um and it provides

governance without friction. Nobody

really wants to have to talk about

governance and so this is a very easy

way to one not have to talk to your

security team because nobody wants to do

that and also not have to talk to your

compliance team. You just give them a

report because also nobody wants to talk

to those guys either. Let's let's be

honest none of us want to talk to them.

Um quickly our architecture we are

admission controller so it runs at

Kubernetes admission. uh it intercepts

your your API request before your

resources are actually persisted to etc.

Um then the policy engine itself works

by web hooks uh which pass that request

onto our Kubernetes policy engine. Uh it

applies either validate, mutate or

generate logic there um based on

whatever policies you've put into

action. And then it is a background

controller um that allows for for

cleanup of your operations and then

provides policy reports. We have our own

reporting but we also have a side

project that we um collaborate with

other open-source tools and policy

engines called open reports uh that you

can leverage with Graphana, Prometheus

or any other reporting provider that you

need. Um there's quickly our our

architecture from our documentation.

Um as all architectures, it's

beautifully designed.

Common use cases for policy. Most people

think of policy as only security and

that's great. You can use policy and you

should for security, but you can also

use it to better your operations. You

can use it for cost management. As silly

as this sounds, most devs are shipping

their things and they put a very high

CPU limit because they just want it to

work. And then you have an S sur that

has to go through and figure all of that

out where you can just set a CPU limit

that's actually what you need and it

cuts your costs exponentially right from

the beginning because people know what

the CPU limit is. This is a very simple

use case that saves a lot of money for a

lot of our end users very quickly. Um

security is clear. We we obviously have

a lot of security use cases specifically

right now a big one is software supply

security supply chain security um

validating image policy is something

that everybody that's adopted Kyiverno

typically uses because you can set what

uh libraries people are pulling images

from and make sure that they've all been

signed with cosign or sig or whatever it

is that you're using. Um and and that

automates your security.

Security

operations

security.

So, if anybody didn't hear his his

question, he's asking if we're a

framework provider that has a framework

that's going to give you policy for

operations, cost management, security,

and automation. Correct? So, we don't

we're not actually a framework provider.

You can pick and choose all the

policies. You create them yourself

because obviously everybody's going to

have niche situations they need to adopt

to their own environment. What we do

have is a library of policies that that

actually cover all of these use cases

plus quite a few more. More than 400

policies that our community maintains on

its own that you can go through the

policy library, find something that fits

for the use case that you have, copy it,

adjust it to what you need and and use

it. And we do maintain all of those as a

framework.

It depends on how your team is actually

set up. In some cases, you have a

platform team. The platform team

determines what policies are going to be

used for each one of those things. Other

people have their security team creates

their own security policies and they

have a team that's working on resources

or their SRES oftent times create

policies that are geared more toward

resource management. Um operations

oftentimes you have a a team leader of

some sort that's that decides what

policies to use and then sometimes

there's a group of people that just

manage the policies for everyone to

provide those golden pathways. So it's

it very much depends on the size of your

company, the industry that you're in,

how compliance checks are set up, if

your focus is more on governance, if

your focus is more on compliance and uh

and to be honest, most people start off

with three or four policies that are

very much security based because policy

is kind of a heavy lift and you have to

get people used to having them. They

start off with something really small

and then that expands very quickly

because they start to realize, oh, I

have a bunch of other use cases. And so

that might be, okay, I'm passing this

information off to another department so

they can use it or another team. Or it

might be that they just start creating

more policies within their pipeline as

new use cases come up.

I'm gonna we can talk I'll come talk to

you and and we can continue the

conversation so that we don't run out of

time for the workshop. Okay, great. Um

so finally main takeaway and we'll wrap

it up here so we can get started.

Kyivero makes policy writing very simple

because it's in a declarative language.

It's very simple to get started with it.

They're super powerful. They execute a

thousand1 use cases, which I'll continue

to discuss in just a moment with him. Uh

they future proof all of your

infrastructure and make everything much

more um self-serviceoriented for your

dev team by giving them actionable ways

to fix whatever it is that that your

guardrail is putting in place for them.

And with that, I will pass it off to

Scott to get started.

>> Awesome. So, let's get started with done

with slides. Let's do the interesting

things. Um,

so let's duplicate screen

and let's yolo this. Uh, hold on.

>> Does anyone else need the QR code that

came in late for the GitHub repo?

>> Scott, do you want to put it up a

second?

>> The QR code. Yeah. One sec.

Find out somewhere.

Right there.

>> Thank you.

>> Okay. So

>> yeah, you could you can find in the the

backstack back-stack GitHub uh

organization.

>> Exactly.

>> Yep.

>> Awesome. So basically now that we have

our uh environment up and running. Um

this basically went and installed the

ACT stack. It did not install backstage

for us. That we're going to do here

during the uh workshop itself. I do

recommend people that are using Chrome

and whatever, there is this install code

spaces button that kind of starts up at

the top. If you click on that, it's just

going to open it in like a web app

that's a bit easier uh and nicer to use.

The functionality is the exact same,

just looks a bit more like an ID and

isn't in a browser tab. Um, with that,

let's get started here with the actual

workshop. So, we can see here basically

we have these instructions for setting

it up. We're going to go with the web

IDE uh implementation here. And there's

two things that we're going to have to

create now on GitHub. Uh if you have a

personal access token, awesome. Um you

can use that. Otherwise, you can just

follow the link here to create a

personal access token uh for your user.

Um

yeah. Uh just give it a name.

I'm very professional there. Um,

yeah, you all can feel very This is

going to be deleted and it's on a fake

user, so there's not much that you can

do with it. Uh, but I'll do this live

here just to show everyone. Um,

but we're just going to create a

nice little one here. That should be

good enough. Generate token.

And I've got my token. And we just want

to copy that off to the side here. Um,

so I'm just going to open up a new file

and just throw it into there for a

moment. Uh, text. It's fine. Just throw

this into here. That's our personal

access token. And the second thing that

we're going to have to do is create an

OOTH app. And this is how we're going to

enable authentication with GitHub

directly to Backstage. Um, so what we're

going to do is click on this create

GitHub OOTH app. Um and this is going to

take you to this register a new ooth app

in GitHub. Um in order to get the values

for this all you need to do is right

under uh that in the readme um there is

this kind of um section here you can

just paste this copy it and paste it

into the terminal.

Um

and this is going to basically give us

three different lines. the application

name. It can be whatever you want it to

be the application name, but what really

matters is this homepage URL and the

authorization callback URL. Those two

have to be set correctly. Um, so just

copy them.

Yeah.

Uh, hold on.

How is that? Is that better

size-wise? Hopefully.

Um,

But, oh, by the way, we're walking

around. So, if you need any help with

anything, just feel free to raise your

hand. I know some have been helped

already, but if you have any questions

or anything or get stuck, uh, we're

walking around there. Oh, I see you. I'm

coming.

Okay. So, what we're going to do is copy

these values out. Um, once we have them,

we just fill them in here, uh, for the

authorization callback URL. Give it a

name, the homepage itself. Um, and with

that, um, we can enable device flow and

then register our application. This is

going to create a new GitHub app for us.

specifically switch it to light mode.

>> What to light mode

>> if you can.

>> That's a good question. I don't even

know how you do that.

Uh, one second. Can try and figure that

out. Never used light mode in my life.

Uh, one second. There's some way here.

>> No, but this is in code spaces. I'm not

sure how I don't know how you do that in

VS Code.

>> Upper left

>> preferences settings. Oh, themes.

Color theme. Maybe

that. There we go. And light visual

studio I guess.

Yeah. Okay. Hey, I guess we'll you work

this way. Um, hopefully this helps

people and I don't go blind. Um,

awesome. So, with that, we've added in

our things. I hope that helps. Uh, we'll

be done in the web browser in one

second, so we won't see the darkness

here. Um, we have our client ID, which

we want to copy. And all we want to do

is generate a new client secret. So,

you're just going to click on generate a

new client secret. And we're going to

copy that value as well. and we're just

going to throw those into our text file

for a second. So, we have our personal

access token, we have our client ID, and

we have our generated client secret. Um,

these are the three things that we need

to generate. Um, that are going to allow

us to render everything we need to

install backstage. Um, so now that we

should have those, all that we're going

to need to do is actually come and

export a few values with those values as

well as your, for example, GitHub user

um, and things like that. So, what we're

going to do here

is just paste uh, that uh, section from

the readme. And what we're going to do

is change your GitHub token to be that

personal access token that we created at

the beginning.

We're going to change the client ID and

client secret as well to be the same

accordingly

and then for your GitHub username and

your repo name. This is the repo that

you have either forked or any other repo

that you want to use as a GitOps

repository that is going to be used by

Flux CD that your user is a part of. So

you can have a separate repo. It can be

the same repo. It's going to go in a

subfolder. So all is good. I recommend

just for the ease you can just use um

the actual repo that you're working on

here. Um and it'll just be in a

subfolder. Obviously in production

you're going to do this in its own repo.

Um but basically so we can see in my

example it's using v Rabbi-demo

uh which is my user. Um and then there's

this CubeCon NA2025. So that's fine. And

we're just going to use that same one

here. So, VRBI demo

and then your repo name is CubeCon

NA

2025.

Or was it 2025 NA? Yeah, that's right.

Okay, awesome. With that, we're just

going to export those variables quickly.

Um, so we're just going to copy that and

run that in the terminal.

And with that done, we're almost done

here with the setup. It's just like two

more steps. Um once we have that, we're

just going to run the rest of these

commands. There's nothing else that we

need to change. We just need to run

steps two through five. Literally just

copy them and paste them into the

terminal. Uh what this is going to do

basically is we're rendering out our

Argo CD uh manifest, which we'll get

into in a moment what we're doing there.

But we're using a uh mechanism called an

application set in Argo CD. What that

allows us to do is basically based off

of different generators, it's going to

automatically generate the Argo CD

applications for us. Um, a really nice

capability um that uh Argo CD has. Um,

so we're going to be using that actually

to generate applications on the fly for

us uh throughout the workshop. Um, and

then we're going to basically template

out our backstage values with those

different uh GitHub based on those uh

GitHub outputs that we had from before.

And we're just going to helm install

backstage basically um with a custom

image uh that we've pre-created into our

environments. This is going to take a

minute or two uh just to start up

Backstage. Um,

so, uh, we'll give this a moment to run.

While this is running, um, there are a

few notes here that are kind of

important. Um,

what you are going to want to make sure

there is, um, when you go to the ports

tab in VS Code. Uh, you can see that

there is port 443. Uh, that is open. Um,

in code spaces, it gives you a public

URL basically that you can use. Um, but

the visibility is set to private for

this workshop. Because of some of the

things we're doing, we have to just

rightclick on that line and change port

visibility from private to public. Um,

and that's going to enable us some of

the features that we need um later on in

the workshop. Um, so that's just uh

important uh to set up um so that things

work correctly. Um basically just so we

can while people are doing this uh I'll

give you guys a minute while backstage

is uh installing also what did we

actually pre-install here right what was

going on during that setup um it is also

important to note this setup has

documentation to be run locally on your

machines um so anyone that wants later

on this is an open repo feel free to use

it it's part of the backstack

organization and you can use this as a

playground to test things in your labs

and anything like that. Um, what the

initial setup script sets up is a kind

cluster. So, our environment is built up

of a kind cluster. Inside of that kind

cluster, we've installed Argo CD. We've

installed Chyoert Manager, ingress

engine X. Um, we've installed

Crossplane. We've also installed a bunch

of sample crossplane configurations,

Chyo policies, um, all ready for us to

be able to do what we need to do in this

environment. Um, typically you're going

to have a different URL for our god and

backstage. I would say um, in our use

case because we're running inside of a

uh, kind cluster and the beauty of how

ingress works within code spaces and

things like that. um and the uniqueness

of projects that need to work on certain

ports. Um everything is set up here with

a subpath. So the same URL/ Argo CD is

going to take you to Argo CD and just

the URL on its own is going to take you

to backstage. Um so there's a bit of a

weird setup in certain levels. We didn't

want to make you guys do that and deal

with the downloads on Wi-Fi networks of

setting up Argo CD helm install. Um it's

really easy, right? Same thing with

Kyivero. Same thing with crossplane, but

all of the files for everything being

installed is all here within these

folders. So if you do want later on to

take a look and see what the Argo CD

configuration looks like, look at what

the backstage source code is that was

built into the container image, you can

see that all here within this repo as

well. Um, so it's not about hiding

anything. It's just making sure that we

can actually uh get done with the

workshop in time. So with backstage

basically installed uh we can come and

just open the browser uh to this URL

here. There's the URL. Uh you can just

click the globe icon or just copy it.

It's going to open up in our browser and

you're going to get this warning from

GitHub. Oh, it's a public URL opened up

from a dev environment. Awesome. Yeah,

we're using dev spa code spaces. So yes,

and we're going to press continue. And

from that point, security doesn't matter

and everything works. Um, and welcome to

Backstage. Uh, this is our backstage

instance. And what we're going to do is

log in with our GitHub user. Um,

press continue,

authorize the application, and we're

logged into Backstage with our own

users. Um, this is hooked up and it's

working. Um,

with that, we're going to actually walk

through the workshop itself now, um, and

really go over some of the use cases

here in the end to end flows. Um, so

basically, if we go back to, uh, where

this was in the readme again, uh, we

went out to

the relevant. Hold on, apologize. One

second.

Uh uh uh um what we want to do is in the

getting started guide um we basically

want to go all the way down to the

bottom and then in this endtoend flow

demo we just want to click follow the

docs here. Uh for anyone that doesn't

get to that exact link, it's simply

under docs.

Hold on.

Wi-Fi internet. We'll give it a moment,

I guess.

Come on, ducks.

Can't be that long.

Okay, it's it's under the docs and then

there's the end to end uh flow

there. Um I don't know why there is

something weird going on with the

internet here. Um but it's under docs

end to end and then there is uh

basically this lab guide here and that's

the guide that we're going to be

following here. Um and basically we're

going to go through two different

setups. We've done the setup the

environment. Um we're not going to do

the explore around the environment but

anyone that wants to later can. And what

we're going to do now is the actual

workshop persona scenario. So we're

going to start with the developer

experience here. first see what the

experience we want to offer our

developers is and then let's look at how

we extend this out as platform

engineers. So the first thing that we're

going to do is actually go and explore

the catalog. If we go to backstage, we

can see we've already installed some uh

composite resources into our

environment. So let's take a look at one

of these. For example, if I go to my

clustered app or my app or my web app,

it doesn't matter which one. We're going

to go into them here.

Why conference Wi-Fi is awesome.

Um,

and in a good situation, this is going

to load.

This is when you take your phone out

into a hot spot.

Give it a moment. Maybe it'll work.

But basically what we will be able to

see when it decides to load um backstage

is really fast by the way typically. Um,

but what we're going to get to see here

in our catalog, um, we can see actually

also throughout the docs itself, we've

added in a bunch of print screens just

to show kind of what this looks like.

Um, so while that's loading in the

background, I'm just going to show you

here also quickly is once we log in and

we get to our catalog itself, we have a

bunch of different resources. And what

we can actually start to see then is the

actual components overview. Um, and if

we take a look at what a component looks

like, we can see in this overview all of

a sudden that here we have a section

that tells us all of the details coming

in from crossplane. We can see that this

is named my app, which name space my

composite resource is installed in. I

can see the cluster this is installed

into because backstage allows us to

connect any number of Kubernetes

clusters to it. um XRS aka composite

resources aka claims rest in peace from

uh crossplane v1 um can be namespaced or

cluster scope. So we're going to mention

here which scope they are so that we

understand that because that's very

important. There are two conditions that

matter in the crossplane world. Synced

and ready. Synced means the resource was

applied to the cluster. Ready means

whatever the target system is says that

the resource is actually in a good

state, right? So that would be AWS

coming back and saying the EKS cluster

is ready, the EC2 instance is ready,

something like that. That's when we'll

be in a ready state. Sync state means it

was applied to the cluster and all is

starting the reconciliation. Um we then

have how many managed resources are

being managed here in the compositions.

We can start to see Chyo policy

overviews, links, about data, right? We

get all of this data available to us.

Um,

let's try again. Come on. Yay, it works.

Um, so we can see all of that same data

here, right? Um, we can even start going

around some of these other tabs. We go

to the Chyo policy reports and we can

see here all the different data. We can

see our different policies and we can

start looking also at our crossplane

data. for example, both in a graphical

view um for any of these resources. I

could click on them, see the YAML, see

the Kubernetes events related with our

specific entity as well. So, we're going

to get this full visibility. Now, that's

great for a developer to get visibility,

but how do they go and create a new

service? So, let's go and create a new

service. So, we're going to go to create

here, and we're going to follow our

guide. Um, so when we go to follow the

guide here, we can scroll down, right?

This is the viewing of the resources.

We've done all of that. That's great.

And we're going to create a new

application. So the first thing that

we're going to do is we're going to

select the app template specifically.

Um, and up here on the screen, we can

see the data. We're going to use the

name demo app01 and the namespace

backstack demo. Now, there is nothing

preventing you from using different

names, but all the commands below to

make things easy use these names. And

therefore, I strongly recommend using

these names because otherwise you're

going to have to manipulate a bunch of

commands. Um, so basically what we're

going to do is go to our app uh template

here and we're going to use demo app

01.

And this is backstack demo.

And then we select the owner. Now we

haven't ingested any groups from

something like entry ID or octa or

active directory or anything into the

system or github teams but you obviously

would in a production environment here.

You can select either guests or

kubernetes autoingested. Those are the

two ones we have set up here. Um we're

just going to select that for now. And

I'm going to select the image that I

want. And we're going to choose engine x

directly. Engine x. no tags, anything.

Um, with that set up, we're just going

to do next. Um, we can select which

composition we would want to use. Um, as

was mentioned before, um, you know, we

have the ability of writing different

compositions. We can use different

languages. Uh, you can later around

look, we've implemented this specific

composition in Go templating, in patch

and transform, which is raw yaml, uh, in

pythonic uh, as well as KCL. Um, in this

case, I'm actually just going to set it

to runtime, though. Um, it doesn't

really matter. Uh, let the system figure

that out. Um, and then the next thing is

where do we want to push this to? Um,

click ops does not mean no to githops.

It simply means that what the clicking

when I click submit that should push

something to git instead of cubectl

applying it. And so that's what we're

doing here. We are going to say I want

to target the demo cluster. Um if we had

multiple clusters here, we could select

that and that's going to go into the

relevant paths in our git repo to be

pulled down. And we're going to click

review and create.

This is going to basically render out a

manifest for us and create a PR. I'm

going to do open pull request here. And

just to save time, because Argo, CD,

Flux, any of them take time to

synchronize because they're running on a

polling interval, we're just going to

merge that pull request. And while it's

merged and we wait for Argo to do its

synchronization, we're going to come

here and we can look at the files

changed. And we can see that we've

generated an application manifest here,

but not an Argo CD application manifest.

This is an app manifest, our custom app

resource from Crossplane. Right. the API

that we have decided

um we have a bunch of annotations here

um that are used by the Kubernetes

ingesttor plugin within backstage that

are going to help us help these

resources be autoingested back into the

catalog. But we can see this is based

exactly off of what we've had in our

form. If I come back to the homepage now

within Backstage,

we can see that we still only have those

same components that we had looked at

before. But let's head back to our docs

now.

And we can see that we've filled

everything out like we looked at before.

And what we want to do

is

what? Yeah. What we want to do now is

get the applications.

Why? What happened here to my screen?

Hold on.

Something happened here to the screen.

One second.

Jeez, hold on.

There we go.

Oh, wifi at a conference. Woo. Um,

so unfortunately this is taking longer

than we expected. Um, but basically this

is going to autosync for us. Um, and in

a moment or two, uh, we should see our

new application pop up into our catalog

in backstage. Um, unfortunately, I will

say we're going to get through as much

of this as we can today. Um, but the

free environment on code spaces should

be good for about eight hours of this

running. Um, you can shut down the

environment afterwards. Just stop it and

you can start it later and play with it

and all of that as well if you want to

continue. We're we still have some more

time, but you can then shut it down um

and start it up later um and just

continue from where you were before. Um

so just note that as well um which is

important. Um now what we can do here

is cubectl

get

application

dash a.

We can see that we have an application

that was automatically created for us.

That new application we can see

automatically synced here demo app01.

And we can see the exact same experience

that we had before is now in our actual

terminal. But what we're going to do

here, just because of the shortness of

time, we're going to skip over some of

the sides of the developer here. I want

to jump over to the platform engineering

perspective just for a moment to kind of

go over what this experience is to add a

new service um to the environment. So

let's go over that just real quickly

here in the endtoend flow of a platform

engineer. And when we look at this here,

um I just want to we're not going to do

this necessarily. Anyone that wants to

can follow along quickly. Um but

basically what we're doing here is

saying if I want to add a new service to

my platform, this what I want you to all

take away from this is how easy it is to

add a new service here, right? We're

basically saying, okay, I want to add a

new service called database to my

platform. And we're going to basically

simply install it into our platform.

Once we've done that, what we're able to

do is simply create new XRDs, a new

composition, and once I do that, it's

going to automatically create that new

software template for you and allow you

to maintain everything going forwards on

your own. There is no extra maintenance

that's going to go on. Everything that

we're using um in this workshop

basically allows us to do an endto-end

flow of managing the life cycle of a

service. One of the real powers here is

that Chaivero traditionally is used as a

tool that allows us to prevent things

from happening and that's great.

Preventing bad things is a good thing.

um I hope but another place that Kyivero

can be used is actually as a amazing way

of giving feedback to our developers.

When we look here, I can see for example

on this application we created, I have a

failure. My developer sees directly when

he goes to view his component that he's

managing his application. I have a

failure here. What is this failure? I've

set the message myself saying validation

error. Engine X image is not allowed.

And it tells me which rule this failed

on. Well, let's fix that. We can go to

the scaffolder content. We can do

choose.

I'm going to do next and login with

GitHub.

And with that, all that we need to do,

obviously, GitHub requires you to do 35

things in order to log in. Um, but from

there, we're just going to change the

image from EngineX to, let's say,

EngineX latest or any other image that

isn't just plain Engine X. Um, and when

we do create, this is going to again

allow our developer to never deal with

GitOps, to never deal with YAML files,

to never need to fill out YAML, but

basically give them here. This is the

form of the YAML schema with all the

values you have in Git right now. Go

make your changes in the form and it's

going to submit a PR with those changes

back to the GitOps repo, allowing click

ops for day zero as well as day two. Um,

and that's what's really important.

We're giving the UI front end through

backstage with the notification to the

user and the feedback coming from

Chaivero powered by Argo CD which is

actually doing the real synchronization

and allowing us to have that GitOps

approach. And in the end, the reason

it's so simple is because we have

crossplane which gave us a simple API

that our developer needs to fill out.

The benefit of all this is that if a

developer wants to use a YAML manifest

directly, they like YAML. I've heard

there are people like that and I heard

there may be some of them here at this

conference this week. They can take

those YAML manifests and get commit and

get push those directly to the GitOps

repo. And you know what's great? they're

going to show up in backstage again and

the next person who needs to edit it can

go to that go to the update Kubernetes

manifest and update it from the UI as

well. Meaning we can work in a symbiotic

relationship whether someone wants to

work directly from the command line,

someone else wants to work from the UI.

Unfortunately, we're going to get

everyone back on stage here just to kind

of sum stuff up and to do a summary

here. Um, but I would love if people

want to afterwards, you know, find me.

I'm here for the rest of the conference

in the evenings. Anywhere where there is

whiskey, that is where I will be found.

Um basically um

not only in the evenings, um, but find

me happy to always answer any questions

on this stuff and play with this. This

repo is part of the backstack or um it's

an org that you know we all really

believe in this idea of the backstack.

Um and we're looking for more

contributors. So come play with it, test

it out, give your feedback. Um and we'd

love to, you know, work together as a

community on building this new version

of a platform. And with that, let's

start out with the summary here. Um

Yeah, sure. Um, I'll start with

backstage. Um, I apologize for you guys

needing to listen to my voice for so

long. Um, but I'll finish quickly and

then pass it on to the better voices

here. Um, Backstage really is a powerful

tool. We didn't get to show here, but

there's a part talking about the AIMCP

integration that Backstage has, which is

extremely powerful as well. I recommend

people to play with that. But Backstage

is the deacto standard, right? I've

heard all of these questions coming

around on you know backstage it's so

complex it's so this is it going to

succeed is it not going to succeed the

answer is backstage is alive and well

and it is an amazing project with a huge

trajectory going forwards if you tried

backstage 2 years ago and got scared so

did I try it again now it is such a good

experience now with the new front-end

system and the new backend system don't

allow the misconception of how it was

previously where it was a real mess to

deal with prevent you from trying it

again. It's in an amazing place now and

it is a great place to start. I

personally maintain about 20 plugins,

well 27 to be exact, of backstage. Um,

but you know, people should try them

out. It's an amazing community. There is

a contrafest for that as well here. Um,

yeah. And let's pass it off to Christian

to summarize on Argo.

>> Yeah. So Argo uh basically to summarize

what it does, right? It is a GitOps

controller providing last mile

deployment, right? So it's not a

complete picture. It's it it literally

just takes care of deploying managing

the the the application YAML on your

Kubernetes cluster, right? You're still

going to need some sort of workflow in

the beginning. So um that really is the

uh the the last mile kind of CD

continuous delivery continuous

deployment um experience uh for in a

GitOps way. So

and for crossplane we are the

abstraction layer for your

infrastructure and applications in a way

that you can like put your eggs in a

basket in the Kubernetes reconciliation

and declarative loop. It's declarative,

consistent, allows for you to be

flexible in the way that you need to do

for your organizations and run it at

scale.

>> And variing

up on the very last part that you saw

there, it closes that feedback loop with

your developer. Makes it a lot easier

for them. They have a guard rail there.

They get some actionable feedback. And

as you saw in the demo at the very end,

that actionable feedback makes it

possible for them to then fix what it is

that went wrong without having to ask

anybody and automates all of your

security, your compliance, your

governance, as well as making it a much

better developer experience for your

team.

And if you want to get involved with any

of the projects, we left GitHub repos

for you to check out, for you to start

if you really became a fan of them

today. And then of course ways that you

can continue learning about Backstage,

Argo, Crossplane, and Civero. Backstage

has a discord server. Argo is over on

the CNCF Slack. Uh there's different

channels for Argo components. So there's

one for Argo CD. Crossplane has its own

Slack at slack.crossplane.io.

And then Kivero is over on the

Kubernetes Slack um with Kiverodev.

And there's also parties that we're

throwing. So you can

>> go to one of these three parties

tomorrow evening. You can go to the Isov

surveillant hive hive mind mango cross.

>> Go to all three parties tomorrow

evening.

>> You'll find us there um

>> with the whiskey, right?

>> Or crossing graduation party or the cube

ATL bash.

>> Exactly.

>> And with that, we would like to thank

you all for coming. Feel free to reach

out if you have any questions and

>> thank you all for putting this together.

>> Enjoy the rest of the conference. Yeah.

Awesome.

>> Have a good Thank you.