Cursor has recently introduced support

for Jupiter notebooks and this feature

has allowed me to work so much faster

inside of Jupiter when I'm doing

analysis. This is something I've

actually been doing in my day job as an

AI engineer. So today I'm going to show

you some of my workflows and my general

approach to doing analysis now inside of

Jupiter using cursor. I think this video

will be relevant to all sorts of data

professionals, not data analysts, not

data engineers, not AI engineers or

software engineers, but all of you guys

because if you're working with data,

then you probably want to just be able

to get your hands on it and see what's

going on and understand it. And that's

what Jupiter allows us to do. And now

cursor allows us to write code inside of

Jupiter. I'm going to show you how I've

been using this and why I love it so

much and just how insane it is. and

saving me time. So, we're going to start

with an empty Jupyter notebook and a

data set and we're going to start

creating visualizations and analysis in

there. So, I'm working in a folder here

called Zazenode season 2/source/juper

notebook cursor analytics and this is

open source. You can find all of this

code on GitHub. Here's what I've got.

We're going to start with this video

game sales data and we're talking about

this VG sales data set. Um, let me have

a look at this. So, I can actually open

this up because it's pretty small data.

It's only 16,000 rows, but it's still

way too big to feed into a language

model. However, we're going to see how

effective cursor is at understanding

this data based on a couple little

tricks. So, the first thing I'll do is

go into video game sales and I'm going

to open up cursor. And this is where

we're going to put our Jupyter notebook.

Right now, there's nothing here except

for access to the data. And I'm going to

open up the assistant and we're going to

add a prompt. And I'm just going to say

um generate me a starter analysis for

this um CSV file in input data.

And I don't have to describe it because

it's going to understand what's going

on. And what I can do is just sort of

copy the first few rows and I'll say um

here's some of the rows. And that's it.

So cursor we'll get to work on this

using claude for sonnet. So, the first

thing I notice it doing is it's trying

to create this directly, like this video

game sales notebook directly. That's not

what we want to do. We want to use its

tools for interacting with Jupyter

Notebooks. So, I'm going to stop it and

I'm going to correct it. So, what I'm

going to do is adjust the initial prompt

to make sure it doesn't do that. Make

sure you only use your Jupyter Notebook

tools um for inserting or editing cells.

New Jupyter Notebook for this. Okay. So

it's gone ahead and created the notebook

over here. It's an iP

file. And now we can see it seems to be

properly using its tools. It's gone

ahead and inserted some markdown. Now

it's inserting some code. And this is

the type of experience we want to see

from cursor. This is how we know it's

working. It's going to be calling these

tools specifically for Jupyter notebooks

and starting to insert code in this way.

Now, while this goes, we could actually

start running the code. And let me just

show you that. Um, we got to have a

kernel selected. And I'm using this ad

hoc Python environment that I have. It's

a virtual environment. And check it out.

I could just start running some of this

stuff. Let's see what we got. Okay, so

we're already getting some data out. And

we've loaded all of our data with the

data types. It's just telling us some

general information about the data. And

it's also nicely formatted, which is

something I just love that language

models do. Sometimes it starts putting

emojis into the output, which to me I

enjoy. Okay, so one thing is it's

difficult to be actually doing this

while it's running because it's going to

keep inserting code below and it's just

not a good user experience. So I'm going

to let it run for another couple minutes

before I get to the demonstration. So

now cursor is working on a correlation

analysis. And what I've seen from cursor

with analytics is it just keeps on

thinking and coming up with new ideas

and then executing on on them. And it

can do this for quite a long time. So,

I'm just going to interrupt it. I'll

just press stop. And we can continue

working in this context. And it just

stops it in this really graceful way

where it just immediately, you know,

ends whatever it's been working on. Um,

so, so now that that's stopped, I can

come to the code and we can actually do

some stuff with this. First thing I'm

going to do is just keep keep

everything. And rather than, you know,

clicking keep keep, what I can do is

just say keep all over here in the chat.

Okay. So, now we've we've got

everything. And I'm going to have to

zoom out a bit because I'm just getting

disoriented. Okay, so here's our

analysis. Now, do you see this cell at

the top? It's a markdown cell, but it's

indicating it's in raw mode. It It's

like not smart enough at this time to

make this into a markdown language cell.

I'll show you how we can do that. Just

click on this and then click markdown.

And now when I actually run this cell,

it's going to like render that markdown.

And you can see I could edit it and

render it. Okay? And but there's a

better way of doing that and let me show

you that. So, I'm going to scroll down

to the next markdown. So, here it is

again, and it's raw. And for me, like I

like using my keyboard more. So, here

are just some 101 keybinding stuff. If

I'm inside of this cell, then using the

arrow keys just like moves me around.

But if I press escape, then I'm out of

the cell. And now moving the arrow keys

moves me through the notebook itself.

And now when I press enter, I'm back in

the cell. Okay? So, here's what I'm

saying. I leave the cell and then I can

press M. And that converts it to a

markdown cell. And so now when I want to

do that, I can say, okay, leave the

cell, press M, renders it as markdown.

Next thing I want to do is start

actually running all of this code. Uh,

and what I can do is just type run all.

I just want to execute everything. So

let's see what we've got. Um, we're

loading our data with pandas. And it's

immediately outputting a bunch of data.

We want to make sure our columns are

correct. We can see the shape of the

data, verify the row counts, we can see

the head of the data. This is all pretty

typical stuff. And say at this point,

this is the first five rows. Maybe I

want to see the last five rows as well.

So pay attention to the key bindings.

I'll press escape then A. That'll insert

a new cell above here. I can press enter

to go into that cell and I to start

editing that. And I'm using Vim

keybindings inside of here. And then I

could say do DF.tail. And you see the

autocomplete that's coming up. We will

talk more about that. But when I execute

this now, I'm getting the tail of the

data set. Just one thing to keep in mind

is this data frame might not necessarily

look the same way it did in this cell

because it may have been changed down

later in the document. If you really

want to get the proper output in a

Jupyter notebook, you've got to restart

it and run all. So I could say this

restart right here. Yes, that restarts

the kernel and then I could run

everything from the top. Now you can see

this little number three right here

means this was the third cell to be

executed. So this is an accurate

representation of the data at this point

in time in the notebook.

Right? So these are like the end of the

data. So so from an analytics point of

view, right? These are like the lowest

selling games inside of the data set

which had like literally yeah very few

sales. Okay, let's continue coming down.

So now we get a bunch of data set

information. And I really like sometimes

when cursor outputs nice styled

information with like emojis and stuff.

So, let's get it to do that. I'm just

going to select all this and I'm going

to type control K. And now I can edit

all of this in place. And inside of

here, I have a limited model selection.

I can do 3.7 sonnet, but I can't do

claude 4 right now at at time of

filming. So, I'm going to select 3.7.

And I'm just going to say um add some

emojis to this output

um to make it easier to uh read.

And now we're going to see that expected

uh experience where it just kind of runs

through line by line. It's going to show

us the diff and we can do this accept

all and output the new the new

information. So it just kind of like

added some emojis whatever. So we got

the data types and now look at this. Do

you see these dots? And what's happening

is cursor is truncating the data in this

cell of the Jupyter notebook. This is

the same experience like VS Code uh

offers. So, what I could do is open this

up in a text editor. And that's

something I like to do sometimes because

then we get a whole separate pane and I

can hop through these panes with my

keyboard and see the full the full data

in in this pane. Uh basic statistics

seems like it's okay, that's not part of

it because the basic statistics is

actually an output down here. I can also

use a scrollable element just to keep

everything in line. Okay, so here's our

basic like DF describe table which is

pretty common to do. Let's carry on. So

now we have our top performer analysis

and immediately we can start just

answering general questions that we

might have about the data. What are the

top selling games? What are the top

publishers by total sales? What are the

top platforms by total sales? Now this

data is kind of difficult to look at. Um

I'd want a chart of this data. So let me

insert something like that. So I'm going

to in uh insert a new cell here. Could

use the agent and I could create a new

tab here. And I'm just going to say

insert plots here. So now I'm going to

take this code and add it into my

context. It's up here. Plot the data uh

from the cell above it. So now it's

using some tools and it doesn't have the

whole notebook in context because

notebooks can be really big and it's not

possible to keep this whole thing in

context all the time. But look at this.

It's like I can see that seven cell has

insert plots here and the cell above it

cell six contains analysis of top

performers and these data sets and these

are the data sets I wanted to plot. So

it looks like this trick is going to

work um for for doing what we wanted.

I'll close the agent and just keep this.

Okay. So we can see the top 10

bestselling games, the top publishers by

total sales. This is more digestible,

but I don't actually like a few aspects

of this. So, I think this is where a

good use case is for this incel editing.

And now I'm going to do this again with

control K. And I'm going to fix these

up. Use a dark theme. Create each chart

as a separate figure. I want to create a

separate plot for each. So, we should

see plot.show, you know, after each is

rendered. So, that these are all

separate images in the notebook. And

okay, so now it's using a dark theme.

It's still not that pretty, but um this

is what I asked for. These are all

separate images. So I can go ahead and

save these all separately, which is what

I'd want to do. Cool. Okay, let's keep

running through our notebook and see

what we got. Okay, so it's given us some

sales trends over time. Maybe this are

these are largely like explaining the

data set because obviously video game

sales didn't just end after 2015. That's

probably the extent of the data set.

That's where the cutoff is. Okay. Okay.

And we have some more plots. And so at

this point, I've noticed um the the

plots, they might not actually be

interesting to us anymore. Like when we

get AI to generate a whole bunch of

stuff, you get to a point where it's

possible that this isn't telling us

stuff that we want to know. And also

another thing is, do you notice how

these charts just don't look very good

visually? Okay, so let me make this a

bit smaller. And do you see how the

style of all these plots is kind of

using this light theme that I don't

really like? And then we tried a dark

mode. But I also don't really love the

style of these plots either. I have a

specific style that I'm going to use and

we're going to apply it to all the

charts in the whole notebook. So I'll

show you how to do that. Um I've got a

claude code in my home user directory

and I've got some agent set up and I've

got this one called mattplotib

visualizer. Let me show you that. So it

has some Python code here for doing

mattplot visualizations in a nice way in

a way that I like. And uh so like what

I'm going to do is open this up. So here

I've got it open mattplot visualizer and

I'm going to copy this right here just

this example and I'm just going to grab

that. So now I'll come over to cursor

and I'm going to paste this in. So I'll

start a new chat and let me make this

bigger. Have a look at all the

visualizations

in this notebook. Standardize style of

the outputs as per the example below.

paste that in and away we go. So, we can

see cursor getting to work and it's

using these colors and this scheme that

I asked for. And by the way, if you end

up liking the looks of these charts, you

can take this exact color scheme the way

that I've formatted it here, as well as

a few other agents that I've set up with

Claude. And you'll get access to all of

this stuff if you sign up for my weekly

email newsletter. I'll just send you all

of those agents and you can start using

this in your own workflow. These are

just markdown files, but my newsletter I

just send once a week and I have some

other bonuses if you sign up. This is

all free and I think you'll like it

because I just talk about my latest

video and explain what it's about. I

include some timestamps and stuff so you

can kind of figure out what's going on

in the video and I also have an

inspirational quote. So, I'd encourage

you to sign up for my weekly email

newsletter. Cursor's finished up. So,

I'll clip all and close that. And I'm

going to restart and I'm going to run

everything. This is our sort of plot

style. It's been defined right off the

start. And as I scroll down now, we're

seeing plots like this. This is in the

exact style that I'm looking for. And if

I come down, we can see our time series

data. Looks good. And some of the data

at the bottom. We've also applied our

styles here. We've got some kind of

broken stuff. And at the bottom, this

stuff didn't run because we stopped

running. So, I can just run that myself.

And yeah, it looks like we're getting

some problems from these charts below.

Now, I could fix that up. And how I'd

approach this is if I wasn't really

thinking about it, I would just take

this error and I would just feed it in

like this. And honestly, I don't even

need to give it context. I'm just going

to dump the error in directly and see if

um cursor can fix this up. So, that

worked better. We've got this sales and

genre by region, which is a really cool

looking plot. So the action games in

North America sell quite a lot and then

shooters and sports and then we can see

like that kind of mirrored. But what I

kind of like is look at Japan. Check out

the role playing game. Those are really

popular there where like role playing

games in Japan are clearly the most

popular. And this is like a really big

discrepancy from the action games which

are most popular in other countries.

Really interesting to see that. And so

if you like this style, uh, I've made a

previous video on how I use clawed

agents to generate these types of plots.

That's what I'm showing here. These are

system prompts for claude agents. But

we're still getting an error here. So

it's something else. Line 43. And um,

the dumb thing to do would just be to

add a follow-up and try and fix that as

well. And while that works, I've also

got an error down here. So I'm going to

feed this one in. And I'll highlight

this and just send that off. Okay, so

that's working too. Let's come on up and

see how this did. I'm going to run this.

Okay, so now it's executing without any

errors. So I've been able to fix those

two minor plotting errors, whatever they

were. And I'll just accept those and

I'll come down. And this to me is saves

a lot of mental overhead thinking about

plotting styles and I can focus more on

the analysis and productivity that I'm

looking for. Now let me show you one

more thing with this data set. I'm just

going to look at the head of it. So,

this is I'm going to change this to a

code cell. I'm going to type escape and

then Y should do it. What's going on?

H. Okay, whatever. That didn't work. So,

then I'll I'll just type Python. Okay,

so this is a Python cell. And now what I

can do is delete this cell below. And

now when I add new cells, they'll all be

Python cells. That's what I like to do.

And let's look at the head of this

notebook. Now I'll demonstrate how we

can do machine learning inside of our

analytics workflow. What I want to do is

try and predict the sales and I'll try

and predict the global sales of a game

based on the genre, the publisher, the

year, the platform, and this these sort

of things, treating these as features.

So I'll open up the agent. I'll do a new

tab and I'm going to say um add a new

cell to the bottom of this notebook.

Create an ML model, a random forest.

We're going to do the platform, the

genre, the publisher. That's it. And

this should not take very long at all

because this is actually not very much

code. And it's going to be using

scikitlearn as a dependency. Okay, it

looks like it's done it. Um, okay. So,

let me run that. I'll accept this. And

we're just going to swing that down. Do

you see all this stuff here where it's

like, okay, here's what it includes. I'm

going to say, um, add this documentation

to a markdown cell above the ML model

cell. So I'm I'm expecting to add

something right above. Uh but now let me

deal with this missing dependency. So we

don't have scikitlearn in this note this

Jupyter notebook my virtual environment.

Uh so let me add that and I'm going to

show you this.

Okay I'll get out of here and let me go

to my home user directory. So what how I

I do stuff on my computer is I have this

virtual M's folder. This is just a

folder I created and I can put

environments in here. I put one in here

called ad hoc. So if I go into this ad

hoc like this would have just been like

python-m

and then I called it ad hoc like that.

And when I ran when I run that it

creates this this virtual environment

for me. Um now what I can do is go into

ad hoc bin pip and I could install

scikitlearn

and once I do this it should be

available in my notebook. So whereas

this cell did not run before it should

run now. I might have to restart but

we'll see. Yeah, looks like it looks

like it was able to import scikitlearn

and it's doing this modeling now.

So we can see our results. Here's the

data set size. We have 16 and a half

thousand games. These are the features

that I asked for and we're predicting

global sales.

So we trained the model and let me put

this into a scrollable element. So we

get some statistics on the training

scores and such the testing scores and

the feature importances which is cool.

So the publisher seems to be more

impactful than the platform or the game.

And we also have some visualizations of

our results which is pretty cool. It's

also gone ahead and done some example

predictions. Now, of course, this model

is not going to perform well in practice

because, you know, just just knowing

that it's an action game from Sony on

PS4 isn't exactly a good way to, you

know, estimate how many copies are sold.

But it's still pretty cool that we can

do this so fast. And since this channel

is about ML and stuff, we can kind of

just peek at the code. So, it's using

this random forest regressor. That's

exactly what I wanted it to do. And of

course, we'd we'd want to do some

hyperparameter tuning if we were doing

real machine learning where we'd want

to, you know, tune the number of

estimators and the max depth and

basically make sure that we're not

overfitting the data um if we actually

did this. And I'd also want to see some

more visualization on our like train

test split. We're doing a random size of

80% for training and 20% for testing.

That's a good idea, but we'd also want

to have a hold out set and maybe test

multiple models. Uh, but overall this is

just incredible that that we've done so

much. And you'll see it's importing job

liib, but we never actually do that. And

Joblib would be used to dump the model.

If we wanted to save this, we have this

RF model right here. And so we could

actually like dump this. Now, let me

demonstrate autocomplete. And for the

most part, I find autocomplete to be

really painful with Jupyter Notebooks

and not extremely helpful, but I'm going

to show you like how it how it works.

Anyway, so after we've trained our

model, that's this part right here. I

might want to save it. So if I type RF

model, and now I'm going to start

getting some autocomplete, I assume. H

autocomplete's kind of tricky. It's it's

hard to like There it is. Okay. So So

see this. It's funny. It's it's it's

already saying, oh, maybe you want to

save the model. It saw that I was

looking at job lib maybe earlier. I

don't know. And it's just like, oh, do

you want to save it? So, I could just

accept that. And now I'll save the model

as this random forest model.pickle. But

maybe I don't want to put it there.

Maybe I want to put it into a folder

called artifacts.

So, artifacts. And I can autocomplete

that. Boom. It kind of figured it out.

And now below it's like, oh, do you want

to load the model? Maybe. Okay, sure.

I'll load the model. What's the benefit

of this? Well, if by saving it and then

loading it, we guarantee that all of the

code below like the predictions and all

of the metrics that we're tracking are

going to be consistent with this model

that we've saved to disk. So, generally

when I save a model, I do like to reload

it in this sense. And this just puts it

into a binary object here on your

computer and then it tests that we're

able to load that binary object back

into the runtime of this Jupyter

notebook. So, yeah, actually I do want

to do these things. Um, but maybe I want

to make this directory. Like if

artifacts doesn't exist yet, I'd like to

make that. Um, so what if I called it

out directory and then yeah, that's

artifacts. Okay, now I want to replace

it. Cool. Yep, that's what I want. But

also this one. Okay, good. And what I

could do is make it. So I'm going to say

um path outdoor maked parents. That's

what I want to do. But look, path is not

defined. So let me go to the top of my

notebook and make sure I import that.

Boom. So, this is the autocomplete

workflow that cursor allows us to do,

which is super powerful. It doesn't

always work well, but when it works,

like I just showed you, it's totally

magic. Uh, let me run this. And this

should all work. And if I look back in

my notebook, let's get out of here. What

do we have? Video games. And we can see

there's this artifacts folder, input

data. So, this is this is like the

random forest model that we just

created. It's right here. And one last

thing I'd like to do is make a folder

for the figures. And I think yeah, I'm

going to run Oh, here's the cell by the

way for um I'm going to just mark

downify this. And this is our ML model

description.

Yeah. So random forest model, some some

benefits, the features, the data

processing pipeline, there's that 80 20%

split. Here's the model hyperparameters

we talked about. And by the way, if you

want to learn more about machine

learning, then you can go to my website

zazencodes.com

and I've got a course called the AI

engineer roadmap. So you can buy this

for a onetime fee and it reads largely

like a book, but it's a little bit more

interactive in a course format. And I've

got this model training and evaluation

section. So I've got four lessons all

about training uh machine learning

models. I want to show you this lesson

like fully unlocked. So here we go. So

this is a model training lesson and you

can see some examples like we're going

to do logistic regression with some

k-fold cross validation and you can see

the results. So like I've plotted the

decision boundary. This is very like

basic ML's getting started type stuff.

Here's what we might expect from a

random forest with the decision boundary

that is nonlinear. And yeah, and so you

can kind of just keep walking through

and we're going to have a few different

lessons here on on training machine

learning models just to get you Yeah,

just to get you up and running. I've got

one on hyperparameter optimization.

And this is what I was talking about

with the holding out a validation set.

Um I think I talk about here. So yeah,

so we'll want to have like a CV set.

This is for training and testing uh with

cross validation. And then we'll have

we'll want to have a hold out set. Uh,

and that's one thing we didn't actually

do in our current workflow. If I go back

to cursor, right, we just do an 8020%

split, but we'd actually want to take

some of this completely out of the mix

as a validation set so we could test

multiple models. Okay, I'm going to do

one more prompt. Save all the figures to

the figures folder. So, cursors just

wrapped up and you can see various

parts. It's just calling save fig for

all of these figures, but it saves me a

lot of work and especially naming all of

these figures. So, let's restart the

notebook. And I'm going to run

everything. Um, and if this is working,

then we should start to see figures

generated. Yep, they're already here.

And they they're all they're all output

in this directory.

Yeah. And by the way, Wii Sports seems

to be the top selling game of all time.

And there's one final thing. This is

more like a bonus that I want to show

you. I'm going to start a new chat and I

wanted to show the the power the real

like power of AI when it comes to

analysis and I'm going to do something

that would be just not even in the realm

of possibility if we weren't using Gen

AI. So, here's what I want to try to do.

I'm going to add a section of the

notebook for a Halo game analysis. I

want to see all the Halo games compared

along with images of the game covers.

All right, here's the cover gallery. to

display the actual cover games, you can

add them. I don't like that. It looks

like it's not actually using the real

covers. It's using these fake covers.

I'm going to stop you right there. I

want the actual game covers. Say render

these actual um Halo game covers using

Python code. So, you can do an along

with the actual data in the data set.

Now, how is it going to get these

images? I actually don't know. I

actually have no idea how it's going to

do this. Maybe it has to do a web search

or something. There's nothing like less

reassuring than a language model telling

me I'm right. It will typically always

tell me that I'm right. It means it

means almost nothing. All it means is

that it's capitulated with whatever idea

I've presented to it. Okay, it looks

like it's got a bunch of Wikipedia URLs

right here. you know, whether or not

these are real. These really do feel

like things that they could have just

hallucinated. Okay, but we've got a

bunch of stuff. Let's accept everything.

And we're gonna try a few of these cells

out.

Okay,

some of them worked. This This original

Halo one worked, but none of the other

ones seem to have worked. Halo 3 was a

good one. Xbox 360 2007.

This was Yeah, this was the most famous

Halo. I played Halo Reach a lot and it's

pretty cool that that actually we see

that in the data that that also reflects

Halo 4 was good and then I liked Halo 2

cuz I got to play that when I was a kid.

So that's that's got a soft spot in my

heart. But um this this this didn't

really work out right this cell right

here. Um for this one, I'd probably want

to bring this to the big guns. And by

the big guns, I mean Open AI. So, how I

would fix this is I'm going to go to

OpenAI 03 and just paste this in. I'll

say um fix this with uh proper URLs,

cover images. The ones below are mostly

uh 404 errors. Update the output the

full updated code. And the reason I like

doing this is 03 has really good web

search features. It's also possible that

I could have just fixed this with 03 in

the notebook. Uh let me actually give

this a shot. I'm gonna try the same

prompt. Let me try these. Let me try

these this this prompt. And I'm gonna

try and use 03 in here if I can. I get

01 or I could use 4.1. Um 4.1 should

have pretty good uh

tool calling abilities. I'll give it a

shot. I don't really have much faith

that this is going to work. It got

reach,

but now none of the other ones rendered.

So 03 is doing a much better job of

actually searching for these images and

hunting all of these files down. Okay,

so it it output some updated stuff. I

try these out.

Oh, let's go. Look at this. How

beautiful is that? Oh, look at this.

Wow.

I love it. So overall, this chart is

totally beautiful, but the we're missing

the emojis on these. Oh look at

this. We also have a whole Halo analysis

with Halo 3 and all the different games.

This is awesome. This chart's wicked.

And we're even saving this Halo

franchise analysis.

So, I've got that somewhere in here.

Right here.

That is truly a thing of beauty. It's

still screwing up these emojis. Okay.

So, we'll just have to get these emojis

out of here. If I look at the title, we

want to get rid of this one.

And do I have any other titles? Yeah, I

do. This one here. Great. Yeah, because

this plot, this chart to me was really

neat. I like that one a lot. Right. So,

I'm going to upload all of this stuff,

including all of these awesome images

that we created to GitHub. So, you can

check all of this stuff out online if

you're interested. So, that's all I have

for you today, but I appreciate you

watching and I would love it if you gave

me a like and consider subscribing to

learn more AI engineering. And speaking

of more AI engineering, I've got a

related video that I'm going to put on

the screen right here for you. Now, this

video is going to have some interesting

and more advanced topics, and it's a

great video to watch next after this one

if you've got some time. Thank you for

watching, and namaste.