The next lightning talk is Victor, and this should be a really fun talk, I think, about
MLOs.
Yes.
So, hello.
This is probably going to be the least serious talk you have seen today, so I'm sorry about
that.
We're going to be automating weight loss with AI.
It's a stupid project I made in a weekend, or like in a few weekends, but I want to talk
about it.
So, who am I?
Lightning talk version.
I'm Victor Zonk.
I work at ClearML.
Hi.
Let's make something.
So, that's the reason why I'm here.
The problem statement was I'm working at home, and I'm not working out enough, like probably
a lot of us are, and so the problem solution is why not lock my PC every hour and force
myself to do push-ups, and then it automatically opens again.
That was the main idea.
I want to do this with AI, obviously, because over-engineering, and I'm a machinery engineer,
so if I am a hammer, everything looks like a nail.
This is going to be the diagram, so left top is an oak one.
It's an AI camera.
I'll talk about it in a second.
That will run one model, and then, because one AI model is not enough, I have two, so
there is a second model that runs on the Raspberry Pi that will lock my PC.
This is what it looks like, so you get like a notification, workout time, lazy bum, and
you have to do push-ups.
It's in Raspberry Pi running in the corner of my room.
You can follow along with the diagram at the right top.
So I'm going to do post-estimation with the oak one.
Now what is the oak one?
The oak one is a 150 bucks open-source hardware AI camera, which is really cool.
I highly recommend it.
They run the Intel Mirrored X, so if you look at the speeds there, if you have the oak one,
because it does the AI, like the AI, on the chip itself, on the camera itself, it can
get a lot higher FPS on the Raspberry Pi, because it doesn't have to go to the Raspberry
Pi to do anything.
Even when compared to another AI accelerator connected to the Raspberry Pi.
It also has excellent documentation, which is a unicorn these days, but yeah, it really
is a nice library.
So they have a bunch of cool examples that you can try, like there's D-plop with segmentation,
there's other stuff.
But luckily for me, I didn't have to do any work, because they also had post-estimation.
So thanks to GXGX for implementing this.
This is an awesome repository.
It's still being maintained, if I remember correctly.
So definitely check that out.
That's really cool.
Now, what does this repository do?
It basically gives me post-estimation, so it films me on that AI camera.
I have one with me, by the way, so after the lightning talks, I can actually give a demo
lightning talk.
I can't do it right now.
Basically it draws like a skeleton on top of you in like seven, eight frames a second
on the Raspberry Pi, which is awesome, and then it even positions them in 3D.
So that's nice.
This is stage one.
We want to go to a pushup detector.
This is stage two.
So we now basically have a skeleton.
If we just throw away the pixels, these are the only points that we actually care about.
And then now it just basically becomes a tabular problem.
So the second part of the machine learning, or like the simple machine learning, is going
to be really simple.
Now we just have a few points, and we want to classify them.
For this second model, though, this is not pre-trained, so I actually have to label.
A few images, it's not a very complex model, but you have to do something.
So what do we want to do is we want to say, okay, this is a pushup, this is a pushdown,
and then we can do some additional logic to actually count them while they're happening.
Right.
But then the question becomes, how do I take a picture when I'm actually doing my pushups?
Because like there is a camera there, do I need a button, but then it might overfit
on me pressing the button or like something else.
So if you're a machine learning engineer, the answer is throw more AI at it.
So basically overengineering using an unnecessary amount of AI, I set up a microphone while
I was pushing up and pushing down.
There's a really cool open source package for Python that can do voice recognition.
It does send it to the proprietary API of Google, but at least the code is there.
And then you can just basically shout label me, and if label, the word label is actually
found inside of what you said, it will take a picture.
So that's that.
Now we have a third AI model that's really useful.
And then I did a lot with ClearML.
So this is actually the MLops part.
I now have two models.
I want to be able to train them.
I want to be able to maintain them.
So what did I do is this is the labeling tool.
So right, left top, left top for you.
Right, left top is the oak one that will take a picture.
When I shout, take a picture, it will send it to ClearML, which is actually an open source
MLops tool, one that I work for.
And they have data versioning, for example.
So every single time I run the labeling tool, it will create a new version of my data set,
which is very useful.
And then I can use this new version of the data set.
I can pull it in.
I can use the experiment manager of ClearML to keep track of all my code.
Every single time I run or I train, I will get all of my output, all of my plots.
And then you can actually build this into pipelines.
You can run this automatically on remote machines.
So I over-engineered the crap out of it, but I can't really tell everything in Lightning
Talk.
The main idea is you have a lot of different tools in ClearML that can help you with that
and automate a lot of that stuff.
Now training my own model.
So now we have all of those points.
We have four each of those sets of points we have if it's a push up or a push down.
Where do you go from here?
Training my own model, it's this.
Like it's super simple.
It's three lines of code these days.
So this is just sklearn.
It's an SVM.
It's a simple classifier.
It takes points in.
Give you one point out.
Push up, push down.
It's not ideal.
I should do a no class, but I was lazy.
No class basically meaning it's nothing, none of the two.
So now it will say when I walk to it, it will maybe register a push up, which is not ideal.
So in order to combat that, I made a very simple, even simpler piece of code that basically
primes it.
So here on the left, you can see one is basically push down, two is push up, and so you can
see it happen.
I think, yeah, you can basically see it happen there in the beginning.
But when I run to my place to start to do the push ups, here you can see that there's
like a bit of jittering going on because it doesn't know the zero class.
So in order to catch that, what you can say is, okay, if the, you can basically say if
the length is, I don't, yeah, if the length is 10, so if you're at least been doing detection
for some time, then you can check if the last 10 of them were push up.
So I'm basically ready in my position, only then prime it.
And then once it's primed, you can start counting.
So that's just a very simple, stupid way of doing it.
Two minutes left.
Excellent.
So actually, that's it, but I have two minutes left, so I'm going to do one more thing.
Locking the computer.
So I use Linux, which allows you to do everything, which is awesome.
So locking the computer was easy, but unlocking was hard, as it probably should be.
You have to put in a password.
So there was no real way to get a custom password.
I tried thinking of like maybe I should like scramble my password and then fill in that
scramble password, but never do that.
You will be locked out if your code is buggy and it happened.
So no way to get a custom password, and there is one big problem.
I know my password, so if I can't change it and I lock my computer and I really don't
want to do push ups, I can just fill it in and be done with it.
So the best and simple solution I can come up with is just to use Xdo tool and then spam
backspace.
So Xdo tool actually allows you to type automatically while your computer is locked.
So you can just spam backspace, not allow you to fill it in because it's like backspace
20 times a second, and then when you do the push ups, it just fills in your password.
And that's it.
So yeah, a lot of over engineering, and I hope you find it interesting and you learned something.
So thank you so much for listening.
One last note before any questions.
There is a YouTube video about it on the channel MLMaker.