You're famous.
Yeah.
That's it.
Close the door behind you.
Okay.
Okay, let's go.
I hope you hear me correctly.
People in the side and in the room and people online.
So, yeah, let's begin.
Okay.
Okay, let's go.
I hope you hear me correctly.
People in the side and in the room and people online.
So, yeah, let's begin.
Thanks for them for having me today
and to talk about cluster API and Kubernetes.
Thanks to you to come here.
That's quite impressive to see the room being fully packed.
Yeah, I hope you will learn things.
I hope you will discover things.
That's the most important.
And you will get some stuff to continue to investigate at home.
So, the goal of this talk
is to give you a brief introduction to cluster API.
To give you a brief introduction to cluster API.
So, yeah, let's begin.
Thanks for having me today
and to talk about cluster API and Kubernetes.
So, yeah, let's begin.
I hope you have a great day.
Thanks for having me.
Thank you.
Thank you.
Thank you very much.
Thank you.
Thank you.
Thank you.
Thank you.
Thank you.
Thank you.
Thank you.
Thank you.
Thanks for joining.
Let me quickly introduce myself.
I work as a flat car engineer
inside Microsoft.
Flat car is an operating system
designed to run container workloads.
If you want to learn more about flat car
you can go at
5.15
see my teammate Thilo talk about flat car.
It's a deep dive introduction.
And it will give you the key elements about this operating system.
But that's not the purpose of this today.
Outside of work
like SRE France, which is a DevOps association
where we organize meetups and conferences in France
and in Paris and in France.
So if you want to talk in a meetup
or if you're interested to organize something,
let me know.
Context, the context is Kubernetes.
Kubernetes is quite the answer to everything today.
So if you want to deploy something small, something big,
there is likely a big chance that you're going to use Kubernetes.
So to me, it becomes a great standard,
I think we can say this term.
So yeah, that's the cool thing with Kubernetes.
You can deploy a small thing and big things and it works.
And it works in the same way if it's a big thing
or if it's small thing.
Something to know about Kubernetes
is that you have two aspects of this technology.
You can consume Kubernetes,
means you deploy your application on it and that's cool.
And you have also to deploy and maintain the Kubernetes cluster.
So you can do both if you want to.
You can do only one aspect of the other.
But today, let's focus on the deploy and maintain
Kubernetes cluster and not how to use Kubernetes cluster.
Two or three weeks ago, I was on Twitter
checking some news, what's going on in the tech industry.
And I've seen a tweet of a person I've met in different conferences.
A tweet about, hey guys, what if I write a book
to describe all the ways to deploy Kubernetes.
So it was an idea like that and he got some traction in the end
about this idea and he started to draft a list
of all the ways to deploy Kubernetes.
So the knee is at the first day of the currently.
So if you want to talk with him about his book or if you want to invest
in his book, it's a great opportunity to meet him.
He has a talk in the Go Dev Room this afternoon.
But we're not talking about Go, we talk about Kubernetes
and the 50 shades of deploying Kubernetes.
So you can use binaries, you can use managed services,
you can use platform, you can use a bunch of ways to deploy Kubernetes.
But today, let's have a look on the line 27 or 26,
something like that, it's the cluster API.
Cluster API, if I quote documentation, it's Kubernetes,
a project focused on providing, you can read.
The most important is the last line, the cluster API project,
use Kubernetes types APIs and patterns to automate cluster life cycle.
In other words, use Kubernetes to deploy Kubernetes.
So that's the cool thing with Kubernetes is you can extend
this technology using CRDs or custom resource definition.
So you can extend the technology and you can leverage,
you can benefit from the reconciliation loop, for example,
the Kubernetes for what you want to do.
It's already available for the basic usage of Kubernetes,
but you have over projects like a cross plane that will leverage
this way of managing the life cycle on the provider side.
So cluster API is this kind of stuff.
So if we take a look on really abstract way of how does it work,
you have two clusters.
On the left, it's the management cluster, this is the pilot of everything.
This is where things happen.
And you have the workload cluster, this is where you run to run your workload.
So your website, your SaaS, whatever, it will run in the workload cluster.
This is what you currently do if you do some Kubernetes cluster.
But before that, we have the management cluster.
So you're going to tell the management cluster,
hey, I need a cluster in these providers,
please deploy everything I need to have to run this Kubernetes cluster.
Because to deploy a Kubernetes cluster, you need networks,
you need security groups,
you need a bunch of things to create on the provider.
Well, the management cluster will take care of that,
and it will deploy things for you, and you don't have to do anything.
So that's the way to see things.
And in this example, my management cluster is running with Kubernetes in Docker, kind.
So this is pretty convenient because I can run my management cluster
on my local laptop, on tiny resource thing,
because I just need to deploy one single control plane.
I don't need to have high availability and stuff like that.
I just want to use the Kubernetes APIs.
And the workload cluster in this case is running on OpenStack just for illustrating.
So as long as you have a network connectivity between those two clusters,
and you have credentials, of course, it will work.
So and you can even decide to migrate the management cluster from one cluster to another,
but that's something else.
That's the key elements to understand and to know
if you want to understand the concept of cluster API.
So this is my kind cluster, so I just have one single control plane running.
That's it. Nothing fancy, nothing to do.
Just kind create cluster, and I have my management cluster.
Nothing to install on it on top of this.
Just a regular Kubernetes cluster.
Really simple.
Now, how can I create things on my cloud provider using cluster API?
Well, for people that already knows Terraform, that already knows cross plane,
all these kind of projects, you know that there is no secret.
You need to know the APIs of the cloud provider to implement them,
to consume them, and to create what we call a contract.
So this is the border between the cluster API logos and the cloud providers.
So you need to teach cluster API.
Okay, so in cluster API, we say that a network, it's this thing.
So a network on GCP will be this thing, on OpenStack,
it will be this thing, and so forth, and so forth.
So yeah, the idea is to teach cluster API how to manipulate and
how to deploy resources on the cloud providers.
And for this, we use what we call a cluster API providers.
So on the Kubernetes SIGS sub project on GitHub repository,
you can see all the various providers supported.
So there is OpenStack, GCP, Public Cloud, on-premise.
So it's a tinker bell on the upper right.
So yeah, you have a bunch of providers and
if you have some knowledge in Go programming,
if you have some knowledge in API consumption and stuff like that,
feel free to start to contribute on this provider because this is
a cool way to start contributing to Kubernetes and Kubernetes ecosystem.
So yeah, that's the idea under the hood, what's going on.
And now I have my management cluster.
I need to create my workload cluster configuration.
So I just use the cluster CTL, cluster Cuddle,
whatever you call it, command to generate this YAML configuration file.
And I just provide a few key elements, the flavor, the Kubernetes version,
how many control plane I want, how many workers I want.
One interesting thing is the flavor.
So cluster API relies on templates.
So these templates are provided by the maintenance of the providers.
So for example, the flat card template will deploy a workload cluster
based on flat card images.
You have some flavor, for example, on the open stack with load balancing,
if you need some load balancing services and stuff like that.
So flavor is a way to customize your deployment of your workload cluster.
You will still have a workload cluster in the end, you get a Kubernetes cluster,
that's fine, but you can decide to customize it.
So this flavor, this variant, are tested using end-to-end testing.
So each time there is a new release of the providers,
you can be sure that it passed the CI, so you can safely update your configuration.
Of course, for clarity, I didn't mention that you need to provide a few more
environment variables to, for example, to provide the credentials.
Of course, cluster API is going to create some things on GCP, on AWS,
on the open stack, whatever.
It needs to get access to this infrastructure, so it needs to get the credentials.
So this is an example of things you can pass, but you can also define which
instance size you want to use for your control plane,
which instance size you want to use for your walkers.
So this is the kind of elements that you need to configure previously calling this command.
But yeah, just for demo purpose, I wanted to show you this command line,
which is the bare minimum to generate the cluster configuration.
And now we have the KAPI quick start.tml file.
We can apply it like any over Kubernetes manifest file.
So KAPI Ctl, KAPI Ctl, apply KAPI quick start.tml.
And it will create, as usual, some resources on my management cluster.
So we can see that there is the cluster definition,
there's machine deployment.
So this is something common to cluster API.
Then we have the open stack specific part.
And this from one provider to the other, of course,
the output will be different.
But that's the idea, you just apply this.
So that's pretty convenient because you have a file.
So you can use this file in a Git repository.
You can use this file in a CI.
You can use this file with whatever you want.
So you have an infrastructure as code in term of cluster API.
Now, if I check on the provider side,
I have some resources that are going to be created by themselves.
Not by themselves, by cluster API.
But you can see that I have some instances.
So I asked for one control plane and three worker nodes.
So we can see that I have four instances between being created.
I have a network, I have security groups,
I have stuff, SSH keys.
So this is for open stack, but once again,
it's the same thing for every provider.
But this is the cool thing about cluster API
is that it does not just deploy a cluster.
It deploys everything to create a cluster.
It's instance, the security groups, the firewall rules,
ingress, egress, whatever you need.
So it works in this way.
When everything is up and running,
you will just get your configuration
that you can inject into Qubectl
and then you have a new cluster ready to be used.
So that's it about open stack.
Now, we can ask yourself what's under the hood
on the operating system side.
I'm a factor engineer.
I work in the operating system field.
So I'm curious to know what's power my nodes.
So with Qubectl, we can inspect the nodes
and see that for example, this one is running Flakar
because I asked for Flakar variant,
but for example, with Flakar, we do not ship QADM.
We do not ship Qubelet service.
We do not ship MISC files.
So how my nodes can start acting like a Kubernetes node.
How things can work.
And on top of that, Flakar is immutable.
There is no package manager.
So there is no way cluster API is going to SSH into that node.
It say, okay, APT install QADM.
No, no, no.
So what's the magic behind?
It's another project called the Image Builder.
So it's on the Kubernetes 6 GitHub repository.
That is the Image Builder project.
So the idea is to take an OS, for example, Ubuntu,
to build it with Pacer.
So nothing new under the sun.
And to inject the QADM, the container runtime,
the MISC files, whatever you need to power Kubernetes nodes.
So it's a three step thing.
You take your OS, you inject the Kubernetes components,
and then you export this new image, this golden image,
like we sometimes call, into your providers of your choice.
Open stack, GCP, AWS.
So you understand that something quite complicated
because in order to use cluster API,
you need to use this kind of image.
So it means I can wait for someone
from the community to build it.
The build of the image is not an issue.
Everyone can build image.
It's more about the storage.
Because storing an image, it's something,
but when you have to store image for each Kubernetes version,
so it's three main versions at each time.
So three Kubernetes version,
then I have to keep the image for each cloud provider
I wanna use, and I have to keep an image
for each different version of Ubuntu.
It can be complicated to store everything
and to have the time and the energy to build these images.
But this is what we currently do with this provider.
So that's, I will not say this is the way to do,
but this is commonly done currently
in the open source world.
So we can think about something, an alternative.
And the truth spirit to me of the open source world
is to have alternative.
So there is no one way or one other way to do things,
there is alternative.
Then you choose which one is the best for you.
So an alternative would be, okay,
I take a Linux based OS, for example,
Ubuntu, Flat Car, whatever.
It's already available on GCP.
It's already available on AWS.
It's already available on Digital Assign Azure.
Because these cloud providers provide these images for you.
So just the vanilla image, a fresh image
is already available.
So what if now we download the Kubernetes components
during the boot of the image?
And in the end, we have the same result.
We have a Linux based OS with the Mixed file,
with the QBGM, everything I need to power my nodes.
So this is something we implemented
on the open stack side.
So you just need to use an over flavor.
It's SystemD CZex, Flat Car dash CZex, sorry.
And it leveraged this new feature of SystemD
called SystemD CZex.
Basically SystemD CZex, it's an image,
raw, squash fs image that you're going to mount
as an overlay on the Linux based system.
And it will bring you new binary files,
new configuration files into your system.
So if you want to have a look to SystemD CZex,
I really encourage you to check this new features
from SystemD and this is what basically
we're going to do with this flavor.
It's during the boot, we're going to download
QBGM SystemD CZex image and everything
will be open running to power my node.
One, just for example, if I SSH on the node,
I can just run SystemD CZex list
and it will give me the output
of Kubernetes image being available.
So what's, what it's cool with this approach
is that you remove the strong bounding
between the Kubernetes version and the image version.
So if you want to update Kubernetes
but you don't want to update your base.OS, you can.
If you want to update your base.OS
but you don't want to update Kubernetes, you can.
Before that, you were supposed to build new images
and stuff like that.
And the cool thing is that SystemD CZex is,
it works in the same way on AWS, on Azure,
on premise, on whatever.
So you have just one configuration
for all the cloud provider.
So that's something to keep in mind.
And we discussed with cluster API folks
to see what could be the new approach of this.
We also attend some office hours of the cluster API
or ecosystem to make some demo of this.
But it's already available on OpenStack
and we hope it will be available in the next,
in the over providers.
A few resources, if you want to continue,
check this at home.
You can have of course the cluster API website,
the cluster API OpenStack.
This is for the example I've shown,
Flattar and SystemD CZex,
which is the main outlook in the end of this talk,
is SystemD CZex.
But yeah, so to conclude,
I would say this talk is to give you the key elements,
what's going on in cluster API, how does it work,
but also to give you an overview of
what we currently working on
on this aspect of cluster API.
So yeah, thanks.
And of course I forgot to add the QR code,
but you can find them on the FOSDEM website.
And yeah, thanks for your attention,
if you have any questions.
So if you have any questions, or maybe on the chat,
if there is some.
I didn't see anything, but we can start with you.
Yeah, do we have a mic?
No, you will pretty much.
Okay, we repeat the question.
Okay, I have a philosophical question
about cluster API.
What's the life cycle of Kubernetes cluster to you?
It's long running, you upgraded,
or for it just temporarily,
you just replace with a new one.
So the question is about the life cycle of the cluster.
Do we need to replace each node
when there is a new version?
That's correct?
Yeah, but what is the intended purpose?
It's like long life cluster,
so it's like just for short term.
Well, the goal is to have, yeah, okay.
So the question is, do we use cluster API
to have long term usage or short term usage?
I'd say that cluster API is in the philosophy
of leveraging Kubernetes,
which means using the reconciliation loop of Kubernetes.
You can just things that by themselves
and if there is, I don't know, a network,
so instance that going down,
it can be restarted by the management cluster
that will take care of,
there is a state and you still want to be in this state.
Because with Terraform, for example,
there is a state, of course, but it's not live.
It's a static state.
So if there is some things missing,
you need to rerun, plan apply,
to be sure that your things get back.
So that's one of the difference.
Yes, Mr.
Why cluster CTL to generate a template
instead of using Helm templating?
That's a great question.
That's a great question.
As you can see, can you repeat?
Yeah, so why use cluster CTL CLI tool
instead of using Helm or customizer or stuff like that?
So the idea of cluster CTL,
it provides some sugar on top of the common generation.
So you can manage your clusters.
So that's one of the features.
But you can also have variable injection.
When you generate a template,
it will check if there is some missing variables
required by the providers.
So I think you can perform the same thing with other tools,
but cluster CTL is just handy and you have it in this way
to just be sure that you don't miss
an environment variable to configure your deployment.
Yes?
In terms of the overlay or the flat card,
how hard or easy is it to build custom overlays?
Say if you've got OEM integration,
what's the tooling to support that?
So the question is about the overlay
and how to build these images.
If I understand correctly, the system is CZX.
So you don't have to build,
because we provide them on...
You've got to say you wrote custom security...
Yeah, you can...
We've all decided to fork the repository I mentioned,
which is called FlatCard CZX bakery,
where we provide these images.
So you can fork and do your stuff
and why not send a peer,
if it's something relevant for the community.
And it's just a matter of SquashFS.
If you have SquashFS utility tools on your system,
you can just build your images.
Basically, everything will be in a directories,
then you convert these directories to a SquashFS image.
Yes?
Does the machine deployment controller do any sort of like...
If forcing reconciliation,
so if you were to delete the instance in OpenStack,
would it be created?
Yeah, not immediately, but in a few seconds,
few minutes after it, we say,
okay, I have to get four machines, I have only three.
One missing is a walker node,
so I need to go to habit.
So the question was about...
The question was about if there is some instance
that is, that disappear from the OpenStack
or the provider's dashboard,
does the management cluster restart the instance?
Yes?
As a Kubernetes admin,
I really love to manage my Kubernetes classes
with Kubernetes resources,
but I always wanted to bootstrapping problems
like how do I manage my management cluster?
I mean, I had so many projects,
but I'd love to use cluster API,
and it never makes sense,
because in the end, I end up using CubeSpray
for the management cluster,
and I can just add it using CubeSpray.
Yeah, so I think the bootstrap issue,
they got the same question.
So the question is about how do you create
the management cluster?
So I think this logo is well representative of the issue.
It's the torter, the torter that you stack,
and in the end, there is no answer,
because your management cluster,
you can define to handle it with another management cluster.
So you can change cluster API if you want to, why not?
That's something you can consider.
And on the new workload cluster,
you can just say, okay, now it's a management cluster.
So I'm going to deploy cluster API on the workload cluster.
That, of course, is just theoretical.
There is no practical way to do that,
and it's not the point.
But your management cluster,
that's what I say, you can use really something simple.
I think I see that there is this new Kubernetes tool
that you can use.
It's like deployment without Cubelet
or something like that,
so maybe, yes, because you just need the APIs
in the end of the Kubernetes.
And so why not try to come with something like that,
to just deploy a set of API, and that's it.
But you can, yeah, we can do things like that.
You can decide to use kind, for example.
That's the best way to deploy things
for the management cluster.
Time's up.
Okay, thanks everyone.
Have a great day.
Thank you.
Thank you very much.
Thank you.
Thank you.
Thank you.
Thank you.
Thank you.
Thank you.
Thank you.
Ooh.
Is it all right?
How's that?
Cold share.
It's all right.
You gonna have it?
Yeah.
Oh, yeah.
Do you also have to use to the...
You'll see it, so.
I think we're gonna have to do the...
So...
We can enable hotspot
for the...
I think we'll stop here.
Yeah, let's go here.
Do you have presentations?
Yeah.