Okay. Take a picture. So, good morning. My name is Daniel Mejado. I'm here today with
my colleague Miguel. Here we are both principals of our engineers at Weihardt. I'm currently
working in the Epsilon DTT team, basically monitoring so, and he's on the OpenShift virtualization
team. I guess we just wanted to speak today a little bit about CNI, which stands for Container
Network Interface, which this is basically all the networking in Kubernetes, but not
only limited to that. Because I think one of the things that this project lacks the most
is documentation and how does this work, how you create a plugin, what are the primitives.
I think that's something that is super simple, but there's little to no documentation besides
the spec. So, let's go watch through that. Yeah, this is a quick intro. So, you may have
noticed and that we are going to be speaking about CNI plugins changed specifically. That
means that we are going to be basically putting a couple of plugins in CNI mode. But first
of all, CNI may stand for three different things and I want to be a little bit clear
here. Like you're going to have the CNI specification itself, which is a document. It's fine. You
can read it. Then you've got the plugins. So, if you go to GitHub, you can see the CNI
plugins. This is a set of plugins basically maintained by the community. Bridge, Mac
BLM, you name it. There's a couple of those. And the third thing, it may be just a couple
of plugins. So, those are basically the three things that you may be interested into. We
are going to be always speaking about the plugins here in this session, but we'll hopefully
discuss anything after that. So, just in case, I mean, here we are going to be speaking about
Kubernetes, but let me take a note that the CNI specification on the plugins, they do
work totally without Kubernetes. So, with any runtime engine such as Rocket, and in fact,
these are started out with Rocket. Well, I guess in case you started that out, you can
be running this out with cryo, Rocket, whatever you want to play with this. So, what is the
CNI plugin specifically? CNI plugins are binaries, which are basically copied over any of the
hosts that are in your Kubernetes clusters. So, if you want to install this out, you probably
need a demo set to deploy it on or do that manually. So, here, who runs this CNI plugins?
You could either run that yourself, but usually, you would need to have the kubelet run those
for you. Is there anybody here who doesn't know what a kubelet is? Okay. The kubelet,
oh, you fuck, anyway. They were trying me out. So, anyways, I'll go ahead. So, basically,
when you go and have the kubelet, which is a mystery thing that you don't know what it's
about, runs the binary here in the host, it basically creates a network name and space,
which is tied to a BF interface. As this guy doesn't know what a BF interface is, I'm
going to be basically speaking out for him. So, it's a B12 point-to-point tunnel, so you
can basically hook this up into whatever you want to. Like, usually, these are connected
to OVN, OpenB streets, or whatever, B12 streets, or it depends on the CNI plugin implementation.
How do I configure the thing? These are used, like, configured by a configuration file.
We're going to be speaking later on a little bit about how this changes. But, yes, FYI,
and Miguel will be speaking later on about this more. Prior to the CNI specification 0.3.0,
you weren't able to, you know, put your plugins in CL mode. It would just break. So, you would
need to know, basically, there's a couple of CNI components that you need to put. Basically,
the name of your CNI plugin, the type, and the type should match the name of your binary
or it would probably break afterwards. Then, later on, you need also some mbars, which
basically are specifically about telling you, like, okay, it can switch both the way around
this, you see, because it may be a part of whatever. If you put all this out, and, again,
recall, a CNI plugin, it's a binary, so you just run it as any binary you would run, goes
to your CNI plugin, and then you can exit code. If you did your things right, like that,
this should be getting you an exit code of zero, not error, and then you would be getting
your standard out here, which is just JSON. Why JSON? And why not using the RPC and the
daemon, like everything in Kubernetes, you may ask? Because, again, this wasn't meant
to be used on a lot of Kubernetes, and if you get to stay at the more networking session,
they'll probably be speaking about some point using YAML for that, too. But here, for the
sake of this session, what you would need to remember here is this prep results, basically,
and advancing some stuff later from my colleague. This is the previous results as per JSON
of the plugin, and if any plugin gets a previous result here, it should output the next one,
just in a serial mode way. I was speaking about the specification. What the hell is
that? So, yes, if you're implementing your plugin, you need to make sure that it observes
several primitives, which is add, del, check, and version. So, for instance, you may want
to add, okay, I want to add a pod. I want to delete a pod network interface. A second
version, to be honest, and even if the specification really requests you to, you know, honor this
and observe this, they aren't really used by most of the commercial appliances. So,
for instance, Celium, I guess they don't do any kind of check, maybe version, but those
aren't really needed, and we'll be discussing that later on. So, this is what a coffee file
usually looks like. So, this is nothing like a big deal. Super simple. CNA version, again,
for example, I'm getting you backwards, like with the things that we were speaking before.
This one would work with CNA plugin change, because it's 0.3.1. If you put 0.2.0, it would
basically fail miserably telling you that he hates you, and it would crash. Then you
got here the name, I don't care, but this type should match the name of your binary
there later on. Then you can put out some plugin-specific things that are, for instance,
this basically comes from the Brits plugin, which Miguel will be showcasing later on.
So you can put here, okay, this, what do I need for a Brits? I need the name. You see
the default gateway, I'm going to be forcing address later on. This is a little bit special,
but I won't be discussing that because you'll be seeing that later on the conflict. Again,
like, okay, so you told me about CNA plugins or binaries, and also I got Jameson's configurations.
So where do I store these things out? So basically, when you define the cubular configuration,
you tell the CNA binary path, and by CNA binary path, it's just a directory where I store
all the plugins. Sadly, there's little to no login in most of the plugins, so if something
breaks, you would have to go and see what's going on here. So you would just go, one of
the things that you'd like to fail the most is just like you don't have the binary here
installed, and it would fail and tell you that it hits again. And the conflict directory
is here, and here's just a couple of Jameson who maps to the name of the binaries usually,
and where do you put this specific plugin configs? Plug-in chains. Let's go for that.
So that's the big deal. Again, available since CNA or CO2-0, those are required since the
version 1, because now everything is the chain, even if it's just a single plugin, and everything
is the conflict, again, even if it's just a single plugin. So if you just want to put
the things in the old way, it would just break. I was also pointing out that now, let's assume
that you've got a couple of plugins here, like, you've got the Bricks plugin, then you've
got a Firewall plugin, which is one of the most used cases for this, and then, let's
say, put a couple of plugins bound with Firewall because I like to, and I'm going to be putting
another Macbillan interface here. So all these plugins are expected to honor, like, that
they're going to be passing the prep results to each other. So even if they don't know
anything about what comes before, because the Bricks plugin will know how to make a
Bricks, but the Macbillan plugin won't. So everything comes through the plugins, all
the info goes through using prep results, and we'll be showing you later on as in with
the demo, if it doesn't fail. This is the config, this is not a config, this is not a
config for CNI plugins, this is a config list. So if you see, this pretty much resembles
what you had in the config file for a plugin before. So now we got CNI version, okay, I
know I'm just saying this a lot, but it should be greater than 3.0, otherwise this would
just break. This is your config list, and then you get, basically, a list of plugins.
At one, we got a few of those, and those would be just working together. So in this case,
we got a name, because all the plugins would need the name, it would need the type, which
would match the binary, and then we got the specific configurations. So in the demo, we'll
be showing you how these plugins get to link with the prep results and so forth. But if
you see, it's just a list of plugins with name, type, and then specific configurations,
but if you just want to do something, okay, I'm buying this, I want to create my own plugin,
you really need a name and type, even if it doesn't do anything. Okay, go ahead, Miguel.
Now, use cases, and then, I guess I spoke maybe a little bit too much, and even if we
are late because we got a rush to the airport and that, but I'm handing over to Miko Miguel,
who's going to be showing you this a little bit around, and then if it works again, demo.
Go ahead, Miguel. Hello? Well, I'll use this. So this is like a list of the plugins that
are provided by the CNI maintainers and are available on the container networking slash
plugins repository. So the first is Tuning CNI. It allows you to configure like a list
of syscuddles. So if you need some syscuddles to happen inside of your pod, you use this
Tuning CNI. The bandwidth CNI, as the name implies, like quite evident, what it does
is to throttle either the ingress or egress traffic to your pod, if you want to do those
kind of things. The firewall plugin, what it does is only allow access to and from the
IP addresses that are referenced in the results that the plugin got, for instance. And the
port mapping, as the name kind of implies, it does port forwarding, configures port forwarding
from the host into the container for the set of ports that you specify in the configuration.
Now, having said this, let's go to the demo, and I'm now wondering how can I do this while
holding the microphone. This doesn't work, I think. Easy. Okay, so the first thing that
I should mention, like Daniel referenced Kubernetes a lot, but please remember that CNI is more
like, like Kubernetes is a user of CNI. Like, it's not that you can use CNI by itself, and
as such, we're going to be using something called CNI tool. It's just a binary that you
point at your, at a certain CNI binary and give it like a set of environment variables,
the plugins configuration via standard then, as Daniel said before, like, you basically
pass the configuration of the plugins, you give it the input parameters, which are the
environment variables, and you execute the binary. And this is how we're going to be
seeing, well, the demo. You can follow it on this, on this link. But yeah, the first
thing I think I should explain is like the scenario we're trying to achieve. And this
can be seen here in the bottom right corner. So very simple thing. We just want to have
like a bridge, two namespaces interconnected via this bridge, and we're going to be configuring
a static IPM on static IP addresses on each of the namespaces. And then we'll run an
IPer from the client to the server, and we're going to see how it fares. So first, I also
need to show you the configuration that I'm using for this. Okay, it's this one. And this
is the configuration that we're going to be using to achieve this scenario. Like, we're
going to be using the plugin of type bridge. This is the name of the binary that will be
invoked on the host to create the bridge. We get the name, we enable the IPs capability
so you can put a static IP in it and you tell it that you want static IPM. And that's pretty
much the configuration that you need to give it. So let's just run the example. Okay, like
this parameter here that you see, can you see it? It's like the font big enough. It's
good. Okay, thanks. Like, you have to give this like the name of the configuration if
you see like unlimited bandwidth, it's the attribute here on the upper left corner that
you see under name. It's the same thing. It must match. And now this configured the scenario
and it's now running the IPer session between the both the client and the server. And as
you see, we're getting like a very big bit rate of around 60 gigabits per second. So
this is with a straight configuration. Now let's use a different configuration where
we put we reuse the first configuration, but we use it in a chain. And afterwards, we put
like the bandwidth plugin. Again, what this does, it will throttle the input traffic into
the network interface of both namespaces. So we're going to be doing the exact same thing.
But with this added with this different configuration and bandwidth limiter. And as we see, like
the bit rate that we're getting is literally a lot less. And it should map somehow to the
values that we've configured here. So what this shows is that you can use like this band
with plug in a chain in order to achieve a different use case than you had before, like
you want to throttle traffic to this, you use this type of plug in. Yeah, I think that's
let me just check the time. Yeah, we're good. We still have 10 minutes. So let's run the
second. The second demo we had, we have actually sorry, okay, the scenario is a lot simpler
this time, like we just have the same bridge as before, but we just have like a network
namespace connected to it. And what we're going to be doing is showing you how the chain
actually works, focusing on the on what Daniel said before, like you need to handle always
the previous result, and you need to account for it. And you need to pass it along the
chain continuously. And okay, let's just show the configuration of this plugin. So this
chain might look a lot more complex than the one before because it has more things in it,
but it's very, very, very simple. So this thing first will in it will call the bridge
plugin, it will create the bridge. Then we'll invoke the debug CNI, like this CNI plugin
is very, very simple. The only thing it does is print the result it got from the previous
plugin. So what we're going to be seeing is here is the result of the first plugin in
the chain. Afterwards, we run the tuning CNI plugin. And what we're going to do is to change
the MAC address that we got on the interface of this dummy namespace that we see here.
So the idea is we first run the bridge plugin. This thing will assign a random MAC address
to the interface that is on this namespace. We'll print that will run tuning to change
that MAC address. And we're going to print that again to see like the result of the previous
plugin. And that's pretty much it. Let's just run the example and show you the, what I actually
mean. So here we see like, and so this log here is the result of the, of the first call
of the debug CNI. And as we see in the, in the pods, in what would be the pod interface
which is identified by this attribute sandbox that points to the name of the namespace or
actually to the path of the namespace. We see that in its result, we have the interface
name that was created on that network namespace. And we see that it was assigned like a random
MAC address that is identified here. We then run the tuning plugin that changed this MAC
address and we finally printed the previous result again. And we see that this changed
to the MAC address that we specifically specified statically in the, in the plugin configuration,
which is exactly what we wanted to show. Like that's like very simple demo, but I think
it kind of illustrates in a very neat way how chaining actually works in a step-by-step
manner. And with this, we arrived at the conclusions. And I'd like to basically tell you again
what we just told you, like focusing on the most important things that we think are about
this slide. First thing is like these plugins, remember what they do. They add more stuff
to the pod. So they enable different use cases. Like they can, you can prevent IP spoofing.
You can throttle bandwidth as we've seen. You can configure port forwarding from the
host to different containers. You can configure different syscuttles. And actually you can
also create an allow list of the syscuttles that you can use inside of the pod. And finally
I think like if you have to, to keep one thing from this presentation is that a meta-plugin
must always handle the result of the previous plugin. Like you need to account for it. First
of all, because you don't know if, if you're a plugin, you don't know if anything will
run after you in a chain. Like the user will configure it. So you don't know what's going
to happen afterwards. So you need to send a result. And if you're somewhere in the middle
of the chain, least you can do, or least you must do is grab the result you got from the
previous one and echo it into the next one. Now finally, remember that two things, plugin
chains are only allowed starting from CNI 0.3. And they're the only configuration type
allowed starting from CNI version 1.0. Like if you use the first example, configuration
example we shown on CNI 1.0, it wouldn't work. It will explode, it will fail, make you miserable.
And like the idea here is know your previous result always because that's probably the
most information you'll get from anything that ran before you. Like as Daniel said,
everything is clearly not the thing that's most prevalent on, at least on the CNI, on
the plugins that are maintained by the CNI maintainers. And yeah, this concludes our,
our talk. And so thanks a lot guys. Questions? Questions? Thank you. Thank you. Can you tell
a little more about use cases of CNI without Kubernetes?
Okay, that's a really good question. And it's going to probably eat all the time that we
have. Like I can, like I work for Qvert, like, and one thing we do in it is, so there's a
pod. And inside the pod, you run a virtual machine in it. Now, CNI, what it does is configure
the pod interface. But what you want to have is like networking inside of VM, like you
need some way to get like the extended connectivity from the pod, from the pod interface and into
the VM, you need to have something there. And we have code in our code base in the Qvert
code base to achieve this, like one of the thing we could do using CNI chains to offload
that entire code to CNI plugins that would create, for instance, the bridge that you
have inside of the pod to extend that connectivity that would create the tap device from which
the VM would create the emulated network device from. So I really think this could be
moduled using CNI. That's something we still need to see, like it's a very rough idea yet,
but it's an example, I think.
And so besides that, I guess the quickest one is, yes, you know, he was mentioning CNI
tool, CNI tool is just used to develop CNI plugins. It doesn't have Kubernetes at all.
And you can see that the plugins have just run there. Even that, Rocket, so Rocket, it
was on its own. He didn't have to use Kubernetes at all. And it's where most of the CNI plugins
were originally developed. You just put any kind of, you know, runtime engine, and it
would work. No Kubernetes needed.
So if I have a CNI plugin that sets up some external state, for example, a firewall that
might even be a separate device, and something goes wrong, and I lose the delete, how do
I recover from that?
So that's an amazing question. Like you need all, like you must design everything assuming
that CNI deletes will be missed. Like it might happen like all the time. So you need
to have a reconciled loops of sorts that knows about your state, relevant state, and reconciles
it somehow.
I didn't see a way to even check if the stage should still be active. How does my CNI plugin
know that something is still there?
It's clueless. Like you need to monitor this little kid and assume that he will not fall
and hit the head in the corner of the table. You need to do that out of bed. It's not designed
to allow for that. I'm sorry.
Hi. During the presentation you mentioned that some CNI plugin like Cilium, but I guess
that you were also mentioning other plugins, they're not doing the logins and they're not
using the entire APIs. How come?
I guess you hear it. It's okay? So it kind of depends on your CNI plugin presentation.
So for instance, some CNI do implement logging, but it's not something that is within implementation
itself at all. So you may be totally fine in doing no logging at all, but then go back
to debugging it so you can have to go to the queue with loops, you can have to go to cry
over loops, and then go back to debugging all those. So it depends on your implementation.
For instance, that, which is here, he implemented some of his logins for enamel, but that's
not something that is on every plugin. For service, not in any of the community maintained
ones.
Okay, thanks.
Any more questions?
Well, thanks a lot for your time and for listening to us and bye-bye. Enjoy your time.