Alright, let's get started.
Is that, I am unmuted, yes.
So yeah, this will be fairly quick, just an update on container D. You're either here
because you're interested in container D or because it's too hard to change dev rooms
and so you're just going to sit here and hear about container D. Hopefully you're somewhat
interested.
I was having a bit of phosom nostalgia like 2018, talking about just like the first year
and a half of container D getting to 1.0.
So now we're on the cusp of our 2-0 release, our first time having kind of a major version
since we started the project.
First just a few stats in case you're unaware.
Container D adoption has been growing a lot.
Some of that's probably due to the Docker shim deprecation in Kubernetes.
This is from DataDogs, ANO report.
The CNCF and Sysdig also put out reports.
They all come out with different numbers so believe whichever one.
This one was positive for container D so I used it.
You can probably find another one.
Maybe more importantly to the project are actual community growth so people actually
contributing, getting involved in the project, becoming maintainers.
This is a crazy eye chart from the CNCF.
You can see Kubernetes way up there at the top.
Again, there's some magic math being done here about how many PRs and issues are flowing
through your project.
How many people are contributing and it comes out to container D being in the top 15 or
so projects.
One of the cool things is that we've had a lot of, I think this captures like the last
nine months, but new maintainers, reviewers, committers from many different companies,
some independents.
So that's awesome to see as well.
The cloud providers you might be using use container D underneath their Kubernetes service
and some other projects as well.
The thing I wanted to focus on is one of the reasons I think container D continues to grow
as a project is that we've built in extensibility in different directions.
I'll talk about three main directions that container D is extensible or how you can build
around it.
One is on the client end and so one of the newest representatives of that is Nerd CTL
written by one of our maintainers, Akahiro Sudha who you've probably heard of because
he's written 100 different projects in the container space and anytime you use rootless
containers it's probably because Akahiro started that work many years ago.
The hero nerd CTL which gives you now kind of a Docker command line for container D.
The other way that we're extensible is in snapshotters and those are, if you remember
Docker's graph drivers, these are the way that your containers file system are actually
stored and so overlay is obviously a very common one that many of the container runtimes
use but we've actually made it.
So we have built in ones which I'll talk about but we also, you're able to extend that with
a remote snapshotter and that's an area where we see a lot of growth where people are writing
their own snapshotters for their own unique use cases.
Then sort of directly down from container D is this layer we call the shim layer that
drives an actual OS level runtime and so obviously many of you have heard of Run C or C Run
that's kind of the common Linux adapter if you will that drives the set of syscalls you
need to name space your process but the container D shim API again is extensible and there's
many different shims available and we'll talk through those.
So these are kind of three directions.
There's also some other pluggable interfaces that I don't have time to get into today but
these are all ways that again as we go into 2.0 we continue to see people expanding container
D in these directions.
I'll spend the least amount of time on clients.
We've had this sort of simple tool in the project since the beginning called CTR.
It was never really meant to be a production client for container D but just an easy way
to poke at the API, get a list of images, list of processes.
Run CTL is much more recent and has its own set of maintainers who are marching along
with new releases that are either bringing better alignment with the Docker command set
so all the flags, all the features or adding features that they can reach because they're
built directly on container D like some of the lazy loading snapshotters, image encryption,
container image signing, all those are built in to nerd CTL.
Cry CTL is from the Kubernetes community that drives the CRI API of which container D has
an implementation obviously CRIO and others have implementations for that API and then
of course the Docker project is also built on container D.
There's some interesting developer platforms built around these clients.
After desktop and CoLima allow you to drive the Docker engine or container D but we have
a team at Amazon who built Finch that's just built on nerd CTL build kit and container
D again that allows you to do macOS and I forgot to add Windows here because we just
launched Windows this past week.
But again these are ways that people are extending the capability by building new clients around
container D. So the other area I mentioned was snapshotters.
There's a bunch of built in ones.
Many of you will recognize things like overlay and device mapper, butter FS but this plugability
of having proxy plugins to a remote snapshot are so now two things you're not tied to container
D's release life cycle.
You don't have to get your snapshot or merged into the container D code base.
You can write your own, you can run it as a separate process with a GRPC listener and
container D will call you for the API of the snapshotter, prepare, diff, unpack and those
operations that are required for the snapshotter.
So there's three main ones that all three of these have now been donated into the container
D GitHub organization.
So they were started as external projects and they've now been donated.
They're all related to lazy loading file system so if you've played around with being able
to run a container but not having to pull the entire image, say it's a 10 gigabyte image
with scientific data sets or some complicated ML model.
These lazy loading snapshots will only pull the files that are needed to start the container
and so Star GZ, overlay BD and NIDUS are all in that family and then there are two, there's
Sochi that was built by one of our teams at Amazon that is seekable OCI so again a lazy
loading snapshotter and that's open source but then GKE also has a feature called image
streaming built around the same ideas of lazy loading but that's at least for my understanding
that's not an open source project today.
So again these are ways that people are extending container D by having their own snapshot technology
and plugging that into container D.
Allison mentioned shims so OCI runtimes, there's several options there.
So we have run C built in, you can also use C run and we test that in our test suite for
container D and there's also some experimental Rust and free BST runtimes but then again
you can have your own shim outside of kind of the container D core project such as the
one for Windows maintained by Microsoft, HCS shim.
Run wasi is one of the more active projects in the container D, you have namespace where
again this is a shim where you can drive container D to the same API and clients but actually
run wasm workloads instead of running a traditional Linux container and again there's a micro
VM based shims, trusted execution environment and Quasar I think is how you pronounce this
shim that deals with a new feature of container D 2.0 called sandboxing which we'll talk about
in a minute.
So again those are just three ways that I think have benefited the sort of container D's growth
of being able to plug in and enable features that don't have to be part of the main container
D code base and allows people to sort of expand for their use cases that maybe we don't even
know about.
So this is kind of the picture of where we are currently in the container D life cycle,
1.5 is now end of life, we created 1.6 as a long term support release that again until
2.0 is released we don't have an official end date but it will at least go out another
few years.
1.7 is an active release cycle right now and then 2.0 should release in a month or two
based on kind of our current set of betas and release candidates that we're in and so
that's where we are as far as releases.
I just mentioned this isn't new news but 1.6 is our first LTS release as it says here support
at least until February 2025 and of course it's always a trick to try and maintain some
integrity about how you get things into the LTS and one of the reasons that's tricky is
that Kubernetes may add features in the CRI we need to implement that CRI endpoint so
it sort of looks like a new feature and so we're having to try and do our best to make
sure that we maintain compatibility with Kubernetes without sort of opening up 1.6 to a lot of
new features so that it's a stable and mostly just back ports of fixes and obviously anything
security related.
So yeah so we have this idea that late this year we'll even make that back port criteria
a little bit stricter so that people can rely on just a long stable release without a lot
of changes to its feature set.
1.7 therefore is the end of our 1.x release cycle and what you'll see here is that we
basically merged a lot of new features in 1.7 before we released it that we marked them
all experimental so that people could start to try and use them and then in 2.0 all those
become supported features and so I already mentioned the sandbox service and the API
around that again we had this extensibility at the shim layer but with micro VMs and other
ideas about how you treat the sandbox and how you configure it several of our contributors
came up with the sandbox service and there's a whole API around that you can read a lot
more about it on our either via the PRs or the documentation that's been merged.
It was a preview in 1.7 but it'll be automatically turned on in 2.0 so in 1.7 there was a split
that we actually had two implementations of the CRI one based on the sandbox and one our
legacy code so that'll go away in 2.0 where it will just have the default sandbox implementation.
NRI is very interesting if you've ever played around with OCI hooks and the ability to you
know modify the specs so say I want to insert a device before my container starts the node
resource interface is the sort of our decided implementation for doing that safely and having
a way to have NRI plug-ins that you can that the administrative your cluster can enable
and give the proper permissions to so NRI was experimental in 1.7 again will be fully supported
in 2.0 and then transfer service so if you think about commands like save or export an
image pull an image push an image in all our previous releases of container D that was
a client side API and so your container client was actually doing those registry interactions
in 1.7 and then of course in 2.0 this is now a service within the demon and so for some
some use cases that was very important that the demon handles credentials of the demon
handles the network connectivity to registries and also gives us a lot more tools for plugability
of sort of source and sync so say I'm trying to copy an image from one place to another
the transfer service gives you all that in a configurable way we also added username
space support which was a new feature coming down so container D core had username space
support but the CRI kept the enabled username spaces and Kubernetes added new API to the
CRI and so those are now plumb through and implemented and supported in container D and
then we had a lightweight RPC mechanism for shims and we've now added full GRPC support
which was important again for certain use cases that people wanted so as I said we're
in the midst of like our 2.0 release plan right now we are just about to I guess I didn't
move that line over far enough because it's February now and we're just about to put out
our first release candidate so we're possibly a little bit delayed from our original thinking
but again 2.0 will be final sometime this spring and like I said all these sort of new
capabilities that were in 1.7 will be final and supported in container D 2.0 it was our
first chance to finally deprecate so we've been insistent on keeping a very stable API
so that you know people aren't surprised that the latest container D release removed something
so you can see that over the years we've deprecated a lot of features or at least mark them deprecated
2.0 will be the chance for us to finally remove those and provide recommendations. One of
our contributors added a nice feature so you can actually turn on deprecation warnings
and you can actually run container D 1.7 or even 1.6 LTS and get notified of all the features
you're using they're deprecated to help you prepare for 2.0. One of the things we were
going to remove was support for our oldest configuration version but then someone wrote
a converter that automatically converts your configuration so we won't actually have to
deprecate that in the sense that you're not going to have to rewrite your config unless
you'd like to it'll do automatically for you. There's still a lot of things we'd like
to do that we're still working on so I mentioned this new transfer service again the CRI is
a plug in implementation within container D that uses container D's APIs to do the work
so when the CRI says pull an image the CRI implementation calls into container D to do
that so one of the things we're trying to migrate that to use the new transfer service
so that's in development to allow plugability for shims themselves and then there's two
there's two kind of API layer enhancements that we're thinking about if you think about
Docker, Docker kind of gives you this higher level API again HTTP based if you ever have
built a tool that uses the Docker API it's at least nice in that you can say run container
and give it all the configuration information and it just does it and when people have come
to container D they're like hey you don't have the Docker API what can I use that's similar
to that and we really don't I have to create a task I have to create a container resource
that I have to start the task and so we're thinking about really creating some of these
abstractions so that when people move to container D they have a higher level image service and
container service so those are things that if you have ideas if you have concepts we're
open to them these aren't things that we've built yet but we're planning to as we go into
the container D to the T to dot oh time frame if you're interested in contributing or getting
involved there's a couple channels in the CNCF slack that we hang out in that we you
know talk about new features or people ask us questions we do have a live community meeting
on zoom twice a month the second and fourth Thursdays if it's bad for your time zone let
us know obviously that's always a tricky thing to handle with time zones and again go to
the repo open issues give us your ideas pull requests and that's all I have thank you