our next talk. I guess I don't have to give a big introduction. A lot of you know Phil,
right? He's going to talk about Turing the container developer tooling landscape.
All right. Hi everybody. I think I'm on. Yeah, you're on. Yeah, so thanks for coming.
It seems like Fossum is back. We've got a packed containers room. I think my talk is mostly a warm-up
for Akahiro after me. So a lot of familiar faces here. A lot of good talk so far. Maybe this will be a little bit different
than the last few talks. We've been talking a lot about containers and various environments, but we haven't really talked
about tools. You've seen a few tools used in some of the demos. And so I'm just going to talk through where we are these days
with development tooling. Interesting that it's 2023 and interesting year in that we're now 10 years since Solomon
Hikes gave this demo during a lightning talk at PyCon. I think it was like April. So getting pretty close to 10 years
since someone saw Docker being run at the command line for the first time. And what an interesting demo was
because he misspelled Hello World and that's now permanently in the history of the Internet forever.
We've been using containers for a long time now. Apologies to those who are big Solaris fans or BSD, obviously containers
and the technologies behind them existed in other operating systems before Docker. But essentially, at this point,
everything that's been demoed today has been on Linux. There was a great kind of intro in one of the earlier talks
about namespaces and C groups. This picture is old because people keep creating new namespaces. So it doesn't work anymore.
This was a cool image way back in the day because it was the perfect number to create like a flat packed box.
So you create your box out of the namespaces and then you shape your box with C groups. What size do you want it to be?
What limits do you want to place on that? And apologies to my friends at Microsoft again. There are containers on Windows
as well these days. But again, for the lion's share of use cases, these are the features and the technologies
we've been using to create containers. But let's not forget there's other pieces to the puzzle, whether you're using Docker
or some other runtime. There's SE Linux or App Armor in use. There's Setcom profiles. The images we've been constructing,
millions of them are constructed around Linux concepts, libraries, binaries that are basically Linux user space file systems.
And then there's the Linux capabilities that we add or remove or default in our container runtimes.
So again, all these things are very Linux specific. And yet, you know, where are developers developing these containers?
What tools are they using on what platforms? And I got a little nervous coming to FOSDOM because I thought, oh boy, everybody in this room,
there's Linux on the laptop actually is alive and well at FOSDOM, but not so much other places in the world as many of you know.
I spent way, way too long trying to create this slide because I kept trying to find better data on the split of who's using
what operating systems for developers. It's pretty easy to find that Windows is still very heavily used if you work for a large
company. They may hand you a laptop and enforce that you use a very specific image of Windows locked down in various ways.
Mac has been growing in popularity for a long time now. A lot of developers use Macs, myself included at the moment.
The problem is it's really hard to gauge how many people use Linux. If you look at the Stack Overflow developer surveys,
you get numbers as high as 30 or 40% in the past few years, but the way they're asking the questions, it's hard to know if people are
saying I'm developing in a Linux instance somewhere in the cloud or I'm actually running Linux on my laptop.
And since we're in Brussels, if you're at dinner somewhere, it turns out someone might overhear your conversation at dinner
and they're also at Fosdum, so someone point me at a new data source, JetBrains has a developer survey that they've been doing
for a number of years and they had slightly different numbers. They had 60% for Windows and Mac and Linux were actually almost
exactly the same at around 25 or 26% each. So regardless, we know that people are on various platforms and they're wanting
to develop Linux containers. The easy solution is, hey, we have tons of virtualization options. I don't know,
it looks like a lot of younger people here. When I was a developer and VMware came out, I was like, wow, this is magic.
I'm like able to run this other operating system on my laptop, Parallels is out there, KVM, VirtualBox, Vagrant,
all these options to be able to run a VM. And obviously that's one very simple solution to, I need to run Linux,
but my physical thing that I have that my manager gave me or that my work provides can't run Linux, so I'll just run a VM.
But this solution brings about some new problems because now I have another OS image to manage and it's got updates
and maybe security issues. And so now I'm managing my laptop or my desktop and also this other OS.
I have these VM boundary issues. So I'm on my host and I've checked out some source code, but it's not in my VM
and I got to figure out this file sharing and figure out how to do networking. I want to run a container in the VM
and I want to access it. And now I figure out how this works with the network. And there's also just my kind of developer workflow.
There's some inhibitors, the fact that this thing's in a VM and I have a tool I want to run, but it's only on my host.
And so again, this becomes potentially clunky to operate in these two worlds.
Way back in the early days of Docker, one of the solutions that someone came up with was Docker Machine.
It was this really nice simple way to sort of do this VM management on your behalf.
You export your Docker host variable and point to the right place.
And all of a sudden it seems like you're using Docker on the host and all the magic of the VM management is done for you.
It was fairly simplistic and so over the years pieces of that are what became Docker desktop.
This is a screen grab, I think, from one of the Windows versions.
But again, 2016, 2017 and beyond for Mac and Windows, a more complete solution was developed.
It also included a ton of other tools. So you didn't just have your runtime.
Runtime you had Docker compose, you had image signing from Notary.
You had a full Kubernetes cluster that you could access that was also being managed by this VM.
So again, there were people sort of trying to make this easier for the developer who wasn't on Linux
to develop their Linux containers that maybe they were going to deploy into a production environment
that ran Linux somewhere in the cloud or in a data center.
So this was great. It felt seamless to the developer.
It felt like I was running my container commands locally.
I'm doing Docker build, Docker run.
The file and networking, people smarter than me had figured out the magic of all this pass-through
that just seemed seamless and easy to use.
And now there was bundling of these other tools, you know, relevant things that I needed to use
were already there in the VM for me.
Meanwhile, everyone wasn't using Docker.
We had the ContainerD project, which I'm wearing my ContainerD t-shirt,
but also a sweater so you can't see it.
We have Podman. There's been some demos today that have used Podman.
Red Hat was building their own suite of tools with Creo and Podman.
And I don't know if we have any high-performance computing HPC folks in the room,
but Singularity was gaining popularity now known as AppTainer.
So again, there was these other technologies, other runtimes, other tools that people were using,
and maybe Docker Desktop was really not meaningful to that group of people.
And so over the years, other solutions for those other runtimes have been developed.
And so obviously Podman Desktop is one of those.
There was just a new release lately.
I think it's been around for about a year, although pieces of how to run Podman on your Mac
and Windows have existed maybe more than that, but the official Podman Desktop project
has been around for about a year.
You get Windows, Linux, and Mac OS support.
And it has Kubernetes. It has a plugin system.
They just recently developed a new DNS and networking service
that is a little more amenable to desktop, laptop environments
than using CNI plugins.
And again, it's built around tools that have been in development for many years.
Podman, Builda, Scopio, and the containers, libraries that these are built around.
And again, because those things were built with certain features,
like the rootless and unprivileged work, the demolus runtime with Podman and CRUN,
you get all those same features, but now you can run it on your Mac or your Windows system
if that's your local developer environment, and you get all the same capabilities
if you were using Podman on a Linux system.
So you get both all the Docker command line compatibility that Podman originally developed,
but with Libpod, you also get the Docker API, which may be important for tools you're using
that try and integrate directly with the Docker API.
If you're not in that world, there's Lima, NerdCTL, and ContainerD as sort of a stack of projects.
NerdCTL, similar to Podman, provides you that same Docker command line API with composed support.
It uses QMU for virtualization, so this is the Lima component.
That handles, again, the file sharing, the network pass-through via some additional projects
that are part of that Lima scope.
Again, this is all focused on macOS so far today.
I think there's some discussions around Windows support and AkaHero is here
and may be able to speak more to that than I can.
One of the benefits of being built around ContainerD is that this stack can also expose
experimental features like lazy loading snapshotters, image encryption,
and other sort of sub-projects of ContainerD that are out there today.
NerdCTL, as it's packaged by default, gives you rootless unprivileged mode,
so if you run it through Lima, you're getting, again, rootless unprivileged containers
running underneath that on Mac.
A few projects that are built on top of that are Rancher Desktop and CoLima.
Rancher Desktop, obviously many of you have heard of the Rancher suite of projects and products.
They created a desktop platform that built on the Lima foundation for their macOS support.
Both of these projects also found that some of their user base either needed the Docker API
or had very specific ties to Docker.
So you can get both of these projects, not just with ContainerD and NerdCTL,
but also get the Docker engine.
In fact, CoLima, if you install it by default, does install Docker.
They both provide Kubernetes clusters.
So again, if local development environments and Kubernetes, that combination is important to you.
They both provide that.
Rancher Desktop also adds Windows and Linux support in addition,
and that's not using Lima underneath.
So the last project I wanted to talk about came out of my team, AWS.
This is a project we just launched in November last year.
So just a few months ago, it's called Finch,
and it builds on the same stack as Rancher Desktop and CoLima,
where we're using Lima, NerdCTL, and BuildKit to provide that Docker command line,
Docker build support, Docker compose support inside of VM on your Mac.
And so there's Homebrew Packaging and Apple Sign Installer packages for that.
It supports ARM64 and Intel.
And also because of QMU and its capabilities, you can build containers.
No matter what your host CPU is, you can build containers for Intel or ARM64.
And again, the host CPU itself can be any of the either Apple Silicon, M1, M2,
or the Intel-based Mac.
So again, we're a young project.
Our plans for an extension framework similar to Podman Desktop and Docker Desktop,
so that we want that same model of you can add features and add capabilities
without having to add them to the Finch project itself
to extend it to other use cases.
And we're also planning similar to Rancher Desktop for adding Windows and Linux support.
Obviously, we're not really building a completely new tool.
We're packaging most of these existing components.
So we're working upstream.
There's myself and a few other container demaintainers.
We're working in Lima.
We have a few pull requests merged in Lima.
We had in the latest Nerd CTL release a few weeks ago,
we had five different Amazon folks mentioned in the release notes.
We're planning to add some features to BuildKit.
And we also have several people working in the OCI specs,
like the recent reference type work.
So again, a lot of the work we do in Finch is really building out capabilities
in these underlying projects,
not so much building a brand new interface on top.
And we want it to be a community open source project.
So we're working on a public roadmap.
Obviously, there's a GitHub repository here where you can go
and see what we're doing, open to external contribution.
And what we'd really love collaboration on is this added operating system support.
Again, some of that work might be in Lima or elsewhere.
But we'd love to add Windows and Linux support.
And then understanding the best way to design this extension system
that you can already use with other tools that I mentioned.
We're also on the CNCF Slack in the channel Finch.
So with that, that was a whirlwind tour
through what's available for desktop tooling today with containers.
And I think we have a few minutes for questions.
APPLAUSE
Yeah, any questions?
Hi.
What was the motivation to create Finch when there was already this whole ecosystem?
If I think I understood the question, why create Finch
when there was Rancher Desktop or Colema or Lima?
Yeah, that's a good question.
So each of those tools kind of has its own natural inclination.
With Rancher Desktop, the focus was great local Kubernetes environment
and a GUI and some management around it and including Docker.
We wanted something simpler that's just the command line tool.
And so we talked to the Rancher folks about maybe having a common upstream.
Maybe Finch becomes that common upstream of Lima,
container D, nerd CTL, build kit.
So that might still be in the works.
And then Colema is a very small project.
There's one maintainer.
He's kind of working on his own.
And again, we were looking at, you know, he's got Docker in there.
He's got Kubernetes.
And we wanted to, again, focus just on the container interloop lifecycle,
build containers, run containers, push containers to registries.
And so essentially it's just a simplification that we think
there's still lots of ability for collaboration with those other projects
because we're all using the same stack below us.
We have time for one more fairly quick question.
How easy it is to pick up Finch for someone who's just started working as a developer?
Yeah, how easy to use?
What's the learning curve compared to Docker?
Yeah, so again, most of these tools are built around the sort of
understood Docker command line tool.
So if you've already used Docker, like it's the same commands, the same flags.
So in that sense, there's no real learning curve.
Now, if you're just brand new to containers, it's really the same effort
that you'd have to do to learn Docker or Podman or Finch or anything else.
So it's really about your understanding of kind of the existing developer
tooling space built around Docker.
Okay, thank you.
Please leave quietly when we are still asking questions.
Other than that, thank you.
Thank you.