[00:00.000 --> 00:10.000] our next talk. I guess I don't have to give a big introduction. A lot of you know Phil, [00:10.000 --> 00:16.000] right? He's going to talk about Turing the container developer tooling landscape. [00:16.000 --> 00:33.000] All right. Hi everybody. I think I'm on. Yeah, you're on. Yeah, so thanks for coming. [00:33.000 --> 00:40.000] It seems like Fossum is back. We've got a packed containers room. I think my talk is mostly a warm-up [00:40.000 --> 00:50.000] for Akahiro after me. So a lot of familiar faces here. A lot of good talk so far. Maybe this will be a little bit different [00:50.000 --> 00:58.000] than the last few talks. We've been talking a lot about containers and various environments, but we haven't really talked [00:58.000 --> 01:06.000] about tools. You've seen a few tools used in some of the demos. And so I'm just going to talk through where we are these days [01:06.000 --> 01:16.000] with development tooling. Interesting that it's 2023 and interesting year in that we're now 10 years since Solomon [01:16.000 --> 01:23.000] Hikes gave this demo during a lightning talk at PyCon. I think it was like April. So getting pretty close to 10 years [01:23.000 --> 01:31.000] since someone saw Docker being run at the command line for the first time. And what an interesting demo was [01:31.000 --> 01:39.000] because he misspelled Hello World and that's now permanently in the history of the Internet forever. [01:39.000 --> 01:48.000] We've been using containers for a long time now. Apologies to those who are big Solaris fans or BSD, obviously containers [01:48.000 --> 01:58.000] and the technologies behind them existed in other operating systems before Docker. But essentially, at this point, [01:58.000 --> 02:06.000] everything that's been demoed today has been on Linux. There was a great kind of intro in one of the earlier talks [02:06.000 --> 02:15.000] about namespaces and C groups. This picture is old because people keep creating new namespaces. So it doesn't work anymore. [02:15.000 --> 02:21.000] This was a cool image way back in the day because it was the perfect number to create like a flat packed box. [02:21.000 --> 02:28.000] So you create your box out of the namespaces and then you shape your box with C groups. What size do you want it to be? [02:28.000 --> 02:36.000] What limits do you want to place on that? And apologies to my friends at Microsoft again. There are containers on Windows [02:36.000 --> 02:44.000] as well these days. But again, for the lion's share of use cases, these are the features and the technologies [02:44.000 --> 02:51.000] we've been using to create containers. But let's not forget there's other pieces to the puzzle, whether you're using Docker [02:51.000 --> 02:59.000] or some other runtime. There's SE Linux or App Armor in use. There's Setcom profiles. The images we've been constructing, [02:59.000 --> 03:10.000] millions of them are constructed around Linux concepts, libraries, binaries that are basically Linux user space file systems. [03:10.000 --> 03:16.000] And then there's the Linux capabilities that we add or remove or default in our container runtimes. [03:16.000 --> 03:24.000] So again, all these things are very Linux specific. And yet, you know, where are developers developing these containers? [03:24.000 --> 03:33.000] What tools are they using on what platforms? And I got a little nervous coming to FOSDOM because I thought, oh boy, everybody in this room, [03:33.000 --> 03:42.000] there's Linux on the laptop actually is alive and well at FOSDOM, but not so much other places in the world as many of you know. [03:42.000 --> 03:51.000] I spent way, way too long trying to create this slide because I kept trying to find better data on the split of who's using [03:51.000 --> 04:00.000] what operating systems for developers. It's pretty easy to find that Windows is still very heavily used if you work for a large [04:00.000 --> 04:09.000] company. They may hand you a laptop and enforce that you use a very specific image of Windows locked down in various ways. [04:09.000 --> 04:17.000] Mac has been growing in popularity for a long time now. A lot of developers use Macs, myself included at the moment. [04:17.000 --> 04:25.000] The problem is it's really hard to gauge how many people use Linux. If you look at the Stack Overflow developer surveys, [04:25.000 --> 04:32.000] you get numbers as high as 30 or 40% in the past few years, but the way they're asking the questions, it's hard to know if people are [04:32.000 --> 04:40.000] saying I'm developing in a Linux instance somewhere in the cloud or I'm actually running Linux on my laptop. [04:40.000 --> 04:46.000] And since we're in Brussels, if you're at dinner somewhere, it turns out someone might overhear your conversation at dinner [04:46.000 --> 04:55.000] and they're also at Fosdum, so someone point me at a new data source, JetBrains has a developer survey that they've been doing [04:55.000 --> 05:04.000] for a number of years and they had slightly different numbers. They had 60% for Windows and Mac and Linux were actually almost [05:04.000 --> 05:15.000] exactly the same at around 25 or 26% each. So regardless, we know that people are on various platforms and they're wanting [05:15.000 --> 05:25.000] to develop Linux containers. The easy solution is, hey, we have tons of virtualization options. I don't know, [05:25.000 --> 05:32.000] it looks like a lot of younger people here. When I was a developer and VMware came out, I was like, wow, this is magic. [05:32.000 --> 05:40.000] I'm like able to run this other operating system on my laptop, Parallels is out there, KVM, VirtualBox, Vagrant, [05:40.000 --> 05:48.000] all these options to be able to run a VM. And obviously that's one very simple solution to, I need to run Linux, [05:48.000 --> 05:59.000] but my physical thing that I have that my manager gave me or that my work provides can't run Linux, so I'll just run a VM. [05:59.000 --> 06:07.000] But this solution brings about some new problems because now I have another OS image to manage and it's got updates [06:07.000 --> 06:15.000] and maybe security issues. And so now I'm managing my laptop or my desktop and also this other OS. [06:15.000 --> 06:23.000] I have these VM boundary issues. So I'm on my host and I've checked out some source code, but it's not in my VM [06:23.000 --> 06:29.000] and I got to figure out this file sharing and figure out how to do networking. I want to run a container in the VM [06:29.000 --> 06:36.000] and I want to access it. And now I figure out how this works with the network. And there's also just my kind of developer workflow. [06:36.000 --> 06:43.000] There's some inhibitors, the fact that this thing's in a VM and I have a tool I want to run, but it's only on my host. [06:43.000 --> 06:49.000] And so again, this becomes potentially clunky to operate in these two worlds. [06:49.000 --> 06:57.000] Way back in the early days of Docker, one of the solutions that someone came up with was Docker Machine. [06:57.000 --> 07:02.000] It was this really nice simple way to sort of do this VM management on your behalf. [07:02.000 --> 07:06.000] You export your Docker host variable and point to the right place. [07:06.000 --> 07:14.000] And all of a sudden it seems like you're using Docker on the host and all the magic of the VM management is done for you. [07:14.000 --> 07:21.000] It was fairly simplistic and so over the years pieces of that are what became Docker desktop. [07:21.000 --> 07:26.000] This is a screen grab, I think, from one of the Windows versions. [07:26.000 --> 07:35.000] But again, 2016, 2017 and beyond for Mac and Windows, a more complete solution was developed. [07:35.000 --> 07:39.000] It also included a ton of other tools. So you didn't just have your runtime. [07:39.000 --> 07:44.000] Runtime you had Docker compose, you had image signing from Notary. [07:44.000 --> 07:50.000] You had a full Kubernetes cluster that you could access that was also being managed by this VM. [07:50.000 --> 07:56.000] So again, there were people sort of trying to make this easier for the developer who wasn't on Linux [07:56.000 --> 08:02.000] to develop their Linux containers that maybe they were going to deploy into a production environment [08:02.000 --> 08:06.000] that ran Linux somewhere in the cloud or in a data center. [08:06.000 --> 08:10.000] So this was great. It felt seamless to the developer. [08:10.000 --> 08:13.000] It felt like I was running my container commands locally. [08:13.000 --> 08:16.000] I'm doing Docker build, Docker run. [08:16.000 --> 08:21.000] The file and networking, people smarter than me had figured out the magic of all this pass-through [08:21.000 --> 08:25.000] that just seemed seamless and easy to use. [08:25.000 --> 08:30.000] And now there was bundling of these other tools, you know, relevant things that I needed to use [08:30.000 --> 08:34.000] were already there in the VM for me. [08:34.000 --> 08:38.000] Meanwhile, everyone wasn't using Docker. [08:38.000 --> 08:43.000] We had the ContainerD project, which I'm wearing my ContainerD t-shirt, [08:43.000 --> 08:46.000] but also a sweater so you can't see it. [08:46.000 --> 08:50.000] We have Podman. There's been some demos today that have used Podman. [08:50.000 --> 08:55.000] Red Hat was building their own suite of tools with Creo and Podman. [08:55.000 --> 08:59.000] And I don't know if we have any high-performance computing HPC folks in the room, [08:59.000 --> 09:04.000] but Singularity was gaining popularity now known as AppTainer. [09:04.000 --> 09:10.000] So again, there was these other technologies, other runtimes, other tools that people were using, [09:10.000 --> 09:16.000] and maybe Docker Desktop was really not meaningful to that group of people. [09:16.000 --> 09:24.000] And so over the years, other solutions for those other runtimes have been developed. [09:24.000 --> 09:28.000] And so obviously Podman Desktop is one of those. [09:28.000 --> 09:30.000] There was just a new release lately. [09:30.000 --> 09:36.000] I think it's been around for about a year, although pieces of how to run Podman on your Mac [09:36.000 --> 09:43.000] and Windows have existed maybe more than that, but the official Podman Desktop project [09:43.000 --> 09:45.000] has been around for about a year. [09:45.000 --> 09:49.000] You get Windows, Linux, and Mac OS support. [09:49.000 --> 09:55.000] And it has Kubernetes. It has a plugin system. [09:55.000 --> 09:59.000] They just recently developed a new DNS and networking service [09:59.000 --> 10:03.000] that is a little more amenable to desktop, laptop environments [10:03.000 --> 10:06.000] than using CNI plugins. [10:06.000 --> 10:10.000] And again, it's built around tools that have been in development for many years. [10:10.000 --> 10:17.000] Podman, Builda, Scopio, and the containers, libraries that these are built around. [10:17.000 --> 10:20.000] And again, because those things were built with certain features, [10:20.000 --> 10:27.000] like the rootless and unprivileged work, the demolus runtime with Podman and CRUN, [10:27.000 --> 10:32.000] you get all those same features, but now you can run it on your Mac or your Windows system [10:32.000 --> 10:36.000] if that's your local developer environment, and you get all the same capabilities [10:36.000 --> 10:40.000] if you were using Podman on a Linux system. [10:40.000 --> 10:46.000] So you get both all the Docker command line compatibility that Podman originally developed, [10:46.000 --> 10:51.000] but with Libpod, you also get the Docker API, which may be important for tools you're using [10:51.000 --> 10:56.000] that try and integrate directly with the Docker API. [10:56.000 --> 11:05.000] If you're not in that world, there's Lima, NerdCTL, and ContainerD as sort of a stack of projects. [11:05.000 --> 11:14.000] NerdCTL, similar to Podman, provides you that same Docker command line API with composed support. [11:14.000 --> 11:20.000] It uses QMU for virtualization, so this is the Lima component. [11:20.000 --> 11:26.000] That handles, again, the file sharing, the network pass-through via some additional projects [11:26.000 --> 11:29.000] that are part of that Lima scope. [11:29.000 --> 11:33.000] Again, this is all focused on macOS so far today. [11:33.000 --> 11:37.000] I think there's some discussions around Windows support and AkaHero is here [11:37.000 --> 11:41.000] and may be able to speak more to that than I can. [11:41.000 --> 11:46.000] One of the benefits of being built around ContainerD is that this stack can also expose [11:46.000 --> 11:51.000] experimental features like lazy loading snapshotters, image encryption, [11:51.000 --> 11:56.000] and other sort of sub-projects of ContainerD that are out there today. [11:56.000 --> 12:02.000] NerdCTL, as it's packaged by default, gives you rootless unprivileged mode, [12:02.000 --> 12:07.000] so if you run it through Lima, you're getting, again, rootless unprivileged containers [12:07.000 --> 12:10.000] running underneath that on Mac. [12:10.000 --> 12:16.000] A few projects that are built on top of that are Rancher Desktop and CoLima. [12:16.000 --> 12:23.000] Rancher Desktop, obviously many of you have heard of the Rancher suite of projects and products. [12:23.000 --> 12:32.000] They created a desktop platform that built on the Lima foundation for their macOS support. [12:32.000 --> 12:39.000] Both of these projects also found that some of their user base either needed the Docker API [12:39.000 --> 12:41.000] or had very specific ties to Docker. [12:41.000 --> 12:46.000] So you can get both of these projects, not just with ContainerD and NerdCTL, [12:46.000 --> 12:48.000] but also get the Docker engine. [12:48.000 --> 12:53.000] In fact, CoLima, if you install it by default, does install Docker. [12:53.000 --> 12:55.000] They both provide Kubernetes clusters. [12:55.000 --> 13:01.000] So again, if local development environments and Kubernetes, that combination is important to you. [13:01.000 --> 13:03.000] They both provide that. [13:03.000 --> 13:08.000] Rancher Desktop also adds Windows and Linux support in addition, [13:08.000 --> 13:12.000] and that's not using Lima underneath. [13:12.000 --> 13:18.000] So the last project I wanted to talk about came out of my team, AWS. [13:18.000 --> 13:22.000] This is a project we just launched in November last year. [13:22.000 --> 13:25.000] So just a few months ago, it's called Finch, [13:25.000 --> 13:30.000] and it builds on the same stack as Rancher Desktop and CoLima, [13:30.000 --> 13:36.000] where we're using Lima, NerdCTL, and BuildKit to provide that Docker command line, [13:36.000 --> 13:42.000] Docker build support, Docker compose support inside of VM on your Mac. [13:42.000 --> 13:47.000] And so there's Homebrew Packaging and Apple Sign Installer packages for that. [13:47.000 --> 13:50.000] It supports ARM64 and Intel. [13:50.000 --> 13:56.000] And also because of QMU and its capabilities, you can build containers. [13:56.000 --> 14:04.000] No matter what your host CPU is, you can build containers for Intel or ARM64. [14:04.000 --> 14:11.000] And again, the host CPU itself can be any of the either Apple Silicon, M1, M2, [14:11.000 --> 14:13.000] or the Intel-based Mac. [14:13.000 --> 14:15.000] So again, we're a young project. [14:15.000 --> 14:22.000] Our plans for an extension framework similar to Podman Desktop and Docker Desktop, [14:22.000 --> 14:28.000] so that we want that same model of you can add features and add capabilities [14:28.000 --> 14:31.000] without having to add them to the Finch project itself [14:31.000 --> 14:34.000] to extend it to other use cases. [14:34.000 --> 14:42.000] And we're also planning similar to Rancher Desktop for adding Windows and Linux support. [14:42.000 --> 14:46.000] Obviously, we're not really building a completely new tool. [14:46.000 --> 14:49.000] We're packaging most of these existing components. [14:49.000 --> 14:51.000] So we're working upstream. [14:51.000 --> 14:55.000] There's myself and a few other container demaintainers. [14:55.000 --> 14:56.000] We're working in Lima. [14:56.000 --> 14:59.000] We have a few pull requests merged in Lima. [14:59.000 --> 15:03.000] We had in the latest Nerd CTL release a few weeks ago, [15:03.000 --> 15:09.000] we had five different Amazon folks mentioned in the release notes. [15:09.000 --> 15:12.000] We're planning to add some features to BuildKit. [15:12.000 --> 15:15.000] And we also have several people working in the OCI specs, [15:15.000 --> 15:18.000] like the recent reference type work. [15:18.000 --> 15:22.000] So again, a lot of the work we do in Finch is really building out capabilities [15:22.000 --> 15:24.000] in these underlying projects, [15:24.000 --> 15:29.000] not so much building a brand new interface on top. [15:29.000 --> 15:32.000] And we want it to be a community open source project. [15:32.000 --> 15:35.000] So we're working on a public roadmap. [15:35.000 --> 15:40.000] Obviously, there's a GitHub repository here where you can go [15:40.000 --> 15:44.000] and see what we're doing, open to external contribution. [15:44.000 --> 15:50.000] And what we'd really love collaboration on is this added operating system support. [15:50.000 --> 15:53.000] Again, some of that work might be in Lima or elsewhere. [15:53.000 --> 15:57.000] But we'd love to add Windows and Linux support. [15:57.000 --> 16:01.000] And then understanding the best way to design this extension system [16:01.000 --> 16:05.000] that you can already use with other tools that I mentioned. [16:05.000 --> 16:09.000] We're also on the CNCF Slack in the channel Finch. [16:09.000 --> 16:13.000] So with that, that was a whirlwind tour [16:13.000 --> 16:17.000] through what's available for desktop tooling today with containers. [16:17.000 --> 16:20.000] And I think we have a few minutes for questions. [16:20.000 --> 16:29.000] APPLAUSE [16:29.000 --> 16:31.000] Yeah, any questions? [16:39.000 --> 16:40.000] Hi. [16:40.000 --> 16:47.000] What was the motivation to create Finch when there was already this whole ecosystem? [16:47.000 --> 16:50.000] If I think I understood the question, why create Finch [16:50.000 --> 16:53.000] when there was Rancher Desktop or Colema or Lima? [16:53.000 --> 16:55.000] Yeah, that's a good question. [16:55.000 --> 16:59.000] So each of those tools kind of has its own natural inclination. [16:59.000 --> 17:04.000] With Rancher Desktop, the focus was great local Kubernetes environment [17:04.000 --> 17:08.000] and a GUI and some management around it and including Docker. [17:08.000 --> 17:12.000] We wanted something simpler that's just the command line tool. [17:12.000 --> 17:18.000] And so we talked to the Rancher folks about maybe having a common upstream. [17:18.000 --> 17:22.000] Maybe Finch becomes that common upstream of Lima, [17:22.000 --> 17:25.000] container D, nerd CTL, build kit. [17:25.000 --> 17:29.000] So that might still be in the works. [17:29.000 --> 17:34.000] And then Colema is a very small project. [17:34.000 --> 17:37.000] There's one maintainer. [17:37.000 --> 17:39.000] He's kind of working on his own. [17:39.000 --> 17:43.000] And again, we were looking at, you know, he's got Docker in there. [17:43.000 --> 17:46.000] He's got Kubernetes. [17:46.000 --> 17:51.000] And we wanted to, again, focus just on the container interloop lifecycle, [17:51.000 --> 17:55.000] build containers, run containers, push containers to registries. [17:55.000 --> 18:00.000] And so essentially it's just a simplification that we think [18:00.000 --> 18:04.000] there's still lots of ability for collaboration with those other projects [18:04.000 --> 18:11.000] because we're all using the same stack below us. [18:11.000 --> 18:28.000] We have time for one more fairly quick question. [18:28.000 --> 18:36.000] How easy it is to pick up Finch for someone who's just started working as a developer? [18:36.000 --> 18:38.000] Yeah, how easy to use? [18:38.000 --> 18:41.000] What's the learning curve compared to Docker? [18:41.000 --> 18:46.000] Yeah, so again, most of these tools are built around the sort of [18:46.000 --> 18:48.000] understood Docker command line tool. [18:48.000 --> 18:54.000] So if you've already used Docker, like it's the same commands, the same flags. [18:54.000 --> 18:56.000] So in that sense, there's no real learning curve. [18:56.000 --> 19:00.000] Now, if you're just brand new to containers, it's really the same effort [19:00.000 --> 19:06.000] that you'd have to do to learn Docker or Podman or Finch or anything else. [19:06.000 --> 19:11.000] So it's really about your understanding of kind of the existing developer [19:11.000 --> 19:15.000] tooling space built around Docker. [19:15.000 --> 19:16.000] Okay, thank you. [19:16.000 --> 19:20.000] Please leave quietly when we are still asking questions. [19:20.000 --> 19:22.000] Other than that, thank you. [19:22.000 --> 19:27.000] Thank you.