[00:00.000 --> 00:07.600]  And our next talk is by Sean, and he's going to talk about
[00:07.600 --> 00:08.600]  Bottle Rocket.
[00:08.600 --> 00:09.600]  Thanks.
[00:09.600 --> 00:10.600]  Thank you.
[00:10.600 --> 00:19.480]  Yeah, so I'm going to talk about the Bottle Rocket Container
[00:19.480 --> 00:22.400]  Optimized Linux Distribution.
[00:22.400 --> 00:23.720]  My name is Sean McGinnis.
[00:23.720 --> 00:27.760]  I work, I'm an engineer on the Bottle Rocket project, and I work
[00:27.760 --> 00:31.320]  at AWS.
[00:31.320 --> 00:35.320]  So I'll just go over what a container optimized Linux is.
[00:35.320 --> 00:39.400]  I'm assuming most people in this track probably have an idea.
[00:39.400 --> 00:42.040]  But go over the basics, talk a little bit about what Bottle
[00:42.040 --> 00:45.680]  Rocket is, show a little bit so you get a feel for it.
[00:45.680 --> 00:51.200]  And what I'd really like is to see others get involved.
[00:51.200 --> 00:56.840]  So the mission statement, I guess, is that it's a free and
[00:56.840 --> 00:59.600]  open-source Linux-based operating system for hosting
[00:59.600 --> 01:00.600]  containers.
[01:00.600 --> 01:02.240]  So what does that mean?
[01:06.640 --> 01:11.080]  Before we get into the motivations behind that, it's
[01:11.080 --> 01:14.520]  interesting to look at general purpose Linux distributions.
[01:14.520 --> 01:21.280]  And some of the challenges with using those, when you have
[01:21.280 --> 01:24.920]  hundreds of nodes in a Kubernetes cluster, you really have
[01:24.920 --> 01:27.240]  a lot of workloads that you're running, containerized
[01:27.240 --> 01:30.000]  workloads that you're running, and you need to optimize how
[01:30.000 --> 01:32.080]  you're using resources.
[01:32.080 --> 01:38.760]  So with most general purpose distros, the configuration is
[01:38.760 --> 01:39.160]  immutable.
[01:39.160 --> 01:42.520]  You can log into the machine, you can make changes, make
[01:42.520 --> 01:47.600]  adjustments, add, install extra services.
[01:47.600 --> 01:51.520]  Out of the box, a lot of them come with a default baseline
[01:51.520 --> 01:54.120]  set of services that you might not necessarily need when
[01:54.120 --> 01:56.520]  you're just trying to run containers.
[01:56.520 --> 02:00.800]  And that uses more system resources and then also
[02:00.800 --> 02:03.720]  creates more of a security risk of there's more attack
[02:03.720 --> 02:06.080]  service available.
[02:06.080 --> 02:08.360]  And because of that, because you can log in, you can change
[02:08.360 --> 02:13.160]  things, you can tweak configuration settings, those
[02:13.160 --> 02:18.960]  kinds of systems are easier to become pets, where you've
[02:18.960 --> 02:22.240]  really customized that node exactly how you want it.
[02:22.240 --> 02:26.560]  And you're less likely to just blow that away and spin up a
[02:26.560 --> 02:29.600]  new one, especially when, OK, I've made some changes there.
[02:29.600 --> 02:31.880]  I don't quite remember what I changed, because I was
[02:31.880 --> 02:32.880]  troubleshooting something.
[02:32.880 --> 02:34.400]  There might be something important there.
[02:34.400 --> 02:38.160]  So now I need to take care of this node, and it
[02:38.160 --> 02:41.440]  becomes my pet.
[02:41.440 --> 02:47.800]  So for container optimized Linux distributions, especially
[02:47.800 --> 02:54.040]  for BioRocket, really try to optimize for just the services
[02:54.040 --> 02:58.960]  that you need running on your Linux machine to be able to
[02:58.960 --> 03:00.880]  run your containers.
[03:00.880 --> 03:04.040]  That means less resource usage for things that aren't very
[03:04.040 --> 03:05.080]  important to you.
[03:05.080 --> 03:10.000]  It means less attack area for someone to compromise that
[03:10.000 --> 03:15.080]  machine, get all kinds of added benefits, faster boot time,
[03:15.080 --> 03:18.320]  smaller image sizes to transfer around.
[03:18.320 --> 03:25.960]  So with BioRocket, we try to make things as small as
[03:25.960 --> 03:31.960]  possible, just what you need, and try to make it more secure
[03:31.960 --> 03:32.840]  by default.
[03:32.840 --> 03:36.080]  And I don't say secure by default, because that's
[03:36.080 --> 03:38.080]  impossible, try to make it more secure.
[03:38.080 --> 03:43.400]  So really locking things down, making sure that if someone
[03:43.400 --> 03:47.080]  were to try to compromise your host, they're going to have a
[03:47.080 --> 03:49.080]  hard time doing it.
[03:49.080 --> 03:52.280]  And it's open source.
[03:52.280 --> 03:54.280]  It's BioRocket is not a general purpose
[03:54.280 --> 03:55.440]  operating system.
[03:55.440 --> 03:59.680]  If you're looking to do other things besides container
[03:59.680 --> 04:02.800]  workloads, BioRocket's probably not the right
[04:02.800 --> 04:05.400]  distro for you.
[04:05.400 --> 04:10.800]  It is backed by AWS, but it is not an AWS-only solution.
[04:10.800 --> 04:18.240]  Coming from AWS, it is very well integrated with AWS, but
[04:18.240 --> 04:25.520]  I hope that doesn't stay the primary case for long.
[04:25.520 --> 04:27.760]  And it is not a container-based OS.
[04:27.760 --> 04:33.000]  So what I mean by that, and this comes up a lot in
[04:33.000 --> 04:36.040]  conversations where there's this confusion where you talk
[04:36.040 --> 04:38.920]  about a container distro.
[04:38.920 --> 04:41.840]  And different people kind of already have a preconception
[04:41.840 --> 04:44.520]  of what that term means.
[04:44.520 --> 04:51.840]  So the two paradigms that come up are the distro, that's
[04:51.840 --> 04:53.920]  kind of the base image that you build on top of.
[04:53.920 --> 04:59.280]  So you've got a file from BioRocket versus your OS.
[04:59.280 --> 05:03.120]  You're running, and then on that you spin up containers.
[05:03.120 --> 05:05.600]  And it really is.
[05:05.600 --> 05:07.960]  When we talk about BioRocket, being a container
[05:07.960 --> 05:11.600]  optimized Linux, it's that second one.
[05:11.600 --> 05:16.240]  BioRocket is not something that you would use to create a
[05:16.240 --> 05:17.560]  container image.
[05:22.920 --> 05:26.600]  A little background of BioRocket, if you see the date
[05:26.600 --> 05:33.800]  in the bottom left there, we launched March 2020, which
[05:33.800 --> 05:35.760]  there are a few other things going on around that time.
[05:35.760 --> 05:41.520]  So didn't quite make the big splash on the launch there
[05:41.520 --> 05:43.480]  that we had hoped for.
[05:43.480 --> 05:47.520]  But it's great now to actually be back in person in front of
[05:47.520 --> 05:50.280]  people, being able to talk about the work that we've done
[05:50.280 --> 05:56.520]  in the last two years, or three years, and hopefully get
[05:56.520 --> 05:59.520]  the awareness out there a little more.
[05:59.520 --> 06:05.680]  BioRocket, right now, we build and distribute
[06:05.680 --> 06:07.480]  different variants.
[06:07.480 --> 06:15.640]  The variants term for us is how we try to optimize things
[06:15.640 --> 06:19.440]  for your specific scenario.
[06:19.440 --> 06:25.280]  So if you're running Kubernetes 1.22, there is a
[06:25.280 --> 06:30.680]  variant specifically for Kubernetes 1.22, and Amazon
[06:30.680 --> 06:33.200]  ECS, and VMware.
[06:33.200 --> 06:36.960]  The reason to have these different variants is back
[06:36.960 --> 06:40.560]  to that idea of no extra overhead.
[06:40.560 --> 06:48.200]  So for these, for the metal variant, we try to limit the
[06:48.200 --> 06:50.720]  number of kernel drivers that are loaded.
[06:50.720 --> 06:53.920]  The kernel drivers that you need between a metal
[06:53.920 --> 06:56.400]  deployment, where you're running on an actual server
[06:56.400 --> 06:59.240]  hardware, versus where you're running in a virtual machine
[06:59.240 --> 07:01.600]  instance, are different.
[07:01.600 --> 07:03.160]  We would know if you're running in a virtual machine,
[07:03.160 --> 07:05.840]  there's a very small subset of the available drivers that
[07:05.840 --> 07:07.120]  you need to actually do that.
[07:07.120 --> 07:10.320]  So anything extra is taken out of there.
[07:10.320 --> 07:14.240]  Any specific agents that are needed to integrate well, like
[07:14.240 --> 07:17.640]  in VMware, we have those baked into that variant.
[07:17.640 --> 07:21.000]  So you pick the variants that you want for your scenario,
[07:21.000 --> 07:26.440]  and that gives you a well-integrated option.
[07:26.440 --> 07:34.600]  Now, Bottle Rocket isn't far from the leaders in container
[07:34.600 --> 07:38.320]  optimized distributions.
[07:38.320 --> 07:42.720]  CoreOS is one that really popularizes the whole idea.
[07:42.720 --> 07:46.320]  FlatCars is very popular.
[07:46.320 --> 07:49.640]  I just wanted to acknowledge that, make sure that
[07:49.640 --> 07:53.440]  everyone's aware, there are other options out there.
[07:53.440 --> 07:57.640]  None of the things here are meant to say one is better
[07:57.640 --> 07:58.240]  than the other.
[07:58.240 --> 08:01.480]  They all approach things from slightly different angles,
[08:01.480 --> 08:04.320]  just because there's maybe a smaller list for one.
[08:04.320 --> 08:09.240]  If you're using that platform, then you're well-integrated.
[08:09.240 --> 08:11.040]  Maybe that is the best option for you.
[08:11.040 --> 08:14.480]  But Bottle Rocket is trying to address all of these similar
[08:14.480 --> 08:17.400]  problem spaces that the other distributions are doing.
[08:17.400 --> 08:19.560]  We all come at it a little different way, just
[08:19.560 --> 08:24.120]  like how you have Ubuntu and Red Hat and many, many other
[08:24.120 --> 08:27.480]  general-purpose distributions.
[08:27.480 --> 08:36.400]  So to dive into Bottle Rocket a little bit, there really
[08:36.400 --> 08:38.760]  isn't too much more than these blocks.
[08:38.760 --> 08:40.760]  We have the base Linux kernel.
[08:40.760 --> 08:45.040]  System.bedu is used to manage things.
[08:45.040 --> 08:49.280]  And we actually run two different container-y instances.
[08:49.280 --> 08:52.920]  And the reason for this is, again, security.
[08:52.920 --> 08:58.360]  Everything on the left-hand side, host containers, are
[08:58.360 --> 09:02.240]  things that are used to manage the node.
[09:02.240 --> 09:05.200]  They're things that have a little more privilege, that
[09:05.200 --> 09:08.040]  might be able to access things that, regular pods that are
[09:08.040 --> 09:10.320]  running on the container-y instance that's used, say,
[09:10.320 --> 09:15.160]  with Kubernetes, that would be a little more locked down.
[09:15.160 --> 09:18.320]  And then if there is any security vulnerability that
[09:18.320 --> 09:19.720]  helps isolate things.
[09:22.400 --> 09:25.440]  It's an API-driven configuration.
[09:25.440 --> 09:29.640]  So when you deploy an instance, you can give a
[09:29.640 --> 09:34.440]  configuration that actually, even though it ends up being a
[09:34.440 --> 09:37.680]  file with settings, everything goes through the API.
[09:37.680 --> 09:42.640]  And that's what actually sets the values for what happens
[09:42.640 --> 09:44.960]  when this instance boots up and runs.
[09:44.960 --> 09:53.920]  The host containers, so again, the things with a little
[09:53.920 --> 10:00.560]  more privilege, host container is really where, how you
[10:00.560 --> 10:02.920]  would access the machine.
[10:02.920 --> 10:07.720]  So the actual ball rocket base itself, if you look at this
[10:07.720 --> 10:11.400]  actual Linux kernel running, all the systems on there,
[10:11.400 --> 10:13.200]  there's no shell.
[10:13.200 --> 10:18.560]  There's no SSHD.
[10:18.560 --> 10:20.680]  It really is isolated.
[10:20.680 --> 10:25.040]  You need to physically connect some way to this instance.
[10:25.040 --> 10:26.960]  So how do you actually do things?
[10:26.960 --> 10:29.520]  And that's where the control container comes in.
[10:29.520 --> 10:34.960]  That provides an environment that you can connect to and
[10:34.960 --> 10:37.680]  has a few of the tools that you may need if you need to
[10:37.680 --> 10:43.760]  actually interact beyond the API configuration.
[10:43.760 --> 10:47.920]  One step above that, then, is from that control container,
[10:47.920 --> 10:49.680]  you can launch an admin container.
[10:49.680 --> 10:55.440]  And that actually does then give you the option to run SSHD.
[10:55.440 --> 10:59.840]  It lets you access more of the system settings.
[10:59.840 --> 11:02.360]  But it's something that's not run by default, and it's not
[11:02.360 --> 11:04.720]  something that we recommend you keep running.
[11:04.720 --> 11:08.680]  Really, it's there, kind of a break glass if you need it.
[11:08.680 --> 11:09.680]  You need to get in the node.
[11:09.680 --> 11:12.800]  You need to be able to troubleshoot some things.
[11:12.800 --> 11:17.760]  This is how you would do it by running an admin container.
[11:17.760 --> 11:20.200]  And then there's also other privileged things that we call
[11:20.200 --> 11:21.320]  bootstrap containers.
[11:21.320 --> 11:24.200]  Those are things that, they're container images that you
[11:24.200 --> 11:27.800]  can customize and spin up to be able to do special things.
[11:27.800 --> 11:33.720]  We've seen some cases where, OK, maybe I have some specific
[11:33.720 --> 11:39.120]  file system requirements, or I need to do some special thing.
[11:39.120 --> 11:42.320]  Bootstrap container is one of those host containers that's
[11:42.320 --> 11:45.400]  a little more privileged that you can have it when the
[11:45.400 --> 11:52.640]  system just starts up, initializes, that can do some
[11:52.640 --> 11:53.960]  custom things for you.
[11:56.560 --> 12:00.280]  So to see what this actually looks like, I talk a bit very
[12:00.280 --> 12:04.800]  vaguely about distributions.
[12:04.800 --> 12:08.480]  I'll just show you what it's like to interact with a
[12:08.480 --> 12:09.520]  ball rocket host.
[12:12.120 --> 12:20.680]  So typically, the only way you would interact with a host is
[12:20.680 --> 12:24.120]  through whatever container orchestration you're using.
[12:24.120 --> 12:26.360]  You shouldn't care what's running, what nodes are
[12:26.360 --> 12:29.040]  actually part of your cluster.
[12:29.040 --> 12:33.680]  You just have a Kubernetes cluster, and you use it.
[12:33.680 --> 12:36.160]  So you can use things like kubectl to get
[12:36.160 --> 12:38.760]  information about nodes.
[12:38.760 --> 12:43.400]  You could describe the nodes if you'd like.
[12:43.400 --> 12:47.000]  And if we do that, you can see here, it's an OS image,
[12:47.000 --> 12:48.760]  ball rocket 1.12.
[12:48.760 --> 12:56.960]  But most of the time, you would just be running your
[12:56.960 --> 13:00.920]  services, your pods, your load bans, demon sets, those
[13:00.920 --> 13:02.160]  types of things.
[13:02.160 --> 13:07.200]  If you need to connect, then you actually need to connect to
[13:07.200 --> 13:07.720]  the console.
[13:07.720 --> 13:09.840]  So if you're on a bare metal instance, this is actually
[13:09.840 --> 13:11.840]  plugging into the display port.
[13:11.840 --> 13:17.000]  If you're using a hosting platform, this is actually
[13:17.000 --> 13:20.200]  a hosting platform, this is whatever console interface
[13:20.200 --> 13:24.200]  that gives you, and you're presented with this.
[13:24.200 --> 13:28.640]  This is the controlled container that lets you
[13:28.640 --> 13:30.720]  actually access the host.
[13:30.720 --> 13:35.840]  So back to that diagram, within these host containers,
[13:35.840 --> 13:39.360]  under that container instance, right now, the shell that I
[13:39.360 --> 13:43.880]  have is actually this controlled container, because
[13:43.880 --> 13:48.160]  there's no shell on the base OS itself.
[13:48.160 --> 13:50.560]  So this gives you a little information to help you get
[13:50.560 --> 13:51.880]  started.
[13:51.880 --> 13:54.560]  There's the API client.
[13:54.560 --> 14:06.000]  And I can use that to get my Kubernetes settings.
[14:06.000 --> 14:13.440]  But if I look at Kubernetes, it puts everything under
[14:13.440 --> 14:17.040]  configuration under Etsy Kubernetes by default.
[14:17.040 --> 14:19.960]  There's nothing there.
[14:19.960 --> 14:23.560]  There's no shell.
[14:23.560 --> 14:32.440]  I can't do, let's see, I can't access file, can't make
[14:32.440 --> 14:34.040]  changes here.
[14:34.040 --> 14:39.120]  But there's nothing on here that shows you that this is part
[14:39.120 --> 14:40.400]  of a Kubernetes cluster.
[14:40.400 --> 14:43.440]  And that's because we're just inside that control container.
[14:43.440 --> 14:49.560]  This isn't actually the host OS, the host file system.
[14:49.560 --> 14:57.280]  The trick that I have there is there is a hidden mount that
[14:57.280 --> 15:02.080]  will give you some access to the root file system.
[15:02.080 --> 15:06.680]  But if you notice, I didn't point it out, there's enable
[15:06.680 --> 15:09.440]  admin container, which is one of the commands that that
[15:09.440 --> 15:12.920]  banner recommends, or lets you know about.
[15:12.920 --> 15:15.160]  And that's what actually gets you into this admin container
[15:15.160 --> 15:17.440]  that has more access.
[15:17.440 --> 15:29.920]  So if I do enter admin container, I
[15:29.920 --> 15:32.240]  knew I should have reset things.
[15:36.600 --> 15:38.760]  Normally, you just enter admin container.
[15:38.760 --> 15:41.480]  So that spins up that container instance, because it's
[15:41.480 --> 15:42.520]  not running by default.
[15:42.520 --> 15:44.080]  It's only when you need it.
[15:44.080 --> 15:46.520]  So I started up an admin container.
[15:46.520 --> 15:49.840]  Now I have a shell within the admin container.
[15:49.840 --> 16:00.760]  So now I'm from there, and now that I'm in that container,
[16:00.760 --> 16:02.040]  I have a little more access.
[16:02.040 --> 16:07.160]  I can see some more files, but it's still not going to give
[16:07.160 --> 16:08.120]  me full access yet.
[16:08.120 --> 16:10.440]  So there's a tool called Shellty.
[16:10.440 --> 16:16.600]  Now that I have that, now I have access to the actual
[16:16.600 --> 16:17.560]  underlying file system.
[16:17.560 --> 16:20.800]  So now I can go in, that's a Kubernetes, I want to take a
[16:20.800 --> 16:25.840]  look at the kubelet config.
[16:25.840 --> 16:27.200]  All that information is there.
[16:30.800 --> 16:35.040]  So yeah, aren't command line demos exciting?
[16:39.320 --> 16:42.480]  So in addition to being able to access the system only
[16:42.480 --> 16:47.520]  through these controlled mechanisms, we try to
[16:47.520 --> 16:51.040]  limit anything else that could be running, that could be
[16:51.040 --> 16:53.600]  used.
[16:53.600 --> 16:57.440]  There's a read-only root file system.
[16:57.440 --> 17:00.960]  So even if a container running in your Kubernetes
[17:00.960 --> 17:04.520]  cluster somehow was able to break out, we'd only have
[17:04.520 --> 17:07.280]  access to the read-only file system.
[17:07.280 --> 17:10.880]  It can't make any changes there.
[17:10.880 --> 17:15.040]  We also use de-imverity as an extra layer of precaution.
[17:15.040 --> 17:21.400]  So even if something happened, that adds some checks.
[17:21.400 --> 17:24.160]  And things are locked down.
[17:24.160 --> 17:29.280]  So really it'd be very difficult to compromise a system.
[17:29.280 --> 17:31.080]  And then we also use SeLinks.
[17:31.080 --> 17:34.760]  So there's multiple layers of protections in place here to
[17:34.760 --> 17:36.600]  try to limit things.
[17:39.520 --> 17:41.080]  There's PCI compliance.
[17:41.080 --> 17:44.800]  Sorry, I don't know what happened to that slide.
[17:44.800 --> 17:47.600]  And we are looking at FIPS compliance in the future to be
[17:47.600 --> 17:53.320]  able to show that the system really is secure.
[17:56.200 --> 18:00.400]  So I mentioned it's a read-only file system.
[18:00.400 --> 18:03.640]  The way Bialarocket is distributed is it's image
[18:03.640 --> 18:04.440]  base.
[18:04.440 --> 18:06.880]  There's no YUM, there's no DNF.
[18:06.880 --> 18:09.880]  You can't go in there and install extra packages.
[18:09.880 --> 18:12.760]  So it is a static image.
[18:12.760 --> 18:16.400]  So one nice thing about that is when you, if you want to
[18:16.400 --> 18:21.080]  upgrade a node to a new release of Bialarocket, the way
[18:21.080 --> 18:24.520]  it works is it'll actually download that newer image
[18:24.520 --> 18:32.080]  and write it to the second partition of the root disk.
[18:32.080 --> 18:35.520]  And then upgrading really is just switching over and
[18:35.520 --> 18:38.520]  pointing at that new partition.
[18:38.520 --> 18:40.880]  Because everything, all the settings that you have are
[18:40.880 --> 18:47.320]  persisted as part of what was set through the API server.
[18:47.320 --> 18:51.280]  It can switch over to this new image, reboot when it comes
[18:51.280 --> 18:51.640]  up.
[18:51.640 --> 18:54.680]  It reads all those settings again, uses the new image,
[18:54.680 --> 18:56.480]  and post is running.
[19:04.080 --> 19:05.920]  We do provide a few tools.
[19:05.920 --> 19:07.440]  There's a command line interface to be able to
[19:07.440 --> 19:08.600]  check for updates.
[19:08.600 --> 19:12.800]  That's a pain, especially if you have hundreds of nodes.
[19:12.800 --> 19:16.120]  But there's things like Bialarocket update operator,
[19:16.120 --> 19:18.280]  which will handle a lot of this for you.
[19:18.280 --> 19:21.640]  So if you have a Kubernetes cluster, you can schedule when
[19:21.640 --> 19:24.080]  you want maintenance activities to happen.
[19:24.080 --> 19:27.040]  That will automatically go out and look for new versions
[19:27.040 --> 19:28.120]  being released.
[19:28.120 --> 19:30.360]  And then it'll take care of interacting with Kubernetes
[19:30.360 --> 19:33.160]  to coordinate off nodes, get workloads drained off into
[19:33.160 --> 19:37.840]  others, upgrade those nodes, and then allow things to move
[19:37.840 --> 19:38.120]  back.
[19:38.120 --> 19:42.320]  So cleanly over a period of time, it'll get all of your
[19:42.320 --> 19:46.400]  nodes within the cluster upgraded to the new version.
[19:46.400 --> 19:52.720]  Or, and this is my preference, just replace the nodes.
[19:52.720 --> 19:53.640]  They're not customized.
[19:53.640 --> 19:56.520]  You know you haven't left any special thing on the file
[19:56.520 --> 19:58.840]  system that you need to worry about, am I going to lose
[19:58.840 --> 20:02.720]  something if I get rid of this?
[20:02.720 --> 20:06.240]  You just spin up new nodes, have them join, and then you
[20:06.240 --> 20:09.880]  can get rid of the old nodes and fresh system every time.
[20:09.880 --> 20:11.000]  Either way works.
[20:14.640 --> 20:17.960]  The configuration, like I mentioned, most of the time
[20:17.960 --> 20:20.480]  you're passing in a user data file.
[20:20.480 --> 20:23.680]  And that's in the Tamil format.
[20:23.680 --> 20:30.920]  But really, we're an equal opportunity markup project.
[20:30.920 --> 20:34.880]  So depending on how you're doing things, there's the
[20:34.880 --> 20:39.360]  YAML, if you use something like EKS, everything's
[20:39.360 --> 20:43.280]  configured in YAML, so you can have settings there.
[20:43.280 --> 20:47.520]  I showed the API client, if you actually do want to go and
[20:47.520 --> 20:51.360]  make changes on the command line, you can use that API
[20:51.360 --> 20:55.560]  client and set and give it a JSON string of the settings
[20:55.560 --> 20:58.280]  that you'd like or the Tamil.
[20:58.280 --> 21:04.320]  And on this URL at the bottom in the repo, there's a full
[21:04.320 --> 21:08.160]  listing of all the different settings that you can do with
[21:08.160 --> 21:11.440]  those configuration files.
[21:16.120 --> 21:21.520]  Now, the ballerocket handles things slightly different
[21:21.520 --> 21:23.280]  than a lot of other distros.
[21:23.280 --> 21:27.840]  So that can be a stumbling block when you look at how, if
[21:27.840 --> 21:29.960]  you want to adopt ballerocket or you're trying it out and
[21:29.960 --> 21:32.360]  trying to see if it works.
[21:32.360 --> 21:35.800]  So there's a few things to be aware of, I guess, when you
[21:35.800 --> 21:37.880]  do that.
[21:37.880 --> 21:43.160]  One common thing that I've heard from users is, well, my
[21:43.160 --> 21:47.440]  company requires that I run this anti-virus agent on all
[21:47.440 --> 21:50.840]  of my hosts.
[21:50.840 --> 21:53.200]  If they've containerized that agent, great.
[21:53.200 --> 21:55.560]  If they haven't, that's an issue.
[21:55.560 --> 21:58.000]  Like I said, there's no DNF, there's no YAML.
[21:58.000 --> 22:00.560]  You can't go in there and install software.
[22:00.560 --> 22:03.920]  So really, anything that you need to run on there, any kind
[22:03.920 --> 22:08.000]  of host agents that integrate with systems, can be run in
[22:08.000 --> 22:10.800]  privileged containers that can do a lot of things.
[22:10.800 --> 22:12.800]  It just has to be containerized to be able to
[22:12.800 --> 22:14.200]  run on ballerocket.
[22:18.680 --> 22:23.040]  And then, like I said, accessing it.
[22:23.040 --> 22:26.720]  A lot of sysadmins, they're used to, OK, I've got all
[22:26.720 --> 22:27.720]  these nodes out there.
[22:27.720 --> 22:31.800]  I know I just SSH in, and that's how I access my system
[22:31.800 --> 22:32.840]  and do things.
[22:32.840 --> 22:35.880]  So that can be a stumbling block, too.
[22:35.880 --> 22:38.720]  Things are done a little bit differently because things are
[22:38.720 --> 22:41.760]  so locked down, pros and cons.
[22:41.760 --> 22:45.120]  So because things are locked down, you need to enable that
[22:45.120 --> 22:46.240]  admin container.
[22:46.240 --> 22:50.280]  Then you can enable SSH if you need to, but it's not going
[22:50.280 --> 22:51.320]  to be there by default.
[22:53.920 --> 22:57.000]  And then the last thing I wanted to bring up, just
[22:57.000 --> 22:59.720]  because these are two AWS-initiated projects,
[22:59.720 --> 23:04.200]  both open source, I'll be talking to someone about
[23:04.200 --> 23:09.200]  ballerocket, and we'll go into some detail.
[23:09.200 --> 23:12.360]  And it seems like we're both on the same page, talking
[23:12.360 --> 23:13.200]  about the same thing.
[23:13.200 --> 23:16.280]  And then, somehow, they say something, and we're like,
[23:16.280 --> 23:19.200]  oh, wait, no, you're actually thinking about a
[23:19.200 --> 23:20.800]  different project.
[23:20.800 --> 23:22.440]  So Firecracker is another thing.
[23:22.440 --> 23:25.920]  That's actually for a virtualized solution.
[23:25.920 --> 23:30.040]  So that's ballerocket, Firecracker, explosive things,
[23:30.040 --> 23:32.840]  but not the same thing.
[23:35.640 --> 23:40.600]  So my main motivation for getting out and talking about
[23:40.600 --> 23:43.720]  things like this is love to see more people get involved.
[23:46.760 --> 23:52.000]  Everything that we can right now is up on GitHub, under the
[23:52.000 --> 23:56.520]  Bottle Rocket OS org, everything is Apache 2 and MIT
[23:56.520 --> 23:58.800]  licensed.
[23:58.800 --> 24:03.920]  We try to publish a roadmap under that org, so if you're
[24:03.920 --> 24:06.680]  curious what's happening, take a look there.
[24:06.680 --> 24:09.720]  But we'd like to, the people working on the project would
[24:09.720 --> 24:14.520]  like to hear from folks about what they'd like to see.
[24:14.520 --> 24:17.280]  If you have ideas and you want to bring pull requests, love
[24:17.280 --> 24:25.040]  that, to actually work on Bottle Rocket, we have the
[24:25.040 --> 24:30.960]  Linux kernel, obviously C, but most of the Bottle Rocket
[24:30.960 --> 24:35.320]  pieces of putting all this together is in Rust.
[24:35.320 --> 24:39.600]  So you will need Linux, you will need Rust, and you will
[24:39.600 --> 24:45.800]  need Docker to be able to do builds and things like that.
[24:45.800 --> 24:48.200]  We have a Bottle Rocket SDK image that we published, so
[24:48.200 --> 24:53.760]  that has the specific Rust, like the version, and there are
[24:53.760 --> 24:57.440]  some Go pieces, so it has the Go tool chain.
[24:57.440 --> 25:01.480]  But you do need a base requirement on your machine
[25:01.480 --> 25:04.360]  to be able to actually run things.
[25:04.360 --> 25:08.360]  And I say a decent amount of CPU memory and storage.
[25:08.360 --> 25:12.200]  I can't really give an exact number.
[25:12.200 --> 25:17.400]  You're compiling, you're building a distro.
[25:17.400 --> 25:22.880]  So if you want to do that in a two core, eight gigabyte VM
[25:22.880 --> 25:27.800]  on your laptop, you're going to have to be patient.
[25:27.800 --> 25:29.360]  That'll take some time.
[25:29.360 --> 25:34.160]  So really, the more CPU, the more memory that you have, the
[25:34.160 --> 25:37.320]  better that whole situation is going to be.
[25:37.320 --> 25:41.160]  But there is a building.md file in the repo.
[25:41.160 --> 25:44.960]  If you are interested in that, take a look, and that will
[25:44.960 --> 25:46.920]  go through everything to get you set up to be able to
[25:46.920 --> 25:50.680]  actually check out the repo, make changes, and compile it.
[25:53.920 --> 25:57.440]  Another area that I hope is going to help to get people
[25:57.440 --> 26:02.800]  involved, we're calling right now Autotree builds.
[26:02.800 --> 26:05.920]  So the variants that I spoke about, having these variants
[26:05.920 --> 26:08.400]  that are very optimized for different situations.
[26:08.400 --> 26:10.600]  If you wanted to build your own, say you have your own
[26:10.600 --> 26:13.440]  container orchestration platform, and you want to
[26:13.440 --> 26:17.080]  integrate BioRocket with it, right now you would have to
[26:17.080 --> 26:24.120]  fork the whole repo and do everything within BioRocket
[26:24.120 --> 26:26.120]  to get repo itself.
[26:26.120 --> 26:28.640]  So we're looking at ways, how can we separate things out
[26:28.640 --> 26:29.440]  and make this easier?
[26:29.440 --> 26:33.480]  So if you have a customized BioRocket image that you'd like
[26:33.480 --> 26:37.560]  to make for your company, for your home lab, how can you do
[26:37.560 --> 26:40.000]  that without having it pulled everything in?
[26:40.000 --> 26:45.040]  So if you are interested, you can subscribe to this, 2669
[26:45.040 --> 26:49.880]  in the GitHub BioRocket issues.
[26:49.880 --> 26:54.920]  And then even if you're not a developer, you don't have the
[26:54.920 --> 26:59.880]  resources to build your own, to be able to compile
[26:59.880 --> 27:04.680]  everything, if you're not interested in that, that's fine.
[27:04.680 --> 27:07.480]  We'd love people to just join our community meetings.
[27:07.480 --> 27:11.000]  Let us know what you're looking for, let us know if
[27:11.000 --> 27:14.840]  there's anything missing from BioRocket, become part of
[27:14.840 --> 27:17.800]  BioRocket itself.
[27:17.800 --> 27:20.720]  So that happens right now every other week.
[27:20.720 --> 27:23.520]  And we manage it through Meetup just to have an easy way to
[27:23.520 --> 27:26.640]  communicate when those are.
[27:26.640 --> 27:30.120]  There's a HackMD, you can throw your ideas in there, and
[27:30.120 --> 27:32.360]  we can discuss them.
[27:32.360 --> 27:36.280]  So I'd love to see anybody join there.
[27:36.280 --> 27:39.040]  So that's meetup.com, BioRocket-community.
[27:42.920 --> 27:46.720]  And with that, I'll open it up if there's any questions.
[27:51.720 --> 27:53.960]  Hey, thanks for a great presentation.
[27:53.960 --> 27:58.400]  Given that the file systems are immutable, where do logs go?
[27:58.400 --> 28:00.320]  Does BioRocket itself log?
[28:00.320 --> 28:03.640]  And I understood that the cubelet is also running
[28:03.640 --> 28:04.760]  non-containerized.
[28:04.760 --> 28:07.440]  So where do the cubelet logs go if you use cubelet?
[28:07.440 --> 28:14.320]  Yeah, there are some very targeted areas where we
[28:14.320 --> 28:16.000]  mount a tempfs.
[28:16.000 --> 28:19.560]  So things like how I was talking about all the settings
[28:19.560 --> 28:23.680]  through the API, you need to use those to spin up
[28:23.680 --> 28:25.160]  cubelet or to run cubelet.
[28:25.160 --> 28:27.080]  It needs to know its settings and needs to read that from a
[28:27.080 --> 28:28.280]  configuration file.
[28:28.280 --> 28:31.120]  So yeah, so if we have a read-only file system, how
[28:31.120 --> 28:33.360]  does it do that?
[28:33.360 --> 28:39.040]  On boot, we mount these tempfs mounts in specific places
[28:39.040 --> 28:40.240]  where they're needed.
[28:40.240 --> 28:45.400]  And then based on reading the configuration settings that
[28:45.400 --> 28:48.960]  gets written out with the template, so if changes
[28:48.960 --> 28:52.200]  somehow happen in there, your reboot comes back up and
[28:52.200 --> 28:56.440]  you're exactly how you have things originally.
[28:56.440 --> 28:59.760]  There's a question in chat is, is there a version of
[28:59.760 --> 29:04.520]  bottle rocket which is built to run on KVM lib world?
[29:04.520 --> 29:12.720]  We, there, in the repo, let me see where did I put that.
[29:12.720 --> 29:20.760]  In the repo, there is, sorry, it must just be
[29:20.760 --> 29:26.760]  under building, there are instructions.
[29:26.760 --> 29:34.040]  Bottle rocket can be run on QMU, so if you, that's a great
[29:34.040 --> 29:38.400]  development tool too, is if you want to make changes and
[29:38.400 --> 29:43.320]  spin things up and just have it running, see how it works,
[29:43.320 --> 29:48.240]  you can run it as a virtual machine, yeah.
[29:48.240 --> 29:56.960]  Thank you for your...