[00:00.000 --> 00:11.360]  Those two are two half boxes, because the other half is in, you know, the vanadaire.
[00:11.360 --> 00:14.360]  But I guarantee those are two boxes.
[00:14.360 --> 00:18.360]  So one for the networking and one for the computing.
[00:18.360 --> 00:20.280]  I bought it years ago on eBay.
[00:20.280 --> 00:27.560]  I think some startups ran out of business and they had like many full-sized rack, full
[00:27.560 --> 00:31.760]  of nukes and, you know, NVIDIA boards.
[00:31.760 --> 00:34.240]  So I bought one and I started to play with it.
[00:34.240 --> 00:39.160]  So now I have ten nukes lying around at home, five GPU and I don't know what to do with
[00:39.160 --> 00:40.160]  them.
[00:40.160 --> 00:41.920]  So this is how everything started.
[00:41.920 --> 00:47.480]  So I have a single point of failure and all the SREs and DevOps here, you know, will hate
[00:47.480 --> 00:51.040]  it, but it's my single point of failure and I love it.
[00:51.040 --> 00:53.760]  And I call it Snowflake because it's unique.
[00:53.760 --> 00:56.240]  And what it does, it does a very minimal setup.
[00:56.240 --> 01:00.120]  It's the only long-living server or nukes, if I can call it server.
[01:00.120 --> 01:01.120]  I would call it server.
[01:01.120 --> 01:05.080]  So it's my only long-living server I have.
[01:05.080 --> 01:07.720]  And it runs a DHCP server.
[01:07.720 --> 01:15.920]  It runs Pixiboot, a TCP server, and is the core of my home lab.
[01:15.920 --> 01:20.840]  It runs there and it distributes the, you know, the NixOS derivation that I want to
[01:20.840 --> 01:27.720]  net boot on the other servers when I hook them to the network.
[01:27.720 --> 01:32.440]  As you know, because you are here and you are like me, I tend to automate everything
[01:32.440 --> 01:34.080]  I can.
[01:34.080 --> 01:40.080]  So as I said, this is the code that I wrote in the NixOS derivation for the Snowflake
[01:40.080 --> 01:41.080]  server.
[01:41.080 --> 01:46.960]  I start OpenSSH, I configure my keys, I didn't, you know, paste that, the public key there
[01:46.960 --> 01:51.960]  because it was too long and not great, but you know how to do it.
[01:51.960 --> 02:00.160]  And I start PixiCore and I configure it with a package called Blackhole that is the distribution
[02:00.160 --> 02:01.960]  I use for net booting.
[02:01.960 --> 02:04.600]  And I'm going to, you know, get to net booting later on.
[02:04.600 --> 02:10.880]  I have my.files shared with the NixOS configuration, so you can check that if you want.
[02:10.880 --> 02:18.720]  So what is, my Snowflake server is used to do Pixi booting and net booting, to distribute
[02:18.720 --> 02:21.200]  the images I want on my Fimmer Labs.
[02:21.200 --> 02:23.480]  And PixiCore is a tool that helps with that.
[02:23.480 --> 02:32.200]  It can be used, Sandalone is a Go project that you run, it exposes a DHTP server, an
[02:32.200 --> 02:38.960]  HTTP server to download the, you know, the kernel and the initram disk.
[02:38.960 --> 02:45.600]  And then this is Blackhole, so Blackhole is the, when you do net booting, you need an
[02:45.600 --> 02:48.640]  operating system to run on the new servers.
[02:48.640 --> 02:52.320]  And I generate that from a NixOS derivation.
[02:52.320 --> 02:56.360]  It's also very lightweight because it stays in RAM, so it's not persistent on disk, so
[02:56.360 --> 02:58.360]  it needs to be as small as it can be.
[02:58.360 --> 03:03.760]  So I use the net boot base profile and I change to minimal, but I didn't update my slides.
[03:03.760 --> 03:10.120]  So if the image is too big, you can use the installer minimal, that is a bit smaller.
[03:10.120 --> 03:14.720]  And then I install OpenSSH because this is what I use to enter into the server when it
[03:14.720 --> 03:16.000]  boots.
[03:16.000 --> 03:18.120]  But what is net booting?
[03:18.120 --> 03:24.440]  So from zero to zero in one minute, because that's how good I am with Nix booting, with
[03:24.440 --> 03:25.920]  net booting.
[03:25.920 --> 03:32.080]  And so every server or many computers boots and they look for, you know, an operating
[03:32.080 --> 03:37.680]  system on disk, and if there is not an operating system on disk, they look for a USB one.
[03:37.680 --> 03:43.360]  And if there is not a USB, the world becomes way bigger and they try to send a DHCP request
[03:43.360 --> 03:48.880]  to the DHCP server, says, oh, I'm alive, what can I do?
[03:48.880 --> 03:54.160]  And if you have a smart enough DHCP server, it replies with a package containing the IP
[03:54.160 --> 03:59.680]  address for the server and then an address that can be used to download an in-it RAM
[03:59.680 --> 04:03.040]  disk and a kernel and boot the server.
[04:03.040 --> 04:06.560]  So this is, you know, very old style, but still in use.
[04:06.560 --> 04:11.720]  You know, all the cloud providers that I had the opportunity to work with use this because
[04:11.720 --> 04:14.720]  hardware doesn't change that quickly.
[04:14.720 --> 04:20.120]  So Pixi booting is still there and this is the same things I use.
[04:20.120 --> 04:28.160]  So I download all the two binaries and the two packages and my server is ready to boot.
[04:28.160 --> 04:34.560]  So for me, I take the, you know, the compute models out from my boxes, I hook it to the
[04:34.560 --> 04:42.320]  network and they power on the DHCP kernel and I'm ready to SSH.
[04:42.320 --> 04:48.200]  This is kind of the log you get from PixiCore when it detects like a new server ready to
[04:48.200 --> 04:50.800]  handle an installation process.
[04:50.800 --> 04:56.840]  So, you know, the IP is assigned 162 and it downloads the kernel, the in-it RAM disk that
[04:56.840 --> 05:03.640]  I generated with the black hole scripts that you just, Nixx derivation that you just see
[05:03.640 --> 05:04.640]  and it boots.
[05:04.640 --> 05:09.240]  So on the end of the booting process, you have an address that you can SSH into and
[05:09.240 --> 05:11.120]  you have a server to play with.
[05:11.120 --> 05:14.040]  Obviously, this is not persisted.
[05:14.040 --> 05:19.960]  So when you reboot the server, the dance would start again and again forever and you would
[05:19.960 --> 05:22.160]  get a fresh new server to use.
[05:22.160 --> 05:24.960]  But for my ephemeral homelab, this is what I need.
[05:24.960 --> 05:26.760]  I just need a quick way to prototype.
[05:26.760 --> 05:36.000]  So if you can get enough RAM to be happy, and for me, 8 GB of RAM is happier than ever,
[05:36.000 --> 05:37.000]  I can play with that.
[05:37.000 --> 05:41.920]  Obviously, all the nooks can have, you know, disk attached to it and you can decide to
[05:41.920 --> 05:44.600]  persist the operating system on it.
[05:44.600 --> 05:50.360]  So you can write a system disk script that does Nixx install at the end of the, when
[05:50.360 --> 05:55.440]  the server is booted and you don't need to touch anything, when the black hole starts,
[05:55.440 --> 06:01.760]  it would run the scripts, the system D scripts and it would have a new persisted and flashed
[06:01.760 --> 06:02.760]  operating system.
[06:02.760 --> 06:07.880]  As I said, I don't do that because I don't need it, but that's how the full cycle, life
[06:07.880 --> 06:10.240]  cycle of the machine can be managed.
[06:10.240 --> 06:15.640]  And how do you, if you persist a disk, how do you reboot the server and get back to the
[06:15.640 --> 06:20.880]  pixie, to the pixie dance is all on you.
[06:20.880 --> 06:25.520]  In my case, I don't have, like the nooks doesn't really have a board, a BMC to control the
[06:25.520 --> 06:29.800]  server, so I can't force the boot loader to net boot.
[06:29.800 --> 06:33.200]  So I wipe the disk and they break the disk, so it will go to net booting.
[06:33.200 --> 06:36.880]  A bit rough, but you know, homelabs, that's how it is.
[06:36.880 --> 06:40.680]  So service discovery, in my case, it's way simpler than what it should be.
[06:40.680 --> 06:44.920]  You can use console core DNS or whatever you want, Type-K even, so you can, you know,
[06:44.920 --> 06:49.400]  in black hole, you can register a Type-K daemon and have it registered there, and you see
[06:49.400 --> 06:53.240]  that in the UI, I use my router.
[06:53.240 --> 06:58.320]  So when I see that there are new servers with a NixOS hostname, I know that there is a server
[06:58.320 --> 07:02.160]  ready to be, to used and that it's booted.
[07:02.160 --> 07:10.560]  In the meantime, when it does the pixie dancing, the hostname is pixie and a random MAC address.
[07:10.560 --> 07:16.280]  So it kind of gives me enough control and I don't need to run anything else, but you
[07:16.280 --> 07:19.360]  can get fans here if you want.
[07:19.360 --> 07:21.360]  So what is net booting used for?
[07:21.360 --> 07:27.480]  Only for ephemeral homelabs, as I said, no, cloud providers or, you know, people managing
[07:27.480 --> 07:32.920]  data centers that I think you know more than me, but you can do that for inventory management.
[07:32.920 --> 07:38.640]  So when you buy a shiny new server, it doesn't do much, it doesn't know what to do, so you
[07:38.640 --> 07:44.000]  just hook it to the network and the pixie dance starts and instead of running a script
[07:44.000 --> 07:50.560]  that persists an operating system, you run a daemon that registers the server to an inventory
[07:50.560 --> 07:53.360]  management like Netbox or others.
[07:53.360 --> 07:57.760]  So you can, you know how many, how much RAM it is, the model, whatever you can get from
[07:57.760 --> 08:02.960]  the system, you expect the system, you push the blob and you register a brand new servers
[08:02.960 --> 08:05.920]  and you have it available in your system.
[08:05.920 --> 08:09.560]  So it's kind of the same process I do for mine, but way cooler because the hardware
[08:09.560 --> 08:11.560]  is way more expensive.
[08:11.560 --> 08:15.320]  You can do that for provisioning, so as I told you before, I don't do provisioning because
[08:15.320 --> 08:21.200]  I don't want a persistent operating system on my homelab, but you can format and partition
[08:21.200 --> 08:29.920]  disks using system if you want or any other way to run a script and you're ready to go.
[08:29.920 --> 08:34.080]  For recovery, the same way, as I said, I don't have a BMC, but you can, you know, break your
[08:34.080 --> 08:39.320]  disk and get back on pixie booting.
[08:39.320 --> 08:44.400]  Some of the hardware have disks attached, but I don't really use disk much and even
[08:44.400 --> 08:52.680]  if you, because I don't need it, so my operating system is small enough and I do simple stuff
[08:52.680 --> 08:58.640]  that don't require me to, you know, swap or go on the disk, so I can use the entire disk,
[08:58.640 --> 09:02.920]  I can format it just for the data if I need to store some data.
[09:02.920 --> 09:09.520]  In order to do orchestration, pixie core, you can declare MAC addresses and the images
[09:09.520 --> 09:12.960]  you want to push to.
[09:12.960 --> 09:20.120]  So if you know the MAC address of your Ethernet cable, you can say, oh, on this MAC address,
[09:20.120 --> 09:25.520]  distribute this image, otherwise distribute a different image, so you can have servers
[09:25.520 --> 09:30.640]  running different net booting images and it's very convenient.
[09:30.640 --> 09:35.800]  So now when it comes to electric wiring, I'm also not really an expert, so what I have,
[09:35.800 --> 09:43.160]  I have a really, you know, a powerful, a powerful enough DC that is connected to a few fuses
[09:43.160 --> 09:48.120]  so I don't, you know, blow up all the nodes at the same time, one at a time maybe, but
[09:48.120 --> 09:53.840]  not all together, so that's why I have the fuse there, easy, cheap, it works.
[09:53.840 --> 10:00.360]  And then to control, you know, the power consumption, I just cut the power at the beginning, so
[10:00.360 --> 10:07.560]  I have a Raspberry Pi connected to a board of 16 relays that I control over an API, so
[10:07.560 --> 10:13.880]  if I want to stop the nuke number one, I just, you know, stop, cut the energy and the power
[10:13.880 --> 10:20.160]  for that nuke, and goodbye nuke, and, you know, playing with the GPIO, the Raspberry Pi is
[10:20.160 --> 10:25.000]  very easy and convenient, so for the prototype, it's good enough.
[10:25.000 --> 10:29.920]  I want to change from a Raspberry Pi from ESP32 to experiment a little bit, but it's
[10:29.920 --> 10:34.920]  not really for NixOS, because it doesn't run an operating system, so, just there, and
[10:34.920 --> 10:40.160]  I got two new fancy boxes that I want to replace with the IKEA one, because I think that would
[10:40.160 --> 10:45.680]  play a better role with my, you know, ambient there, and I don't know if I'm late or earlier,
[10:45.680 --> 10:46.680]  but that's it.
[10:46.680 --> 10:54.480]  Okay, so, yeah, this is my home lab and what I'm playing with, so if you have any question
[10:54.480 --> 11:03.840]  or if you want to know more, I will just go back to the, if I can, my invite is to have
[11:03.840 --> 11:12.920]  a look at the.file repo, if I can, oh, I can go there, at the.file, because it has
[11:12.920 --> 11:22.000]  all the code in there and you can play with it, so coming back to these slides, what I
[11:22.000 --> 11:28.280]  really like is the convenience of, you know, I have my NixOS derivation, for these experiments
[11:28.280 --> 11:35.160]  I'm building two NixOS derivations and the flake, there's no flake one encapsulates the
[11:35.160 --> 11:40.760]  other one, so I can distribute it conveniently and it's a single flake file that contains
[11:40.760 --> 11:47.360]  and run both, so it's, you know, it's very easy and it's very easy when you get to it.
[11:47.360 --> 11:54.040]  To get there, it took me 135 questions on this course, 36,000 messages on metrics, but
[11:54.040 --> 12:03.640]  I'm there as it's there for you, so play with it if you want, so thank you.
[12:03.640 --> 12:23.240]  Yeah, because it's very, oh yeah, how NixOS plays with the pixie booting and, you know,
[12:23.240 --> 12:29.000]  net booting, you have, I didn't find, there are other packages or other projects that
[12:29.000 --> 12:34.200]  helps you to package an operating system, like HashiCore Packer is another one, there
[12:34.200 --> 12:39.760]  is Infrakit from Docker and all those kinds of stuff, so, but NixOS gives you full control
[12:39.760 --> 12:44.720]  on the distribution that you want to provide, so for example at the beginning I wasn't really
[12:44.720 --> 12:49.240]  looking at the size of the image, so I was pushing in there everything I needed, but
[12:49.240 --> 12:54.280]  then I started to realize that my image was like four gigabytes and my RAM, I didn't have
[12:54.280 --> 13:00.360]  enough RAM or I didn't have enough RAM left to do the actual work, so I started to iterate
[13:00.360 --> 13:05.120]  and play on what I was able to remove, so I started to investigate about how to do a
[13:05.120 --> 13:09.800]  minimal operating system, so I see that minimal profiles, I disabled the documentation, I
[13:09.800 --> 13:14.560]  disabled, you know, everything that I don't need it, and I went back to like a, now the
[13:14.560 --> 13:20.840]  image is 400 megabytes in RAM when it's fully unpacked and it works, so it's good enough,
[13:20.840 --> 13:25.840]  so I think that's really the power you get from NixOS compared with other systems.
[13:25.840 --> 13:31.840]  For getting questions answered, I can also suggest the Discord server which is also pretty
[13:31.840 --> 13:32.840]  great.
[13:32.840 --> 13:33.840]  Okay, yeah, I will look at it.
[13:33.840 --> 13:34.840]  Thank you.
[13:34.840 --> 13:35.840]  One more round of applause please.
[13:35.840 --> 13:36.840]  Thank you.
[13:36.840 --> 13:37.840]  Thank you.
[13:37.840 --> 14:02.840]  Okay, I'll produce some more.
[14:02.840 --> 14:27.840]  One more round of applause please.
[14:27.840 --> 14:53.840]  One more round of applause please.
[14:53.840 --> 15:19.840]  One more round of applause please.
[15:19.840 --> 15:44.840]  Thank you.
[15:44.840 --> 16:11.840]  Thank you.
[16:11.840 --> 16:36.840]  Thank you.
[16:36.840 --> 17:03.840]  Thank you.
[17:03.840 --> 17:28.840]  Thank you.
[17:28.840 --> 17:38.840]  Thank you.
[17:38.840 --> 17:48.840]  We ready for the next?
[17:48.840 --> 17:49.840]  Great.
[17:49.840 --> 17:50.840]  Yes.
[17:50.840 --> 17:51.840]  Hi.
[17:51.840 --> 17:52.840]  I'm Philip.
[17:52.840 --> 18:03.840]  I'm here on behalf of Circle, a company, so we specialize on, you know, software delivery,
[18:03.840 --> 18:08.840]  customized to the clients, you know, like what people want.
[18:08.840 --> 18:14.840]  This talk is about NixOS, so like, you know, I'm here in Boston 2023, of course, and my
[18:14.840 --> 18:18.840]  talk is pitfalls of NixOS and how to avoid them, right?
[18:18.840 --> 18:25.840]  So at Circle, we use NixOS for like infrastructure, we use NixOS for CI CD and setting up deaf
[18:25.840 --> 18:26.840]  environments.
[18:26.840 --> 18:31.840]  Basically, everywhere you can use NixOS, we try to adopt it just because, you know, NixOS
[18:31.840 --> 18:32.840]  is great.
[18:32.840 --> 18:36.840]  You have audio visibility and everything that comes with that.
[18:36.840 --> 18:37.840]  Right.
[18:37.840 --> 18:38.840]  So this is me.
[18:38.840 --> 18:39.840]  All my socials are here.
[18:39.840 --> 18:44.840]  If you want to reach out to me, my GitHub.
[18:44.840 --> 18:45.840]  Right.
[18:45.840 --> 18:46.840]  Right.
[18:46.840 --> 18:47.840]  Yes.
[18:47.840 --> 18:54.840]  So Circle, as I said, is, you know, a software company focused on creating customized high-performance
[18:54.840 --> 18:55.840]  solutions.
[18:55.840 --> 18:56.840]  Yes, I can.
[18:56.840 --> 18:57.840]  Right.
[18:57.840 --> 18:58.840]  Sorry.
[18:58.840 --> 19:02.840]  So this is mainly in like the like functional programming realm of like Haskell and like
[19:02.840 --> 19:09.840]  stuff like that, where, you know, people want customized solutions for certain problems.
[19:09.840 --> 19:10.840]  Exactly.
[19:10.840 --> 19:11.840]  Right.
[19:11.840 --> 19:12.840]  So again, this hurts about Nix.
[19:12.840 --> 19:16.840]  And this is, right.
[19:16.840 --> 19:20.840]  The goal of this talk is to make Nix more approachable.
[19:20.840 --> 19:21.840]  Right.
[19:21.840 --> 19:25.840]  So when we adopted Nix, I mean, I wasn't dead then.
[19:25.840 --> 19:30.840]  Obviously, developers weren't using Nix, right.
[19:30.840 --> 19:33.720]  And they still, you know, they try to, right.
[19:33.720 --> 19:39.040]  So there has to developers, they're not dead into Nix, right.
[19:39.040 --> 19:44.800]  And they had some issues and I want to touch on that just so that, you know, a case study
[19:44.800 --> 19:48.800]  of how Nix works in Circle.
[19:48.800 --> 19:49.800]  Exactly.
[19:49.800 --> 19:50.800]  Right.
[19:50.800 --> 19:51.800]  Sorry.
[19:51.800 --> 19:52.800]  I'm right.
[19:52.800 --> 20:01.000]  So I am pretty new at Circle and I'm part of the SID team.
[20:01.000 --> 20:03.840]  So I try to push Nix too.
[20:03.840 --> 20:17.520]  In the often times you have developers.