Okay, good morning everyone.
I'm Dominic Mills-Holl and today I'll be presenting on fortifying the foundation's
elevating security in Nixson and XOS.
To give, well, before I get into the talk, I'll give a brief introduction to myself.
I'm a software engineer that's broadly interested in application development, build system compilers,
and algorithms.
And I've served in varying capacities in different open source projects such as being a mentor
under the Palasadus Foundation and Haskell.org under the Google Summer of Code program for
three years.
In addition to that, I've also been a participant of Google Summer of Code-like programs such
as Summer of Haskell and Summer of Nix, which I both participated in last year.
And now to get to the meat of the matter.
Today I'll be presenting on this talk, which is about the various features that were implemented
during the sovereign tech fund's Contribute Back Challenge, which occurred in the fourth
quarter of last year.
And the sovereign tech fund is essentially a fund mandated by the German government that
seeks to support the development, improvement, and maintenance of open digital infrastructure.
So Nix was one of the nine selected projects in 2023, and the focus was on three aspects.
But this talk will be mainly focused on the first aspect, which is a proper boot security
chain for Nix OS.
This was chosen because it's easily the most expansive and arguably the more interesting
of the three.
So let's get into it.
So I assume many of you use Linux, because why wouldn't you?
And I know that when you first installed, like whenever that was, when you first installed
Linux, you had to disable secure boot in order to proceed with installation.
And it's a very common place for us to just disable secure boot and just completely relinquish
that thought of ever having a dual booted Windows and Linux machine, because why would
you?
And essentially what we're interested in is implementing this boot security feature in
Nix OS.
And that consists of a number of different facets, but historically, there were two
earlier implementations, Lands Boot and another one developed by the Terminet Systems, Boot
Spec, Boot Spec Secure Boot.
Yeah, sorry, it's literally on the screen.
But there were a bit unsatisfactory in the sense that they required one to first install
Nix OS in an insecure manner and then modify the configuration file in order to apply, well,
Lands Boot in particular, in order to apply Lands Boot, then Nix switch rebuild and then
you'd have secure boot after you've made the necessary configurations for your machine.
And for you as an end user, that maybe is fine, but say you're using Nix OS on some kind of
network or some kind of cloud server or you're trying to find a way to kind of determine
if you could build some kind of server form.
It's not really satisfactory because if you're providing some kind of service like that,
then you pretty much leave yourself susceptible to things such as boot kits that can basically
take control of your entire system from the boot process onwards.
And essentially you kind of want to adapt yourself to a secure boot story in Nix OS in
the sense that you want secure boot to be the default option.
Once you get the Nix OS installed and put it in your device and you boot it up on a new
machine, you don't really want to have to turn off secure boot.
You want to essentially have an option where you can have secure boot by default on Nix
OS the moment you've begun to install it.
And this consists of a number of different steps.
I mean, the secure boot is essentially just a chain of trust such that various components
from the inception of the booting process just kind of sign one another.
And the methodology that we kind of took in this approach was basically to use a unified
kernel image because it would keep everything in one particular place.
You won't really have to worry about booting any RD or MS-D on the kernel and whatnot.
You basically have everything in one place and you just kind of have to sign that and
then you go forward from there.
And how that works is that essentially secure boots is a very Microsoft-centric thing.
So you kind of have to adapt the Nix image that you're building to accommodate for that.
And what that means is that you translate the language into the language of portable executables.
So what that means is that you kind of have to create new structures in order to manage
these portable executables in such a way that you can have Microsoft keys on your machine
and secure boot will work.
You don't necessarily have to have Microsoft keys per se.
You can install your own personal keys.
But the problem with that is that the chances of you breaking something are astronomically
high.
So it's probably best to have something that has Microsoft keys.
And loosely speaking, what I mean by has Microsoft keys means that somewhere on your laptop there's
like a sticker that says Microsoft somewhere.
And that usually means the machine.
The UFE has unified extensible firmware interface, has Microsoft keys in its database.
And how this was done implementation-wise was that we essentially used something called
a meta-writer structure because we couldn't think of a better name.
And what this does is not necessarily modified portable executable in and of itself, but
more or less kind of begins to recreate it from scratch, kind of looking at every single
aspect of it and then modifying that accordingly to accommodate for what changes we want.
So the reason why we had to do this is because portable executables are very, for lack of
a better word, say volatile.
It's not really meant to be cross-platform or anything like that.
It's more or less meant to be made for just Microsoft-type products and Microsoft-type
products only.
And basically you kind of have to look at a lot of the headers and see how they're related
to one another.
If you mess up, if you write the wrong data to the wrong header or if you have a data
section somewhere and it doesn't link to somewhere else, you've ripped your system again so you
don't want to have this type of issue anymore.
You basically want something that is already developed, quote-unquote, in-house and just
works from the get-go.
And essentially this was accomplished.
If you actually look on the NixOS issue, you'll see that Linus Heckelmann, whose GitHub name
is Linux Ackerman, and I often forget what his real name is, but actually did manage
to construct a NixOS shim with an embedded self-signed certificate.
So that part is done.
But there are actually two aspects of the story.
The second aspect is to send the shim to the shim review committee, which is basically
a bunch of independent, well, they're affiliated with Microsoft in somewhere or another.
And this allows one to kind of, by the way, for those, for the uninitiated, a shim is
just a first-stage bootloader.
So just a bootloader that can load an operating system or another bootloader.
And what this essentially means is that once this gets reviewed by the shim review committee,
it means that they say, yeah, okay, we can give you some Microsoft keys and you can sign
the ISO and everything should work.
So we're currently in that stage where we're able to construct a NixOS shim that can embed
a self-signed certificate.
But we haven't yet got the green light from the shim review committee, but there's nothing
to indicate that we shouldn't.
And more or less, this is so far the secure boot story in the broadest possible way.
But more specifically, there are other aspects such as the boot spec specification.
So this is boot spec version one.
And a part of the work done during this project was creating a boot spec version two.
So if you're probably wondering, why do we need a version two?
Version one looks good already.
It's not, there's nothing dynamic about it.
It's just a JSON document.
What could be wrong?
Well, NixOS only has one, it can only take one argument, a string.
When in actuality, that doesn't fill the entire gamut of the entire spectrum of, you
could have multiple in-entities, essentially.
And that's usually the case if you have, say, some global user settings, some specific user
settings, some environment settings that you've changed.
And usually these end up becoming some CPIO archive files somewhere.
And so you essentially want it to be a list.
And also you can also dynamically generate initRD from the EFS.
So yet you could just end up with a lot of initRD files that just spring out of nowhere.
So that one isn't very satisfactory because it doesn't consider all the use cases.
And you also see initRD secrets.
But here's the funny thing.
There's actually nothing secret about initRD secrets.
The secret in initRD secrets is a plain text file.
So it doesn't actually provide any form of security.
And ultimately this is what boot spec version two looks like.
So initRDs now has, it's a list.
You can add things to your list to eventually put them together and then that satisfies the
most general case possible.
FDT third, that's not really going to be discussed here, but essentially it's just to consider
the case of using that's you boot.
And device trees, we didn't have support for device trees previously.
But we do now with boot spec version two.
So what we've done with initRD secrets is that we've basically used a hashing method.
It's a very boring hashing method.
So it's now more secret.
It's actually a secret.
You don't have a plain text file anymore.
And this is the difference between boot spec version one and boot spec version two.
And this is somewhat of a segue from security.
It's tangentially related, but it's still a very important feature nonetheless.
So we define AB schema to refer to a type of primary and secondary boot partition, wherein
if the primary partition fails, you'll switch to secondary.
And then this is generalized into something that's called automatic boot assessment in
NixOS.
And what this means is essentially if I want to load a NixOS generation and it fails for
some reason, and the most practical way this would fail, to give a concrete example, is
if you went into Nix packages unstable and got that package AMD GPU and you realized you
can't boot into your generation and notice from experience.
So this is an example of where this would be useful.
And essentially what it does is that it generalizes AB schema to NixOS generation.
So in that case, you had a very simple case before, and it's designated as an indeterminate
with a specific number of boot attempts.
So you have some predeterminate number of boot attempts that you want to have.
And then eventually, once you've gone past that, then it's no longer deemed as a good
boot.
You can't decryment it every single time, and then it's designated bad.
And this is useful in the case that, say, you have some kind of procedure where you have
unattended boot.
So maybe you have some kind of build farm, maybe you have some kind of service where
you have to boot into some specific NixOS generation.
Maybe you're using NixOS as a replacement for Ansible.
And you add things to NixOS, you add new services, maybe some of the services are from Nix packages
unstable, and you just, it's more or less everything is automated.
This is a use case where that would be useful.
If somehow you magically find a way to upgrade things without you having to touch the Nix
configuration file, then automatic boot assessment will take care of the fact that your generation
may be corrupts and you may have to switch back to an earlier generation after certain
more boot attempts.
And this is an example for an authoritative DNS.
The parameter boot.com specifies what you call a synchronization point.
That means that it eventually just cycles back.
So we haven't specified the number of boot attempts here.
I don't believe it's a default.
This work was actually driven by Julian Malka, who's giving a talk on this.
I really hope it's not the same example he uses, because I didn't ask him before.
But more or less, this is an example of where you would have a type of service that has
a specific number of boot attempts that needs to be done.
And you can use this automatic boot assessment to just count the number of times that it
would be completed, and if not, your failure action is to reboot.
You could also specify the number of times it could be rebooted, but I don't believe
that's done here.
And another feature is integrity checks for the next store.
So when implemented secure boot and transitioning to stage two, limitations do arise.
You can mess with the file system, essentially.
If essentially you don't really have any place to prevent that from happening.
And a number of different times things were tried.
So for example, the invariant which works on the block device layer was used, but it
was unsatisfactory in the sense that it basically creates an entire copy of your NixOS generation,
which takes up a lot of space and doesn't really provide any means of flexibility to
work around that.
FS very to works on the file system level, but it doesn't...
The invariant is also read only.
That's the reason why it takes the entire NixOS generation.
FS very to works on the file level, so that's a big improvement over the invariant, but
it doesn't prevent you from kind of just switching files at that point either.
So you could easily switch between bash and Perl and whatnot.
IMA and EVM, IMA is intermediate measured assessment and EVM is extended verification
measurement for system integrity.
They more or less have the same problems as FS verity and they're really only good for
auditing purposes.
Like they don't really help us that much in terms of our intended goal of looking at the
file system and seeing if it gets past stage two.
So what was essentially chosen was the simplest method, which is just to use NixStoreVerify.
And the problem with this is that there's a penalty for it.
If you have a low-end device such as a Raspberry Pi, you probably have to wait two minutes
extra for it to boot.
And if you have a high-end device more or less like a desktop machine, there's still
a five-second penalty.
And there are, we're currently considering this such as Apple's signed file system, which
looks promising but has to be combined with something like BCatchFS or BCatchFN or something
to kind of really make it a viable option.
And this isn't, this is just speculation.
This work is actually driven by Will Fanature, whose GitHub name is Elvish Jericho.
So he's done the research for this, but he hasn't, and this is the end result of it.
But in terms of future work, he's looking at considering whether to use that mechanism
that I just mentioned a while ago, which is to use Apple's signed signature volumes to
verify the NixStore in stage two.
So enabling integrity checks looks as such.
You have to first create a public file key, and then in the NixOS modules, you essentially
import that into the trusted public file keys, trusted public keys, using boot when during
the, in the boot, in it verify a configuration.
And it's really just as simple as that.
The only issue is that you may have to find a way to kind of hide your file key somewhere
if you're in a situation where you have the boot unattended.
And lastly, interpreters in Nix.
So Nix essentially has a lot of bash and pearl and Python scripts all about the place, and
this leaves a lot of room for vulnerabilities.
And in phase one, essentially all the pearl scripts were removed.
So set up, etc.pl, which sets up users and whatnot.
That's replaced by overlay in it already.
Update users, groups, that's replaced by system, the system, the users, this user's functionality.
And the broad replacement of activation scripts, that actually isn't necessary for this, but
it does work in terms of performance and in terms of gaining performance and maintainability
benefits.
So this is, I failed to mention that this was just like phase one of this challenge.
We're still waiting on the status of phase two.
But phase two would also involve removal of bash scripts.
I'm not entirely sure how Python scripts would be removed because they're more tightly integrated.
For example, if you tried to use a Nix OS test, you'd end up having to use Py.
You'd have to write it in Python at some point.
So it's not clear to me exactly how that path forward there.
But pearl has been removed in this phase of the sovereign tech funds country-by-back challenge
and in the future, eventually bash should be removed.
So yeah, that's my talk.
Any questions?
Okay.
Well, feel free to reach out and push.
What is store-fairification?
How about a little huge cloud system wrapper that's a verified store, perhaps on first
access?
Well how would you verify, like what would we use to verify them?
I mean the Nix data base hashes there.
I mean, we have to, obviously, have that hashed somewhere.
Yeah, yeah.
Yeah, so Elko is asking how would we verify, can we use just the hashes from the Nix store
to verify the files themselves?
And I'm not sure where the hashes are stored to be honest, but I mean.
Nix for Nix dv.sqlite.
Okay.
Yeah, but you need the signature somewhere of the hashes.
Okay.
I'm not sure.
I'd have to circle back and get back to you on that.
If there's no other questions, let's give it up again for Don.
Feel free to reach out.
Thank you.