Okay, all right. Hello, everyone. Meet Bob. Bob's a software engineer and Bob just had
the idea of a century for a new startup. That's Gray Cat. Gray Cat is a service that
given the picture of a cat will reach in the same picture, but grayscale. Bob's really
excited about that and just got some funding to start working on that. So Bob gets started.
Chooses to write that in Rust because that's cool and trendy and use GitHub because that's the
standard. And so just writes the initial Rust boilerplate, the initial boilerplate for having
that built by GitHub Actions, then Git commit, Git push. The first TI runs green. That's wonderful.
Champagne. Now Bob decides to do something useful with that code and so he pulls in image2,
which is the Rust library, for doing image manipulations, uses that in the code, builds it.
The build is just fine locally. Wonderful. Now Git commit, Git push. The TI runs. It's not green.
It's complaining about some missing data files somewhere. Okay, so Bob, no big deal about
the software engineers. He knows how to use Google. So he searches, finds out that actually this
image2 library is mostly a wrapper around C++ library, which is OpenImage.io. And it turns out
that Bob had that installed on his laptop, but the CI runners don't, which is why it's failing
on the CI. But no big deal. Bob just tweaks a bit the CI config to install OpenImage.io before
running the build. And now the CI is green. Wonderful. So fast forward one year later, Bob
Crop has grown quite a bit and also did the tech stack of Graycat. It's getting a bit complex,
but no big deal. I mean, it's just a matter of having the right CI config file to make sure that
everything's gets installed. So the config file has grown a bit out of hand with 5,000 lines of
code, but it's not real problem. People just treat it as happened only whenever they need a new piece
of software. They just add that to the config file, add a few lines to install it. Generally forget
to remove it if they don't need it anymore. But I mean, no big deal. It works, right? And I mean,
it's not like anyone would want to maintain that because just the feedback loop of having to do a
change, push, push, wait for the CI, get the results. Yeah, now. Okay, better keep it like that. It
would need to move fast anyway. Because if some troubles, if we know them, like for instance,
when GitHub decides to update the base image of the runners because given the complexity of the
setup, obviously something's breaking here and there. No big deal. It takes a couple of days
sometimes to fix, sometimes a bit more blocking, blocking things a bit. That's annoying, but I
mean, have to move fast. No, one day big deal happens. Microsoft, it's now 2025. Microsoft is in a
slightly difficult financial situation and decides that this GitHub action thing is actually, yeah,
it's wasting money on that. So just decides to shut it down. Well, no, not such a big deal. I mean,
it's not like Bobcops married to GitHub actions. They just have to migrate this little config file
and use another CI provider. That's what they do. And three months after, they actually managed to
migrate that to a new CI config by a new provider. But now by the time competition has caught up,
Graycat is definitely beyond. Bobcops is leaking, bleeding money everywhere. And this is the end of
the Graycat dream. So very sad story for Bob. But could you have avoided that? So there's a bunch
of things that went wrong. You might have noticed. But most of these are just like natural
consequences of we want to do things the quick, quickly as possible. And so we just, like, we
can't take care of everything. But there's one practical choice that Bob did at the very beginning,
which was what caused the ultimate failure. And that was just being stuck on one single service
provider and being at its mercy. And Bob could have avoided that, hopefully. The first, like,
the elephant in the room, if I may say, is just the blue whale docker, which would have given Bob a
way to just have a, like, agnostic way of defining this CI environment that doesn't depend on GitHub.
And Bob could just have written a docker file instead of the CI config file. So now some things
that wouldn't have solved is that the feedback loop for a big docker file is not that much better
than that of the official CI. Docker's layering is great for caching when you don't need to touch the
latest lines of the docker file if you touch up things at the top. You're pretty much screwed up.
The other thing it would also not have solved is that unless you're very, very careful, it's easy
to just have lines here and there that might break at any time because upstream decides to change
something. But, I mean, this is okay. The big thing, the big problem that Bob would still have is that
the libopementimage.io issue we had at the beginning. I mean, Bob has his laptop, he's working on that.
On his laptop, he has the code for the project. He has tool chains needed to build the project. Then
the CI of the container has the same code for the project, checking for the same, from the same
commit, also has tool chains to build it, but not exactly the same ones. They are provided by a different
means, so obviously there's going to be some differences that might break things down on the line.
If you're lucky, it breaks your build. If you're unlucky, your build still succeeds, but then the
underlying something behaves slightly differently and you have absolutely no clue why. So what would
have been nice would have been to just have Bob's laptop and whatever is running the code in the
CI, use exactly the same tool chains. And there's an obvious solution to that, just ask Bob to do all
his development on the Docker container. That is great. The thing that's not great here is that Bob's
laptop doesn't only have tool chains encoded, also has his text editor, his config, his whole
development environment fine-tuned for years to just make Bob as efficient and productive as can be.
And if Bob has to develop in the container, he mostly can't access that easily. And now we get a
very sad and angry Bob and a very efficient Bob. So now there's one bit of the infrastructure that
I barely mentioned in passing, but not made much attention to. That bit is cargo, the rest package
manager. And the reason why I haven't really talked about it much is not that it's not important.
I mean, it's probably the most crucial part of the infrastructure because it's the thing that
pulls in the bulk of the dependencies of Gray Cat. But the reason I didn't talk about it that much
is that it just walked. I mean, I was talking about broken things because it's always funnier to
talk about broken things. And cargo was not broken, not nothing. And the reason cargo just walked,
I think it's like there's two reasons. The first one is that cargo has been very transparent of
the role. In the CI, Renskago to provide the dependencies for the build, and that's fast enough
for the CI that just walks fine. Bob on his machine runs cargo for that, and that's works that doesn't
prevent him from using all the rest of his tooling. And so Bob is happy using that. And beyond that,
cargo is also declarative. Like there's one file, two files, that exactly define the set of rest
dependencies that your code has. And when Bob runs cargo on his laptop, cargo just reads that file
and provides the exact environment needed. When the CI clones the project, Renskago, it reads that
same file and provides exactly the same environment. And that's why it walks. Now, there's one thing
that cargo doesn't do properly. And that thing is everything except rest packages. And yeah,
that's a problem because like the, yeah, that's why we have this open image, your problem. And so
it means that actually the declarative aspect of cargo is limited because it's declarative
to some point. You really have to read the terms and conditions for that. And so at that point,
like it would be great if only we could have something a bit like cargo, but more generic,
you know, that would be so awesome if only such a tool could exist. Okay, so meet my friend, Nick.
You can think of it, Nick, if you don't know about it in that context as something exactly like cargo
or NPM or whatever you want, except that it's fully generic. So you can use it to package and provide
your rest crates if you want, but you can also use it to provide the C libraries that your rest
crates depend on. And the C compiler used to compile these C libraries and the mini server
you're using to run the test or the PostgreSQL database you're using for your deployment server.
And so really now like declarative is not just a vain word. It is fully declarative up to the
lowest level you might want to think of. And so what could happen for Bob if he were using Nick's
is that he has this laptop with everything set up and then he can just use Nick's to provide these
tool chains. And because Nick's is transparent, it will it won't prevent Bob from still using his
editor with his all his tools just on top have the required tool chains to build the code.
And that makes a very, very happy Bob. And then the CI system can just use the same
Nick's with the same Nick's config files to get the tool chain. And then the CI builds exactly
the same thing as Bob on his laptop and the world is wonderful. So now assuming that,
now I mean I'm not assuming Bob is convinced that Nick's is the great thing and probably you are too,
right? And what would that look like in practice then for Bob to use Nick's? So Bob would essentially
drop a shell.nix file at the root of his environment saying, hey, I want a shell. So calling the make
shell function to get a deployment shell saying, I want in my shell this set of packages,
cargo, rest, open image, whatever you want. And the little bit of magic here is this PKGS thing
from which everything comes, which you import which points to Nick's package collection,
big repository with recipes for all the packages that exist in Nick's, which you can. So I've hidden
that, but you can import that in this Nick's package.nix file and pointing to a very specific
commit of the Nick's package's repository which will pin down every single version of all your
transitive dependencies. And now if Bob wants to use that, he can just run the Nick's shell command,
be dropped into a new shell in which, for instance, cargo will be available at a path that is
managed by Nick's. And once Bob exits the shell, no cargo anymore, that's what we wanted. But then,
I mean, Docker also does that. The bit that Nick has in terms of transparency, the extra coolness,
is that the shell doesn't tell anything about Vine, Bob's editor. But still, I mean,
Bob can still access his Vine, install Globody on his laptop, even inside the shell,
which is what you want for development because it's much, much, much nicer. And when Bob wants
some extra guarantees that really his shell is complete, he's not just accidentally leaking
things from his computer. You have an extra pure mod that you can use for building things with
more guarantees. And so that's Bob's machine, but this talk about CI. So let's look at the CI side
of things. On the CI, so Bob would still be using GitHub CI because that's the standard at that time.
But then beyond the initial mandatory boilerplate to just fetch the repository and all that,
there's only two things that Bob would need to have in his CI config file. The first one is install
Nick's because it's not yet part of the default GitHub image. And the second one is render build
within a Nick shell, a pure one because you really want to be strict at that point. And then if Bob
needs to migrate, all he needs to do is on his new CI system install Nick's again and copy that
exact Nick's shell command. And now Bob is fine. And we can all send great cat pictures over the
internet. So this was like scratching the edge of the tip of the iceberg. We could go a bit
much further, although I only had 15 minutes, so I won't cover all that. But the first thing we could
do to go a bit further is to improve the pinning situation. I mentioned like I hand wave this,
oh, you have this file that pins things down to a very specific version. There's ways to make that
much nicer and using a proper log file like all modern package managers would do so that you have
full control over when you want to upgrade. But it's also trivial to upgrade. More interestingly,
you can also use Nick's a bit further and not only use it to provide you some development or
some CI environment, but you can build your thing fully with Nick's, which gives you, I mean, first
time extra guarantees that, oh, this is really the right thing I've built in the right environment,
in the right set of dependencies. But more interestingly, now you can integrate that and
use Nick's a bit further, for instance, to build OCI images on top of that with only a few extra
lines of Nick's code, or use that to build AMIs for whichever cloud provider you want to use.
At that point, you probably want to care about caching. Nick's is pretty great at caching things
for that today. If you have a know-how build, it's really going to be a know-off and take you
a few seconds rather than the whatever time it takes to build the project. And you can also get
that cache to be distributed, meaning that your developers, if that's built on the CI,
then your developers can just reuse the pre-built results, which makes their life quite nicer.
And the last thing that could be done to go further is to use Nick's OS, which is a Nick's-based
Linux distribution, which follows that same philosophy of being purely declarative,
which means that you just have one config file that describes the whole system, and you just
rebuild the system based on that, which is useful for deployment, because that's infrastructure
at the core, but really down to the deepest level, not just scratched on top of something that existed
before that and was never meant to be that way. And you can also use that for testing things further.
There's in particular a really nice testing framework that allows you to always declaratively
declare a whole network of virtual machines that you can spawn, rinse, comment on, and then just
read the results. And that's really useful, whereas as soon as you start to want to test some
weird multi-tenant applications. So all that I've been talking about, Bob, but maybe a few words
about me, but you know who's talking to you. So I'm Theo Fan. I'm the leader of the Nick's Tech
Group at TWIG, which is the open source program office of Modisk Create, and it's pretty big on
Nick's, as you might have guessed. And I'm also a maintainer of Nick's and a member of the Foundation
Board, the Nick's Foundation Board. And you can reach me in all these places, and more concretely,
you can also meet me right here in the AW building where we have the Nick's West End. And that's all for me.