Okay, and so our final in-person talk, the last talk will be recorded playback.
And now we have Ignas Foster talking about open-source and micro-S and the technical
details.
So thanks for coming.
Thanks for the introduction and thanks for joining.
As said, this talk will be about open-source and micro-S.
We have several topics to cover, and not much time, so I'll just go over them quickly.
We can't go into too much detail over here, but yeah, we'll see.
Just one slide back.
My name's Ignas.
I'm working for SUSE, and yeah, I'm working as a research engineer.
So what are today's topics?
First of all, I'll give you a short introduction on open-source and micro-S itself.
How is it built up, and on which foundations is it built up?
Then we'll have a short look about the update and rollback mechanisms we are using there,
and something we haven't talked a lot about today on configuration file management on
our approach to handle these in image-based systems.
We probably won't make it to the full-disk encryption and trusted boot part, but I can
just refer to Len's talk from earlier today.
So when we are talking about open-source and micro-S, we are basically talking about some
product of the whole open-source universe.
As the most known products, there are tumbleweed and leap.
Tumbleweed is our rolling release distribution, and leap is our stable distribution.
Stable not in a sense of tumbleweed crushes all the time, but stable in a sense of the
API doesn't change all the time.
Leap itself is based on Susie Lenox Enterprise, Susie Lenox Enterprise in turn is based on
a tumbleweed snapshot which will then merge into a stable product.
So when we are talking about micro-S, we still have the same products basically.
We have open-source and micro-S which is based on tumbleweed, and we have open-source and
leap micro which is based on leap.
So basically we also have the same ingredients in there.
We still have the RPM packages as a base.
To be exact, exactly the same packages we have in the tumbleweed or leap distributions,
and we are building up on these.
You may know open-source as one of the large users of B3FS with leap and tumbleweed.
You can use any file system you want, B3FS is just the default, but when we are talking
about micro-S, then B3FS will be mandatory.
You have to use it because we rely on these features really deeply.
First of all, these are the snapshots.
When we are creating an update, we will need those snapshots to have the forward and rollback
handling.
And B3FS has a huge advantage, we have heard that from a previous talk already, B3FS has
copy and write functionality, so when we do an update, it will be the smallest update
possible basically.
We don't have AP partitioning where we have to have two partitions the same size but can
have a minimal snapshot for that.
So when we are talking about micro-S, if it's based on tumbleweed and leap, what's
the actual difference compared to them?
First of all, it's a read-only root file system, which is common for all the image-based
systems we heard of today, and it contains a minimal package selection.
Minimal doesn't necessarily mean that it's the smallest distribution possible, but a
minimal package set for what it's trying to do.
What it's trying to do is being a single-purpose system.
It evolved from a project called OpenSusieCubic, which was a Kubernetes distribution, until
we discovered that you can use it for so much more basically, what it's nowadays is
a single-purpose system including running containers, but you can use it for any system
or any use case you actually want to have.
You also have a micro-S desktop, so your single-use case would be running a desktop there with
flat packs as packages for using your applications.
That single-purpose system approach also means you can install packages from the tumbleweed
and leap distributions, but it's not designed to be used that way, but you should use containers
for having your actual use for your actual load, or you can say you install one dedicated
package, for example, if you want to run it as a mail server or whatever.
If you have a look at our commercial site, we have the SusieLynx Enterprise Micro-Product.
We even have a very limited package set.
You can't even install all the packages you can from tumbleweed and leap to make that
clear that it's really meant to be used in a quite restricted way, for example, for workloads,
for example, for cluster nodes, or for embedded systems.
Especially for embedded systems, we also have another very important point, namely it's
a self-maintaining system.
So you basically install it, and then you, in theory, can't forget about it, because
it will just update and maintain itself.
We'll have a short look at that later, how that works.
So when we are talking about updates, I said it's a read-only system, so you still have
to be able to update the system somehow if it's not image-based.
For that, we are basically using the BDRIV-S snapshot functionality, which the BDRIV-S
snapshots are snapshots of the root file system.
That's the important part.
If you have a workload, for example, containers, it's running in VAR.
Of course, you also have ETC for your configuration files, but the root file system, basically
what's below user, is that part that is actually snapshotted and read-only.
So when we have an update, we will just create a new snapshot, the blue square on the second
point, and perform the update in there.
The blue snapshot is a BDRIV-S snapshot, so it has minimal overhead.
It will just be created in microseconds.
There's no performance penalty for doing that.
And then the update will be performed in that snapshot, for example, by just calling the
open source world superdup for doing a system update.
If anything should go wrong in that step, then that snapshot can just be discarded again,
and we are back at square one, we just have the currently running system, and the active
system won't even have seen the new snapshot.
If the update was successful, then that snapshot will be marked as the new default snapshot.
That means if we reboot the system, just like AB partitioning, that snapshot will be used
as the new root file system.
We have a second step, but that will be later, just a second.
Yeah, let me talk about it immediately.
We have a second step at boot called health checker.
If a health checker should detect that something is not working as expected, for example, a
service you expected doesn't come up, then an automatic rollback will be performed again.
We will be back at square one again, and it behaves as nothing would have been changed,
and we can wait for the bug fix for whatever broke the actual update.
So all the magic happens in an application called transactional update, and a new library
called tuket.
Transaction update was originally a shell script.
It called all the open SUSE-specific stuff, for example, calling SIPR for all the package
management, or calling MKInitad for rebuilding the Initad, everything which needed write
access, so had to modify the root file system was just wrapped with that wrapper script.
Out of that, a new library emerged called libtuket.
That one is supposed to be platform agnostic.
It includes C, C++, and divas bindings, and the only implementation currently is for BDRIF
as in snapper, but if you want to, you could also just write another backend, it's prepared
for that for more general snapshot management.
We have, for example, a DNF plugin, or micro DNF plugin to be exact, so you could just
use that one library for that.
So I just said, if you want to activate that snapshot, you would usually reboot and have
that snapshot activated.
That's the atomic part, which is important for making sure that you don't touch the currently
running system, but make sure the update is activated in one atomic step.
With the next transaction update version, there will be also a new option called transaction
update apply, what that one does is it will just mount basically user into the currently
running system.
That was possible if you heard the talk from Ludwig two talks ago.
That was possible because of the user merge, because practically everything the system
requires is below user, if it isn't, it should be changed in the future, but for micro s,
that one's working very reliably already.
If you just update or if you just installed a new package in the new snapshot, that one
will be totally painless, because you just have the new package available immediately.
If you change system services and I don't know what, then maybe you should still think
about rebooting afterwards, because it's basically the same when running an update in a running
system, basically an update or supered up in a running system.
So I talked about rollback already, if you reboot the system and something doesn't work
as expected, then it will perform an automatic rollback.
Health checker, you can see the URL at the bottom, itself provides an interface basically
for custom plugins, so you have to know yourself which services are running on your system,
and you now have to know yourself how to check them.
As said, it's a single purpose system, so I hope you know what you're doing.
And you can just extend it with custom plugins, where you define what defines a correctly
running system.
So let's have a look at the time, excellent.
Let's have a look at the configuration file management.
When we are talking about read only root file systems, we are talking, we still have to
have some directories of files writable, we've just seen that with the Ubuntu core talk,
in contrast to Ubuntu core for open source and micro as all of var and all of etc are
writable.
Var is a separate subvolume, for etc we are using a mechanism called overlays, it's basically
just the kernel overlays file system, and that one may need a bit more explanation.
So if we have a look at the etcfs tab entry for etc, it looks overly long and overly complicated,
but in the end it's just the two lines which are colored.
We have the upper there, when we are just performing an update, that's the overlay of
the next snapshot.
The lower there is the directory of the currently running system, and to avoid having stacks
over stacks over stacks, you'll just sync the state of the previous system as the base
of the new snapshot.
So that's the three layers we are talking about.
If we have a look at that in detail, what's happening on configuration file changes?
If we have the case of file 1, that's a really old file, that file has always existed, was
never changed, that's no problem at all, basically we are just having a look from the
top to the bottom, nothing is in upper there, nothing is in lower there 1, so the version
of lower there 2 will be used.
File 2 is interesting because that one has an old state, during the update that file
was actually changed, it seems, either the package refreshed it or some postscript modified
it or whatever happened to it.
In any case, we have a new version of that file, so as soon as you reboot into the new
snapshot, you will see that version of the file, older snapshots will see that version
of the file.
I mean, file 4, file 5 and file 6 are similar, those are quite easy, that one is new in the
next snapshot, that one was new in the previous snapshot and those were modified in the snapshot
before.
The interesting thing is file 3, because that one exists both in the current snapshot
and in the new snapshot and that may indicate a conflict, because as soon as you perform
an update or create the new snapshot, the state of the file will be frozen basically
and that one will be used as the base for the new snapshot.
If you change the file before rebooting the system, then the new layer won't see that
new file anymore and either you are operating on an old version of the file or you'd have
to check, there will be a warning when you boot the system that there may be a conflict
if the file in lower tier 1 is newer than the file in upper tier.
That's our approach to handling current systems, current packages as we have them from vendors,
from our own distribution and having a pragmatic approach to handling those configuration files,
but in the end we don't want to have that at all.
In the end we want to have the files which are packaged to be completely in user, we
don't want to have them in ETC at all.
Only changes that were explicitly done by the admin are supposed to be in ETC.
That's where we are really approaching our final goal and for OpenSus and MicroS we almost
achieved that goal.
There are still some problematic files, for example ETCFS tab of course is such a singular
file which you can't extend.
If you have an application and also want to make it possible to split those files, we
have developed a library called LibEcon.
You can just embed it and we'll do all the things automatically just in case you're interested
in that.
We have made several pull requests and patches for projects which are used by OpenSus and
MicroS to fix those use cases for us.
So in the end I think we took a quite pragmatic approach using existing technologies to just
be able to use your existing infrastructure, your existing packages and preserve the compatibility
with all the legacy software which may exist out there and we are basically doing the opposite
of most other distributions by extending the existing distributions to the new use cases.
As I've heard, I just referred to Len's talk from earlier today, we do support full disk
encryption, we do support trustee boot and we also have measurement support so let's
better get to the questions part and trust in case you have some.
Thank you very much for your talk.
I was pestering your colleagues out here moments before this talk.
Do you have any plans to migrate the old cubic docs to MicroS because first someone was new
to transactional systems, it isn't immediately obvious how the classical administration style
translates to transactional one?
Yes, we have a new documentation server.
The problem is MicroS doesn't include documentation.
If you're in an embedded use case or using it for containers, the main pages are just
overhead which is not needed.
So we have a separate documentation server which will just provide all the documentation
but I forgot which one it is.
You have to ask me later, I'll search it for you.
And I hope the search engines have picked it up meanwhile it's a few weeks old.
There's a question in the chat.
How is MicroS related to ALP?
Okay, basically the current state of open SUSE ALP which is supposed to be the next generation
of our SUSE links enterprise, whatever emerged out of that, basically open SUSE ALP is based
on open SUSE MicroS in its current state.
There will be more use cases than just the container use cases but that's the base which
it was initially put onto.
So a lot of things you can do with open SUSE MicroS, you can also do with open SUSE ALP.
We even share parts of the documentation currently but open SUSE ALP is more than just open SUSE
MicroS.
It's just the same foundation currently.
Questions?
Going once?
Going twice?
Thank you very much.