[00:00.000 --> 00:14.960]  Okay, and so our final in-person talk, the last talk will be recorded playback.
[00:14.960 --> 00:21.520]  And now we have Ignas Foster talking about open-source and micro-S and the technical
[00:21.520 --> 00:22.520]  details.
[00:22.520 --> 00:25.760]  So thanks for coming.
[00:25.760 --> 00:28.240]  Thanks for the introduction and thanks for joining.
[00:28.240 --> 00:32.520]  As said, this talk will be about open-source and micro-S.
[00:32.520 --> 00:38.960]  We have several topics to cover, and not much time, so I'll just go over them quickly.
[00:38.960 --> 00:44.920]  We can't go into too much detail over here, but yeah, we'll see.
[00:44.920 --> 00:46.160]  Just one slide back.
[00:46.160 --> 00:47.160]  My name's Ignas.
[00:47.160 --> 00:53.840]  I'm working for SUSE, and yeah, I'm working as a research engineer.
[00:53.840 --> 00:55.760]  So what are today's topics?
[00:55.760 --> 01:00.240]  First of all, I'll give you a short introduction on open-source and micro-S itself.
[01:00.240 --> 01:03.760]  How is it built up, and on which foundations is it built up?
[01:03.760 --> 01:09.520]  Then we'll have a short look about the update and rollback mechanisms we are using there,
[01:09.520 --> 01:14.080]  and something we haven't talked a lot about today on configuration file management on
[01:14.080 --> 01:17.200]  our approach to handle these in image-based systems.
[01:17.200 --> 01:23.240]  We probably won't make it to the full-disk encryption and trusted boot part, but I can
[01:23.240 --> 01:27.480]  just refer to Len's talk from earlier today.
[01:27.480 --> 01:33.840]  So when we are talking about open-source and micro-S, we are basically talking about some
[01:33.840 --> 01:37.360]  product of the whole open-source universe.
[01:37.360 --> 01:41.400]  As the most known products, there are tumbleweed and leap.
[01:41.400 --> 01:46.560]  Tumbleweed is our rolling release distribution, and leap is our stable distribution.
[01:46.560 --> 01:51.480]  Stable not in a sense of tumbleweed crushes all the time, but stable in a sense of the
[01:51.480 --> 01:55.640]  API doesn't change all the time.
[01:55.640 --> 02:02.200]  Leap itself is based on Susie Lenox Enterprise, Susie Lenox Enterprise in turn is based on
[02:02.200 --> 02:07.520]  a tumbleweed snapshot which will then merge into a stable product.
[02:07.520 --> 02:13.560]  So when we are talking about micro-S, we still have the same products basically.
[02:13.560 --> 02:18.160]  We have open-source and micro-S which is based on tumbleweed, and we have open-source and
[02:18.160 --> 02:21.680]  leap micro which is based on leap.
[02:21.680 --> 02:26.840]  So basically we also have the same ingredients in there.
[02:26.840 --> 02:31.800]  We still have the RPM packages as a base.
[02:31.800 --> 02:39.080]  To be exact, exactly the same packages we have in the tumbleweed or leap distributions,
[02:39.080 --> 02:41.680]  and we are building up on these.
[02:41.680 --> 02:48.120]  You may know open-source as one of the large users of B3FS with leap and tumbleweed.
[02:48.120 --> 02:52.920]  You can use any file system you want, B3FS is just the default, but when we are talking
[02:52.920 --> 02:56.880]  about micro-S, then B3FS will be mandatory.
[02:56.880 --> 03:04.680]  You have to use it because we rely on these features really deeply.
[03:04.680 --> 03:11.640]  First of all, these are the snapshots.
[03:11.640 --> 03:19.040]  When we are creating an update, we will need those snapshots to have the forward and rollback
[03:19.040 --> 03:20.040]  handling.
[03:20.040 --> 03:26.320]  And B3FS has a huge advantage, we have heard that from a previous talk already, B3FS has
[03:26.320 --> 03:31.200]  copy and write functionality, so when we do an update, it will be the smallest update
[03:31.200 --> 03:33.200]  possible basically.
[03:33.200 --> 03:40.440]  We don't have AP partitioning where we have to have two partitions the same size but can
[03:40.440 --> 03:44.080]  have a minimal snapshot for that.
[03:44.080 --> 03:50.680]  So when we are talking about micro-S, if it's based on tumbleweed and leap, what's
[03:50.680 --> 03:53.520]  the actual difference compared to them?
[03:53.520 --> 03:58.320]  First of all, it's a read-only root file system, which is common for all the image-based
[03:58.320 --> 04:05.000]  systems we heard of today, and it contains a minimal package selection.
[04:05.000 --> 04:09.360]  Minimal doesn't necessarily mean that it's the smallest distribution possible, but a
[04:09.360 --> 04:14.680]  minimal package set for what it's trying to do.
[04:14.680 --> 04:17.760]  What it's trying to do is being a single-purpose system.
[04:17.760 --> 04:24.280]  It evolved from a project called OpenSusieCubic, which was a Kubernetes distribution, until
[04:24.280 --> 04:29.960]  we discovered that you can use it for so much more basically, what it's nowadays is
[04:29.960 --> 04:36.520]  a single-purpose system including running containers, but you can use it for any system
[04:36.520 --> 04:41.280]  or any use case you actually want to have.
[04:41.280 --> 04:47.240]  You also have a micro-S desktop, so your single-use case would be running a desktop there with
[04:47.240 --> 04:54.360]  flat packs as packages for using your applications.
[04:54.360 --> 05:05.520]  That single-purpose system approach also means you can install packages from the tumbleweed
[05:05.520 --> 05:15.120]  and leap distributions, but it's not designed to be used that way, but you should use containers
[05:15.120 --> 05:23.880]  for having your actual use for your actual load, or you can say you install one dedicated
[05:23.880 --> 05:28.560]  package, for example, if you want to run it as a mail server or whatever.
[05:28.560 --> 05:33.360]  If you have a look at our commercial site, we have the SusieLynx Enterprise Micro-Product.
[05:33.360 --> 05:36.000]  We even have a very limited package set.
[05:36.000 --> 05:41.760]  You can't even install all the packages you can from tumbleweed and leap to make that
[05:41.760 --> 05:51.040]  clear that it's really meant to be used in a quite restricted way, for example, for workloads,
[05:51.040 --> 05:56.640]  for example, for cluster nodes, or for embedded systems.
[05:56.640 --> 06:00.600]  Especially for embedded systems, we also have another very important point, namely it's
[06:00.600 --> 06:02.240]  a self-maintaining system.
[06:02.240 --> 06:07.120]  So you basically install it, and then you, in theory, can't forget about it, because
[06:07.120 --> 06:10.760]  it will just update and maintain itself.
[06:10.760 --> 06:16.880]  We'll have a short look at that later, how that works.
[06:16.880 --> 06:23.200]  So when we are talking about updates, I said it's a read-only system, so you still have
[06:23.200 --> 06:28.840]  to be able to update the system somehow if it's not image-based.
[06:28.840 --> 06:38.000]  For that, we are basically using the BDRIV-S snapshot functionality, which the BDRIV-S
[06:38.000 --> 06:41.640]  snapshots are snapshots of the root file system.
[06:41.640 --> 06:44.160]  That's the important part.
[06:44.160 --> 06:49.600]  If you have a workload, for example, containers, it's running in VAR.
[06:49.600 --> 06:55.400]  Of course, you also have ETC for your configuration files, but the root file system, basically
[06:55.400 --> 07:02.680]  what's below user, is that part that is actually snapshotted and read-only.
[07:02.680 --> 07:12.240]  So when we have an update, we will just create a new snapshot, the blue square on the second
[07:12.240 --> 07:17.200]  point, and perform the update in there.
[07:17.200 --> 07:23.640]  The blue snapshot is a BDRIV-S snapshot, so it has minimal overhead.
[07:23.640 --> 07:28.320]  It will just be created in microseconds.
[07:28.320 --> 07:32.200]  There's no performance penalty for doing that.
[07:32.200 --> 07:36.600]  And then the update will be performed in that snapshot, for example, by just calling the
[07:36.600 --> 07:41.860]  open source world superdup for doing a system update.
[07:41.860 --> 07:50.680]  If anything should go wrong in that step, then that snapshot can just be discarded again,
[07:50.680 --> 07:59.480]  and we are back at square one, we just have the currently running system, and the active
[07:59.480 --> 08:02.920]  system won't even have seen the new snapshot.
[08:02.920 --> 08:11.480]  If the update was successful, then that snapshot will be marked as the new default snapshot.
[08:11.480 --> 08:18.400]  That means if we reboot the system, just like AB partitioning, that snapshot will be used
[08:18.400 --> 08:22.800]  as the new root file system.
[08:22.800 --> 08:29.320]  We have a second step, but that will be later, just a second.
[08:29.320 --> 08:35.400]  Yeah, let me talk about it immediately.
[08:35.400 --> 08:40.320]  We have a second step at boot called health checker.
[08:40.320 --> 08:47.480]  If a health checker should detect that something is not working as expected, for example, a
[08:47.480 --> 08:54.160]  service you expected doesn't come up, then an automatic rollback will be performed again.
[08:54.160 --> 09:02.680]  We will be back at square one again, and it behaves as nothing would have been changed,
[09:02.680 --> 09:09.920]  and we can wait for the bug fix for whatever broke the actual update.
[09:09.920 --> 09:17.400]  So all the magic happens in an application called transactional update, and a new library
[09:17.400 --> 09:18.760]  called tuket.
[09:18.760 --> 09:21.560]  Transaction update was originally a shell script.
[09:21.560 --> 09:27.280]  It called all the open SUSE-specific stuff, for example, calling SIPR for all the package
[09:27.280 --> 09:37.760]  management, or calling MKInitad for rebuilding the Initad, everything which needed write
[09:37.760 --> 09:45.280]  access, so had to modify the root file system was just wrapped with that wrapper script.
[09:45.280 --> 09:50.000]  Out of that, a new library emerged called libtuket.
[09:50.000 --> 09:54.040]  That one is supposed to be platform agnostic.
[09:54.040 --> 10:02.480]  It includes C, C++, and divas bindings, and the only implementation currently is for BDRIF
[10:02.480 --> 10:08.280]  as in snapper, but if you want to, you could also just write another backend, it's prepared
[10:08.280 --> 10:15.640]  for that for more general snapshot management.
[10:15.640 --> 10:25.960]  We have, for example, a DNF plugin, or micro DNF plugin to be exact, so you could just
[10:25.960 --> 10:30.800]  use that one library for that.
[10:30.800 --> 10:37.680]  So I just said, if you want to activate that snapshot, you would usually reboot and have
[10:37.680 --> 10:39.680]  that snapshot activated.
[10:39.680 --> 10:44.960]  That's the atomic part, which is important for making sure that you don't touch the currently
[10:44.960 --> 10:53.280]  running system, but make sure the update is activated in one atomic step.
[10:53.280 --> 10:59.400]  With the next transaction update version, there will be also a new option called transaction
[10:59.400 --> 11:07.600]  update apply, what that one does is it will just mount basically user into the currently
[11:07.600 --> 11:08.600]  running system.
[11:08.600 --> 11:14.880]  That was possible if you heard the talk from Ludwig two talks ago.
[11:14.880 --> 11:20.040]  That was possible because of the user merge, because practically everything the system
[11:20.040 --> 11:30.560]  requires is below user, if it isn't, it should be changed in the future, but for micro s,
[11:30.560 --> 11:35.000]  that one's working very reliably already.
[11:35.000 --> 11:39.720]  If you just update or if you just installed a new package in the new snapshot, that one
[11:39.720 --> 11:46.360]  will be totally painless, because you just have the new package available immediately.
[11:46.360 --> 11:51.560]  If you change system services and I don't know what, then maybe you should still think
[11:51.560 --> 11:58.000]  about rebooting afterwards, because it's basically the same when running an update in a running
[11:58.000 --> 12:09.920]  system, basically an update or supered up in a running system.
[12:09.920 --> 12:14.920]  So I talked about rollback already, if you reboot the system and something doesn't work
[12:14.920 --> 12:19.360]  as expected, then it will perform an automatic rollback.
[12:19.360 --> 12:27.200]  Health checker, you can see the URL at the bottom, itself provides an interface basically
[12:27.200 --> 12:32.760]  for custom plugins, so you have to know yourself which services are running on your system,
[12:32.760 --> 12:35.680]  and you now have to know yourself how to check them.
[12:35.680 --> 12:42.000]  As said, it's a single purpose system, so I hope you know what you're doing.
[12:42.000 --> 12:48.720]  And you can just extend it with custom plugins, where you define what defines a correctly
[12:48.720 --> 12:51.640]  running system.
[12:51.640 --> 12:56.600]  So let's have a look at the time, excellent.
[12:56.600 --> 13:01.880]  Let's have a look at the configuration file management.
[13:01.880 --> 13:08.920]  When we are talking about read only root file systems, we are talking, we still have to
[13:08.920 --> 13:15.840]  have some directories of files writable, we've just seen that with the Ubuntu core talk,
[13:15.840 --> 13:22.960]  in contrast to Ubuntu core for open source and micro as all of var and all of etc are
[13:22.960 --> 13:23.960]  writable.
[13:23.960 --> 13:33.040]  Var is a separate subvolume, for etc we are using a mechanism called overlays, it's basically
[13:33.040 --> 13:42.880]  just the kernel overlays file system, and that one may need a bit more explanation.
[13:42.880 --> 13:50.760]  So if we have a look at the etcfs tab entry for etc, it looks overly long and overly complicated,
[13:50.760 --> 13:55.320]  but in the end it's just the two lines which are colored.
[13:55.320 --> 14:03.880]  We have the upper there, when we are just performing an update, that's the overlay of
[14:03.880 --> 14:07.080]  the next snapshot.
[14:07.080 --> 14:14.000]  The lower there is the directory of the currently running system, and to avoid having stacks
[14:14.000 --> 14:22.280]  over stacks over stacks, you'll just sync the state of the previous system as the base
[14:22.280 --> 14:24.640]  of the new snapshot.
[14:24.640 --> 14:30.160]  So that's the three layers we are talking about.
[14:30.160 --> 14:40.000]  If we have a look at that in detail, what's happening on configuration file changes?
[14:40.000 --> 14:48.920]  If we have the case of file 1, that's a really old file, that file has always existed, was
[14:48.920 --> 14:52.840]  never changed, that's no problem at all, basically we are just having a look from the
[14:52.840 --> 14:59.240]  top to the bottom, nothing is in upper there, nothing is in lower there 1, so the version
[14:59.240 --> 15:04.160]  of lower there 2 will be used.
[15:04.160 --> 15:09.560]  File 2 is interesting because that one has an old state, during the update that file
[15:09.560 --> 15:17.520]  was actually changed, it seems, either the package refreshed it or some postscript modified
[15:17.520 --> 15:19.440]  it or whatever happened to it.
[15:19.440 --> 15:25.000]  In any case, we have a new version of that file, so as soon as you reboot into the new
[15:25.000 --> 15:29.960]  snapshot, you will see that version of the file, older snapshots will see that version
[15:29.960 --> 15:33.480]  of the file.
[15:33.480 --> 15:39.880]  I mean, file 4, file 5 and file 6 are similar, those are quite easy, that one is new in the
[15:39.880 --> 15:45.320]  next snapshot, that one was new in the previous snapshot and those were modified in the snapshot
[15:45.320 --> 15:46.560]  before.
[15:46.560 --> 15:54.840]  The interesting thing is file 3, because that one exists both in the current snapshot
[15:54.840 --> 16:02.600]  and in the new snapshot and that may indicate a conflict, because as soon as you perform
[16:02.600 --> 16:08.520]  an update or create the new snapshot, the state of the file will be frozen basically
[16:08.520 --> 16:13.040]  and that one will be used as the base for the new snapshot.
[16:13.040 --> 16:20.680]  If you change the file before rebooting the system, then the new layer won't see that
[16:20.680 --> 16:29.640]  new file anymore and either you are operating on an old version of the file or you'd have
[16:29.640 --> 16:34.360]  to check, there will be a warning when you boot the system that there may be a conflict
[16:34.360 --> 16:44.080]  if the file in lower tier 1 is newer than the file in upper tier.
[16:44.080 --> 16:51.720]  That's our approach to handling current systems, current packages as we have them from vendors,
[16:51.720 --> 16:57.920]  from our own distribution and having a pragmatic approach to handling those configuration files,
[16:57.920 --> 17:01.520]  but in the end we don't want to have that at all.
[17:01.520 --> 17:10.040]  In the end we want to have the files which are packaged to be completely in user, we
[17:10.040 --> 17:13.480]  don't want to have them in ETC at all.
[17:13.480 --> 17:21.320]  Only changes that were explicitly done by the admin are supposed to be in ETC.
[17:21.320 --> 17:32.600]  That's where we are really approaching our final goal and for OpenSus and MicroS we almost
[17:32.600 --> 17:33.600]  achieved that goal.
[17:33.600 --> 17:40.840]  There are still some problematic files, for example ETCFS tab of course is such a singular
[17:40.840 --> 17:43.560]  file which you can't extend.
[17:43.560 --> 17:49.320]  If you have an application and also want to make it possible to split those files, we
[17:49.320 --> 17:52.520]  have developed a library called LibEcon.
[17:52.520 --> 17:56.960]  You can just embed it and we'll do all the things automatically just in case you're interested
[17:56.960 --> 17:57.960]  in that.
[17:57.960 --> 18:06.600]  We have made several pull requests and patches for projects which are used by OpenSus and
[18:06.600 --> 18:14.080]  MicroS to fix those use cases for us.
[18:14.080 --> 18:21.480]  So in the end I think we took a quite pragmatic approach using existing technologies to just
[18:21.480 --> 18:28.560]  be able to use your existing infrastructure, your existing packages and preserve the compatibility
[18:28.560 --> 18:37.040]  with all the legacy software which may exist out there and we are basically doing the opposite
[18:37.040 --> 18:51.880]  of most other distributions by extending the existing distributions to the new use cases.
[18:51.880 --> 18:58.720]  As I've heard, I just referred to Len's talk from earlier today, we do support full disk
[18:58.720 --> 19:06.360]  encryption, we do support trustee boot and we also have measurement support so let's
[19:06.360 --> 19:21.080]  better get to the questions part and trust in case you have some.
[19:21.080 --> 19:22.480]  Thank you very much for your talk.
[19:22.480 --> 19:28.800]  I was pestering your colleagues out here moments before this talk.
[19:28.800 --> 19:35.600]  Do you have any plans to migrate the old cubic docs to MicroS because first someone was new
[19:35.600 --> 19:44.400]  to transactional systems, it isn't immediately obvious how the classical administration style
[19:44.400 --> 19:46.960]  translates to transactional one?
[19:46.960 --> 19:51.200]  Yes, we have a new documentation server.
[19:51.200 --> 19:53.560]  The problem is MicroS doesn't include documentation.
[19:53.560 --> 20:00.160]  If you're in an embedded use case or using it for containers, the main pages are just
[20:00.160 --> 20:02.320]  overhead which is not needed.
[20:02.320 --> 20:07.920]  So we have a separate documentation server which will just provide all the documentation
[20:07.920 --> 20:11.280]  but I forgot which one it is.
[20:11.280 --> 20:16.000]  You have to ask me later, I'll search it for you.
[20:16.000 --> 20:24.400]  And I hope the search engines have picked it up meanwhile it's a few weeks old.
[20:24.400 --> 20:26.400]  There's a question in the chat.
[20:26.400 --> 20:32.040]  How is MicroS related to ALP?
[20:32.040 --> 20:42.480]  Okay, basically the current state of open SUSE ALP which is supposed to be the next generation
[20:42.480 --> 20:54.200]  of our SUSE links enterprise, whatever emerged out of that, basically open SUSE ALP is based
[20:54.200 --> 20:57.040]  on open SUSE MicroS in its current state.
[20:57.040 --> 21:03.720]  There will be more use cases than just the container use cases but that's the base which
[21:03.720 --> 21:06.720]  it was initially put onto.
[21:06.720 --> 21:13.160]  So a lot of things you can do with open SUSE MicroS, you can also do with open SUSE ALP.
[21:13.160 --> 21:18.680]  We even share parts of the documentation currently but open SUSE ALP is more than just open SUSE
[21:18.680 --> 21:19.680]  MicroS.
[21:19.680 --> 21:22.040]  It's just the same foundation currently.
[21:22.040 --> 21:28.400]  Questions?
[21:28.400 --> 21:29.400]  Going once?
[21:29.400 --> 21:30.400]  Going twice?
[21:30.400 --> 21:54.480]  Thank you very much.