[00:00.000 --> 00:20.440]  Hello, everyone, thanks for listening to my topic, URF5 system update and its future.
[00:20.440 --> 00:27.960]  Due to my visa application issue, I didn't find a proper way to go to Russell on site,
[00:27.960 --> 00:33.240]  therefore I have to upload a video for online presentation.
[00:33.240 --> 00:39.400]  My name is Xiang Gao, and I've been working on Lin's kernel stuff for most seven years,
[00:39.400 --> 00:42.880]  mainly focusing on the local file system stuff.
[00:42.880 --> 00:48.600]  I guess URF5 is still familiar stuff for some people,
[00:48.600 --> 00:54.520]  and here I try to give more useful information of URF5.
[00:54.520 --> 00:59.480]  Hopefully it is helpful to everyone.
[00:59.480 --> 01:04.120]  So URF5 stands for Enhanced Read-Only File System.
[01:04.120 --> 01:14.640]  It was originally started in the late 2017, and available since Lin's 4.19.
[01:14.640 --> 01:19.400]  It is designed to be a generic high-performance read-only file system,
[01:19.400 --> 01:25.040]  with a very simple but effective core on-disk format design.
[01:25.040 --> 01:36.560]  As a result, it almost has a powerful performance among the current in kernel read-only file system.
[01:36.560 --> 01:47.280]  URF5 is kernel-mountable as a SQL-occurval format replacement of traditional CPIO and TAR.
[01:47.280 --> 01:51.360]  And it is currently contributed by community lovers,
[01:51.360 --> 02:00.600]  like Google Cloud, Dance, Corepad, Google, Huawei, Oracle, and more.
[02:00.600 --> 02:16.640]  So as an option, URF5 supports core file LV4 and LVMA transparent data compression.
[02:16.640 --> 02:24.840]  However, URF5 can live without compression as well.
[02:24.840 --> 02:30.200]  It is targeted for various high-performance read-only solutions,
[02:30.200 --> 02:35.640]  such as system partitions and APX for Android, smartphone,
[02:35.640 --> 02:48.040]  and other embedded system, new CDs, and container images, as well as AR data sets.
[02:48.040 --> 02:52.760]  There are many useful features which are actively underdeveloped,
[02:52.760 --> 03:05.240]  so if you have any suggestions or contributions, always welcome.
[03:05.240 --> 03:09.920]  There are several main use cases for URF5.
[03:09.920 --> 03:14.960]  The first main use case is Android system partitions.
[03:14.960 --> 03:22.040]  So Android has several read-only partitions, which behave as system firewall,
[03:22.040 --> 03:28.880]  which means Android core can only be changed by way of an update.
[03:28.880 --> 03:36.360]  So in this way, it has many benefits, such as it is easy for vendors to shape, distribute,
[03:36.360 --> 03:41.560]  keep original signing for the images to its instance.
[03:41.560 --> 03:51.920]  And it is easy to roll back to the original shaped state or do incremental updates.
[03:51.920 --> 04:02.480]  And it is easy to check data traction or do data recovery even in a very low level,
[04:02.480 --> 04:05.240]  such as hardware.
[04:05.240 --> 04:14.440]  Also, it is easy for real storage devices to do hardware write protection and even more.
[04:14.440 --> 04:17.560]  So why we introduce URF5?
[04:17.560 --> 04:23.880]  Basically, because it exists, we read only compression solutions.
[04:23.880 --> 04:31.000]  In kernel, we cannot meet our performance requirements,
[04:31.000 --> 04:39.640]  but we need to do compression for our low-ended Android smartphones at that time.
[04:39.640 --> 04:50.280]  That is why we design URFS and sort it out from the beginning.
[04:50.280 --> 05:05.840]  We handled many basic common issues of generating read-only use cases to get high performance read-only file system.
[05:05.840 --> 05:13.760]  In addition, it is good to switch APX to URFS on disk format as well.
[05:13.760 --> 05:19.080]  Also, currently, APK is also another archival format.
[05:19.080 --> 05:29.080]  If it becomes URFS-mountable, that may leverage the latest long-demand review polling as well.
[05:29.080 --> 05:38.080]  So here is the first demo, which URFS is running on Android Cartofish emulator.
[05:59.080 --> 06:09.080]  URFS is running on Android Cartofish emulator.
[06:09.080 --> 06:19.080]  It is running on Android Cartofish emulator.
[06:19.080 --> 06:46.080]  URFS is running on Android Cartofish emulator.
[06:46.080 --> 06:56.080]  Our second main landed use case for URFS is container image with a user-space program called NEDAS.
[06:56.080 --> 07:05.080]  Private fly NEDAS is a user-space example which uses in kernel URFS to leverage its functionality
[07:05.080 --> 07:13.080]  to do faster container image distribution like lazy polling and data duplication across layers and images.
[07:13.080 --> 07:27.080]  Currently, NEDAS can do lazy polling for NEDAS URFS images as well as star GZ images and original OCI images with an extra minimal index,
[07:27.080 --> 07:33.080]  which is much similar to another project which is called SOCI.
[07:33.080 --> 07:50.080]  For more details of NEDAS itself, you could also refer to another topic which is called NEDAS Image Service for Confidential Containers at Confidential Computing Devroom.
[07:50.080 --> 08:04.080]  On the left-hand side, it is NEDAS architecture. You could see that an image format could be built with advanced features such as lazy polling,
[08:04.080 --> 08:11.080]  data duplication, and native or OCI compatible modes.
[08:11.080 --> 08:29.080]  And then a read-only file system for containers such as RunC, Cata, Cata CC, AMOs, and software package can be run by NEDAS-D with Linux,
[08:29.080 --> 08:48.080]  URFS Fuse, VTARO-FS, and URFS over-FS cache with pitch cache sharing. On the left-hand side, it is some partners which are learning NEDAS and driving fly solutions.
[08:48.080 --> 09:04.080]  The second demo, URFS is running with NEDAS 4 container images.
[09:18.080 --> 09:45.080]  So firstly, the run NEDAS container.
[09:45.080 --> 10:14.080]  And it finished in 16 seconds.
[10:14.080 --> 10:28.080]  Then it runs OCI container.
[10:28.080 --> 10:41.080]  You can see that it finished in 27 seconds. So that it induces times due to lazy polling.
[10:41.080 --> 10:55.080]  So this is the third demo. In this demo, URFS is running with original OCI and NEDAS slimy indexes for lazy polling.
[10:55.080 --> 11:21.080]  Note that this use case is still under development so that we could optimize it even further.
[11:21.080 --> 11:32.080]  So firstly, we start already in OCI container.
[11:32.080 --> 11:41.080]  And it costs 26 seconds.
[11:41.080 --> 11:57.080]  And we build NEDAS zero run indexes for OCI images.
[11:57.080 --> 12:05.080]  So next we start zero run OCI container.
[12:05.080 --> 12:18.080]  And you can see it costs 21 seconds. And that is the file system.
[12:18.080 --> 12:26.080]  You could see that the NEDAS slimy indexes is very small.
[12:26.080 --> 12:36.080]  So next I will go into take some minutes to give a brief introduction of URFS core internals.
[12:36.080 --> 12:42.080]  So as an effective with only internal solutions, core URFS on disk format is quite simple.
[12:42.080 --> 12:49.080]  Almost all URFS on disk structures are well aligned and laid within your single block,
[12:49.080 --> 12:54.080]  which means they are never across two blocks for performance.
[12:54.080 --> 12:59.080]  So on the left-hand side, this is on disk superblock format,
[12:59.080 --> 13:06.080]  which contains the overall file system statistics and the root I-node NID.
[13:06.080 --> 13:17.080]  Each URFS I-node is aligned in an I-node slot so that the basic I-node information can be in the same block.
[13:17.080 --> 13:30.080]  And they can be read and wins. On the right-hand side, there are URFS on disk I-node format.
[13:30.080 --> 13:39.080]  Short extended attributes can be kept just next to the core on disk I-node as well as chunk,
[13:39.080 --> 13:49.080]  compress, indexes, and inline data. Here is URFS on disk directory format.
[13:49.080 --> 13:54.080]  URFS directories consist of several directory blocks.
[13:54.080 --> 14:07.080]  Each block contains two parts called deranged part and name part so that with such on disk design,
[14:07.080 --> 14:12.080]  URFS can do a name lookup with binary search,
[14:12.080 --> 14:21.080]  which makes URFS more effective than other existing internal read-only file systems
[14:21.080 --> 14:30.080]  and kept in a simple implementation.
[14:30.080 --> 14:41.080]  So here is an overview of NIDUS use case. You can see that it has almost two parts.
[14:41.080 --> 14:50.080]  One part is called bootstrap or also called primary device,
[14:50.080 --> 15:05.080]  which has meta-blocks and data-blocks. So the meta-blocks could have super-blocks, I-nodes, and some inline data.
[15:05.080 --> 15:20.080]  The other data blocks could have directory blocks or some blocks for regular files.
[15:20.080 --> 15:41.080]  And the other part is called the blocks, which could have external data, which is separated with chunks
[15:41.080 --> 15:55.080]  so that in such designs, blocks can be referred with the metadata.
[15:55.080 --> 16:00.080]  And the details of compressed data is somewhat not quite trivial,
[16:00.080 --> 16:13.080]  but it could be referred from the following links as well if you have more interest in.
[16:13.080 --> 16:22.080]  Here is an URFS recent update. The first two features are called chunk-based file,
[16:22.080 --> 16:33.080]  which could implement sparse files and data deduplicated plain files.
[16:33.080 --> 16:38.080]  The next feature is called multiple devices and blocks,
[16:38.080 --> 16:44.080]  so URFS image can refer to other external data as well.
[16:44.080 --> 16:54.080]  Since 5.19, URFS over FS cache has been already landed,
[16:54.080 --> 17:04.080]  which is already mentioned by some materials available online as the following links.
[17:04.080 --> 17:14.080]  Since 6.1, URFS has been introduced a special I-node called piped I-node for TEL data
[17:14.080 --> 17:22.080]  so that TEL data or the whole of files can be deduced or compressed together.
[17:22.080 --> 17:36.080]  Also, since 6.1, URFS supported global compressed data deduplication by using ruling hush,
[17:36.080 --> 17:46.080]  URFS over FS cache page cache sharing is still working in progress.
[17:46.080 --> 17:52.080]  Here is a URFS compressed data deduplication test result.
[17:52.080 --> 18:03.080]  You can see that compared with scratch FS, URFS is more space saving
[18:03.080 --> 18:11.080]  by using this new optimization.
[18:11.080 --> 18:15.080]  In the next year, we've already planted some new features.
[18:15.080 --> 18:20.080]  Many of them are already working in progress, such as verification solutions
[18:20.080 --> 18:24.080]  and data deduplicated encryption solutions.
[18:24.080 --> 18:31.080]  We also have FS cache improvements together with bad dance folks,
[18:31.080 --> 18:37.080]  such as failover, multiple demons, and directories, as well as demoners.
[18:37.080 --> 18:44.080]  And more features can be referred to with the following links.
[18:44.080 --> 18:49.080]  So that's all of my topic. Thanks for listening again.
[18:49.080 --> 19:00.080]  If you have more interest in URFS, please kindly contact and join us. Thank you.
[19:00.080 --> 19:04.080]  We actually have time for five minutes of question.
[19:04.080 --> 19:12.080]  We don't know how bad the lag actually is, but we can type the question into the chat if you have one.
[19:12.080 --> 19:33.080]  Or you can just ask it.
[19:33.080 --> 19:34.080]  Thanks for the talk.
[19:34.080 --> 19:38.080]  There was mention of self-contained verification solution.
[19:38.080 --> 19:53.080]  Can you compare us with the severity and what advantages do you see in the verification solution you are working on?
[19:53.080 --> 19:58.080]  I mean, you can also write it, yeah?
[19:58.080 --> 20:09.080]  You have no idea what the lag is.
[20:09.080 --> 20:13.080]  Sure. Do you have the app installed, like the FOSM app?
[20:13.080 --> 20:17.080]  If you go into the schedule, then you just need to click a link.
[20:17.080 --> 20:29.080]  Ah.
[20:29.080 --> 20:54.080]  Thank you.
[20:54.080 --> 21:23.080]  Thank you.
[21:23.080 --> 21:51.080]  This is a text only development room, by the way, as you can see.
[21:51.080 --> 22:20.080]  Thank you.
[22:20.080 --> 22:49.080]  Thank you.
[22:49.080 --> 23:01.080]  Just saying, just saying, just saying.