Hello, everyone, thanks for listening to my topic, URF5 system update and its future.
Due to my visa application issue, I didn't find a proper way to go to Russell on site,
therefore I have to upload a video for online presentation.
My name is Xiang Gao, and I've been working on Lin's kernel stuff for most seven years,
mainly focusing on the local file system stuff.
I guess URF5 is still familiar stuff for some people,
and here I try to give more useful information of URF5.
Hopefully it is helpful to everyone.
So URF5 stands for Enhanced Read-Only File System.
It was originally started in the late 2017, and available since Lin's 4.19.
It is designed to be a generic high-performance read-only file system,
with a very simple but effective core on-disk format design.
As a result, it almost has a powerful performance among the current in kernel read-only file system.
URF5 is kernel-mountable as a SQL-occurval format replacement of traditional CPIO and TAR.
And it is currently contributed by community lovers,
like Google Cloud, Dance, Corepad, Google, Huawei, Oracle, and more.
So as an option, URF5 supports core file LV4 and LVMA transparent data compression.
However, URF5 can live without compression as well.
It is targeted for various high-performance read-only solutions,
such as system partitions and APX for Android, smartphone,
and other embedded system, new CDs, and container images, as well as AR data sets.
There are many useful features which are actively underdeveloped,
so if you have any suggestions or contributions, always welcome.
There are several main use cases for URF5.
The first main use case is Android system partitions.
So Android has several read-only partitions, which behave as system firewall,
which means Android core can only be changed by way of an update.
So in this way, it has many benefits, such as it is easy for vendors to shape, distribute,
keep original signing for the images to its instance.
And it is easy to roll back to the original shaped state or do incremental updates.
And it is easy to check data traction or do data recovery even in a very low level,
such as hardware.
Also, it is easy for real storage devices to do hardware write protection and even more.
So why we introduce URF5?
Basically, because it exists, we read only compression solutions.
In kernel, we cannot meet our performance requirements,
but we need to do compression for our low-ended Android smartphones at that time.
That is why we design URFS and sort it out from the beginning.
We handled many basic common issues of generating read-only use cases to get high performance read-only file system.
In addition, it is good to switch APX to URFS on disk format as well.
Also, currently, APK is also another archival format.
If it becomes URFS-mountable, that may leverage the latest long-demand review polling as well.
So here is the first demo, which URFS is running on Android Cartofish emulator.
URFS is running on Android Cartofish emulator.
It is running on Android Cartofish emulator.
URFS is running on Android Cartofish emulator.
Our second main landed use case for URFS is container image with a user-space program called NEDAS.
Private fly NEDAS is a user-space example which uses in kernel URFS to leverage its functionality
to do faster container image distribution like lazy polling and data duplication across layers and images.
Currently, NEDAS can do lazy polling for NEDAS URFS images as well as star GZ images and original OCI images with an extra minimal index,
which is much similar to another project which is called SOCI.
For more details of NEDAS itself, you could also refer to another topic which is called NEDAS Image Service for Confidential Containers at Confidential Computing Devroom.
On the left-hand side, it is NEDAS architecture. You could see that an image format could be built with advanced features such as lazy polling,
data duplication, and native or OCI compatible modes.
And then a read-only file system for containers such as RunC, Cata, Cata CC, AMOs, and software package can be run by NEDAS-D with Linux,
URFS Fuse, VTARO-FS, and URFS over-FS cache with pitch cache sharing. On the left-hand side, it is some partners which are learning NEDAS and driving fly solutions.
The second demo, URFS is running with NEDAS 4 container images.
So firstly, the run NEDAS container.
And it finished in 16 seconds.
Then it runs OCI container.
You can see that it finished in 27 seconds. So that it induces times due to lazy polling.
So this is the third demo. In this demo, URFS is running with original OCI and NEDAS slimy indexes for lazy polling.
Note that this use case is still under development so that we could optimize it even further.
So firstly, we start already in OCI container.
And it costs 26 seconds.
And we build NEDAS zero run indexes for OCI images.
So next we start zero run OCI container.
And you can see it costs 21 seconds. And that is the file system.
You could see that the NEDAS slimy indexes is very small.
So next I will go into take some minutes to give a brief introduction of URFS core internals.
So as an effective with only internal solutions, core URFS on disk format is quite simple.
Almost all URFS on disk structures are well aligned and laid within your single block,
which means they are never across two blocks for performance.
So on the left-hand side, this is on disk superblock format,
which contains the overall file system statistics and the root I-node NID.
Each URFS I-node is aligned in an I-node slot so that the basic I-node information can be in the same block.
And they can be read and wins. On the right-hand side, there are URFS on disk I-node format.
Short extended attributes can be kept just next to the core on disk I-node as well as chunk,
compress, indexes, and inline data. Here is URFS on disk directory format.
URFS directories consist of several directory blocks.
Each block contains two parts called deranged part and name part so that with such on disk design,
URFS can do a name lookup with binary search,
which makes URFS more effective than other existing internal read-only file systems
and kept in a simple implementation.
So here is an overview of NIDUS use case. You can see that it has almost two parts.
One part is called bootstrap or also called primary device,
which has meta-blocks and data-blocks. So the meta-blocks could have super-blocks, I-nodes, and some inline data.
The other data blocks could have directory blocks or some blocks for regular files.
And the other part is called the blocks, which could have external data, which is separated with chunks
so that in such designs, blocks can be referred with the metadata.
And the details of compressed data is somewhat not quite trivial,
but it could be referred from the following links as well if you have more interest in.
Here is an URFS recent update. The first two features are called chunk-based file,
which could implement sparse files and data deduplicated plain files.
The next feature is called multiple devices and blocks,
so URFS image can refer to other external data as well.
Since 5.19, URFS over FS cache has been already landed,
which is already mentioned by some materials available online as the following links.
Since 6.1, URFS has been introduced a special I-node called piped I-node for TEL data
so that TEL data or the whole of files can be deduced or compressed together.
Also, since 6.1, URFS supported global compressed data deduplication by using ruling hush,
URFS over FS cache page cache sharing is still working in progress.
Here is a URFS compressed data deduplication test result.
You can see that compared with scratch FS, URFS is more space saving
by using this new optimization.
In the next year, we've already planted some new features.
Many of them are already working in progress, such as verification solutions
and data deduplicated encryption solutions.
We also have FS cache improvements together with bad dance folks,
such as failover, multiple demons, and directories, as well as demoners.
And more features can be referred to with the following links.
So that's all of my topic. Thanks for listening again.
If you have more interest in URFS, please kindly contact and join us. Thank you.
We actually have time for five minutes of question.
We don't know how bad the lag actually is, but we can type the question into the chat if you have one.
Or you can just ask it.
Thanks for the talk.
There was mention of self-contained verification solution.
Can you compare us with the severity and what advantages do you see in the verification solution you are working on?
I mean, you can also write it, yeah?
You have no idea what the lag is.
Sure. Do you have the app installed, like the FOSM app?
If you go into the schedule, then you just need to click a link.
Ah.
Thank you.
This is a text only development room, by the way, as you can see.
Thank you.
Just saying, just saying, just saying.