[00:00.000 --> 00:09.920]  At our final talk for this session, we have Pierre here. He's going to discuss about
[00:09.920 --> 00:16.400]  Loup, a tool that he and we have been using to measure compatibility for different OSs.
[00:16.400 --> 00:20.000]  Pierre, you have the floor. Thank you, Resvan, and thanks everyone for
[00:20.000 --> 00:26.240]  attending my talk. This is joint work with a bunch of colleagues and students, including
[00:26.240 --> 00:31.520]  Hugo, my PhD student. He's a key player behind his work. I'm just, you know, getting all the
[00:32.800 --> 00:37.280]  medialization, but he has built all this stuff, so all the credits go to him.
[00:37.840 --> 00:44.000]  So in this brief talk, I want to speak a bit about application
[00:44.560 --> 00:51.920]  compatibility for custom operating systems. So I guess most of you don't need to be convinced
[00:51.920 --> 00:58.000]  that we still need custom operating systems today. When I say custom, I mean both, like
[00:58.000 --> 01:03.760]  research operating systems and prototypes operating systems from the industry, right?
[01:05.600 --> 01:14.240]  The thinking that Linux has solved everything is not true, in my opinion. We still need things
[01:14.240 --> 01:19.440]  like Unicraft if you want to go fast, or if you want to specialize like crazy, we still need things
[01:19.440 --> 01:25.600]  like Rusty-Armet if you want security, or SEL4, so we still need custom operating systems,
[01:26.560 --> 01:32.960]  and the thing is with these operating systems, they're only as good as the application that they
[01:32.960 --> 01:42.720]  can run, right? So compatibility is key. Compatibility with existing application is extremely important.
[01:42.720 --> 01:48.560]  If you want to build a community, you want your user to go to your website, compile your custom OS,
[01:48.560 --> 01:55.120]  and then try some of their favorite applications, or try some of the highly popular applications
[01:55.120 --> 02:00.720]  in a given application domain like Enginings or Redis for cloud, if you want to attract
[02:00.720 --> 02:07.120]  sponsors or investors, or even if you, like me, are scientists, you want to gather some early
[02:07.120 --> 02:12.400]  numbers to make a publication, well, you need to do that on standard applications, right?
[02:12.400 --> 02:22.560]  So compatibility is important, and another argument would be like how many times did you hear the
[02:22.560 --> 02:28.800]  one POSIX spoken today, right? There were some slides, there was POSIX like three or four times
[02:28.800 --> 02:38.400]  written in a single slide. So compatibility is important, and it can be achieved in a few different
[02:38.400 --> 02:46.160]  ways as we have seen with Simon. But one important thing to note is, in my opinion, porting is not
[02:46.160 --> 02:52.480]  sustainable. So porting is what many of us do. We build a custom operating system, and then we take
[02:52.480 --> 02:58.880]  Redis, and obviously it doesn't work as is with our operating system, so we modify Redis a bit,
[02:58.880 --> 03:04.240]  we disable some features because we know that they make our OS crash, and then we have Redis
[03:04.240 --> 03:08.880]  like a version customized for our operating system. This is not sustainable because you can't
[03:08.880 --> 03:14.480]  maintain like a branch of Redis for every operating system out there, right? In the long
[03:14.480 --> 03:24.080]  term, it doesn't work so well. So porting also basically means that you as a OS developer,
[03:24.080 --> 03:30.560]  you ask the users of your application to make some effort to the application developers,
[03:30.560 --> 03:34.640]  they need to make some effort to be compatible with your operating system. This doesn't work,
[03:34.640 --> 03:41.440]  nobody is ready to make that kind of effort. Maybe if you give them 10x performance speed up,
[03:41.440 --> 03:48.640]  but this is unrealistic. So what you want to do is, once again, in my opinion,
[03:50.000 --> 03:55.120]  as an OS developer, you want to provide compatibility as transparently as possible.
[03:55.120 --> 04:05.040]  And this means you emulate a popular operating system, for example Linux, or a popular abstraction
[04:05.040 --> 04:13.200]  like POSIX or the standard C library, and then you can be compatible at three different levels.
[04:13.200 --> 04:22.240]  The first level is source level or API level compatibility. So you ask the users to compile
[04:22.240 --> 04:31.040]  their application code against the sources of your kernel, in the case of a unique kernel.
[04:31.040 --> 04:36.880]  So this is, you're still asking some effort from the users, right? In many scenarios,
[04:36.880 --> 04:41.280]  you don't have access to sources, right? If you have proprietary binary or pre-compiled
[04:41.280 --> 04:48.640]  binaries, well, you can't have source level compatibility. So it's not perfect and binary
[04:48.640 --> 04:55.120]  compatibility is generally a more, let's say, pure version of compatibility. And there are many
[04:55.120 --> 05:01.840]  two ways to achieve it. You can do that at the level of the standard C library, like OSV. So you
[05:01.840 --> 05:12.800]  will dynamically link your kernel plus a standard C library against your application, compile as a
[05:12.800 --> 05:21.040]  position independent executable or as a shared library itself. This is great, but if the application
[05:21.040 --> 05:26.480]  is making directly system calls to the kernel without going through the standard C library,
[05:26.480 --> 05:31.120]  well, once again, it doesn't work, right? And as a matter of fact, I have counted
[05:31.120 --> 05:36.720]  more than 500 executables in the Debian repository that contain the C score instruction, right? So
[05:36.720 --> 05:42.720]  they make C scores directly to the kernel. They don't go through the C library. Go, for example,
[05:42.720 --> 05:48.720]  is making most of its C score, put directly to the kernel and not through the C library. So what
[05:48.720 --> 05:54.800]  you want to do is to be compatible at the level of the system calls. So your kernel needs to
[05:54.800 --> 06:01.440]  emulate the C score API that Linux is providing. This is the most transparent way of achieving
[06:01.440 --> 06:10.480]  compatibility. Now, this is scary, right? Linux has more than 350 system calls. Do we need to
[06:10.480 --> 06:18.160]  implement them all? Will we be and aren't we going to reimplement Linux by doing so? And some of them
[06:18.160 --> 06:24.000]  are extremely scary by themselves, right? You have like hundreds of IO controls and each of
[06:24.000 --> 06:32.480]  them probably require its own implementation. The Linux API even goes beyond system calls. You
[06:32.480 --> 06:40.720]  have things like slash proc slash devs that are, you know, used by many applications. Like the
[06:40.720 --> 06:49.120]  first thing a muscle binary does when it runs is to look in, I believe it's slash proc or slash
[06:49.120 --> 06:53.680]  this to get the size of the terminal, right? So you need to have emulation for this part of the
[06:53.680 --> 07:01.840]  API too. And this, because it seems like a big engineering effort, this creates, it hinders
[07:01.840 --> 07:09.360]  the development of custom operating systems. So this is inspired by the keynote by Timati Roscoe
[07:09.360 --> 07:17.680]  at ATC and OSDI 2021. We looked at all the papers. So these are top tier operating systems
[07:17.680 --> 07:25.760]  conferences. And we look over the past 10 years, over a total of more than 1,000 papers,
[07:25.760 --> 07:31.680]  how many were about proposing a new operating system as opposed to things like security or
[07:31.680 --> 07:38.240]  machine learning. And among them, how many were just hacking Linux versus proposing an actual
[07:38.240 --> 07:45.360]  operating system implemented from scratch. And the numbers are similar to what we saw earlier,
[07:45.360 --> 07:51.520]  right? You have just a very, very, very few papers proposing a new operating system because
[07:51.520 --> 07:58.800]  it's a significant engineering effort. And part of the effort is to be providing compatibility
[07:58.800 --> 08:04.240]  to an application like Apache already to get a few numbers at the end of the paper, right?
[08:04.240 --> 08:12.720]  So this is the problem. Now, the particular problem that I want to talk about is how
[08:14.560 --> 08:19.600]  I'm sure several people in this forum have attempted to build some form of compatibility
[08:19.600 --> 08:26.000]  layer for an operating systems. And we are all kind of working on the same thing in parallel
[08:26.560 --> 08:33.680]  with some form of ad hoc processes that may benefit from some optimization. So I've just listed
[08:33.680 --> 08:40.960]  here a few projects that have a Cisco level binary compatibility layers, but actually there are many
[08:40.960 --> 08:50.160]  more. And from what I understand, it is a very organic process. So first of all, it is application
[08:50.160 --> 08:56.480]  driven, right? People have a few sets of application in mind that they want to support. If you are
[08:56.480 --> 09:02.400]  doing cloud, you want to support the user's suspect, ready, Apache, whatever. And the process
[09:02.400 --> 09:10.320]  basically looks like that. You take an app, you try to run it on top of your operating system.
[09:10.320 --> 09:16.960]  Obviously, it fails. You investigate. You're like, oh, I'm missing the implementation for this system
[09:16.960 --> 09:22.640]  code. So you implement whatever operating system features are required to fix that particular
[09:22.640 --> 09:29.840]  issue, rinse and repeat until the app is working. And then you go to the next app. So it's a very
[09:30.640 --> 09:35.840]  intuitive and organic process. So when I built the Armitage, this is exactly what I was doing.
[09:35.840 --> 09:46.720]  So something that comes to mind is, can't we have some form of generic compatibility layers
[09:46.720 --> 09:54.640]  that we could plug? Like something a bit like New Lib that would provide a generic interface.
[09:55.280 --> 10:00.880]  And I believe it's not really possible because most of this implementation to support the system
[10:00.880 --> 10:06.640]  code is very specific to whatever operating system you are using. And it's not clear if a
[10:06.640 --> 10:15.120]  generic compatibility layer can be achieved. But can we still somehow optimize that process?
[10:16.880 --> 10:25.280]  Some have tried static analysis. So they take the binary of the application they want to support
[10:25.280 --> 10:28.720]  and they look, okay, so what are the system codes that are made by these applications?
[10:28.720 --> 10:38.080]  So this has been done in the best paper in Eurosis 2016, analyzed all the binaries from
[10:38.080 --> 10:46.080]  Ubuntu, I believe it was 14 or four repositories. And they concluded that every Ubuntu installation,
[10:46.080 --> 10:53.360]  including the smallest one, require more than 200 system codes and 200 IO controls,
[10:53.360 --> 11:00.640]  five controls, and PRCTL codes, and hundreds of 2.0 files. So this doesn't help. It is still
[11:00.640 --> 11:13.520]  quite scary. It still represents a gigantic engineering effort. But do we want full compatibility
[11:13.520 --> 11:22.160]  with Ubuntu installation? In the end, especially in the early stage of the development of an
[11:22.160 --> 11:28.640]  operating system, you just want to get a few applications up and running. And do you even
[11:28.640 --> 11:34.160]  100% compatibility? When I write a paper, I don't really care if everything is stable. I just want
[11:34.160 --> 11:44.000]  to get some numbers. So isn't there a better way? And obviously, maybe you think about, yeah,
[11:44.000 --> 11:48.800]  let's do dynamic analysis. Let's run the applications that we want to support.
[11:48.800 --> 11:53.600]  We send them some input that we want to support, like I'm running
[11:53.600 --> 11:58.800]  engineering and I'm submitting some HTTP for something like that. And then we trace the system
[11:58.800 --> 12:04.800]  codes that are done. So this is going to give us a subset of the system codes that can be identified
[12:04.800 --> 12:11.600]  through static analysis that has a tendency to overestimate. So with this trace, the engineering
[12:11.600 --> 12:17.600]  effort to support an application and a set of input is a bit lower. But it's still not a panacea
[12:17.600 --> 12:23.600]  because it's not taking into account two things that we do when we implement compatibility layers.
[12:23.600 --> 12:32.320]  So this is my code. Don't judge me. One thing that I did with Hermitux was at some point,
[12:32.320 --> 12:38.080]  it was an app that was calling MNCore to check if some page of memory wasn't swapped or not.
[12:38.080 --> 12:43.840]  It has actually, you know, there is no swap in most unique kernels. So it really didn't matter
[12:43.840 --> 12:50.880]  to implement this. So you know this means operation not supported. So stopping a system
[12:50.880 --> 12:55.440]  code is just saying, yeah, we don't support it. And you cross your finger that the application
[12:55.440 --> 13:01.040]  has some kind of fallback path to do something else if the system code fails. And it works
[13:01.040 --> 13:07.120]  in some cases. And then we can do something even more nasty. Don't judge me again. You can fake
[13:07.120 --> 13:15.680]  the success of a system code, right? Surprisingly, in some situation, returning a success code,
[13:16.880 --> 13:21.840]  even if the system code doesn't have any implementation in your operating system,
[13:21.840 --> 13:27.120]  it's going to work in some cases. You know, I'll tell a bit more about why this works sometimes.
[13:27.120 --> 13:32.800]  So stubbing and faking lets you implement even less system calls than what you would trace with
[13:32.800 --> 13:39.520]  it trace. So in the end, you know, if you want to support an app or a set of application in your
[13:39.520 --> 13:44.560]  custom operating system, the amount of system calls that you actually need to implement.
[13:46.000 --> 13:51.840]  So obviously, it's smaller than the entire Linux SQL API. Static binary analysis will,
[13:53.040 --> 13:56.960]  on the binaries of the applications you want to support, will identify a subset of that.
[13:56.960 --> 14:02.960]  Still pretty big. It's an overestimate. Source analysis gets you more precise results.
[14:03.840 --> 14:11.680]  But it is pretty hard to achieve. And it is still overestimating. S trace will give you,
[14:11.680 --> 14:17.360]  once again, a subset. Things start to look better. And among these trace by S trace,
[14:17.360 --> 14:22.640]  you actually don't need to implement everything. You can stub and fake some of this SQL. So
[14:22.640 --> 14:32.240]  can we measure that? Yes, with loop. So loop means magnifying glass in French. It's a tool
[14:32.240 --> 14:37.120]  that was built by Hugo, my student, and it's some kind of super S trace that is measuring
[14:37.120 --> 14:42.080]  the system calls that are required to support an application. And that can also tell you which
[14:42.080 --> 14:51.040]  one you can stub and which one you can fake. So we used it to build a database of measurements
[14:51.040 --> 14:59.520]  for a relatively large set of applications. And with loop, if you give me a description
[14:59.520 --> 15:03.920]  of your operating system, basically the list of system calls that you already support,
[15:03.920 --> 15:08.800]  and you give me the list of applications that you would like to support, we run them through loop
[15:08.800 --> 15:14.480]  and loop can derive a support plan, which basically will tell you, okay, for this set of target
[15:14.480 --> 15:21.760]  application. And given the set of system calls that you already support, what is the optimized
[15:21.760 --> 15:27.280]  order of system calls to implement to support as many applications as soon as possible? Okay,
[15:27.280 --> 15:31.360]  so I will give you an example of support plan by the end of the presentation.
[15:32.880 --> 15:38.080]  So from the user point of view, loop needs two things to perform its measurement on a given
[15:38.080 --> 15:45.280]  application. You give it a Docker file that is describing how you want to build and run the
[15:45.280 --> 15:51.920]  application for which you want to measure the system calls needed. And optionally, you may need
[15:51.920 --> 15:56.560]  an input workload. Think about a web server. It's not going to call many system calls until you
[15:56.560 --> 16:03.440]  actually start to send requests to it. Loop will instantiate the application, launch it on a,
[16:03.440 --> 16:09.600]  you know, standard Linux kernel and analyze the system calls that are done and with a few tricks
[16:09.600 --> 16:17.920]  we'll be able to know which ones can be faked or stubbed. The results are, it's basically just a
[16:17.920 --> 16:26.640]  CSV file for each system call that is made by the application. Can it be faked? Can it be stubbed? Or
[16:26.640 --> 16:34.000]  does it require a full implementation? We start that in a database and later, so, you know,
[16:34.000 --> 16:40.000]  we populate the database with as many measurements as possible. And this database can
[16:40.960 --> 16:45.520]  given the list of these calls that is already supported by your operating systems, give you
[16:45.520 --> 16:50.800]  like some form of optimized super plan given which of the applications you want to support.
[16:50.800 --> 17:00.720]  Okay, so how does it work? When loop runs the application, first it does a quick pass of S-Trace
[17:01.440 --> 17:06.960]  to measure all the system calls that are done by the application and then for each system call
[17:06.960 --> 17:14.640]  that we identify, we use SecComp to hook into the execution of each of the system calls and
[17:14.640 --> 17:24.240]  rather than actually executing them through the Linux kernel, we emulate the fact that
[17:24.240 --> 17:29.760]  the Cisco is stubbed, so we just return EnoSys without executing the Cisco. We can also emulate
[17:29.760 --> 17:35.680]  the fact that the Cisco is faked, we return zero. And then we check if the application
[17:37.520 --> 17:43.520]  works or not following the stubbing or the faking of this particular Cisco. And then we do that
[17:43.520 --> 17:50.560]  for each system call that we have identified with S-Trace. How do we actually check for the success
[17:50.560 --> 17:57.440]  of the execution of the application? So we identified two types of apps. Some we call them
[17:57.440 --> 18:02.160]  run to completion. There'll be something like FIO when you know you start FIO, it runs for one
[18:02.160 --> 18:09.200]  minute and then it exits outputting some kind of some stuff on the standard output. So with run to
[18:09.200 --> 18:15.920]  completion apps, we run the app instrumented with loop, we check its exit code. If it's different
[18:15.920 --> 18:20.480]  from zero, we consider that the run was a failure, could have been killed by a signal or things like
[18:20.480 --> 18:28.480]  that. And we can also run a script optionally in addition to that after each run of the application
[18:28.480 --> 18:35.680]  to check its standard output. We can grab for error values, we can grab for success printing,
[18:35.680 --> 18:42.080]  something like, you know, 50 requests per second have been achieved. The files that may have been
[18:42.080 --> 18:48.640]  created by the application and so on. And then another type of application is client servers.
[18:49.200 --> 18:55.520]  So with client servers, we run the app instrumented by loop and in parallel we run a workload,
[18:55.520 --> 19:02.560]  could be WRK, HTT path, the Redis benchmark for Redis and so on. And we check the success of both,
[19:02.560 --> 19:08.000]  we check that the app doesn't crash, generally servers are not supposed to exit. So we check that
[19:08.000 --> 19:13.040]  it doesn't crash and we check the success of the workload. Like, you know, if Redis benchmark returns
[19:13.040 --> 19:19.120]  something different than zero, probably something went wrong. And then we are able to see, okay,
[19:19.120 --> 19:24.640]  so I'm currently trying to stub the read system call, is the application succeeded or not?
[19:24.640 --> 19:38.480]  So really the database, let me check the time, okay. And we analyzed the results. So these results
[19:38.480 --> 19:45.200]  are made on a relatively small database of about 12 highly popular, sorry, 15 highly popular cloud
[19:45.200 --> 19:53.520]  applications. So this is just a subset. So what you have on the y-axis is a number of system calls
[19:53.520 --> 20:04.800]  that are identified by static analysis in purple on the binary, on the sources in yellow. And then
[20:04.800 --> 20:10.960]  dynamic analysis. And we run for each of these applications, both the standard benchmarks,
[20:10.960 --> 20:17.440]  that will be Redis benchmark for Redis, WRK for engineering, and so on. And we also run the
[20:17.440 --> 20:25.040]  entire test suite. So the key idea with the test suite is if you, you know, support, I mean,
[20:25.040 --> 20:30.720]  if you measure what's going on during the entire test suite, you get a very good idea of what are
[20:30.720 --> 20:35.600]  all the possible system calls that could be done by the application. Obviously, you need to assume
[20:35.600 --> 20:41.120]  that the test suite has a good coverage, but it is the case with these very popular applications.
[20:41.120 --> 20:49.040]  And, and what we see is, first of all, you know, static analysis overestimates. This is not very
[20:49.040 --> 20:53.360]  surprising. The amount of system calls that is identified by static analysis is relatively high
[20:53.360 --> 20:59.680]  compared to what we get with dynamic analysis. And if something interesting, too, is that the
[20:59.680 --> 21:06.160]  amount of system calls that can be stirred or faked, so the grain bits on the dynamic analysis
[21:06.160 --> 21:12.160]  pass, it is actually quite non-negligible, right? So, so what this means is that if you want to
[21:12.160 --> 21:18.320]  support Redis with a Redis benchmark, where binary level static analysis tells you that you
[21:18.320 --> 21:24.320]  should implement 100 system calls, if you just want to run the Redis benchmark to get, you know,
[21:24.320 --> 21:30.000]  performance numbers for your paper, you actually need to implement just 20, right? So that's what,
[21:30.000 --> 21:37.200]  like, divided by five, right? And if you want to pass the entire test suite of Redis, you need
[21:37.200 --> 21:43.920]  to implement about 40. It's still like half what static analysis is telling you. So it's kind
[21:43.920 --> 21:49.200]  of a message of hope, right, for building compatibility layers and for developing custom
[21:49.200 --> 21:54.640]  operating systems in general. So, yes, static analysis overestimates a lot of the engineering
[21:54.640 --> 22:01.120]  effort to support an app. And even naive dynamic analysis does measure much more these calls
[22:01.120 --> 22:07.440]  than what is actually required if you know that you can stop and fake these calls.
[22:08.880 --> 22:15.120]  Another view at these results can be seen here. So for each of the system calls, you know, zero
[22:15.120 --> 22:25.200]  is read, one is write, two is open, I guess, and so on, among our dataset of about 15 apps,
[22:25.200 --> 22:30.880]  how many of these apps require the implementation of the system calling question, right?
[22:32.400 --> 22:36.560]  And then so you have here the result for static analysis at the binary level.
[22:36.560 --> 22:46.240]  At the source level, this is S trace without counting which system calls you can stop or fake.
[22:46.960 --> 22:51.840]  And this is what is actually required. So if you consider that you will not implement what you
[22:51.840 --> 22:56.240]  stop or fake, this is what you actually need to implement. And as you can see, you know,
[22:56.240 --> 23:01.840]  it's much, much, much, much less engineering effort versus what static analysis is telling.
[23:01.840 --> 23:10.160]  And why does stopping and faking work? So here you get some code snippet from Redis.
[23:11.360 --> 23:19.360]  So if you stop, get our limit, the C library wrapper will return minus one.
[23:19.360 --> 23:24.960]  And as you can see, Redis will actually fall back on some kind of safe value, you know,
[23:24.960 --> 23:30.720]  so I'm not able to understand the maximum number of files that I can open. So I'm going to fall
[23:30.720 --> 23:43.200]  back on 100, sorry, 1000. And the fact that faking works is actually that you have quite a bunch
[23:43.200 --> 23:49.360]  of system calls. So this is for each system call and each app in our dataset, what is the
[23:49.360 --> 23:57.920]  percentage of apps that are actually checking the return value of the system calls. And some system
[23:57.920 --> 24:03.840]  calls are almost never checked the return value. It kind of makes sense, right, when you see this,
[24:04.480 --> 24:12.560]  why check the return value of close. And this is why, you know, faking work in many cases.
[24:15.360 --> 24:22.400]  Another question that we asked is, okay, so when you speak about providing binary
[24:22.400 --> 24:28.480]  compatibility and you don't do porting anymore, basically, all the effort of supporting apps
[24:28.480 --> 24:34.000]  is on you, the operating system developer. And this is how it should be, in my opinion, but how
[24:36.240 --> 24:41.840]  much effort does that mean in the long term, right? So we had a look at versions of Redis
[24:41.840 --> 24:48.400]  and Jennings and Apache over the last 10 years and what, you know, what are these calls that
[24:48.400 --> 24:54.720]  actually needs to be implemented in purple. And we saw that this number does not change very much,
[24:54.720 --> 25:03.120]  right? So once you make an app and you make it work, it actually means that you need to
[25:03.120 --> 25:09.120]  keep up to date with the most recent version of this app that are coming up, but it doesn't
[25:09.120 --> 25:16.880]  necessarily mean a very big engineering effort either. And these are the support plans. So we
[25:16.880 --> 25:25.040]  had a look at Unicraft, Fushia, which are some operating systems that have already a
[25:25.040 --> 25:29.200]  relatively good support for a good number of system calls. And we look at Kerala,
[25:29.200 --> 25:34.720]  so Kerala is another Unicernel written in Rust. And it's very, I wouldn't say immature,
[25:34.720 --> 25:40.320]  but it doesn't have support for a lot of system calls. And for a set of 15 apps that we had in
[25:40.320 --> 25:47.680]  the database, we derive a support plan. So for Unicraft, for example, in its current state,
[25:47.680 --> 25:53.760]  it's already supporting most of the apps of our data set. If you want to support an additional
[25:53.760 --> 26:01.040]  app, what you need to do is to implement system call number 290 and stop these, and then you'll
[26:01.040 --> 26:08.000]  get memcached. And next, if you implement this syscall, you get H2O, and then you need to implement
[26:08.000 --> 26:14.880]  these two syscalls, and then you stop that, and you get MongoDB. So same thing for Fushia and
[26:14.880 --> 26:19.440]  Kerala. Obviously, it's a bit more interesting because this one doesn't support many applications
[26:20.080 --> 26:27.200]  out of the box. And I believe I have time to do a quick demo. I'm going to do it real quick.
[26:31.840 --> 26:36.720]  So I'm going to do a test with LS, which is like the simplest test because we don't have a lot of
[26:36.720 --> 26:47.200]  time. In the Docker file, I just copy a test that I'm going to show you, and then I call like the,
[26:47.200 --> 26:53.680]  this is kind of the top level script of loop with a few options that don't matter that much. And I
[26:53.680 --> 26:59.520]  say, okay, the binary that we are going to instrument is slash bin slash LS, and this is the
[26:59.520 --> 27:05.360]  parameter. So I'm going to do LS slash, and we are going to check if it works or not with every
[27:05.360 --> 27:14.480]  possible syscalls that can be invoked by LS. And the test, which should be there, the test that
[27:14.480 --> 27:20.880]  we are going to run after each execution of LS to see if things have worked. So this share script
[27:20.880 --> 27:27.680]  will take the standard output of LS as parameters, and to make things simple, I'm just checking that
[27:27.680 --> 27:33.040]  LS actually outputs something, right? I'm doing LS slash, so something should be outputted. If
[27:33.040 --> 27:38.480]  nothing is output, there is a problem. And keep in mind that loop is also checking the return
[27:38.480 --> 27:50.240]  value of LS itself. So, okay, so I'm launching loop like this, so it should work. So what happens
[27:50.240 --> 27:56.480]  under the hood is that we build the container that we've seen the Docker file for. We are starting
[27:56.480 --> 28:03.680]  two containers in parallel. Each one is running a full set of tests trying to stop and fake all
[28:03.680 --> 28:12.160]  the system calls. And we use this to check for differences between the replicas in case there
[28:12.160 --> 28:18.640]  is a problem. Most of the time, there is no differences. So it takes a bit of time. And then,
[28:18.640 --> 28:31.520]  okay, it's done. So, if we go to the database, so we have now much more than 12 apps. And if we go
[28:31.520 --> 28:43.600]  to LS, the most interesting result is this CSV file, which contains, for HCC call, 0 being read,
[28:43.600 --> 28:53.440]  1 being write. Is it called by LS or not? Can we fake it? Can we stop it? Or can we
[28:55.280 --> 29:00.320]  both fake and stop it? Or it's more like, does the application works when it's fake? Does it
[29:00.320 --> 29:06.000]  works when it's stubbed? And does it work when it's both fake and stubbed? And as you can see,
[29:06.000 --> 29:13.680]  some CSV calls, like 11, I don't know which one it is, can be both stub and fake, same thing for
[29:13.680 --> 29:19.200]  12, same thing for 16. Some CSV calls, like this is read, for example, it is called, but you can't
[29:19.200 --> 29:25.680]  stop or take it, which kind of makes sense. LS wouldn't work if it can read. And yeah, that's
[29:25.680 --> 29:31.760]  pretty much it. So briefly, what we are currently working on is some more fine-grained measurements.
[29:31.760 --> 29:39.360]  Some system calls have kind of sub features, like a lot of programs require at least a map anonymous
[29:39.360 --> 29:45.280]  for a map to allocate memory, but not really to map a file. So we are looking at, you know,
[29:45.280 --> 29:50.080]  checking which flags can be stubbed or fake and things like that. And we are also looking at the
[29:50.080 --> 29:57.920]  virtual file system API. That's it. So building compatibility layers is important for custom
[29:57.920 --> 30:04.080]  operating system. It seems a bit scary, but actually, it's not that much engineering effort.