[00:00.000 --> 00:12.000]  Hi, I am Vijay Dandraj and welcome to a talk on Grameen.
[00:12.000 --> 00:16.200]  I am one of the contributors to Grameen project and I have been working in this project for
[00:16.200 --> 00:19.000]  past few years.
[00:19.000 --> 00:25.600]  There was a talk on Grameen in last year's FOSDEM by one of the Grameen maintainers,
[00:26.600 --> 00:35.600]  on the technical details of Grameen as well as a demo showing how to use Grameen as well as GSE.
[00:35.600 --> 00:45.600]  So, in this talk I will quickly go over the technical details and focus more on the current status as well as the new features that were added in Grameen.
[00:45.600 --> 00:54.600]  We will focus more on EDMM which is one of the new features that was added and we will also discuss on the future plans.
[00:54.600 --> 01:09.600]  I am assuming the audiences are aware of Intel SGX as well as DEs in general and so I will skip the background on this and move further.
[01:09.600 --> 01:17.600]  So, this is the presentation from an Intel representative and thus I should show the following disclaimer.
[01:18.600 --> 01:29.600]  Grameen is an open community project and currently Intel and ITL are the most active members of this community.
[01:29.600 --> 01:42.600]  There have been sizable contributions by IBM as well as Alibaba and there is also other contributors from several other companies and academic partners.
[01:42.600 --> 01:47.600]  Now, productizing Grameen is currently done by Intel.
[01:47.600 --> 01:52.600]  Let us discuss on the technical details of Grameen.
[01:52.600 --> 01:57.600]  Grameen is a library OS that emulates Linux kernel.
[01:57.600 --> 02:08.600]  In other words, Grameen can be thought of as a tiny Linux kernel re-implementation to run unmodified guest in the user space.
[02:08.600 --> 02:25.600]  Grameen supports multiple processes, applications with multiple processes such as a bash script with many child workers or an engine X application with many worker processes.
[02:25.600 --> 02:34.600]  This is possible because Grameen emulates syscalls like clone, fork, exec v.
[02:34.600 --> 02:44.600]  Another important aspect of Grameen is that it is architected in such a way to support different backends.
[02:44.600 --> 02:47.600]  Currently, there are two supported backends.
[02:47.600 --> 03:03.600]  Grameen is Linux also called as direct as this backend directly forwards the calls to the host OS and the Linux SGX which runs an application inside an SGX on clay.
[03:03.600 --> 03:09.600]  In future, we also plan to support other backends like TDX.
[03:09.600 --> 03:21.600]  Grameen is a community maintained open source project and it joined the confidential compute consortium in September 2021.
[03:21.600 --> 03:27.600]  Grameen can be easily deployed in cloud environment like Azure.
[03:27.600 --> 03:34.600]  One can get the Docker image from Docker Hub with a few additional packages.
[03:34.600 --> 03:39.600]  One can get started with a simple hello world program very quickly.
[03:39.600 --> 03:49.600]  The latest production ready Grameen was released in September 2022 and we plan to release in a quarterly cadence.
[03:49.600 --> 03:55.600]  One dot release is actually planned this week so keep an eye out for it.
[03:55.600 --> 04:09.600]  We have successfully enabled multiple widely used applications in Grameen and we have also enabled support for many popular languages like Python, Go, Rust and Java.
[04:09.600 --> 04:14.600]  One of the recent applications that we have added was scikit-learn.
[04:14.600 --> 04:20.600]  We plan to support more use cases as we grow.
[04:20.600 --> 04:34.600]  One of the benefits of Grameen is that it's both modular as well as it's small which allows unmodified applications to run on different platforms in different environments.
[04:34.600 --> 04:49.600]  So technically Grameen can take an x8664 based application and it's dependent libraries without any modification or recompilation and let it run on another environment.
[04:49.600 --> 05:01.600]  So currently in Grameen we can run an application in an Intel SGX environment on top of an untrusted Linux host.
[05:01.600 --> 05:13.600]  Hypothetically we could also run an application in Grameen on a RISC-5 Keystone environment on top of a Windows kernel.
[05:13.600 --> 05:21.600]  But this may need a new backend to enable such environment.
[05:22.600 --> 05:33.600]  The takeaway here is with minimal effort Grameen enables to lift and shift application from one environment to another environment.
[05:33.600 --> 05:41.600]  Grameen is a library OS and so it runs as a normal user level process on the host OS.
[05:41.600 --> 05:47.600]  Grameen is modular and it runs as two closely interacting components.
[05:47.600 --> 05:55.600]  So one is the Grameen main component and the other one is the Grameen backend component.
[05:55.600 --> 06:04.600]  The backend component can be switched to different implementation without any modification to the Grameen main component.
[06:04.600 --> 06:14.600]  To allow switching between different backends Grameen specifies an ABI interface between the main component and the backend.
[06:14.600 --> 06:23.600]  Applications enabled inside Grameen should be a Linux application that uses Linux system calls.
[06:23.600 --> 06:31.600]  Grameen intercepts these system calls and tries to emulate them in the Grameen main component.
[06:31.600 --> 06:38.600]  Grameen currently implements around 169 CIS calls.
[06:38.600 --> 06:45.600]  A few examples of Grameen emulating CIS calls are affinity related CIS calls.
[06:45.600 --> 06:51.600]  And there are also a few calls which Grameen forwards to the host OS.
[06:51.600 --> 06:56.600]  These being the input output CIS calls such as receive or send.
[06:56.600 --> 07:06.600]  As Grameen cannot communicate with outside world without the help of the host OS.
[07:06.600 --> 07:13.600]  When Grameen forwards to the host OS it doesn't simply call the host OS.
[07:13.600 --> 07:18.600]  There is one more indirection. This is the backend, the Grameen backend.
[07:18.600 --> 07:26.600]  The backend is needed to adjust Grameen to request to the capabilities of the underlying platform.
[07:26.600 --> 07:31.600]  So the backend is kept minimal and stateless.
[07:31.600 --> 07:37.600]  Grameen calls into the backend via 50 functions.
[07:37.600 --> 07:43.600]  So this is basically the Grameen ABI that's defined between the two components.
[07:43.600 --> 07:52.600]  The main code base remains exactly the same between different backends.
[07:58.600 --> 08:10.600]  To run a Linux application inside SGX on-tly, we can simply use Grameen with an SGX specific backend.
[08:10.600 --> 08:16.600]  Linux SGX is the backend that is primarily used in Grameen.
[08:16.600 --> 08:28.600]  As Intel SGX dictates separation of a process into trusted and untrusted part, Grameen's SGX backend is also divided into two parts.
[08:28.600 --> 08:36.600]  The trusted part that runs within the on-tly and the untrusted part that runs outside the on-tly.
[08:36.600 --> 08:46.600]  The trusted part performs OCOLs to exit the on-tly and pass the control to the untrusted part.
[08:46.600 --> 08:51.600]  There are approximately 42 OCOLs in total.
[08:51.600 --> 08:59.600]  The untrusted part forwards the request to the host kernel, gets the results back and then re-enters the on-tly.
[08:59.600 --> 09:07.600]  It is important to know that these 42 OCOLs are the only attack surface in Grameen.
[09:07.600 --> 09:20.600]  So we audit all these OCOLs and also add checks and validations to make sure there are no attacks from the untrusted host.
[09:21.600 --> 09:36.600]  We believe having these 42 OCOLs between the trusted and the untrusted world is a good compromise between security and performance.
[09:36.600 --> 09:45.600]  But when talking about trusted execution environment, the TCB plays a main role.
[09:45.600 --> 09:56.600]  On the metric of TCB, Grameen offers the main component, which is around 27K lines of code.
[09:56.600 --> 10:03.600]  This is basically implemented in C with few assembly.
[10:03.600 --> 10:10.600]  The direct Linux backend is about 15K lines of code.
[10:10.600 --> 10:20.600]  For comparison, the open-source version of Redis without any additional models is about 144K lines of code.
[10:20.600 --> 10:27.600]  The tiny configuration of Linux is about 270K lines of code.
[10:27.600 --> 10:34.600]  When we look at the SGX environment, the main Grameen component remains the same.
[10:34.600 --> 10:47.600]  It is around 27K lines of code and the SGX trusted background is around 15K lines of code and the untrusted backend is around 4K lines of code.
[10:47.600 --> 10:59.600]  Since the trusted background only runs within the on-tly, the total TCB for Intel SGX is around 42K lines of code.
[10:59.600 --> 11:09.600]  And this is small for a framework that can run a wide range of applications.
[11:09.600 --> 11:16.600]  Another important aspect is the SGX attestation.
[11:16.600 --> 11:26.600]  But for this, I would recommend to go and look into docs, which very nicely explains the SGX attestation flow.
[11:26.600 --> 11:34.600]  And we can also look at last year's talk, which also captures the SGX attestation.
[11:34.600 --> 11:41.600]  So next, let's talk about the enhancements since the last 4-stem talk.
[11:41.600 --> 11:52.600]  So in Grameen, now we have started releasing packages for popular Linux distributions such as the Ubuntu and CentOS.
[11:52.600 --> 11:58.600]  And we have started releasing official Grameen Docker images.
[11:58.600 --> 12:08.600]  And we have also introduced some curated applications with several workloads and it is available at Azure Marketplace.
[12:08.600 --> 12:20.600]  So few of the core Grameen changes that were done are a major overhaul to the memory management and a total rewrite of sockets.
[12:20.600 --> 12:25.600]  And also ELF and ELF parsing and loading.
[12:25.600 --> 12:31.600]  We have also added support for Intel AMX instructions.
[12:31.600 --> 12:37.600]  And there was also a sanitization of CPU and NUMA topology done.
[12:37.600 --> 12:42.600]  And we have added support for executable scripts.
[12:42.600 --> 12:49.600]  Recently, we added support for EDMM, which is on-tly dynamic memory management.
[12:49.600 --> 12:54.600]  And we will discuss about this in the next few slides.
[12:54.600 --> 12:57.600]  So why EDMM?
[12:57.600 --> 13:03.600]  So SGX-1 requires all on-tly memory to be committed at on-tly build time.
[13:03.600 --> 13:06.600]  This causes few limitations.
[13:06.600 --> 13:18.600]  So now the developer needs to predict maximum memory that he needs, thus hogging the critical EPC resources.
[13:18.600 --> 13:26.600]  Also, the developer needs to specify maximum number of threads that can be created inside an on-tly.
[13:26.600 --> 13:34.600]  And since the page permissions cannot be changed, all the on-tly memory must have read, write, execute permissions.
[13:34.600 --> 13:47.600]  To improve on this, SGX-2 extensions were introduced and as part of that, we have EDMM, which allows runtime management of on-tly resources.
[13:47.600 --> 13:52.600]  So with EDMM, on-tly memory is added and removed on-demand.
[13:52.600 --> 14:04.600]  And as part of that, we can reduce the startup time because we don't add any EPC pages during on-tly build.
[14:04.600 --> 14:09.600]  And so for bigger on-tly sizes, this is really useful.
[14:09.600 --> 14:17.600]  And it also enables new functionality such as dynamic thread creation.
[14:17.600 --> 14:21.600]  So these are few of the SGX-2 instructions.
[14:21.600 --> 14:35.600]  The blue table shows supervisor instructions and these comprise of allocating EPC pages, restricting page permissions and modifying EPC page types.
[14:35.600 --> 14:40.600]  And the yellow table shows user-level instructions.
[14:40.600 --> 14:47.600]  So these are basically to accept the page inside the on-tly to enhance page permissions.
[14:47.600 --> 15:01.600]  And an E-accept copy is a special instruction wherein as part of the accept, you also copy the contents into the augmented EPC page.
[15:01.600 --> 15:04.600]  So a quick overview of EDMM.
[15:04.600 --> 15:11.600]  So as part of EDMM, we can dynamically allocate and deallocate EPC pages.
[15:11.600 --> 15:18.600]  We can dynamically create and destroy threads and we can restrict or relax page permissions.
[15:18.600 --> 15:25.600]  By restrict, I mean reducing the permissions, for example, from read-write executable to just read.
[15:25.600 --> 15:33.600]  And by relax, I mean the opposite wherein you change from read to maybe read-write executable.
[15:33.600 --> 15:37.600]  But dynamic memory management requires a new approach.
[15:37.600 --> 15:42.600]  The OS is involved in page table management and also TLB flushes.
[15:42.600 --> 15:56.600]  But since the OS cannot be trusted, the EDMM architecture ensures that the on-tly confirms the request before any changes are taken into effect.
[15:56.600 --> 16:06.600]  So EDMM provides a new instruction, the E-accept, to make sure on-tly accepts the new modifications.
[16:07.600 --> 16:15.600]  So here I will quickly discuss how typical page allocation flow happens.
[16:15.600 --> 16:20.600]  This is a page fault-based allocation.
[16:20.600 --> 16:27.600]  So what I mean by that is we act E-accept on an uncommitted page.
[16:27.600 --> 16:35.600]  This triggers an asynchronous exit to the SGX driver calling the page fault handler.
[16:35.600 --> 16:45.600]  And as part of the page fault handler, the SGX driver eogues the page and it returns back to the untrusted backend.
[16:45.600 --> 16:55.600]  The untrusted backend resumes back to the on-tly and then we retry the instruction again.
[16:55.600 --> 17:07.600]  Since it is a fault instruction, it will be retried and this time it will get accepted and the EPC page is accepted as part of the on-tly.
[17:07.600 --> 17:22.600]  So as part of each page allocation, we encounter one AEX, two context switches and one ECOL.
[17:22.600 --> 17:33.600]  And in case of page DL allocation, we call an iOcton to the SGX driver to trim the page.
[17:33.600 --> 17:39.600]  What that means is we tell the on-tly not to use those pages.
[17:39.600 --> 17:49.600]  As part of this iOcton, the SGX driver invokes an instruction called eTrack to remove all the TLB addresses from the CPUs.
[17:49.600 --> 17:59.600]  It issues IPIs to flush the stale linear address to physical address translation and it returns to the untrusted backend.
[17:59.600 --> 18:09.600]  The untrusted backend now resumes the flow and we accept this new modification that was done to the on-tly.
[18:09.600 --> 18:27.600]  Then we issue another iOcton to remove the page from the EPC and the driver removes the page from the EPC and goes back again to the untrusted backend which resumes the on-tly and we continue our execution.
[18:27.600 --> 18:41.600]  So this is a simple deallocation flow and as part of the deallocation flow, we end up having two OCOLs, four context switches and two ECOLs.
[18:41.600 --> 18:51.600]  So this is a pretty expensive operation and might lead to some performance impact.
[18:51.600 --> 18:56.600]  So in Grameen, we have enabled EDMM.
[18:56.600 --> 19:15.600]  So the EDMM support was enabled in the Linux kernel starting from v6.0 and Grameen currently supports the nape page allocation and deallocation that we just saw as well as restricting and relaxing page permissions.
[19:15.600 --> 19:26.600]  To turn on EDMM, we use a manifest option called EDMM underscore enable but this is by default set to false.
[19:26.600 --> 19:36.600]  So if this is set to true and we run on CPU which does not support EDMM, Grameen will refuse to start.
[19:37.600 --> 19:57.600]  So on the performance implication as we saw, the name EDMM implementation is fairly expensive as we see adding and removing page at runtime is more expensive than adding the page at the on-tly creation or build time.
[19:57.600 --> 20:05.600]  Currently we are working on optimizations to improve this performance.
[20:06.600 --> 20:24.600]  So our current ongoing work is we are working on EDMM optimizations to support better quicker allocation, deallocation flows and we are also working on adding support for dynamic thread creation and destruction.
[20:24.600 --> 20:41.600]  We are also continuing to develop support for additional runtimes and workloads and we are currently working on integrating with confidential container deployments like Cata containers and Enclave CC.
[20:41.600 --> 20:52.600]  We are also working on an interoperable RA TLS and also standardization of this interoperable TLS.
[20:53.600 --> 21:04.600]  As part of our future work, we want to support additional TEs such as Intel TDX and we want to support communication with hardware accelerators.
[21:05.600 --> 21:27.600]  So core screen partitioning for applications to enable use cases like DPDK where control plane is executed inside the enclave and the data plane is executed in the untrusted region.
[21:27.600 --> 21:31.600]  So we are also exploring such use cases.
[21:31.600 --> 21:38.600]  To get a roadmap on Grameen, please visit this following link.
[21:38.600 --> 21:56.600]  So you can reach us in the GitHub and for those who are interested to join us, you can contribute in the form of pull requests, issues, discussion or tutorials as well.
[21:56.600 --> 22:06.600]  And you can read more about Grameen in the Grameen website and there is also a doc section which is pretty useful.
[22:06.600 --> 22:18.600]  You can reach us also through GitHub and yeah, I think the end of the talk. Thank you for your time.
[22:26.600 --> 22:28.600]  Thank you.