All right, our next speaker is Andrew Jeffery from University of Cambridge talking about
LSKB.
Hello.
So yes, I'm Andrew Jeffery from University of Cambridge, emails there if you want to
email me about any questions that I can't answer today.
As a brief kind of precursor, I kind of come from the distributed systems world, not necessarily
confidential computing world, so this is kind of like a hybrid of both worlds here.
So today we're going to talk about LSKB aiming to democratise confidential computing from
the core.
So first of all, we've got to work out what this core actually is that we kind of want
to start replacing.
And so we're going to start working with distributed key value stores.
In particular, we're going to look at CD.
And as the CD website defines itself, it's a distributed, reliable key value store and
importantly, it's for the most critical data of your distributed systems.
So CD runs as a cluster, it's distributed, so you have this one leader node and you might
have some more followers in this setup as well.
CD is also not alone, it's the core, so you have some applications around that.
Some of those applications might be some sort of orchestration, so using Kubernetes on
top is like one of the main candidates.
Otherwise you might use M3 or Rook or Core DNS or other applications that use CD internally
as well.
So effectively, it's really widely used, it's quite a critical piece of a lot of infrastructure.
And so you have to interact with CD in some way, even if you're just using one of these
services.
And so primarily you use some key value operations like you can put, so you might write foo one
equals bar into the data store and it keeps some history so you kind of have this revision
system.
So when you do that first write, it'll be stored as a version five and you write to
be six, seven, eight going on like that.
So you can, after you've written, you can get something back out using the range queries
and with this you can say, I'd like all keys between foo and foo five in history, so you
would be able to get multiple keys at once.
And you can also specify the revision here if you wanted to go back in time just to see
what it was some previous point.
After you've read something, you might no longer need stuff, so you can delete it.
So you can also delete with this range as well, so you can delete say foo to foo five.
Transactions are a nice ability here, so you can do some kind of conditional logic at the
data store side.
So if you can use put range of deletes or more or less the transactions internally to
do kind of bulk operations here, so you can say write foo two and foo three in the same
revision.
Additionally, you can have leases on top of the data store, so these can be used for building
high level primitives and distributed systems, and primarily you might want some like leadership
mechanism.
One final thing here is the watch API that Etsy provides, similar to ranges you can do
a range between a start and an end, and you can also do a watch, a certain point in history.
So for watches, that history is where you start watching from.
So if you start revision five, you'll be notified that foo equals bar, and then you'll be notified
of the things in revision six, foo two and foo three, and everything that kind of comes
in after that as well while you keep the connection open.
And this is just like a really core API that's used by lots of these other systems, so this
is kind of something we might want to mimic if we want to like replace Etsy.
So Etsy is a big system, we want to run it somewhere, primarily lots of people run it
in the cloud.
You don't always want to trust this cloud, because it's run by cloud providers.
They might themselves be trustworthy, but the things that they're operating might not
be.
So if a high provider gets a weakness, then might get attackers going through to the lower
layers and being able to access some of the hardware themselves.
Clients that are interacting with your service might be within the cloud themselves, they
might have already accepted some of the cloud primitives, they might also just be outside
of the cloud and just having to use your service for some reason, so they might not be wanting
to use the cloud directly themselves.
Additionally, they might not necessarily speak directly to your data store.
Lots of things they talked through a proxy, so if you're in Kubernetes as well, you might
have Cubelets and Kube CTL, they speak through the API server, which basically terminates
the CLS connections before passing and doing some logic back on the data store itself.
And so today we're going to speak about two problems.
Problem one here is this LCD cluster that's running in the cloud.
If we're not trusting in this cloud, then all of the data in memory is currently unencrypted
and so we want to be able to do something about that.
And problem two is this proxy.
If this proxy is terminating some CLS connections, we don't really want that to be able to happen.
We'll actually see how the proxy can be a bit distrustful with our interactions.
We want to be able to show when it's not being very trustworthy.
So diving into problem one.
So we've got this XED cluster, XED like any kind of storage service has some storage,
it has some memory and some processing application.
It's also distributed, so we have some CLS connections between the peers.
And effectively, as you can see in yellow here, we have some level of security.
So recommended setups have XED communicating for XED nodes over TLS.
And we also can have this optional sort of file system encryption that gets put down
to storage.
One main problem with this, all of those keys are in memory.
So the TLS key that we were using for TLS is now in memory, so the attackers got that.
And they've also got the private key for any file system encryption.
So basically renders our TLS connections and our storage encryption pretty much worthless
if we're not trusting that someone can't access our memory.
So if we actually swap out CD for LSKV, which is our data store, we run LSKV inside of
an SGX enclave, which gives us those confidentiality properties that we just mentioned in the
previous talk, then we can build some of these privileges to be a bit more trustworthy.
So now that our memory is encrypted and integrity protected, we can store our TLS keys and file
system keys there and trust that they're not going to be able to be accessed.
The actual application itself is running in a secure enclave, so we can show you that
it's not going to be able to be modified.
While TLS connections, we can be sure that they're actually secure TLS because our TLS
keys are in memory, and we can actually upgrade those to a tested TLS where rather than just
trusting the other application to ever end, we can make sure the other application is
running in a secure enclave as well, so it's in the same environment that we trust.
And finally, we've got this file system key in memory, so we can trust that anything we
write to disk is actually going to be encrypted properly as well and safe.
So this is like one nice solution to that problem one.
So if we delve into a bit of what LSKV is a bit more, we can see that it builds on something
called CCF.
It runs in the SJX enclave, and like most services in the cloud, it will run on top
of a hypervisor and has some memory and storage and other resources attached as well.
So if we quickly jump into CCF, which is the confidential consortium framework, it's
a pretty nice project that basically splits up the interactions and the management of
it into three distinct roles.
So the first role is the operator.
So this is the kind of cloud provider, the person who's standing up VMs for you.
You might say, I'd like one LSKV node to run with, and then you might later want more
to join into the network.
And so this operator is untrusted, so all they can do is stand those nodes up.
They can't do any sort of giving those nodes access to the data in the cluster.
They don't auto join the cluster.
That's the responsibility of the governance, who we partly trust.
There's a few of them, and so any interactions they do have to be done in some sort of majority
way.
And so these people will be responsible for things like once a node has been stood up,
except checking the configuration of it, and finally accepting it into the network
so it can start serving application requests and handle some of the data in the cluster.
And finally, we have users that need to actually access the application that we're running.
And so these are treated as kind of trusted towards the application, but the application
itself can have internal access controls inside.
And all the data is stored on an encrypted ledger that gets written to the disks.
Governance requests are stored publicly in this ledger, and they're also signed so that
everyone can see those and verify those.
User interactions that go through the application are normally stored encrypted by default.
You can make them public if you have certain use cases for those.
So LSKV actually has an HCD-compatible API.
This slide might look pretty familiar.
It's basically the same API at the core.
One asterisk is that this watch is at the bottom, is currently requiring a patch on
CCF because it hasn't got around to being merged in yet, but that's something that should
be fixed and should be expected to work.
So effectively, this basically means that we can switch out HCD for LSKV in most cases,
solving problem one.
So before we quickly go into problem two, we've just swapped out HCD and LSKV, and
it might not always be as simple as this.
So some quick trade-offs.
LSKV is actually optimistically consistent in the way it replicates data rather than
HCD is strongly consistent, and otherwise we have the normal things we might expect
that LSKV actually gives us confidentiality properties of our data, makes more operations
transparent so those governance operations we can see.
It also has this HCD API at its core.
It also has some extra features that we're not going to cover too much here.
There's one later.
So quickly on this optimistic sense, rather than replicating data in a synchronous way,
when you write to LSKV, it will replicate asynchronously, and in turn, you get an ID
back.
You can later follow up with this ID to say, was this replicated properly or was it not
allowed?
And this basically puts the decision at the client's side, so they can either be optimistic
and say, I'm going to trust that that's fine, or they can come back later and say, no,
I wanted to make sure that was replicated first.
So this is just a key difference.
If we go quickly on to problem two, we have this proxy that we might want to communicate
to the data store too, but in this instance, Alice wants to write 500 pounds into an account,
but the proxy is going to intercept that and write that money into Bob's account instead.
LSKV is none the wiser here.
It's just gone on the request.
It's going to process that request and handle the response.
The proxy in turn has an opportunity to return to the client and say, OK, Alice, we've written
your money into the account.
Now hopefully you can see here that Alice is not equal to Bob, and so she hasn't actually
got the money.
So LSKV gives us this receipt functionality where we can actually kind of expose untrustworthy
proxies.
So when the client first does the request to write money into Alice's account, they
can also ask for a receipt for that operation.
This goes through the proxy.
The proxy can rewrite the normal, write request as normal, so the LSKV actually puts the money
into Bob's account, but we still want to get that receipt back.
So when the receipt actually comes back from LSKV, the client can actually detect that
either the proxy's manipulated this receipt in some way, so it's no longer valid because
it's a signature, or it is valid, and in which case it says that the money went into Bob's
account, which is not what they wanted.
So you can use this to kind of flag to someone else that this proxy's not trustworthy and
you'd probably stop talking to it.
So that's also how we can kind of solve problem two quickly with LSKV.
Just a quick summary of things here is now that we don't think current data stores are
really suited for confidential operation, primarily looking at CD.
We don't think that lifting and shifting them into confidential environments gets you all
the properties that you necessarily want.
You get some kind of memory encryption integrity just by running them in enclaves, but you don't
get some of the other properties that we kind of get from building on the ledger, and also
like having a different trust model compared to what some systems have.
We've introduced LSKV, which is a new confidentiality store we've built on CCF, and it actually
has an HD compatible API, which basically means that you can swap out HD for LSKV in
most cases.
LSKV is also able to highlight these untrustworthy proxies using receipts, and yeah, it's kind
of fast.
Thanks to that optimism, it basically has a higher throughput and lower latency than
an HDD, so if you do replace an HDD with it, hopefully your performance of stuff will
start increasing as well.
And that's pretty much me, so I'm around for a few questions, and please check out the
GitHub repo, its open source and everything.
And yeah, my email is there if you have any questions after the talk, so thank you.
We definitely have time, so feel free to go ahead and answer your first question.
On the proxy side, the receipts, is that an API extension, or is that an API change?
If I'm using the proxy lab, I switch it to LSKV, do I need to change my application?
So that's a question about the HDD API, does that include the write receipts by default?
No, it does not.
They're a separate GRPC service on top, so your clients would need to be manipulated
with that.
Either your clients or you'll build that into a proxy, so you could say, because the idea
of the proxy is that it's a different kind of exposing different API to a client, right?
So if you do a write request to a proxy, your proxy, you could by default ask for a receipt
and present that to the user as well, they might need some extra functionality to be
able to verify that.
Is that because it's, I don't know how the receipt works, with extending the HDD API
to have some sort of, I don't know, a non-source or a request for signature, is that something
that you're planning to, would it make sense to extend the actual HDD API?
Questions, would it make sense to extend the HDD API for receipts?
We don't really think so, we're not really planning on doing changes directly in HDD,
just primarily because HDD has a different kind of threat model and trust model, so some
of the things that we have, HDD doesn't necessarily support against.
So that's one reason we're not going to stop putting some of this back into HDD itself.
It's kind of designed to be a separate service.
Yeah.
I'm wondering how the authentication would work, like if I have confidential services
that and only those should be able to access the confidential storage, how would the mutual
authentication work here?
Between, so do you mean, the HDD should just give out secrets to those confidential services
and the confidential service should know that they access the correct HDD storage essentially.
By HDD storage, do you mean a key or?
I mean the LSKV as an HDD replacement in the Kubernetes state.
Okay.
So I think the question is about how do you make sure you access the right HDD cluster
from a confidential client?
Yes.
Yeah.
So you'd be speaking through the API server right in Kubernetes?
Yeah.
Or you're saying you want to connect directly?
Maybe you have a confidential controller or you connect directly, I guess.
Okay.
So we wouldn't, I'm not sure if we can do that directly in LSKV.
It has a cluster ID like a normal HDD cluster, so you can go about trusting that.
But we don't, I don't think we've really worked through that use case directly.
The main thing is that if you want to write a proxy around it, then you might want that
in a proxy.
Okay.
Thanks.
What are the things that LSKV have to be careful of as compared to like HDD?
In what context?
Like, what are the things that LSKV need to be taken care of that GTC don't take care
of?
Okay.
Why do you need to do that?
Yeah, yeah.
So the question is about why do we, well, some things that LSKV kind of supports that
doesn't, HDD doesn't.
So one of the main things is storage.
In HDD, everything gets written to disks right away and that's part of the consensus
process.
You won't get returned until it's written to disk.
In LSKV, that's not the case, we don't write to disk synchronously, which basically means
that we're not trusting this host to necessarily persist our data or even give it back to us
correctly when we ask for it.
It's only used for like disaster recovery scenarios.
So that's like one primary difference, which basically means that HDD is open to rollback
attacks.
So if you trust the host to do the data, they can cut some off and when you load back, you've
got a subset of what you had before.
Okay.
Yeah?
So I saw in the trade-offs that LSKV cannot have a strong replication like HDD.
So what was the cause of LSKV, what's the case to have to do that trade-off?
Yeah.
So the question is about why does LSKV not have strongly consistent replication?
Primarily, if we're not trusting the host, then A, we don't want to write things to disk,
like we just said.
So that's one reason for that part.
And then on the replication side, we're not trusting the host, so we don't want to block
everyone on wanting to replicate everything.
And so it does do the replication, and you can follow up again with that ID from your
operation.
So if you do care about it being strongly replicated, you can just follow up the ID and
wait until it's committed.
It's just kind of giving the users a bit more flexibility in that operation choice.
Okay.
So I have one more, so Tom will set up while we're in the last session, but I have one
more.
So I'm going to write for your performance numbers, because 50% latency loss I expect,
but then three and a half times gain in performance, something I didn't expect.
Can you say why that is, or how well it keeps them consistent?
It's actually consistent.
I mean, maybe I missed that.
Yeah, of course you don't see the slide anymore.
Oh, that's a good slide, yes.
Yes.
Great.
Yes.
So the question is about why it's so fast.
Unfortunately, it was a consistency slide, but I think I missed it, but thank you.
Yeah.
I kind of won.
Yeah.
Even the slides are confidential now, but yeah, slides are on there for some pitch.
One minute, second question.
Otherwise.
Yeah.
So one of the first slides, when you get an example of a typical architecture, there
were like simple types.
Mm-hmm.
Yeah, so primarily you can replace ETD with LSKV at the moment, if you took the clusters
and swap them, because the ETD API is still there, your things will still work.
If you wanted to take advantage of the extra things like the receipts for the proxies
and things, you would need to add some logic into your proxy or into your client's fave
route.
There are, I think, apps that support the specificity support as well.
No, there's not currently apps that support those receipts and extra bits at the moment.
That's, we haven't got to add a bit.
It's just on the data store focus at the moment.
Cool.
All right.