Okay, hello everybody. Can you hear me? Yes, I see thumbs up at the back. Please come in,
come in, roll up, roll up for the Matrix show or introducing Matrix 2.0 or how we are going
to make or how we have made Matrix go boom, very technical term. Please take your seats,
ladies and gentlemen. Right, so hi everybody. I'm Matthew. I'm the project lead and co-founder
of Matrix and I'm here to tell you all about the work we've been doing to fix Matrix's performance
problems and a few other things as well. So I'm guessing that a bunch of people know what Matrix
is, given that Vostem has been running on Matrix during the pandemic and we're doing
four hard to lead doing a hybrid Matrix version of Vostem right now as we speak. So hello to
everybody following along on the HLS stream by Matrix. You probably know that Matrix is an open
network for secure decentralized real-time communication. We have to say this because
people watching the video might not know later on. You can use it for interoperable chats,
so following along on Vostem, bridge through to ILC or X and PP, etc. You can use it for
interoperable VoIP, but Matrix at its core is a real-time communication fabric for any kind
of real-time comms. So you could use it for communication within VR or AR. You could use it
to synchronize world data within VR and AR. You could use it for IoT. It is basically meant to be
the missing communication layer of the open web. So no single party in Matrix owns your conversations.
The second you talk to somebody on a different server, your conversations get replicated equally
between the different servers, so there cannot be a gatekeeper. There cannot be some big tech company
going and holding your conversations hostage. Instead, the conversations are shared between
all the participants. To apologize, the network is a mesh of servers like Vostem.org, say,
on the right, which might be talking to Mozilla.org, which might be talking to Matrix.org, or Nome.org,
KDE.org, or whatever it is, and you have native Matrix clients like Element, or Fluffy Chat,
or Mecca, or Katernion, or hundreds of others these days, which connect through to the Matrix
server. Then you have application services, which glue additional things onto the Matrix server,
like bridges, or bots. You have identity servers, which we don't talk about because they're a mess,
and we need to get rid of them. And then you've got application services that bridge through to
things like Slack, or Teams, or Telegram, or ILC, or XMPP, et cetera. And that is a schematic view
of the public Matrix network. Now, the Matrix ecosystem, as it sits today, looks something
like this. You've got the Matrix spec still being the kind of one true commonality across the whole
ecosystem, spec.matrix.org, a whole bunch of markdown, and swagger, or open API, I should say,
that defines how the server's on the bottom, talk to the clients on the top. So you've got Synapse,
as our first-gen Python server, which has been proving to be a bit more than first-generation,
as we've invested a lot of time making it fast and generally performant. So Synapse is not going
away anytime soon. If anything, it is corroding into Rust as the Python is augmented cybernetically
with a bunch of Rust. Then we have Dendrite in Go, which is also doing very well and is
ending up focusing more on embedded use cases. So you use Synapse for massive servers and use
Dendrite for ones you embed into an app. Then we have application services and bridges for
things like ILC bridging, et cetera. And then many other servers and bots and bridges from the
wider community. The green stuff is stuff that we published as the matrix.org foundation,
all Apache licensed for people to build on top of. We have, then, the clients. On the far left,
we have our original SDKs of matrix JS SDK, React SDK on top of it, and then iOS SDK and
Android SDK too. These are relatively venerable SDKs now. JS SDK is eight years old, which is,
you know, enough to probably get a degree or something. iOS SDK is about the same age. Android
SDK is a little bit younger, but this is what the current generations of element use today.
Then we have Hydrogen, which is a relative newcomer. This is a progressive web app SDK,
super small. It's about 70, 80 kilobytes in total for the whole thing, including all end-to-end
encryption. And it's designed for embedded web matrix instances. So we have Hydrogen itself,
hydrogen.element.io, as a very lightweight PWA for playing around with matrix. We have chatterbox
as intercom style embeds a matrix chatbox into your website. We have Surderoom, which is our
crazy science fiction spatial collaboration on top of matrix platform, where you want to have
the matrix bit as lightweight as possible, which is why we use Hydrogen, the lightest element,
to make it happen. And then the thing we'll be talking about a lot today is element X. So this
is a total rewrite of element mobile on iOS and Android. And for maximum cliche, we have rewritten
it in Rust using the matrix Rust SDK. I hear we have Rust fans in the audience. So we'll be talking
a lot about that. Meanwhile, on the community side, there are just more and more clients all over
the place, like Thunderbird released its native matrix implementation. We've got Watch the Matrix,
which is an excellent Apple Watch matrix client, which doesn't tether to your phone. It's actually
running on the watch itself. NeoChats, and KD, and QTE, and C++. I wish I could name them all.
I can't. I don't have time. So help for the network. Here is the total number of users we've
ever seen on matrix, which is, looked as if it was going to hit 90 million, and then somebody
turned off the server. Kids, this is what happens if you turn off the server, because it means
the graph goes down. So please never, ever turn off your matrix servers ever again. This is
literally the phone home stats that Synapse reports. So it doesn't include Android. It doesn't
include Conduit or other servers. It also doesn't include all of the paranoid people,
which is quite a lot of people running matrix who obviously aren't going to phone home their usage
stats to us. It does include guest users and bridge users. So it's a little bit of an overestimate.
But the important thing is the shape of the graph, which, as you can see, is continuing to go well,
apart from that guy who turned off his server. In terms of the spec, we have fallen into this
cadence of quarterly releases of matrix since the big 1.0 back in 2020, and then particularly in
2022, we've managed to crank out a point release every quarter. Just in, I think, the day before
Last Foster, we had spaces, restricted joins, and actually the matrix ERI scheme, which is now
being implemented all over the place. Then a few months later, we added aggregations and restricted
knocking. Now, these features have often been around for years, but this is actually formalizing
them into the proper long-term supported spec. 1.4 added threads, massive feature that has been
an epic to get done, but it landed. Edits, private read receipts, a very long time
complained about matrix being that you couldn't turn off read receipts, turns out it's surprisingly
hard to do, but we now have it specced and implemented. Then 1.5 in November, we added in
formalized references, we fixed up some things in the ASAPI, and I think we have basically
maintenance release on 1.6 any day now. So nothing too exciting in the next release,
mainly because we're building up to the big 2.0. Nice stat is that there were 120 MSCs in 2022,
of which 39 came from the community, 27 from new contributors, 30 from the gray beards of the
spec core team, and then 51 from the folks who are paid to work on matrix.org, mainly by element.
So it's a reasonable mix of community and core project work. In terms of uptake,
other than that, we obviously help the world's best open source conference, dodge COVID.
Hopefully, apologies if matrix was painful over the last couple of years, but it's probably
better than no false to me at all. Lots of government uptake. New news is Germany,
we have bundles messenger rolling out matrix across the whole German government in November,
also good martyred, the German healthcare agency, proposing it as a neutral standard for
secure messaging in healthcare. Lots and lots of associated deployments in healthcare, education,
utilities, manufacturing. Basically, if you are an organization who cannot put their data
unencrypted into some proprietary silo, like teams or Slack, I think matrix hopefully provides
a good alternative. Moodle is busy integrating matrix into the learning management system.
Automatic have got a project called chat tricks, which embeds matrix into WordPress,
so you can literally dump your little chat console based on hydrogen into your WordPress blog.
Reddit is rumored to be building chat capability powered by matrix, mainly because I think they
had public who sign up enabled and somebody logged in and discovered a matrix over there,
and there's a lot of interest over matrix being potentially the communication there
for the digital market sites. So, last year, we put out this slide to basically say the plan
for 2022. In the early days of matrix, we were just trying to make it work. Then we tried to make
it work right and managed to exit beta and launched the 1.0. The last year particularly
has been trying to make it work fast, and I hope we have now made it work fast. I will attempt
to prove this to you with a demo. We haven't seen anything yet. So, there is one of my cats
helping me here as a kid. Now, along the bottom here, you might see three, four icons in fact,
element X, element R, element X itself, so that's element X nightly, and then element normal.
We're going to talk about element X here. So, I've got the nightly here, and I'm going on the ship.
This is what you see as a little splash screen. I'm going to hit continue on that, and hopefully I've
got enough connectivity to connect to the server. That's a good start. If it takes that long to
discover that there's a server out there, then this demo might not go so smoothly. I'm going to
log in as my actual main real matrix account. I'm not going to type in my password in front of you,
but I'm going to pull it out of my password manager. Hit continue. If the server is there,
there's too many people in the room. It hasn't even started talking to the server yet, okay? And
that's it. I'm in. So, my account... So, if I was going to try to log in on my normal element
account, it would take 20 minutes, because I'm in 4,000 rooms that date back to literally day one,
or actually day minus two weeks or something of matrix, and I can go and scrub through all of these
gazillion rooms there, and they are all actually there. I can go and find somewhere, I know,
try not to expose anything too sensitive, but you can see it's actually already pulled in
room previews on all of these things. Going to, I know, this week in matrix, and there is the chat.
There is just no spinner here other than the slow network at the beginning, which really was the
slow network. And you can see we've got reactions in here. We've got some nice bubbles. We've got
replies. We've got joins and parts and day markers, read markers. We've got map markdown.
This is the SwiftUI incarnation of element X, but all of the heavy lifting here is done by Rust,
and it is transformative. The whole point here is to be faster than Telegram, and I think that we
might have got it, although if anybody's in a Telegram account with 4,000 rooms, please tell me
how long it takes to log into it afresh and how long it takes to launch. So talking about how long
it takes to launch, if I go and quit the app like that and relaunch it, we're in. That was it.
And I'm going to risk doing one other thing, which is to launch my non-nightly, which I haven't
actually used for a couple of hours. And again, it's synced almost instantly. And what I'm going to
do is that this is on a custom build, which is hooked up to Yeager. And if I go over to Yeager,
and if I have enough internet connection to even load the Yeager UI, this is going to be really
fun for demoing later if this is how bad the connectivity is. And I search for the well-known
element X Matthew app in the last, actually, let's do the last 15 minutes or five minutes even,
then we've actually hooked up Rust SDK so that all of the logging is structured and all of the
logging gets uploaded via LTP to Yeager. And if I had internet access, I should start tethering,
I would be able to show you a blow-by-blow account of what happened when I launched the app
then a minute ago. So what we're going to see when it finally loads, hopefully,
is first of all, it has to pull up a local cache of my messages if you're already logged in from
disk. For this, we currently use sled, which is a key value database native to Rust. It hasn't
been going amazingly well for us, and let's hope that's the right one. And as you can see here,
at the top, we've got the build operation. And the 410 milliseconds there, frankly,
should be more like 40, because all it's doing is loading up 20 rooms also out of sled. We're
going to move to SQLite because if nothing else, sled spends its entire life rebuilding itself
and defragmenting itself when you launch it, which is a bit unfortunate when you're trying to launch
an app quickly. Then it restores your session and gets a whole bunch of events out of it,
which is the first couple of events on the page. And if you scroll past those,
the really interesting one here is doing the sync. So this on the server is 90 milliseconds
to calculate your sync response. It's ended up being 900 over the wire because of all you people
with your electronic magnetic interference and your mobile phones. But still, you saw that it was
very usable. It's like a second to get to the point that you're viewing stuff. And in fact,
we already are interactive before the sync response returns thanks to the local store having been
resumed. So we have gone deep down the rabbit hole, so the saying goes, to try to optimize
the performance on element X. So it is as snappy as iMessage or WhatsApp or Telegram,
rather than the slightly clunky beast that we've had historically.
So before it looked like this, you got a synapse on the right. We've all sorts of fun
workers to do the various bits and bobs. And then we had element iOS with iOS SDK, mainly written
in Objective-C, matrix kit in the UI layer with more Swift in it. You had MXCrypto, again written,
I think, in Objective-C, and LibOm as the encryption library in C++ and C sitting below it.
And then the database layer was horrific with a mix of flat files, realm, core data, carrier
pigeon, element iOS has some issues. In our brave new sliding sync world, everything has changed.
On the left-hand side, we now have on iOS SwiftUI for the funky app I just showed you.
On Android, we have Jetpack Compose. Then we have Unify bindings to the Rust SDK,
which has been a lot of fun. Even on our Rust team, it's been hacking way, contributing async
bindings through to Swift, to Unify, so that we could expose Rust SDK, complete with nice futures
and async through to Swift and Kotlin. And then you've got Rust SDK itself, which is doing all
the heavy lifting. It's got the crypto crate within it. And then within that is Vdozomats,
which is our native Rust encryption implementation for matrix. Below that, you've got sled and in
futures equalize. This then talks through to a sliding sync proxy. And this is written in Go,
and it implements MSE 3575, which is sliding sync. And this is the magic for how this works so quickly.
It's basically storing, well, it's going and talking normal sync to normal Synapse. So this
could be Synapse or Dendrite or Conduit or anything on the right-hand side. The Golang thing is an
intermediary that is going and sucking up the state of your account, storing it in a local
Postgres and then talking the very, very responsive API in order to pull that data into Element X
itself. It does it by looking like this. Sliding sync lets the clients request the bare minimum
of data that they need to render the UI. So here's an almost real request where we say,
I want to see the currently visible rooms. I want to see the first page to preload it. So I want
20 rooms, 0 to 20. I want it sorted by recency and then name. I only want the avatar and whether
it's encrypted. I'm going to get the calculated name whatever. I don't want any messages because
we've done a waste time actually downloading scroll back. And we're going to filter it to
not have invites, not have old rooms, and not have those pesky space rooms. And whilst we're at it,
we want to have end-to-end encryption. We want to have two device messages and we want account
data. And the server or the sliding sync proxy will literally just return about 10k of data,
which is those 20 rooms with the bare data, bare essentials that you requested.
The key design criteria for sliding sync is the performance is constant with the number of rooms.
And this was the horrible mistake with the old API and frankly the whole design of matrix
historically that as you join more rooms, it gets linearly slower for basically everything.
And that was fine for the first few years when people are in a couple of hundred rooms,
but obviously we don't want to predicate the success of the protocol on, yeah, it's fine as
long as you're not a power user or it's fine as long as you don't actually use it. And if you
think of matrix being a bit like a file system, imagine how awful, and I'm looking at you,
EXT2, a file system would be if it just slowed down linearly with the number of files that you put
in a directory or some other characteristic. And as more and more rooms pop up in matrix,
it's not just chat rooms, it could be spaces. Now imagine that you go and join the EU and the,
you know, you're working in the EU government and the EU has got a massive space of hierarchy over
all of the countries and all of their public sector bodies. Even before you've talked to somebody,
you might need to have visibility over this big hierarchy of like a thousand rooms. You do not
want your matrix client to take a thousand times longer to log in or sync. So that's basically
the entire idea here that you can have an infinite number of rooms, bit like IMAP, where you can
have massive mail folders and yet you're only going to care about the subset that your client
wants to actually manipulate. Having requested this range of 20 rooms, you then get updates from
the server, and this is why it's called sliding sync, that you basically have requested a window
over these rooms, these 20 rooms or whatever it might be. And then as the state changes on the
server, you get updates of inserts of room here, delete one here, invalidate it here. It doesn't
have to be rooms, in future it could be members or other sort of characteristics. This has been
shamelessly stolen from Discord, so apologies to Discord folks if you're listening, but thank you
also for coming up with this nice approach for how to maintain performance and scrolling on apps.
And the end result is just many more. It makes login instant, sync instant, and also time to
view rooms instant. Element X is only going to talk sliding sync. There is no point in us wasting
time implementing both approaches, but they're really quite different, and we want everybody
using Element X for it to have a snappy snappy snappy experience. We've done a lot of iterations.
I've been driving the poor Rust team and sliding sync team and Element X team mad by constantly
demanding us to try to get the launch time down to 100 mils or something, and there's gone through
probably 10 iterations to see how we actually drive the API. And it's been really interesting.
The end conclusion is, first of all, when you launch the app, you sync that first
screen's worth of rooms, but without any timeline. Literally the request I just showed you.
Next, you immediately increase the timeline limit on that window to one. So you'll fill in the
room previews, and it was happening so fast earlier that we didn't even have time to spot it happening.
And then you pre-cache a page's worth of history of the visible rooms. So when I jumped into TWIM
and the history was already all loaded there, it's because the background and pre-cache had already
happened because I'd stopped scrolling the room list and immediately jumped in to pre-populate
the history for those pages. Then you also incrementally build the big list of all your rooms
in the background, which I guess technically is ON with a number of rooms, but because it's
happening in the background, it's not on the critical path. It means you can do the scroll
for all your rooms or search for a room by name instantly and be able to find them. And finally,
you cache it in Sled or SQLite. Rust SDK is doing all the heavy lifting here.
The code base is maturing really well. We got it audited at the Vdozomat Slayer last year,
thanks to Least Authority, funded by Gamatik. And we have, I think, three other audits planned
this year. They were meant to happen last year, but we had disruption along the way.
Then, high-quality bindings are critical for this. I mentioned that we've added AsyncFuture
to UniFFI. I think this stack could be the ultimate stack for building cross-platform
mobile apps going forwards. I mean, you can use Rust for the heavy lifting,
and then you hook up at the top a very thin but very native performance layer based on whatever
the iOS gives you. And at the beginning, UniFFI was a little bit, it didn't have everything we
needed, and so we invested to go and particularly hook up the future support. And the end result,
I think, is quite transformational. So that's Rust SDK. Meanwhile, whilst Element X is maturing,
we need to keep the existing clients secured too. But so it's going to take us a while to get to
parity between Element and Element X. And the project for this for crypto is called Element R,
confusingly. So this replaces the old cryptography implementations in JS, iOS, and Android SDK
with the same Rust crate that powers Rust SDK. So it's just the crypto crate that is providing
a consistent encryption implementation across all the platforms. So this means that if we have
hypothetically horrible CVEs popping up, we only have to fix them in one place in the Rust SDK,
rather than having to do it four times over between where by West Android and Rust.
And you can use this today. It is still beta, so it may kill your cat and flood your house.
I've been using it, it occasionally logs me out, which is a bit frustrating because initial sync
on V2 takes 20 minutes. But I recommend having a play if you're interested, enable Rusty
and Twen's encryption in labs on Element iOS. Androids will be coming fairly soon and Web
started working on Friday. We sent the first and received the first encrypted messages by Rust
crypto in a Wasm blob inside Element Web then. Rather than climatically, it looks and feels
identical to the current crypto, except it's written in Rust. Then another big thing we've
done to speed things up is faster remote room joins. So this is a huge internal change to Synapse.
So again, you only receive the subset of state you need to participate in a room.
Breaks all the assumptions that Synapse had. The rooms are typically atomic.
Instead, you basically trickle in the membership of the room in the background after having got
the minimum subset to join the room. So for instance, matrix HQ right now, there are 92,948
state events for every user who has ever joined or changed their name or left and a whole bunch
of other things. If you actually look at the subset you need to participate in the room,
it's 152. So this speeds up the room join time from 15 minutes to 14 seconds. So finally,
we will hopefully have fixed the problem where somebody gets and stores matrix Synapse immediately
tries to join matrix HQ, sits there for 15 minutes looking at errors as their computer
explodes and wonders why everybody thinks matrix is as amazing as it is. So I mean,
the computer will still explode because they're still having to talk to all of the servers in
the room, but at least they will be responsive in 14 seconds. And we hope that the Synthelating
conversation in matrix HQ will be such that it distracts them from the smoke coming out to their
server. There's still a lot of room for improvement here. We shouldn't be hammering dead servers,
which is where a lot of that time is going. And also, we should be caching the partial state.
So, you know, if I want to join matrix HQ, the server can just go and hand me the events I need
to do that. Another big thing is matrix RTC. So this is the name we're referring to MSE3401
and 3898 as many because those were awful names, whereas matrix RTC is a bit more snappy. And this
is multi-party native VoIP. We've always had one-to-one VoIP. Here, we are standardizing the
multi-party Zoom teams, Jetsy style experience, but in a heterogeneous way. So you can use
different clients. This is in lamps and element web. Video rooms looks like this on the right
hand side, powered by element call. And it works as what we call a matroshka widget, where you
embed element call here as a widget. So this is an iframe on the left hand side. But even though
element call itself is a matrix client, it is piggybacking on the host's matrix client. So it
shares the encryption and the identity. You don't have two logs in sessions. And really
excitingly, we have multiple independent implementations of this in element call, hydrogen, third room,
and also, I believe, famously, it has an independent implementation in their healthcare
product for Germany. And I'll very quickly try to show you a demo of this. And I'm going to show
you, where am I going? Oh, cool. That's good. Fozdom 2023. If anybody wants to heckle along
on this, then feel free. Oh, I don't have any internet access, do I? Video conferencing demo, so
when you don't have any Wi-Fi, it's always a good idea, right? Let's see how it does. So I'm going
to pop into that room there. And here is element call. And then I'm also going to spin up a local
hydrogen. Here's what I made earlier. Oh, hello, Amundee. Found some meeting here. This is Amundee
and everybody. She co-founded Matrix. And I'm going to wish that this thing was telling me that a
video call was happening in the room. And still being one that's ended, but in practice, there's
one happening right now. I wonder if this is... I wonder why? Okay. Perhaps we'll use a different
room. Let's go to Fozdom 2024. Back to the future, everybody. And go over to hydrogen here. And I
think I should be able to just be able to join Fozdom 2024 on call.ams.host. Yeah, okay, here
I can actually join it. And hey, Presto, you've got me staring at myself like a muppet because
nobody else is responding to my comedy joke of moving to 2024. So everybody, oh, hello. Perfect.
Somebody at the back. Thank you. Oh, and there's Amundee. So this thing is really cool because
it is completely standardized multi-party void signaling. We have two entirely different
code bases. There is not a line of codes in common between hydrogen here on the left
and element call here on the right. And just like back in the day on the phone network,
we could call each other on different things or have different set clients or whatever it might
happen to be. Oh, we've got somebody remote. That's awesome. Then here we actually have a proper
heterogeneous thing. So unlike JETC or some other conferencing system where everybody has to end up
using the same system to work, this is providing an interoperable thing. And crap out on this one
because I've got something better. I've got something better. So one of the other projects we
have worked at, which we're just releasing now is called Waterfall. Now what we just did then was
full mesh. All the clients were talking to one another. I'm amazed that it worked as well as
it did. What you want to do, though, is to have what's called an SFU. So these guys which go and
mix together the local video calls. And Sean Dubois, who's the project leader at Pion,
the Golang WebRTC, wrote one of these called SFU to SFU based on reading MSC3401. And we
renamed it Waterfall. We've fleshed it out and we've hooked it up to element call. And I will
endeavour to show you what that looks like. And it's going to be quite similar in some ways.
Let me actually try a demo, demo room in here. Again, if someone is already in there, I'm going
to try a fresh one. Let's call it fresh demo. Again, if anybody wants to try following along
on this URL, if you can see it, please do so. Now this looks a little bit different because
it's connecting to the SFU instead. Oh, hello and hello. And hopefully we will get some video
off and on. So this has been bounced off the go SFU. But perhaps I'll distract everybody by
zooming. So this has got a completely different layout on it. And it thinks it's connected.
Oh, there's Simon. I'm glad that Simon of all people is able to connect because he has written
the vast majority of Waterfall. So thanks for demoing Simon. And I suspect it is not like in
the packet loss here on my side. People are trying to connect in there. But it's working for some
folks. It's very, very new, very alpha, but it's exciting to actually see this thing working.
And if I quickly turn on debug here in developer mode, you'll see that it actually supports
simulcast. So here you can see that Simon is 640 by 360 pixels. Whereas this guy here in dandelion,
hello dandelion, is 320 by 240. And if I go and zoom embarrassingly, then it should catch up. There
we go. 640 by 480. And it's going to renegotiate the higher resolution stream through. And all
the people are going and actually uploading multiple low resolution and high resolution ones.
So this is very early, but it's the shape of the future for doing proper massive scalable
SFU based conferences. And that's actually looking good. I'm going to take a quick selfie.
There we go. Right. Next demo or next stuff. I'm running out of time. I've got a lot of demos.
OpenID Connect. Matrix is subject to MSE approval. Moving over to OpenID Connect. It rocks and gives
us 2FA, multifactorial, pass keys, QR code login. You don't have to implement the weird matrix
off APIs on the server or the clients anymore. ElementX has native OIDC on iOS already and
will be OIDC first. First room is the first OIDC only matrix client. Go to our OIDC yet.com for
the gory details. So SigningSync faster joins native VoIP and OIDC is a big change. Like this is
fundamentally changing how federation works, how VoIP works, and critically how servers sync data
to clients, and how you log in. Couldn't be a bigger change. So we are taking the liberty of
declaring it matrix 2.0 when we finally release it. So this is not a breaking change. This is pure
enthusiasm basically on my behalf because I think it's worth saying, hey guys, come back to matrix,
give it another go because we fixed all the crap stuff and we're calling it matrix 2.0. Okay,
I'm not doing well on time. We're halfway through in theory. We're going to go now to the future.
Flying cars and jet packs and all that good stuff. First of all, digital markets act. Where we're
going, we do not need gatekeepers. If you haven't heard about the DMA, it is a fascinating piece
of legislation that mandates the big tech companies must interoperate together, particularly for
their communication services. The whole idea is that it empowers users to pick the services they
want to use and trust without sacrificing the ability to talk to other people. Now frankly,
forces the gatekeepers to differentiate based on quality and features, rather than relying on a
monopolistic situation where all of their users happen to be trapped in the silo. This is happening
right now. The rules came into force in November. The rules start to apply in May and then gatekeepers
get designated and it will start getting enforced, which is chunky GDPR style finds in March 2024
if the gatekeepers have not gone and interoperated things together.
Turns out the matrix already provides a secure interoperable communication protocol. Who knows?
The DMA requires the gatekeepers to maintain end-to-end encryption, which is good news. There's
been a lot of paranoia that DMA is a secret attack on end-to-end encryption. It really isn't. Having
spoken to the folks responsible, they really want to make sure that if WhatsApp does E2E today,
then an interoperable WhatsApp also needs to do E2E. They don't want to be responsible for
destroying privacy. So to re-encrypt, you need to either do it on the client side, so you would
install something like a WhatsApp to matrix app if you want to link your WhatsApp account into
matrix, or you could do a multi-head approach effectively, open APIs and have your app talk
to WhatsApp as well as matrix or whatever. Or everybody ends up talking the same protocol,
which means on the server side, the gatekeepers would have to add support for matrix or XNPP
or RCS or whatever it might be alongside their somewhat legacy proprietary protocol.
ITF is established a working group called MIMI, more instant messaging interoperability targeting
the everybody talks the same protocol approach. We've proposed matrix as an ITF draft for content
and transport layers, and we're trying to work with them to make the most of the mix.
Decentralized MLS, this is another big thing where we're going, we don't need salamanders,
because if you haven't noticed, all of the encryption historically have been called axolotl
or ohm or proteus, all of which are species of salamander. MLS is another ITF working group,
in fact the one from which MIMI has emerged for next generation end-to-end encryption.
We have figured out how to make MLS work in a decentralized model. We have implemented it
in a toy type script stack called MLSTS. It is currently being re-implemented on top of
open MLS and Rust. At which point, when it works, we will integrate it into Rust SDK and build it
into real clients and write an MSE. Are we MLS yet dot com for all the gory details?
Here are some early performance testing, which is pretty interesting. Let me get the key right
for you here. If you look at MEGOM creating new sessions, this is obviously the log log scale
for all of you mathematicians. You can see this red dashed line here, showing how MEGOM
sessions scale with a number of users, and it's up at 100 seconds if you've got 100,000 users
in the room, which is pretty slow. However, if you look at an MLS update or adding new members,
then they're down at 1,000 milliseconds, so a couple of seconds. This is a major
algorithmic improvement for certain operations over even VODOSMATS. Support of VODOSMATS,
which feels relatively new, may get displaced by open MLS and DMLS when we get there, hopefully
later this year. In peer-to-peer matrix, where we're going, we don't need servers. Matrix is way
too dependent on servers, and the admins and the risks of internet shutdowns and censorship.
This is because home servers have to store their users' chat history and metadata. Peer-to-peer
matrix exists to fix this. This is a long-running blue sky project, so to speak, where we go and
embed your home server inside the app in order to not have a server running in the cloud.
Dendrite is the server we use. Big news on Dendrite is, as of a few weeks ago, it passes 100%
server API compliance, so it has a parity of a good old Synapse, and 93% client server API,
and the 7% is boring stuff we don't really care about for this. Pinecone, the peer-to-peer overlay
network, is going really well as well. We just switched to soft state routing for reliability,
and as of about 4 a.m. this morning, not me. Initial store and forward relaying is here.
I will very rapidly try to demo that. That's still my phone there. If I go and launch Peer-to-peer
matrix, Dendrite is not running. Now it is running. This has literally got a Dendrite running inside
my phone here, and if I go over to Visor here, I should hopefully also be able to run it on
my Android thing. Here it is, and did it already crash? So it is a bit crashy. It is still beta.
I've got Fosdom demo here, Fosdom demo here, and slightly, okay, when I first say hello there,
then you can see yay, messages flowing back and forth peer-to-peer. Now if I take my iPhone
and put it on to airplane mode like so, and send some messages through from Android,
they still go. So this is Peer-to-peer matrix running over Bluetooth flow energy.
It silently fell over from running over IP into Bluetooth mode. Now the exciting thing is that
if I also run... Well, this is going to be an interesting demo. So here is Element Peer-to-peer
running in iOS or macOS, because you can do that on an M1 Mac, and apparently there are five
connected peers, which is three more than I was expecting, so hello everybody out there who's
about to screw up my demo, and you can see the same history coming on here, and I can
send a message through, and you can see both on iOS it came through on Android too. Now the really
interesting thing is if I go and... Now you are going to screw up my demo. If I go and kill the
Peer-to-peer app on Android, and I send a message here saying hello relaying, and actually not in
that room, I'm going to do it in my DM through to Android. I'm going to say testing relays,
or very badly typed, and critically my... Where has it gone? Has it crashed? No, there is. So I'm
not actually in that room. I'm only in one room on my Mac here. However, this hopefully has gone
and peered, relayed through to my Android phone. So if I now go and kill the app here on iOS,
and relaunch it here on Android, if I knew how to use Android, come on. This is going to work,
or is it going to search Google? There it is. Perfect. Hopefully the first thing this thing will do
is to... I'm in the wrong room. If Dendrite launches, the red bar is telling me that it can't
tell and talk to its own Dendrite. You can do it. Yes! Testing relaying. This is huge because
historically Peer-to-peer matrix has had a massive problem that the other guy has to be online
running the app at the same time. And a long story, but I ended up on an aircraft carrier
a couple of months ago going and trying to explain a bunch of people how they could use
matrix in that environment. And there are a bunch of us on board this thing. And it turns out that
an aircraft carrier is a massive floating Faraday cage. And we went and fired up Peer-to-peer
matrix on it. And we're very smart that we can talk to one another, but you had to physically
wave at the other guy to get them to launch the app so they could receive the message.
Whereas with a relay, you can obviously talk to them even if the app isn't running.
So that's big. Right. We're almost there. Matrix is not just for chat and VoIP. This is
the final thing. Third Room is a decentralized consular for, well, matrix is a consular for
any kind of real-time data. Third Room is this tiny project done by Robert May and AJ to provide
an open decentralized platform for spatial collaboration of any kind built on matrix.
It uses hydrogen, 3JS, bit ECS, and rapier for a new engine called manifold. And the idea is you
take a matrix room. You upload some GLTF 3D assets to it. You upload some WASM or JavaScript scripts
to it. They use matrix RTC to spell up spatial VoIP and physics synchronization over WebRTC.
And you get WebFirst, open decentralized virtual worlds and spatial apps without any of the
cryptocurrency or token or Facebook stuff, apart from possibly the hardware. And it looks like this.
So you can literally go to it right now, thirdroom.io, and I'm going to go to a world called
third room presentation here. And it's going to hopefully pull this up out of my index DB,
because I don't want to wait to download 50 megabytes over the network. And if the demo
gods are smiling, then you find yourself in this rather snazzy demo world. Now, this is running
in Braille. So no plugins or anything. And it's going at 60 frames a second. It does
this with proper multi-threading using shared array buffers and atomics to synchronize together
the WebGL thread and also the game thread and the main thread in the UI. You can see I'm indeed
wandering around there wearing a placeholder avatar. I can flip into third person view here.
And you can see I'm also wearing the same beautiful thing. We haven't got customizable avatars yet.
Some of the things I can show you here is that you can go and click on buttons. And
this is actually a script showing the layout of the different threads. I don't have time to show
you that right now, but the game thread has got like rapier physics and WebAssembly. But a really
fun thing is that you can just do freeform scripting of any kind. So one example could be
this silly, silly, silly demo, which hopefully will load up rapidly. Oh, that's what happens if
you this is a bug where you have your worlds overlaying on one another. I'm going to keep it
like this, but this is pretty cool. So we've got the construct room from the matrix,
so I'll see for in place on top of Mars down here. And if I go over to the TV and I click on it,
predictably enough, the entire world goes matrix. So the script for that is literally just sitting
there as a bunch of JavaScript uploaded into the media repository. And it's compiled down
to Wasm in real time by the engine by QuickJS, thanks to Fabrice Bellard. And it's like four
lines of code. You use WebSG, which is our new API called WebSceneGraph that we're going to propose
to W3C as a 3D API for manipulating JLTF scene graphs. Now you make it intractable and then every
tick, every frame, you see if it's being pressed and then you toggle the state and you enable
the matrix material on the room. It's slightly cheated by hard coding it on the in the engine
for now like this. Something that we haven't hard goaded though is this guy, which is really
exciting. I'm going to refresh his time. And here you can see a big scary black blob with
Poland noise, which is pulsating according to my voice as I bellow at it. And this thing is
actually a huge chunk of C, which is compiled down to Wasm and is going and programmatically
changing in real time the JLTF scene. So this is like the first proper, more advanced capabilities
on top of third room. The whole idea is you can build any old app on top of this. You could
build Figma on this. You could do multiplayer blender. In fact, we have an in-world editor in
here where I can go and select this guy at the bottom and they will have a big white bagel
around it. And we don't yet have the property editor, but you should be able to go in and
directly manipulate it, change the opacity, the transformations, et cetera, and all that sort
of thing. And it really ends up feeling a lot like the web. Rather than a DOM, you've got JLTF,
rather than the DOM API, you've got the WebSG API, rather than JavaScript, you've got Wasm,
Sandblocks, Blobs, with Rust and Zig and C and JavaScript within it. And there is one final thing
I'm going to try to show you, which is probably going to go horribly wrong, which is that we've
just added WebXR into third room. So if I go and put on my Facebook device, and there we go.
And I back away a bit. Probably unplug it. You'll see, hopefully, in fact, I need to go full
screen on that. I guess I do. There we go. Is that coming through? You can see that here I am
wondering around third room. Probably can get them full off the stage and break my neck. And I can
go and, like, spawn object. So you can have big crate. I can throw the crate away. I can spawn
some other big crate. Let's run away from that one. Go and pick this guy up and throw it away.
Go and pick that one up. It's over here. And throw it away, et cetera. And I mean,
this is running at 90 frames a second. 90 frames a second. So that's a bit weird. It is
as least as good as the native MetaHorizon stuff, the Facebook of ships, except it's running within
WebXR in a browser in a completely open environment. So we're kind of hoping this provides a really
viable platform to build a genuine open spatial collaboration plane for the web. I've already
spoken about that. Coming up next is persistence. So we don't yet persist the changes into the
matrix room, but we will by uploading little bits of JLTF files so you can even have bots which
go into that. Another thing I should have shown you, but forgot, was this guy somewhere. I've
lost it already. Never mind. Well, I was going to show you it's an echo. This guy here. But if in
this room I go in, and I think this is going to be a comedy, yeah, that's what happens if London
and Mars get mixed together for everybody. If I go and say hi because, look, it's matrix room,
then if I do slash echo something, I get an echo back. That echo has come from a
widget in Wasm running inside the world. So you can program matrix now from within Wasm
blobs sitting within the room. Nothing to do with third room. You could use this in clients,
et cetera, to start doing client-side widgets. So what's next? Loads of stuff. One big PSA
is that Gitter is going native matrix roughly next week. We basically can't afford to run
both the Gitter infrastructure and a bunch of matrix infrastructure. So Gitter will become
a branded element instance. The API will go away. Please use matrix instead. And finally,
we need help. Friends, don't let friends use proprietary chat services. Please use matrix.
And critically, and this is new and it's really important, if you're benefiting commercially
from matrix, please financially support the foundation because it's stuck in this horrible
feedback loop at the moment where the better we make matrix, the less inclined it seems
that people want to pay for support or pay for things if they can just grab it on GitHub.
This can end up being a disaster where we run out of cash. So please, please, please contribute
back, particularly if you're a government. You've got loads of money. Also, run a server,
run bridges bots, build on matrix, follow us on MasterDone, and thank you very much.
Thanks for listening. Sorry for running on time, as always.