[00:00.000 --> 00:14.160]  Okay, hello everybody. Can you hear me? Yes, I see thumbs up at the back. Please come in,
[00:14.160 --> 00:20.240]  come in, roll up, roll up for the Matrix show or introducing Matrix 2.0 or how we are going
[00:20.240 --> 00:26.320]  to make or how we have made Matrix go boom, very technical term. Please take your seats,
[00:26.320 --> 00:33.760]  ladies and gentlemen. Right, so hi everybody. I'm Matthew. I'm the project lead and co-founder
[00:33.760 --> 00:39.280]  of Matrix and I'm here to tell you all about the work we've been doing to fix Matrix's performance
[00:39.280 --> 00:45.520]  problems and a few other things as well. So I'm guessing that a bunch of people know what Matrix
[00:45.520 --> 00:50.000]  is, given that Vostem has been running on Matrix during the pandemic and we're doing
[00:50.000 --> 00:55.200]  four hard to lead doing a hybrid Matrix version of Vostem right now as we speak. So hello to
[00:55.200 --> 01:01.280]  everybody following along on the HLS stream by Matrix. You probably know that Matrix is an open
[01:01.280 --> 01:06.800]  network for secure decentralized real-time communication. We have to say this because
[01:06.800 --> 01:11.120]  people watching the video might not know later on. You can use it for interoperable chats,
[01:11.120 --> 01:16.240]  so following along on Vostem, bridge through to ILC or X and PP, etc. You can use it for
[01:16.240 --> 01:22.240]  interoperable VoIP, but Matrix at its core is a real-time communication fabric for any kind
[01:22.240 --> 01:27.120]  of real-time comms. So you could use it for communication within VR or AR. You could use it
[01:27.120 --> 01:32.960]  to synchronize world data within VR and AR. You could use it for IoT. It is basically meant to be
[01:32.960 --> 01:40.800]  the missing communication layer of the open web. So no single party in Matrix owns your conversations.
[01:40.800 --> 01:45.040]  The second you talk to somebody on a different server, your conversations get replicated equally
[01:45.040 --> 01:50.720]  between the different servers, so there cannot be a gatekeeper. There cannot be some big tech company
[01:50.720 --> 01:55.600]  going and holding your conversations hostage. Instead, the conversations are shared between
[01:55.600 --> 02:02.480]  all the participants. To apologize, the network is a mesh of servers like Vostem.org, say,
[02:02.480 --> 02:07.200]  on the right, which might be talking to Mozilla.org, which might be talking to Matrix.org, or Nome.org,
[02:07.200 --> 02:12.640]  KDE.org, or whatever it is, and you have native Matrix clients like Element, or Fluffy Chat,
[02:12.640 --> 02:17.280]  or Mecca, or Katernion, or hundreds of others these days, which connect through to the Matrix
[02:17.280 --> 02:23.200]  server. Then you have application services, which glue additional things onto the Matrix server,
[02:23.200 --> 02:27.920]  like bridges, or bots. You have identity servers, which we don't talk about because they're a mess,
[02:27.920 --> 02:32.800]  and we need to get rid of them. And then you've got application services that bridge through to
[02:32.800 --> 02:38.800]  things like Slack, or Teams, or Telegram, or ILC, or XMPP, et cetera. And that is a schematic view
[02:38.800 --> 02:44.480]  of the public Matrix network. Now, the Matrix ecosystem, as it sits today, looks something
[02:44.480 --> 02:50.640]  like this. You've got the Matrix spec still being the kind of one true commonality across the whole
[02:50.640 --> 02:57.200]  ecosystem, spec.matrix.org, a whole bunch of markdown, and swagger, or open API, I should say,
[02:57.200 --> 03:01.840]  that defines how the server's on the bottom, talk to the clients on the top. So you've got Synapse,
[03:01.840 --> 03:06.880]  as our first-gen Python server, which has been proving to be a bit more than first-generation,
[03:06.880 --> 03:12.480]  as we've invested a lot of time making it fast and generally performant. So Synapse is not going
[03:12.480 --> 03:19.520]  away anytime soon. If anything, it is corroding into Rust as the Python is augmented cybernetically
[03:19.520 --> 03:25.600]  with a bunch of Rust. Then we have Dendrite in Go, which is also doing very well and is
[03:25.600 --> 03:30.400]  ending up focusing more on embedded use cases. So you use Synapse for massive servers and use
[03:30.400 --> 03:36.080]  Dendrite for ones you embed into an app. Then we have application services and bridges for
[03:36.080 --> 03:41.440]  things like ILC bridging, et cetera. And then many other servers and bots and bridges from the
[03:41.440 --> 03:46.960]  wider community. The green stuff is stuff that we published as the matrix.org foundation,
[03:46.960 --> 03:52.880]  all Apache licensed for people to build on top of. We have, then, the clients. On the far left,
[03:52.880 --> 03:59.520]  we have our original SDKs of matrix JS SDK, React SDK on top of it, and then iOS SDK and
[03:59.520 --> 04:06.480]  Android SDK too. These are relatively venerable SDKs now. JS SDK is eight years old, which is,
[04:06.480 --> 04:12.160]  you know, enough to probably get a degree or something. iOS SDK is about the same age. Android
[04:12.160 --> 04:17.200]  SDK is a little bit younger, but this is what the current generations of element use today.
[04:18.000 --> 04:24.000]  Then we have Hydrogen, which is a relative newcomer. This is a progressive web app SDK,
[04:24.000 --> 04:29.360]  super small. It's about 70, 80 kilobytes in total for the whole thing, including all end-to-end
[04:29.360 --> 04:35.360]  encryption. And it's designed for embedded web matrix instances. So we have Hydrogen itself,
[04:35.360 --> 04:42.000]  hydrogen.element.io, as a very lightweight PWA for playing around with matrix. We have chatterbox
[04:42.000 --> 04:48.240]  as intercom style embeds a matrix chatbox into your website. We have Surderoom, which is our
[04:48.240 --> 04:54.640]  crazy science fiction spatial collaboration on top of matrix platform, where you want to have
[04:54.640 --> 04:58.560]  the matrix bit as lightweight as possible, which is why we use Hydrogen, the lightest element,
[04:58.560 --> 05:04.160]  to make it happen. And then the thing we'll be talking about a lot today is element X. So this
[05:04.160 --> 05:11.200]  is a total rewrite of element mobile on iOS and Android. And for maximum cliche, we have rewritten
[05:11.200 --> 05:20.640]  it in Rust using the matrix Rust SDK. I hear we have Rust fans in the audience. So we'll be talking
[05:20.640 --> 05:24.800]  a lot about that. Meanwhile, on the community side, there are just more and more clients all over
[05:24.800 --> 05:30.800]  the place, like Thunderbird released its native matrix implementation. We've got Watch the Matrix,
[05:30.800 --> 05:36.160]  which is an excellent Apple Watch matrix client, which doesn't tether to your phone. It's actually
[05:36.160 --> 05:44.320]  running on the watch itself. NeoChats, and KD, and QTE, and C++. I wish I could name them all.
[05:44.320 --> 05:50.480]  I can't. I don't have time. So help for the network. Here is the total number of users we've
[05:50.480 --> 05:54.800]  ever seen on matrix, which is, looked as if it was going to hit 90 million, and then somebody
[05:54.800 --> 05:59.440]  turned off the server. Kids, this is what happens if you turn off the server, because it means
[05:59.440 --> 06:05.280]  the graph goes down. So please never, ever turn off your matrix servers ever again. This is
[06:05.280 --> 06:09.760]  literally the phone home stats that Synapse reports. So it doesn't include Android. It doesn't
[06:09.760 --> 06:15.840]  include Conduit or other servers. It also doesn't include all of the paranoid people,
[06:15.840 --> 06:19.600]  which is quite a lot of people running matrix who obviously aren't going to phone home their usage
[06:19.600 --> 06:25.360]  stats to us. It does include guest users and bridge users. So it's a little bit of an overestimate.
[06:25.360 --> 06:30.240]  But the important thing is the shape of the graph, which, as you can see, is continuing to go well,
[06:30.240 --> 06:36.080]  apart from that guy who turned off his server. In terms of the spec, we have fallen into this
[06:36.080 --> 06:45.280]  cadence of quarterly releases of matrix since the big 1.0 back in 2020, and then particularly in
[06:45.280 --> 06:53.280]  2022, we've managed to crank out a point release every quarter. Just in, I think, the day before
[06:53.280 --> 06:58.000]  Last Foster, we had spaces, restricted joins, and actually the matrix ERI scheme, which is now
[06:58.000 --> 07:03.680]  being implemented all over the place. Then a few months later, we added aggregations and restricted
[07:03.680 --> 07:08.080]  knocking. Now, these features have often been around for years, but this is actually formalizing
[07:08.080 --> 07:14.960]  them into the proper long-term supported spec. 1.4 added threads, massive feature that has been
[07:16.000 --> 07:21.360]  an epic to get done, but it landed. Edits, private read receipts, a very long time
[07:21.360 --> 07:26.080]  complained about matrix being that you couldn't turn off read receipts, turns out it's surprisingly
[07:26.080 --> 07:31.760]  hard to do, but we now have it specced and implemented. Then 1.5 in November, we added in
[07:32.480 --> 07:37.760]  formalized references, we fixed up some things in the ASAPI, and I think we have basically
[07:37.760 --> 07:43.680]  maintenance release on 1.6 any day now. So nothing too exciting in the next release,
[07:43.680 --> 07:52.480]  mainly because we're building up to the big 2.0. Nice stat is that there were 120 MSCs in 2022,
[07:52.480 --> 07:58.480]  of which 39 came from the community, 27 from new contributors, 30 from the gray beards of the
[07:58.480 --> 08:05.440]  spec core team, and then 51 from the folks who are paid to work on matrix.org, mainly by element.
[08:05.440 --> 08:12.480]  So it's a reasonable mix of community and core project work. In terms of uptake,
[08:12.480 --> 08:16.160]  other than that, we obviously help the world's best open source conference, dodge COVID.
[08:16.160 --> 08:20.720]  Hopefully, apologies if matrix was painful over the last couple of years, but it's probably
[08:20.720 --> 08:26.800]  better than no false to me at all. Lots of government uptake. New news is Germany,
[08:26.800 --> 08:31.280]  we have bundles messenger rolling out matrix across the whole German government in November,
[08:31.280 --> 08:36.080]  also good martyred, the German healthcare agency, proposing it as a neutral standard for
[08:36.080 --> 08:41.520]  secure messaging in healthcare. Lots and lots of associated deployments in healthcare, education,
[08:41.520 --> 08:47.360]  utilities, manufacturing. Basically, if you are an organization who cannot put their data
[08:47.360 --> 08:53.440]  unencrypted into some proprietary silo, like teams or Slack, I think matrix hopefully provides
[08:53.440 --> 09:00.080]  a good alternative. Moodle is busy integrating matrix into the learning management system.
[09:00.080 --> 09:04.480]  Automatic have got a project called chat tricks, which embeds matrix into WordPress,
[09:04.480 --> 09:10.160]  so you can literally dump your little chat console based on hydrogen into your WordPress blog.
[09:10.160 --> 09:15.200]  Reddit is rumored to be building chat capability powered by matrix, mainly because I think they
[09:15.200 --> 09:19.520]  had public who sign up enabled and somebody logged in and discovered a matrix over there,
[09:19.520 --> 09:24.080]  and there's a lot of interest over matrix being potentially the communication there
[09:24.080 --> 09:30.800]  for the digital market sites. So, last year, we put out this slide to basically say the plan
[09:30.800 --> 09:36.240]  for 2022. In the early days of matrix, we were just trying to make it work. Then we tried to make
[09:36.240 --> 09:42.000]  it work right and managed to exit beta and launched the 1.0. The last year particularly
[09:42.000 --> 09:48.240]  has been trying to make it work fast, and I hope we have now made it work fast. I will attempt
[09:48.240 --> 09:58.720]  to prove this to you with a demo. We haven't seen anything yet. So, there is one of my cats
[09:58.720 --> 10:06.720]  helping me here as a kid. Now, along the bottom here, you might see three, four icons in fact,
[10:06.720 --> 10:12.480]  element X, element R, element X itself, so that's element X nightly, and then element normal.
[10:13.280 --> 10:17.840]  We're going to talk about element X here. So, I've got the nightly here, and I'm going on the ship.
[10:18.720 --> 10:24.560]  This is what you see as a little splash screen. I'm going to hit continue on that, and hopefully I've
[10:24.560 --> 10:30.560]  got enough connectivity to connect to the server. That's a good start. If it takes that long to
[10:30.560 --> 10:34.640]  discover that there's a server out there, then this demo might not go so smoothly. I'm going to
[10:34.640 --> 10:39.680]  log in as my actual main real matrix account. I'm not going to type in my password in front of you,
[10:39.680 --> 10:48.880]  but I'm going to pull it out of my password manager. Hit continue. If the server is there,
[10:48.880 --> 10:55.280]  there's too many people in the room. It hasn't even started talking to the server yet, okay? And
[10:55.280 --> 11:06.160]  that's it. I'm in. So, my account... So, if I was going to try to log in on my normal element
[11:06.160 --> 11:12.160]  account, it would take 20 minutes, because I'm in 4,000 rooms that date back to literally day one,
[11:12.160 --> 11:17.760]  or actually day minus two weeks or something of matrix, and I can go and scrub through all of these
[11:17.760 --> 11:22.800]  gazillion rooms there, and they are all actually there. I can go and find somewhere, I know,
[11:22.800 --> 11:27.360]  try not to expose anything too sensitive, but you can see it's actually already pulled in
[11:27.360 --> 11:32.640]  room previews on all of these things. Going to, I know, this week in matrix, and there is the chat.
[11:32.640 --> 11:36.880]  There is just no spinner here other than the slow network at the beginning, which really was the
[11:36.880 --> 11:43.920]  slow network. And you can see we've got reactions in here. We've got some nice bubbles. We've got
[11:43.920 --> 11:50.400]  replies. We've got joins and parts and day markers, read markers. We've got map markdown.
[11:51.120 --> 11:58.080]  This is the SwiftUI incarnation of element X, but all of the heavy lifting here is done by Rust,
[11:58.080 --> 12:03.360]  and it is transformative. The whole point here is to be faster than Telegram, and I think that we
[12:03.360 --> 12:09.120]  might have got it, although if anybody's in a Telegram account with 4,000 rooms, please tell me
[12:09.120 --> 12:13.920]  how long it takes to log into it afresh and how long it takes to launch. So talking about how long
[12:13.920 --> 12:21.040]  it takes to launch, if I go and quit the app like that and relaunch it, we're in. That was it.
[12:26.640 --> 12:32.720]  And I'm going to risk doing one other thing, which is to launch my non-nightly, which I haven't
[12:32.720 --> 12:37.920]  actually used for a couple of hours. And again, it's synced almost instantly. And what I'm going to
[12:37.920 --> 12:46.000]  do is that this is on a custom build, which is hooked up to Yeager. And if I go over to Yeager,
[12:46.000 --> 12:50.880]  and if I have enough internet connection to even load the Yeager UI, this is going to be really
[12:50.880 --> 12:55.600]  fun for demoing later if this is how bad the connectivity is. And I search for the well-known
[12:55.600 --> 13:00.880]  element X Matthew app in the last, actually, let's do the last 15 minutes or five minutes even,
[13:00.880 --> 13:08.640]  then we've actually hooked up Rust SDK so that all of the logging is structured and all of the
[13:08.640 --> 13:17.200]  logging gets uploaded via LTP to Yeager. And if I had internet access, I should start tethering,
[13:17.200 --> 13:23.280]  I would be able to show you a blow-by-blow account of what happened when I launched the app
[13:23.280 --> 13:27.760]  then a minute ago. So what we're going to see when it finally loads, hopefully,
[13:27.760 --> 13:33.120]  is first of all, it has to pull up a local cache of my messages if you're already logged in from
[13:33.120 --> 13:40.000]  disk. For this, we currently use sled, which is a key value database native to Rust. It hasn't
[13:40.000 --> 13:45.920]  been going amazingly well for us, and let's hope that's the right one. And as you can see here,
[13:45.920 --> 13:50.800]  at the top, we've got the build operation. And the 410 milliseconds there, frankly,
[13:50.800 --> 13:56.960]  should be more like 40, because all it's doing is loading up 20 rooms also out of sled. We're
[13:56.960 --> 14:02.400]  going to move to SQLite because if nothing else, sled spends its entire life rebuilding itself
[14:02.400 --> 14:06.000]  and defragmenting itself when you launch it, which is a bit unfortunate when you're trying to launch
[14:06.000 --> 14:10.560]  an app quickly. Then it restores your session and gets a whole bunch of events out of it,
[14:10.560 --> 14:14.080]  which is the first couple of events on the page. And if you scroll past those,
[14:14.080 --> 14:21.120]  the really interesting one here is doing the sync. So this on the server is 90 milliseconds
[14:21.120 --> 14:25.840]  to calculate your sync response. It's ended up being 900 over the wire because of all you people
[14:25.840 --> 14:31.280]  with your electronic magnetic interference and your mobile phones. But still, you saw that it was
[14:31.280 --> 14:36.400]  very usable. It's like a second to get to the point that you're viewing stuff. And in fact,
[14:36.400 --> 14:42.000]  we already are interactive before the sync response returns thanks to the local store having been
[14:42.880 --> 14:48.480]  resumed. So we have gone deep down the rabbit hole, so the saying goes, to try to optimize
[14:48.480 --> 14:53.760]  the performance on element X. So it is as snappy as iMessage or WhatsApp or Telegram,
[14:53.760 --> 14:57.040]  rather than the slightly clunky beast that we've had historically.
[14:58.960 --> 15:04.320]  So before it looked like this, you got a synapse on the right. We've all sorts of fun
[15:04.320 --> 15:10.160]  workers to do the various bits and bobs. And then we had element iOS with iOS SDK, mainly written
[15:10.160 --> 15:16.720]  in Objective-C, matrix kit in the UI layer with more Swift in it. You had MXCrypto, again written,
[15:16.720 --> 15:22.400]  I think, in Objective-C, and LibOm as the encryption library in C++ and C sitting below it.
[15:22.400 --> 15:29.040]  And then the database layer was horrific with a mix of flat files, realm, core data, carrier
[15:29.040 --> 15:37.040]  pigeon, element iOS has some issues. In our brave new sliding sync world, everything has changed.
[15:37.040 --> 15:41.440]  On the left-hand side, we now have on iOS SwiftUI for the funky app I just showed you.
[15:42.000 --> 15:47.760]  On Android, we have Jetpack Compose. Then we have Unify bindings to the Rust SDK,
[15:47.760 --> 15:53.280]  which has been a lot of fun. Even on our Rust team, it's been hacking way, contributing async
[15:54.240 --> 16:02.080]  bindings through to Swift, to Unify, so that we could expose Rust SDK, complete with nice futures
[16:02.080 --> 16:08.000]  and async through to Swift and Kotlin. And then you've got Rust SDK itself, which is doing all
[16:08.000 --> 16:12.640]  the heavy lifting. It's got the crypto crate within it. And then within that is Vdozomats,
[16:12.640 --> 16:18.000]  which is our native Rust encryption implementation for matrix. Below that, you've got sled and in
[16:18.000 --> 16:23.360]  futures equalize. This then talks through to a sliding sync proxy. And this is written in Go,
[16:23.360 --> 16:29.440]  and it implements MSE 3575, which is sliding sync. And this is the magic for how this works so quickly.
[16:29.440 --> 16:34.720]  It's basically storing, well, it's going and talking normal sync to normal Synapse. So this
[16:34.720 --> 16:39.920]  could be Synapse or Dendrite or Conduit or anything on the right-hand side. The Golang thing is an
[16:39.920 --> 16:44.480]  intermediary that is going and sucking up the state of your account, storing it in a local
[16:44.480 --> 16:51.120]  Postgres and then talking the very, very responsive API in order to pull that data into Element X
[16:51.120 --> 16:56.800]  itself. It does it by looking like this. Sliding sync lets the clients request the bare minimum
[16:56.800 --> 17:01.440]  of data that they need to render the UI. So here's an almost real request where we say,
[17:01.440 --> 17:05.600]  I want to see the currently visible rooms. I want to see the first page to preload it. So I want
[17:05.600 --> 17:11.120]  20 rooms, 0 to 20. I want it sorted by recency and then name. I only want the avatar and whether
[17:11.120 --> 17:15.360]  it's encrypted. I'm going to get the calculated name whatever. I don't want any messages because
[17:15.360 --> 17:20.480]  we've done a waste time actually downloading scroll back. And we're going to filter it to
[17:20.480 --> 17:25.520]  not have invites, not have old rooms, and not have those pesky space rooms. And whilst we're at it,
[17:25.520 --> 17:29.760]  we want to have end-to-end encryption. We want to have two device messages and we want account
[17:29.760 --> 17:35.440]  data. And the server or the sliding sync proxy will literally just return about 10k of data,
[17:35.440 --> 17:40.000]  which is those 20 rooms with the bare data, bare essentials that you requested.
[17:40.800 --> 17:47.120]  The key design criteria for sliding sync is the performance is constant with the number of rooms.
[17:47.680 --> 17:52.880]  And this was the horrible mistake with the old API and frankly the whole design of matrix
[17:52.880 --> 17:58.240]  historically that as you join more rooms, it gets linearly slower for basically everything.
[17:58.240 --> 18:01.520]  And that was fine for the first few years when people are in a couple of hundred rooms,
[18:01.520 --> 18:05.680]  but obviously we don't want to predicate the success of the protocol on, yeah, it's fine as
[18:05.680 --> 18:10.400]  long as you're not a power user or it's fine as long as you don't actually use it. And if you
[18:10.400 --> 18:15.680]  think of matrix being a bit like a file system, imagine how awful, and I'm looking at you,
[18:15.680 --> 18:20.320]  EXT2, a file system would be if it just slowed down linearly with the number of files that you put
[18:20.320 --> 18:27.360]  in a directory or some other characteristic. And as more and more rooms pop up in matrix,
[18:27.360 --> 18:32.720]  it's not just chat rooms, it could be spaces. Now imagine that you go and join the EU and the,
[18:33.600 --> 18:38.240]  you know, you're working in the EU government and the EU has got a massive space of hierarchy over
[18:38.240 --> 18:42.800]  all of the countries and all of their public sector bodies. Even before you've talked to somebody,
[18:42.800 --> 18:47.440]  you might need to have visibility over this big hierarchy of like a thousand rooms. You do not
[18:47.440 --> 18:54.720]  want your matrix client to take a thousand times longer to log in or sync. So that's basically
[18:54.720 --> 18:59.760]  the entire idea here that you can have an infinite number of rooms, bit like IMAP, where you can
[18:59.760 --> 19:03.760]  have massive mail folders and yet you're only going to care about the subset that your client
[19:03.760 --> 19:10.720]  wants to actually manipulate. Having requested this range of 20 rooms, you then get updates from
[19:10.720 --> 19:14.560]  the server, and this is why it's called sliding sync, that you basically have requested a window
[19:15.600 --> 19:21.280]  over these rooms, these 20 rooms or whatever it might be. And then as the state changes on the
[19:21.280 --> 19:26.720]  server, you get updates of inserts of room here, delete one here, invalidate it here. It doesn't
[19:26.720 --> 19:32.400]  have to be rooms, in future it could be members or other sort of characteristics. This has been
[19:32.400 --> 19:36.880]  shamelessly stolen from Discord, so apologies to Discord folks if you're listening, but thank you
[19:36.880 --> 19:42.400]  also for coming up with this nice approach for how to maintain performance and scrolling on apps.
[19:43.280 --> 19:48.640]  And the end result is just many more. It makes login instant, sync instant, and also time to
[19:48.640 --> 19:54.800]  view rooms instant. Element X is only going to talk sliding sync. There is no point in us wasting
[19:54.800 --> 19:59.200]  time implementing both approaches, but they're really quite different, and we want everybody
[19:59.200 --> 20:05.360]  using Element X for it to have a snappy snappy snappy experience. We've done a lot of iterations.
[20:05.360 --> 20:10.880]  I've been driving the poor Rust team and sliding sync team and Element X team mad by constantly
[20:10.880 --> 20:16.320]  demanding us to try to get the launch time down to 100 mils or something, and there's gone through
[20:16.320 --> 20:21.920]  probably 10 iterations to see how we actually drive the API. And it's been really interesting.
[20:21.920 --> 20:26.640]  The end conclusion is, first of all, when you launch the app, you sync that first
[20:26.640 --> 20:30.560]  screen's worth of rooms, but without any timeline. Literally the request I just showed you.
[20:31.280 --> 20:36.880]  Next, you immediately increase the timeline limit on that window to one. So you'll fill in the
[20:36.880 --> 20:41.920]  room previews, and it was happening so fast earlier that we didn't even have time to spot it happening.
[20:41.920 --> 20:48.080]  And then you pre-cache a page's worth of history of the visible rooms. So when I jumped into TWIM
[20:48.080 --> 20:54.240]  and the history was already all loaded there, it's because the background and pre-cache had already
[20:54.240 --> 20:59.200]  happened because I'd stopped scrolling the room list and immediately jumped in to pre-populate
[20:59.200 --> 21:05.200]  the history for those pages. Then you also incrementally build the big list of all your rooms
[21:05.200 --> 21:09.920]  in the background, which I guess technically is ON with a number of rooms, but because it's
[21:09.920 --> 21:13.600]  happening in the background, it's not on the critical path. It means you can do the scroll
[21:13.600 --> 21:18.560]  for all your rooms or search for a room by name instantly and be able to find them. And finally,
[21:18.560 --> 21:24.000]  you cache it in Sled or SQLite. Rust SDK is doing all the heavy lifting here.
[21:24.640 --> 21:30.400]  The code base is maturing really well. We got it audited at the Vdozomat Slayer last year,
[21:30.400 --> 21:36.880]  thanks to Least Authority, funded by Gamatik. And we have, I think, three other audits planned
[21:36.880 --> 21:42.160]  this year. They were meant to happen last year, but we had disruption along the way.
[21:43.040 --> 21:48.400]  Then, high-quality bindings are critical for this. I mentioned that we've added AsyncFuture
[21:48.400 --> 21:53.760]  to UniFFI. I think this stack could be the ultimate stack for building cross-platform
[21:53.760 --> 21:58.880]  mobile apps going forwards. I mean, you can use Rust for the heavy lifting,
[21:58.880 --> 22:04.000]  and then you hook up at the top a very thin but very native performance layer based on whatever
[22:04.000 --> 22:11.440]  the iOS gives you. And at the beginning, UniFFI was a little bit, it didn't have everything we
[22:11.440 --> 22:16.880]  needed, and so we invested to go and particularly hook up the future support. And the end result,
[22:16.880 --> 22:23.120]  I think, is quite transformational. So that's Rust SDK. Meanwhile, whilst Element X is maturing,
[22:23.120 --> 22:27.840]  we need to keep the existing clients secured too. But so it's going to take us a while to get to
[22:27.840 --> 22:34.240]  parity between Element and Element X. And the project for this for crypto is called Element R,
[22:34.240 --> 22:42.000]  confusingly. So this replaces the old cryptography implementations in JS, iOS, and Android SDK
[22:42.000 --> 22:49.200]  with the same Rust crate that powers Rust SDK. So it's just the crypto crate that is providing
[22:49.200 --> 22:55.200]  a consistent encryption implementation across all the platforms. So this means that if we have
[22:55.200 --> 23:00.960]  hypothetically horrible CVEs popping up, we only have to fix them in one place in the Rust SDK,
[23:00.960 --> 23:05.920]  rather than having to do it four times over between where by West Android and Rust.
[23:06.880 --> 23:13.280]  And you can use this today. It is still beta, so it may kill your cat and flood your house.
[23:13.280 --> 23:17.680]  I've been using it, it occasionally logs me out, which is a bit frustrating because initial sync
[23:17.680 --> 23:23.360]  on V2 takes 20 minutes. But I recommend having a play if you're interested, enable Rusty
[23:23.360 --> 23:28.640]  and Twen's encryption in labs on Element iOS. Androids will be coming fairly soon and Web
[23:28.640 --> 23:34.480]  started working on Friday. We sent the first and received the first encrypted messages by Rust
[23:34.480 --> 23:40.960]  crypto in a Wasm blob inside Element Web then. Rather than climatically, it looks and feels
[23:40.960 --> 23:46.480]  identical to the current crypto, except it's written in Rust. Then another big thing we've
[23:46.480 --> 23:52.320]  done to speed things up is faster remote room joins. So this is a huge internal change to Synapse.
[23:52.320 --> 23:56.960]  So again, you only receive the subset of state you need to participate in a room.
[23:56.960 --> 24:00.960]  Breaks all the assumptions that Synapse had. The rooms are typically atomic.
[24:00.960 --> 24:05.120]  Instead, you basically trickle in the membership of the room in the background after having got
[24:05.120 --> 24:12.160]  the minimum subset to join the room. So for instance, matrix HQ right now, there are 92,948
[24:12.160 --> 24:17.280]  state events for every user who has ever joined or changed their name or left and a whole bunch
[24:17.280 --> 24:20.880]  of other things. If you actually look at the subset you need to participate in the room,
[24:20.880 --> 24:28.240]  it's 152. So this speeds up the room join time from 15 minutes to 14 seconds. So finally,
[24:28.240 --> 24:33.440]  we will hopefully have fixed the problem where somebody gets and stores matrix Synapse immediately
[24:33.440 --> 24:38.160]  tries to join matrix HQ, sits there for 15 minutes looking at errors as their computer
[24:38.160 --> 24:44.160]  explodes and wonders why everybody thinks matrix is as amazing as it is. So I mean,
[24:44.160 --> 24:47.600]  the computer will still explode because they're still having to talk to all of the servers in
[24:47.600 --> 24:52.080]  the room, but at least they will be responsive in 14 seconds. And we hope that the Synthelating
[24:52.080 --> 24:57.120]  conversation in matrix HQ will be such that it distracts them from the smoke coming out to their
[24:57.120 --> 25:02.160]  server. There's still a lot of room for improvement here. We shouldn't be hammering dead servers,
[25:02.160 --> 25:06.560]  which is where a lot of that time is going. And also, we should be caching the partial state.
[25:06.560 --> 25:11.280]  So, you know, if I want to join matrix HQ, the server can just go and hand me the events I need
[25:11.280 --> 25:19.760]  to do that. Another big thing is matrix RTC. So this is the name we're referring to MSE3401
[25:19.760 --> 25:27.280]  and 3898 as many because those were awful names, whereas matrix RTC is a bit more snappy. And this
[25:27.280 --> 25:33.840]  is multi-party native VoIP. We've always had one-to-one VoIP. Here, we are standardizing the
[25:33.840 --> 25:40.960]  multi-party Zoom teams, Jetsy style experience, but in a heterogeneous way. So you can use
[25:40.960 --> 25:45.440]  different clients. This is in lamps and element web. Video rooms looks like this on the right
[25:45.440 --> 25:50.400]  hand side, powered by element call. And it works as what we call a matroshka widget, where you
[25:50.400 --> 25:55.120]  embed element call here as a widget. So this is an iframe on the left hand side. But even though
[25:55.120 --> 26:00.960]  element call itself is a matrix client, it is piggybacking on the host's matrix client. So it
[26:00.960 --> 26:05.200]  shares the encryption and the identity. You don't have two logs in sessions. And really
[26:05.200 --> 26:10.720]  excitingly, we have multiple independent implementations of this in element call, hydrogen, third room,
[26:10.720 --> 26:14.480]  and also, I believe, famously, it has an independent implementation in their healthcare
[26:14.480 --> 26:21.680]  product for Germany. And I'll very quickly try to show you a demo of this. And I'm going to show
[26:21.680 --> 26:28.480]  you, where am I going? Oh, cool. That's good. Fozdom 2023. If anybody wants to heckle along
[26:28.480 --> 26:35.200]  on this, then feel free. Oh, I don't have any internet access, do I? Video conferencing demo, so
[26:35.200 --> 26:39.600]  when you don't have any Wi-Fi, it's always a good idea, right? Let's see how it does. So I'm going
[26:39.600 --> 26:47.200]  to pop into that room there. And here is element call. And then I'm also going to spin up a local
[26:47.200 --> 26:53.680]  hydrogen. Here's what I made earlier. Oh, hello, Amundee. Found some meeting here. This is Amundee
[26:53.680 --> 27:01.520]  and everybody. She co-founded Matrix. And I'm going to wish that this thing was telling me that a
[27:01.520 --> 27:06.080]  video call was happening in the room. And still being one that's ended, but in practice, there's
[27:06.080 --> 27:13.200]  one happening right now. I wonder if this is... I wonder why? Okay. Perhaps we'll use a different
[27:13.200 --> 27:22.480]  room. Let's go to Fozdom 2024. Back to the future, everybody. And go over to hydrogen here. And I
[27:22.480 --> 27:29.280]  think I should be able to just be able to join Fozdom 2024 on call.ams.host. Yeah, okay, here
[27:30.000 --> 27:34.640]  I can actually join it. And hey, Presto, you've got me staring at myself like a muppet because
[27:34.640 --> 27:40.800]  nobody else is responding to my comedy joke of moving to 2024. So everybody, oh, hello. Perfect.
[27:40.800 --> 27:51.760]  Somebody at the back. Thank you. Oh, and there's Amundee. So this thing is really cool because
[27:51.760 --> 27:57.360]  it is completely standardized multi-party void signaling. We have two entirely different
[27:57.360 --> 28:00.960]  code bases. There is not a line of codes in common between hydrogen here on the left
[28:00.960 --> 28:05.520]  and element call here on the right. And just like back in the day on the phone network,
[28:05.520 --> 28:09.120]  we could call each other on different things or have different set clients or whatever it might
[28:09.120 --> 28:15.680]  happen to be. Oh, we've got somebody remote. That's awesome. Then here we actually have a proper
[28:15.680 --> 28:21.520]  heterogeneous thing. So unlike JETC or some other conferencing system where everybody has to end up
[28:21.520 --> 28:31.200]  using the same system to work, this is providing an interoperable thing. And crap out on this one
[28:31.200 --> 28:36.800]  because I've got something better. I've got something better. So one of the other projects we
[28:36.800 --> 28:42.640]  have worked at, which we're just releasing now is called Waterfall. Now what we just did then was
[28:42.640 --> 28:45.920]  full mesh. All the clients were talking to one another. I'm amazed that it worked as well as
[28:45.920 --> 28:52.400]  it did. What you want to do, though, is to have what's called an SFU. So these guys which go and
[28:52.400 --> 28:59.440]  mix together the local video calls. And Sean Dubois, who's the project leader at Pion,
[28:59.440 --> 29:06.400]  the Golang WebRTC, wrote one of these called SFU to SFU based on reading MSC3401. And we
[29:06.400 --> 29:10.880]  renamed it Waterfall. We've fleshed it out and we've hooked it up to element call. And I will
[29:10.880 --> 29:17.040]  endeavour to show you what that looks like. And it's going to be quite similar in some ways.
[29:17.040 --> 29:24.400]  Let me actually try a demo, demo room in here. Again, if someone is already in there, I'm going
[29:24.400 --> 29:29.040]  to try a fresh one. Let's call it fresh demo. Again, if anybody wants to try following along
[29:29.040 --> 29:35.360]  on this URL, if you can see it, please do so. Now this looks a little bit different because
[29:35.360 --> 29:41.360]  it's connecting to the SFU instead. Oh, hello and hello. And hopefully we will get some video
[29:41.360 --> 29:47.600]  off and on. So this has been bounced off the go SFU. But perhaps I'll distract everybody by
[29:47.600 --> 29:52.880]  zooming. So this has got a completely different layout on it. And it thinks it's connected.
[29:54.400 --> 29:59.440]  Oh, there's Simon. I'm glad that Simon of all people is able to connect because he has written
[29:59.440 --> 30:05.760]  the vast majority of Waterfall. So thanks for demoing Simon. And I suspect it is not like in
[30:05.760 --> 30:11.360]  the packet loss here on my side. People are trying to connect in there. But it's working for some
[30:11.360 --> 30:17.680]  folks. It's very, very new, very alpha, but it's exciting to actually see this thing working.
[30:17.680 --> 30:23.440]  And if I quickly turn on debug here in developer mode, you'll see that it actually supports
[30:23.440 --> 30:31.840]  simulcast. So here you can see that Simon is 640 by 360 pixels. Whereas this guy here in dandelion,
[30:31.840 --> 30:39.040]  hello dandelion, is 320 by 240. And if I go and zoom embarrassingly, then it should catch up. There
[30:39.040 --> 30:44.640]  we go. 640 by 480. And it's going to renegotiate the higher resolution stream through. And all
[30:44.640 --> 30:49.760]  the people are going and actually uploading multiple low resolution and high resolution ones.
[30:49.760 --> 30:54.560]  So this is very early, but it's the shape of the future for doing proper massive scalable
[30:55.600 --> 30:59.520]  SFU based conferences. And that's actually looking good. I'm going to take a quick selfie.
[30:59.520 --> 31:04.800]  There we go. Right. Next demo or next stuff. I'm running out of time. I've got a lot of demos.
[31:06.000 --> 31:12.320]  OpenID Connect. Matrix is subject to MSE approval. Moving over to OpenID Connect. It rocks and gives
[31:12.320 --> 31:18.080]  us 2FA, multifactorial, pass keys, QR code login. You don't have to implement the weird matrix
[31:18.080 --> 31:23.200]  off APIs on the server or the clients anymore. ElementX has native OIDC on iOS already and
[31:23.200 --> 31:30.640]  will be OIDC first. First room is the first OIDC only matrix client. Go to our OIDC yet.com for
[31:30.640 --> 31:38.480]  the gory details. So SigningSync faster joins native VoIP and OIDC is a big change. Like this is
[31:38.480 --> 31:43.360]  fundamentally changing how federation works, how VoIP works, and critically how servers sync data
[31:43.360 --> 31:48.880]  to clients, and how you log in. Couldn't be a bigger change. So we are taking the liberty of
[31:48.880 --> 31:54.880]  declaring it matrix 2.0 when we finally release it. So this is not a breaking change. This is pure
[31:54.880 --> 31:59.360]  enthusiasm basically on my behalf because I think it's worth saying, hey guys, come back to matrix,
[31:59.360 --> 32:13.920]  give it another go because we fixed all the crap stuff and we're calling it matrix 2.0. Okay,
[32:13.920 --> 32:18.640]  I'm not doing well on time. We're halfway through in theory. We're going to go now to the future.
[32:18.640 --> 32:23.280]  Flying cars and jet packs and all that good stuff. First of all, digital markets act. Where we're
[32:23.280 --> 32:28.240]  going, we do not need gatekeepers. If you haven't heard about the DMA, it is a fascinating piece
[32:28.240 --> 32:33.040]  of legislation that mandates the big tech companies must interoperate together, particularly for
[32:33.040 --> 32:37.520]  their communication services. The whole idea is that it empowers users to pick the services they
[32:37.520 --> 32:42.560]  want to use and trust without sacrificing the ability to talk to other people. Now frankly,
[32:42.560 --> 32:47.360]  forces the gatekeepers to differentiate based on quality and features, rather than relying on a
[32:47.360 --> 32:52.080]  monopolistic situation where all of their users happen to be trapped in the silo. This is happening
[32:52.080 --> 33:00.240]  right now. The rules came into force in November. The rules start to apply in May and then gatekeepers
[33:00.240 --> 33:07.120]  get designated and it will start getting enforced, which is chunky GDPR style finds in March 2024
[33:07.120 --> 33:10.320]  if the gatekeepers have not gone and interoperated things together.
[33:11.360 --> 33:15.520]  Turns out the matrix already provides a secure interoperable communication protocol. Who knows?
[33:16.560 --> 33:21.760]  The DMA requires the gatekeepers to maintain end-to-end encryption, which is good news. There's
[33:21.760 --> 33:27.200]  been a lot of paranoia that DMA is a secret attack on end-to-end encryption. It really isn't. Having
[33:27.200 --> 33:32.960]  spoken to the folks responsible, they really want to make sure that if WhatsApp does E2E today,
[33:32.960 --> 33:37.520]  then an interoperable WhatsApp also needs to do E2E. They don't want to be responsible for
[33:37.520 --> 33:42.960]  destroying privacy. So to re-encrypt, you need to either do it on the client side, so you would
[33:42.960 --> 33:46.880]  install something like a WhatsApp to matrix app if you want to link your WhatsApp account into
[33:46.880 --> 33:52.400]  matrix, or you could do a multi-head approach effectively, open APIs and have your app talk
[33:52.400 --> 33:57.280]  to WhatsApp as well as matrix or whatever. Or everybody ends up talking the same protocol,
[33:57.280 --> 34:01.760]  which means on the server side, the gatekeepers would have to add support for matrix or XNPP
[34:01.760 --> 34:07.200]  or RCS or whatever it might be alongside their somewhat legacy proprietary protocol.
[34:07.200 --> 34:12.080]  ITF is established a working group called MIMI, more instant messaging interoperability targeting
[34:12.080 --> 34:17.600]  the everybody talks the same protocol approach. We've proposed matrix as an ITF draft for content
[34:17.600 --> 34:21.600]  and transport layers, and we're trying to work with them to make the most of the mix.
[34:23.280 --> 34:27.520]  Decentralized MLS, this is another big thing where we're going, we don't need salamanders,
[34:27.520 --> 34:32.800]  because if you haven't noticed, all of the encryption historically have been called axolotl
[34:32.800 --> 34:39.840]  or ohm or proteus, all of which are species of salamander. MLS is another ITF working group,
[34:39.840 --> 34:44.160]  in fact the one from which MIMI has emerged for next generation end-to-end encryption.
[34:45.040 --> 34:49.920]  We have figured out how to make MLS work in a decentralized model. We have implemented it
[34:49.920 --> 34:54.640]  in a toy type script stack called MLSTS. It is currently being re-implemented on top of
[34:54.640 --> 34:59.600]  open MLS and Rust. At which point, when it works, we will integrate it into Rust SDK and build it
[34:59.600 --> 35:05.120]  into real clients and write an MSE. Are we MLS yet dot com for all the gory details?
[35:05.120 --> 35:11.760]  Here are some early performance testing, which is pretty interesting. Let me get the key right
[35:11.760 --> 35:16.720]  for you here. If you look at MEGOM creating new sessions, this is obviously the log log scale
[35:16.720 --> 35:22.240]  for all of you mathematicians. You can see this red dashed line here, showing how MEGOM
[35:22.240 --> 35:28.720]  sessions scale with a number of users, and it's up at 100 seconds if you've got 100,000 users
[35:28.720 --> 35:36.560]  in the room, which is pretty slow. However, if you look at an MLS update or adding new members,
[35:36.560 --> 35:41.280]  then they're down at 1,000 milliseconds, so a couple of seconds. This is a major
[35:41.280 --> 35:46.880]  algorithmic improvement for certain operations over even VODOSMATS. Support of VODOSMATS,
[35:46.880 --> 35:54.080]  which feels relatively new, may get displaced by open MLS and DMLS when we get there, hopefully
[35:54.080 --> 36:00.000]  later this year. In peer-to-peer matrix, where we're going, we don't need servers. Matrix is way
[36:00.000 --> 36:05.120]  too dependent on servers, and the admins and the risks of internet shutdowns and censorship.
[36:05.120 --> 36:09.360]  This is because home servers have to store their users' chat history and metadata. Peer-to-peer
[36:09.360 --> 36:17.040]  matrix exists to fix this. This is a long-running blue sky project, so to speak, where we go and
[36:17.040 --> 36:22.320]  embed your home server inside the app in order to not have a server running in the cloud.
[36:22.320 --> 36:29.040]  Dendrite is the server we use. Big news on Dendrite is, as of a few weeks ago, it passes 100%
[36:29.040 --> 36:35.120]  server API compliance, so it has a parity of a good old Synapse, and 93% client server API,
[36:35.120 --> 36:39.920]  and the 7% is boring stuff we don't really care about for this. Pinecone, the peer-to-peer overlay
[36:39.920 --> 36:46.080]  network, is going really well as well. We just switched to soft state routing for reliability,
[36:46.080 --> 36:52.720]  and as of about 4 a.m. this morning, not me. Initial store and forward relaying is here.
[36:52.720 --> 37:02.960]  I will very rapidly try to demo that. That's still my phone there. If I go and launch Peer-to-peer
[37:02.960 --> 37:08.880]  matrix, Dendrite is not running. Now it is running. This has literally got a Dendrite running inside
[37:08.880 --> 37:16.880]  my phone here, and if I go over to Visor here, I should hopefully also be able to run it on
[37:19.200 --> 37:26.480]  my Android thing. Here it is, and did it already crash? So it is a bit crashy. It is still beta.
[37:27.680 --> 37:33.600]  I've got Fosdom demo here, Fosdom demo here, and slightly, okay, when I first say hello there,
[37:33.600 --> 37:39.120]  then you can see yay, messages flowing back and forth peer-to-peer. Now if I take my iPhone
[37:39.120 --> 37:43.600]  and put it on to airplane mode like so, and send some messages through from Android,
[37:43.600 --> 37:49.040]  they still go. So this is Peer-to-peer matrix running over Bluetooth flow energy.
[37:49.040 --> 37:55.120]  It silently fell over from running over IP into Bluetooth mode. Now the exciting thing is that
[37:55.120 --> 38:09.040]  if I also run... Well, this is going to be an interesting demo. So here is Element Peer-to-peer
[38:09.040 --> 38:13.440]  running in iOS or macOS, because you can do that on an M1 Mac, and apparently there are five
[38:13.440 --> 38:17.200]  connected peers, which is three more than I was expecting, so hello everybody out there who's
[38:17.200 --> 38:22.080]  about to screw up my demo, and you can see the same history coming on here, and I can
[38:22.080 --> 38:27.360]  send a message through, and you can see both on iOS it came through on Android too. Now the really
[38:27.360 --> 38:36.560]  interesting thing is if I go and... Now you are going to screw up my demo. If I go and kill the
[38:36.560 --> 38:45.200]  Peer-to-peer app on Android, and I send a message here saying hello relaying, and actually not in
[38:45.200 --> 38:49.760]  that room, I'm going to do it in my DM through to Android. I'm going to say testing relays,
[38:49.760 --> 38:59.360]  or very badly typed, and critically my... Where has it gone? Has it crashed? No, there is. So I'm
[38:59.360 --> 39:04.960]  not actually in that room. I'm only in one room on my Mac here. However, this hopefully has gone
[39:04.960 --> 39:11.840]  and peered, relayed through to my Android phone. So if I now go and kill the app here on iOS,
[39:11.840 --> 39:23.440]  and relaunch it here on Android, if I knew how to use Android, come on. This is going to work,
[39:23.440 --> 39:29.280]  or is it going to search Google? There it is. Perfect. Hopefully the first thing this thing will do
[39:30.000 --> 39:35.520]  is to... I'm in the wrong room. If Dendrite launches, the red bar is telling me that it can't
[39:35.520 --> 39:48.880]  tell and talk to its own Dendrite. You can do it. Yes! Testing relaying. This is huge because
[39:50.480 --> 39:54.240]  historically Peer-to-peer matrix has had a massive problem that the other guy has to be online
[39:54.240 --> 40:01.520]  running the app at the same time. And a long story, but I ended up on an aircraft carrier
[40:01.520 --> 40:04.880]  a couple of months ago going and trying to explain a bunch of people how they could use
[40:04.880 --> 40:08.640]  matrix in that environment. And there are a bunch of us on board this thing. And it turns out that
[40:08.640 --> 40:14.160]  an aircraft carrier is a massive floating Faraday cage. And we went and fired up Peer-to-peer
[40:14.160 --> 40:19.760]  matrix on it. And we're very smart that we can talk to one another, but you had to physically
[40:19.760 --> 40:23.680]  wave at the other guy to get them to launch the app so they could receive the message.
[40:23.680 --> 40:27.600]  Whereas with a relay, you can obviously talk to them even if the app isn't running.
[40:27.600 --> 40:34.960]  So that's big. Right. We're almost there. Matrix is not just for chat and VoIP. This is
[40:34.960 --> 40:40.720]  the final thing. Third Room is a decentralized consular for, well, matrix is a consular for
[40:40.720 --> 40:45.840]  any kind of real-time data. Third Room is this tiny project done by Robert May and AJ to provide
[40:45.840 --> 40:50.160]  an open decentralized platform for spatial collaboration of any kind built on matrix.
[40:50.160 --> 40:55.280]  It uses hydrogen, 3JS, bit ECS, and rapier for a new engine called manifold. And the idea is you
[40:55.280 --> 41:01.200]  take a matrix room. You upload some GLTF 3D assets to it. You upload some WASM or JavaScript scripts
[41:01.200 --> 41:07.120]  to it. They use matrix RTC to spell up spatial VoIP and physics synchronization over WebRTC.
[41:07.120 --> 41:11.600]  And you get WebFirst, open decentralized virtual worlds and spatial apps without any of the
[41:11.600 --> 41:17.840]  cryptocurrency or token or Facebook stuff, apart from possibly the hardware. And it looks like this.
[41:17.840 --> 41:24.480]  So you can literally go to it right now, thirdroom.io, and I'm going to go to a world called
[41:24.480 --> 41:30.800]  third room presentation here. And it's going to hopefully pull this up out of my index DB,
[41:30.800 --> 41:35.600]  because I don't want to wait to download 50 megabytes over the network. And if the demo
[41:35.600 --> 41:42.640]  gods are smiling, then you find yourself in this rather snazzy demo world. Now, this is running
[41:42.640 --> 41:46.800]  in Braille. So no plugins or anything. And it's going at 60 frames a second. It does
[41:46.800 --> 41:53.360]  this with proper multi-threading using shared array buffers and atomics to synchronize together
[41:54.160 --> 42:00.960]  the WebGL thread and also the game thread and the main thread in the UI. You can see I'm indeed
[42:00.960 --> 42:06.960]  wandering around there wearing a placeholder avatar. I can flip into third person view here.
[42:06.960 --> 42:11.440]  And you can see I'm also wearing the same beautiful thing. We haven't got customizable avatars yet.
[42:11.440 --> 42:14.960]  Some of the things I can show you here is that you can go and click on buttons. And
[42:14.960 --> 42:18.960]  this is actually a script showing the layout of the different threads. I don't have time to show
[42:18.960 --> 42:24.000]  you that right now, but the game thread has got like rapier physics and WebAssembly. But a really
[42:24.000 --> 42:30.560]  fun thing is that you can just do freeform scripting of any kind. So one example could be
[42:30.560 --> 42:37.360]  this silly, silly, silly demo, which hopefully will load up rapidly. Oh, that's what happens if
[42:37.360 --> 42:40.960]  you this is a bug where you have your worlds overlaying on one another. I'm going to keep it
[42:40.960 --> 42:46.880]  like this, but this is pretty cool. So we've got the construct room from the matrix,
[42:46.880 --> 42:53.200]  so I'll see for in place on top of Mars down here. And if I go over to the TV and I click on it,
[42:53.200 --> 43:04.560]  predictably enough, the entire world goes matrix. So the script for that is literally just sitting
[43:04.560 --> 43:10.880]  there as a bunch of JavaScript uploaded into the media repository. And it's compiled down
[43:10.880 --> 43:16.960]  to Wasm in real time by the engine by QuickJS, thanks to Fabrice Bellard. And it's like four
[43:16.960 --> 43:21.680]  lines of code. You use WebSG, which is our new API called WebSceneGraph that we're going to propose
[43:21.680 --> 43:28.560]  to W3C as a 3D API for manipulating JLTF scene graphs. Now you make it intractable and then every
[43:29.200 --> 43:32.880]  tick, every frame, you see if it's being pressed and then you toggle the state and you enable
[43:32.880 --> 43:38.880]  the matrix material on the room. It's slightly cheated by hard coding it on the in the engine
[43:38.880 --> 43:45.120]  for now like this. Something that we haven't hard goaded though is this guy, which is really
[43:45.120 --> 43:52.960]  exciting. I'm going to refresh his time. And here you can see a big scary black blob with
[43:52.960 --> 43:58.480]  Poland noise, which is pulsating according to my voice as I bellow at it. And this thing is
[43:58.480 --> 44:03.840]  actually a huge chunk of C, which is compiled down to Wasm and is going and programmatically
[44:03.840 --> 44:12.240]  changing in real time the JLTF scene. So this is like the first proper, more advanced capabilities
[44:12.240 --> 44:16.320]  on top of third room. The whole idea is you can build any old app on top of this. You could
[44:16.320 --> 44:21.120]  build Figma on this. You could do multiplayer blender. In fact, we have an in-world editor in
[44:21.120 --> 44:26.160]  here where I can go and select this guy at the bottom and they will have a big white bagel
[44:26.160 --> 44:29.840]  around it. And we don't yet have the property editor, but you should be able to go in and
[44:29.840 --> 44:34.320]  directly manipulate it, change the opacity, the transformations, et cetera, and all that sort
[44:34.320 --> 44:41.920]  of thing. And it really ends up feeling a lot like the web. Rather than a DOM, you've got JLTF,
[44:41.920 --> 44:47.040]  rather than the DOM API, you've got the WebSG API, rather than JavaScript, you've got Wasm,
[44:47.040 --> 44:52.640]  Sandblocks, Blobs, with Rust and Zig and C and JavaScript within it. And there is one final thing
[44:52.640 --> 44:56.240]  I'm going to try to show you, which is probably going to go horribly wrong, which is that we've
[44:56.240 --> 45:05.520]  just added WebXR into third room. So if I go and put on my Facebook device, and there we go.
[45:07.440 --> 45:12.080]  And I back away a bit. Probably unplug it. You'll see, hopefully, in fact, I need to go full
[45:12.080 --> 45:20.560]  screen on that. I guess I do. There we go. Is that coming through? You can see that here I am
[45:20.560 --> 45:27.840]  wondering around third room. Probably can get them full off the stage and break my neck. And I can
[45:27.840 --> 45:34.240]  go and, like, spawn object. So you can have big crate. I can throw the crate away. I can spawn
[45:34.240 --> 45:41.040]  some other big crate. Let's run away from that one. Go and pick this guy up and throw it away.
[45:41.040 --> 45:46.400]  Go and pick that one up. It's over here. And throw it away, et cetera. And I mean,
[45:46.400 --> 46:02.400]  this is running at 90 frames a second. 90 frames a second. So that's a bit weird. It is
[46:03.680 --> 46:11.600]  as least as good as the native MetaHorizon stuff, the Facebook of ships, except it's running within
[46:11.600 --> 46:16.720]  WebXR in a browser in a completely open environment. So we're kind of hoping this provides a really
[46:16.720 --> 46:23.520]  viable platform to build a genuine open spatial collaboration plane for the web. I've already
[46:23.520 --> 46:27.440]  spoken about that. Coming up next is persistence. So we don't yet persist the changes into the
[46:27.440 --> 46:32.160]  matrix room, but we will by uploading little bits of JLTF files so you can even have bots which
[46:32.160 --> 46:41.280]  go into that. Another thing I should have shown you, but forgot, was this guy somewhere. I've
[46:41.280 --> 46:46.560]  lost it already. Never mind. Well, I was going to show you it's an echo. This guy here. But if in
[46:46.560 --> 46:51.120]  this room I go in, and I think this is going to be a comedy, yeah, that's what happens if London
[46:51.120 --> 46:56.160]  and Mars get mixed together for everybody. If I go and say hi because, look, it's matrix room,
[46:56.160 --> 47:02.640]  then if I do slash echo something, I get an echo back. That echo has come from a
[47:02.640 --> 47:08.720]  widget in Wasm running inside the world. So you can program matrix now from within Wasm
[47:08.720 --> 47:12.800]  blobs sitting within the room. Nothing to do with third room. You could use this in clients,
[47:12.800 --> 47:20.240]  et cetera, to start doing client-side widgets. So what's next? Loads of stuff. One big PSA
[47:20.240 --> 47:25.520]  is that Gitter is going native matrix roughly next week. We basically can't afford to run
[47:25.520 --> 47:29.760]  both the Gitter infrastructure and a bunch of matrix infrastructure. So Gitter will become
[47:29.760 --> 47:34.800]  a branded element instance. The API will go away. Please use matrix instead. And finally,
[47:34.800 --> 47:41.040]  we need help. Friends, don't let friends use proprietary chat services. Please use matrix.
[47:41.040 --> 47:45.440]  And critically, and this is new and it's really important, if you're benefiting commercially
[47:45.440 --> 47:51.840]  from matrix, please financially support the foundation because it's stuck in this horrible
[47:51.840 --> 47:56.560]  feedback loop at the moment where the better we make matrix, the less inclined it seems
[47:56.560 --> 48:02.560]  that people want to pay for support or pay for things if they can just grab it on GitHub.
[48:02.560 --> 48:07.680]  This can end up being a disaster where we run out of cash. So please, please, please contribute
[48:07.680 --> 48:11.920]  back, particularly if you're a government. You've got loads of money. Also, run a server,
[48:11.920 --> 48:33.680]  run bridges bots, build on matrix, follow us on MasterDone, and thank you very much.
[48:33.680 --> 48:43.920]  Thanks for listening. Sorry for running on time, as always.