Okay, so this is the Matrix Dev Room on FOSDEM 24, I guess, in case you are in the wrong
room then take the chance now and leave. And we have an afternoon packed full of information
about Matrix. It's only an afternoon, so if you want to look up more information in the
air is an internet full of it. But if you are lazy and don't want to collect all the
information yourself then there is this wonderful people, they collected the information for
you and will give you a presentation now about the state of the union. Matthew and
Amadin give them a warm welcome and the stage is yours.
Thank you Jan.
So we honestly weren't sure what to talk about because if folks came to the main stage
talking Jan's in this morning, basically the first 25 minutes was the state of the union
of Matrix. So we have a bit of a question mark on the subject here. Also Jan just promised
that we will transfer the contents of the internet into your brains which we also hadn't
really prepared for. Anyway, basically if you don't know who we are, I am Matthew, I
am the technical co-founder side of Matrix, day job CEO at Element.
Amadin, the non-technical co-founder side of Matrix and day job COO at Element.
But we would like to at least try to tell you something new about what's going on here.
And actually realized that we have never done a brief history of Matrix which begins before
many of you were born in the year 2000 and free.
Now seriously, the actual backstory is a bunch of us were at the University together at Cambridge
and we were messing around and instant messaging on a project called Project Foxtrot. Now the
idea of Foxtrot is that it was written in Java 1.3, new fresh off the books or something
at that point in the late 90s. And what it did was to serialize hunks of Java, put it
over TCP sockets except it was end to end encrypted using manually written Diffie Hellman
RSA exchanges. So that is where I at least got the bug for Matrix and instant messaging
and after we either got kicked out or left or graduated from Cambridge we ended up working
at the little company doing APIs for the PSTM. So that's 2003. Fast forward rapidly to 2010.
Well, my company were doing mobile app development and Matthew's company both got acquired about
a month apart by a big telco vendor. You would find them in the depth of AT&T doing all their
billing systems. So small startups having fun getting into a very big company.
After a few years of rattling around inside Amdox, I'm not sure why we're not mentioning
Amdox by name but it was Amdox, we discovered a new found desire to burn the phone network
to the ground and annihilate it and replace it with something that would be open and decentralized
and federated that anybody can join rather than the cabal of the phone companies where
it's almost impossible to connect into them. And so that was where the idea of Matrix came
from. We basically took the combined folks in Hren and London went to Amdox and said,
hey, a little bit of a crazy idea but what if we build an entirely new communications
protocol and if we pull it off, then you, my friends at Amdox, can go and sell it to AT&T
and many other big telcos and you can replace the PSTN. And meanwhile, at the same time,
the rest of the world would get a big benefit of the existence of Matrix. And amazingly,
they said yes, with no strings attached, they allowed us to go and switch the business unit
from selling clones of WhatsApp and Skype to telcos to instead building out Matrix. And that's
what we did from starting depressingly in May 2014. So we are a couple of months off having
been doing this for 10 years. Not sure whether that's something to celebrate or not in the grand
scheme of things. What happened in 2014? So 2014, we all gathered in Hren and sat down and had a big
brainstorm on how this thing would look like and ended up with mostly what Matrix is today.
Not much has changed in terms of overall idea and architecture and these sort of things.
We started in May and the goal was, September 2014, we're going to launch this. Like four months
to figure out a high-level working Matrix proof of concept. And we did it.
Yeah, it was a disaster really because we rushed incredible speeds. It was like the best gig
possible that your day job is suddenly told you and all your mates at work that you can
suddenly go wild to create something like Matrix and everybody's sprinted in slightly different
directions naming no names. We might have ended up with three different versions of synapse at
first. We had the bit that taught the client server API. We had the bit that spoke the server to
server API and we had the bit in the middle that sort of meant to funnel stuff around the place.
Each one had a different database schema. Each one had a different object model. They were all
written in Python which was honestly a win but it's possible we might have sprinted a little
too over enthusiastically into this and spent about six years playing off the technical debt
that we accumulated in those three months and then ran up to launching synapse.
Worth noting the end-to-end encryption wasn't there on day one but we did start in 2015 and we
always designed as part of the protocol because if you are going to replicate data equally over
many, many home servers, obviously it needs to be end-to-end encrypted such that if one gets owned
all the messages don't go out the door. Then in 2016 it says we launched Element. I'm not sure
I pulled the slide from but it definitely wasn't called Element. Basically when we launched we
were using Matrix console at the beginning and then we said okay we need a very glossy app to
actually drive the usage of this. We launched something which became Element at some point
but initially was called Vector. What was the second name? Okay let's quiz the audience. What
was the second name of Vector before Element? Yay well done and here we go. Element now is the
flagship client and still growing there. Eventually in 2017 we set up shop as properly independent
both like we started with the commercial company Element and also a bit later like in 2019.
Yeah I think technically the foundation I think was incorporated in 2018 but we didn't do anything
with it until 2019 to try to make sure that there is a clear split between Governance,
the open source project and the protocol versus us practically trying to fund the bloody thing,
Element running around, doing commercial stuff but that was the point where things started
to split properly into your classic open source foundation versus startup trying to build stuff
on top. We eventually turned on and went encryption by default in 2020 alongside Matrix 1.0
which I guess was June 2019 and then fast forward to 2023 when we announced the idea of Matrix 2.0
as showcase development X last foster and here we are today in 2024 the year of mainstream
Matrix. Who knows maybe if you saw the DMA bit of the talk earlier it may or may not be getting the
but yeah we'll talk a bit about it and Travis afterwards is going to have an amazingly very
deeply technical talk all about everything you wanted to know about DBA. DBA? Well you could
do one on DBA or you could do one on DMA up to you. I haven't asked permission for any other
people in this photo to put this photo up but this is the original Matrix team on our way to REN
from the London side playing Magic the Gathering or something. No it was all of us. Yeah that point
the front and side is all in REN. Yes because we hadn't got to France yet. We're literally at some
crappy travel lodge I think in Luton or Gatwick or somewhere on the way through to REN and so
yeah basically that was the vibe at the beginning of Matrix back in May 2014 and more of a vibe
is this which was the whiteboard in the Jupiter project room in the offices in REN where we
basically drew up the possible architectures that we could use for Matrix. You will notice
that there are four if not five architectures here the simplest one is just client to server to
client and this was almost just mapping out the various options we had on the table but at that
point we hadn't really decided how decentralized it would be. Then we had the one that honestly
I came into this with which was assuming that it would be a little bit like SMTP and IMAP that
you'd have like or just SMTP your client would talk to a home server that would cash in rooms
which would talk to another home server that would be a single point of failure that would talk to a
client. I mean it's a bit like a mark in XMPP sounds pretty easy. What I did not expect was for
some of the folks on the previous slide to turn out looking really excited saying you know I think
we might be able to do it like it we can actually go and replicate this between the home servers
which I christened at the time the distributed sink nightmare in terms of active active replicated
version of a protocol and then there is another one down here when you've got sort of two inboxes
that sort of synchronize ish together but you basically have queues rather than DAGs and you
had this one which has got lots of double arrows and I have no idea oh it's a mix net I think is
basically all that was talking about you'd have a personal home server you have a bunch of relays
which trusted maybe or not trusted I don't know I can't remember it was 10 years ago but either way
that was the level of sort of whiteboard diagram that we were playing with at that point.
So basically as Matthew said earlier fast forward almost 10 years now 2023 was very much
focusing on the basics to work well thanks to the limits of funding which is good sometime
like if you have a bit less money then you do focus on the thing more important. So we have
posed a lot of things how do you want to do this do you want to go through the list Matthew?
We have no you don't. I do it okay I will do it so the focus was very much on 2.0 CNAPs the SDKs
Rust and GS SDK the otherwise peer-to-peer matrix is on the side pseudo IDs as well crypto IDs
accountability however we still hope like very very soon we'll be able to get back to all of this
low bandwidth as well and some of the done right work funded by Elements. The legacy
Elements apps and dsdk that are based on are just put on bug fixes only and hopefully we'll
be able to switch to everything to Rust soon and LiboM as well now that we have those amounts
taking over and PortoDrum is on the side waiting for someone to take it and bring it up to
what it all the power it can do. Yeah third room is particularly frustrating we got an email from
the W3C after we announced that we had had to lay off the team element who are working on it and
that nobody picked it up saying what this is meant to be our promised land of Web SG the Web
scene graph API that we created I thought this was how the future of the spatial web as Apple
would call it is meant to be and I said well I'm really sorry but we literally could not find anybody
to fund it whatsoever even people like Rolls Royce who promised that they really needed this and would
fund it then proceeded well first of all to lay off the team that we were talking to on their side
and be not funded at all anyway it's been a really fun year so that said I'm gonna disgrace myself
as you probably expect by wanting to talk a little bit about the projects which are shelved
because it's really frustrating that an awful lot of work went into them last year until around
November they got forcibly parked one of them is some pseudo IDs MSC4014 so this is the project
to replace MXIDs with arbitrary identifiers per room now the reason for doing this is well first
of all GDPR at the moment MXIDs get baked into the conversation history of your room and they are
things like at Matthew got on matrix.org whereas if you had a different unique identifier on a per
room basis that problem goes away and it's up to me whether I want to publish a mapping of my matrix
ID onto the sender key or not the idea of this MSC is that it works out of the box with existing
clients no code change is needed because the CS API maps the sender keys back to MXIDs when it
hands it to the client however this does not provide account portability it's just replacing
the MXIDs and it got implemented in dendrite in June of last year and if you're feeling particularly
creative go and turn on the feature flags on dendrite and have a play with it but as I said
unfortunately it is currently on ice I'm not going to force Amundoon to do the crypto ID one
for the sake of alternating slides so for crypto IDs is an extension of pseudo IDs highly experimental
the idea is that your sender keys become your end-to-end encrypted identity so we finally unite
together end-to-end encryption in matrix with the idea of your MXIDs so the idea is that when you
join a room for the first time you get a crypto ID generated for that room interestingly and
perhaps controversially your client then signs everything it does or the events with the crypto ID
so that you can basically prove that you own those events and as you move between servers in
future you can prove that it came from me as an individual Matthew rather than being signed by
your home server which you don't really care about if you're migrating between home servers
this has the gbs side effect that we no longer have cryptographic deniability because by definition
you would be able to see that a given client owned by a given user has sent a given message
so there's going to be an interesting trade-off to be had there right now we do technically
have cryptographic deniability but practically speaking it really depends on the trust model
I'm not sure just how useful it really is other than on paper whereas this would obviously throw it
away again implemented in dendrite and was just being drafted in Rust SDK when it got shelved in
November the idea is that if you take pseudo IDs and add crypto IDs and add some magic glue which
probably means storing account data in a room so it can replicate between servers then you would
have client controlled account portability also a prerequisite for peer-to-peer matrix which is also
on hold and again is currently on hold how am I doing on time Jan?
More than 15 minutes?
No no no no I am don't worry I am can I do a demo then?
Okay so whilst we're talking about daily departed projects I know this is probably going to piss
off a bunch of people but I really want to very briefly show the final bits that the third room
guys did before they got killed so here's our third room using OIDC as oh that's a great start
this is what happens if something is busy a bit rotting away let me try to sign into this
using OIDC because third room was the first thing that we used to test out native OIDC
we might have to wait a little minute for that server to wake up because I haven't logged in very
recently so this is definitely a dangerous demo talk amongst yourselves imagine that a server is
actually working here which it is right so where we left you last year at Fozden was that this
thing had just launched and the next big thing was actually to make the whole thing
scriptable and do fun stuff with it and it got to the point here where you could go and enter
a world like this and this is just a matrix room stored in gltf with the sorry with the world data
stored in gltf itself but what Robert and AJ implemented is if you press the tilt button
at any point you go and get an inworld inspector up you can go and select things like buildings
and you can do things like move them around and manipulate them in real time I think I
showed this last year the next thing though was to make the entire thing scriptable by Wasm
so you have a script editor now built in here which gives you a little bit of javascript
what you can do is to go in and grab something like the buildings you just drop it straight in
there and it right see the javascript to grab the buildings and then for every time the world
updates you get a delta timestamp and absolute timestamp and I can go in there and do I do not
know what this API is how back in this go let's assume it has a translation button and say that
y is going to be what 10 units times or the sign of the current timestamp that will work right
and if you head um save as run what it will do is compile the javascript down to Wasm using quickjs
written by the amazing Fabrice Miller and reload the world and there you go
let's upload in your dance I think
this is so cool this is so cool you can see why w3c got in touch afterwards saying
hang on this is how the future of the web is meant to be and where are the people and it's like
well this is what it is so if you're watching this and you think this deserves to exist well first
of all I'm not sure I'm gonna ever persuade the guys to work on it again because they feel pretty
pissed off obviously that the project collapsed but the code is all there still and it's so
antelizingly close to being absolutely amazing right sorry back on to what we're talking about
cryptoidies or yeah
what's the next time
matrix 2.0 I mean who was in the thing in Nansen this morning do I need to go through this again
oh crap yeah only about half of you yeah yeah perhaps we should have done that at the beginning
of the talk in 20 minutes through anyway right so matrix 2.0 very quickly first of all this is not
a spec release this is a state of mind a bit like web 2.0 it's made up of various MSCs and the status
is sliding sink so instant launch and instant logging and instant sink it kicks ass but it's too
fiddly we are currently performing slidectomy which is the technical term for removing the sliding
bit from sliding sink and there is in fact a PR against the Rust SDK which basically shifts all
of the ordering onto the client rather than doing it on the server and this is all my fault being
stupid and over enthusiastic going and trying to do this over optimized implementation where the
server figures out the best possible ordering and then the client tweaks it at the end and it turns
out that having two different things fighting over control of the order of a list doesn't work very
well so we've basically said the client gets to order it entirely the server does a very approximate
probably based on time stamp thing and the good news is that it is just a subset of the current
API so it's not a yes or no rewrite it's just basically simplifying the API so it's easier to
implement then you've got end to end encrypted VoIP which again kicks ass we demoed it and Janssen
and it worked this morning need to update the MSC because it's on its 6 or 7th iteration now and I
think it's stabilized enough that we should actually spec it properly faster joins so synapse
rapidly joining rooms and other home servers for that matter if they implemented them
incrementally lazy loading the data in it would kick ass if we actually finished it so we got the
hard bit done the kind of infrastructure and made rooms non-atomic in synapse and then actually
didn't get to the point where we would get it to go faster significantly faster and then IDC which
does kick ass but it's going to be a big migration as we need basically everything to support it
before we start turning it on our matrix org etc but there is lots of stuff in progress if I have
more time and try to show the QR single hop login demo which is super cool then Rust SDK is the brave
new world that goes and wraps us all together on the client side and as of Friday as I mentioned in
Janssen the JS SDK and therefore Element Web and anything else using JS SDK now uses Rust SDK for
crypto so we are finally at the point where the old Le Bon c++ library is in maintenance mode and
then some whereas Vadosmats the Rust implementation is our brave new future and I spoiled that Demir
has already produced a post quantum PR draft for Vadosmats using the kyber primitives wrapped around
I think curve 25519 so a kind of hybrid approach which should be compatible with the signal and
PQXDH and key exchange stuff and what else are we doing in Vadosmats it was another big thing
but I can't remember what it was another PR that landed basically we fixed all the crypto bugs in
one place and a huge huge focus in the coming months is making the crypto finally suck a lot
lot less. Should I keep going on MLS you can do the whole end bit of it okay on MLS people might
be wondering hey it's not talking about MLS anymore what's that all about first of all we are
still doing this you can track the progress on our MLS yet.com MLS is the group encryption that
scales much much much better than normal double ratchet and almond Vadosmats we have it largely
working on matrix has huge key bundles you have to store the keys and the media repository they're
so big at the moment however there's been a lot of discussion on the meme side which we'll talk
about briefly and Travis will talk about a lot more in a few minutes in terms of what if you
actually used MLS to synchronize everything so rather than having a matrix tag for tracking
no synchronizing data between servers what if you just chuck everything into MLS. TBD so there's
a little bit of a do you put MLS over matrix or do you put matrix or meme over MLS and debate going
on right your slides. Yeah basically as we said at the beginning 2024 could really be the year
where our prediction was the convention would come to and the prediction was this that this is a
slide taken off investor pitch deck saying in 2019 in five years everyone will communicate over
matrix that's why we did this right. In 2019 it said 10 years and now because we're five years later
and now it says five years just saying also this is written in R and it's real traffic from 2019
showing the I think the top 100 home servers talking to one another just saying if you're
investor decks aren't written in R you're doing it wrong. So basically killing email and the phone
network so why the digital market tag you may have heard of it they demand the big communication
services called gatekeepers to actually interoperate with the rest of the world. Two of them have been
named so far WhatsApp and Facebook Messenger iMessage is pushing back saying no no no we're not a
gatekeeper but let's see where it goes. To the business yeah business to a user. So last year
it was coming into force in the 7th of March they will have to actually expose these APIs as
production ready and anyone in here who actually wants to interoperate with WhatsApp because they
don't want to create an account there will be able to come to them and say hello can I please
integrate against your APIs to talk to your users. She's a little bit ironic because it starts to look
an awful lot like the PSTN in terms of you have great big telecoms providers and you go to someone
at AT&T and said hello please can I talk SS7 to you so my little telco can talk to the big telco
and they make you sign a massive contract and there's all sorts of back and forth to happen.
Obviously we can't say what that will look like with method but there may be a risk oh well there
could be entire spectrum between open federation versus closed federation versus everything in
between and we just don't know what will happen. Let's see in a month and basically yeah one we
may get to a point where matrix becomes the glue and between all the communication system and
matrix them together. Yeah I mean I'm not counting on it honestly on this architecture particularly
because everybody would need to agree both on the same dialect of double ratchet as well as the
content payloads within but you never know if we get critical mass in some places perhaps everybody
will follow. So yes we mentioned it this morning already still a lot a lot a lot of things to do
especially on the core making sure the core is funded we're trying to put a big call out for
fundraising and honestly the goal is really to get the big guys who actually are using it for
hundreds of thousands of users millions of users without contributing a sense to the project itself
and funding the core trying to raise the alarm. At the same time there is a public policy dev room
there where we're trying to figure out how do we get the proven source projects actually funded so
I'm going to run there to try to solve that problem very shortly after this. Cool. Thank you guys so
this morning a lot of people actually let's go thank you to everyone who is supporting it and
everyone who jumped in live this morning during the talk to actually become a member of the
foundation and thank you to all the already how do we call them supporters organizational
supporters as well in here. Yeah honestly if your organization is just happening to use element
and matrix as its common system it really doesn't cost that much to put some money behind the bar
to keep it going like we met x wiki on Friday and said oh how's it going and they said
stuck notifications are the bane of my life and he said oh well if you actually want us to have more
member to go and work on certain notifications and perhaps you can become a silver member of the
matrix.org foundation and that is why there is a x wiki logo and a cryptpad logo on the slides there
seriously it's meant to be relatively modest but if we get all the organizations doing it as well
as the individuals then if nothing else is really a lot easier to go to the really big people like
the EU and say look we've got already got 800 people supporting this this is an important thing it
matters therefore you should match 20 fold 50 fold 100 fold and as a narrative it may work.
So yeah meanwhile we have an awesome community a lot a lot a lot of things are happening around
and this is the menu for this afternoon where everyone will be able to tell us a bit more
about what they're working on looking forward to it thank you everyone.
Any questions? Two I'm allowed two questions but they can't come from me.
Kim.
Excellent question. So the excellent question which I shall repeat is where the hell is
multiple accounts support in element. Now most of the best of clients out there have it already
however we've never got round to it in either element or element x there's not a good reason for it
other than everything else taking slightly higher priority we did have it in matrix console the very
first matrix client that we wrote before producing vector and riot or whatever it is now.
Yeah no good answer other than we need to add it and element x would be a great time to do that
it's built with it in mind we just haven't put it in the UI yet.
Everyone can do it apart from element pretty much.
Is there any indication that other assorted governments are looking at following on something
So the question is whether other governments are going to take inspiration from the digital
market tax there are some movements in the US around it trying to remember the name it's not
the interoperability bill but something along these lines so there is definitely it's like
GDPR then has been looked at by the US and European Europe is leading on these sort of things and
yeah there is movement in that direction.
So yeah the question is whether we are lobbying within the EU to make the API's the gatekeeper
offer open so the DMA is forcing gatekeepers to open their APIs the big lobbying we've been doing
in the last four years has been please please please don't ask them to only open their APIs
but try to converge towards an open standard so that as small companies who want to integrate
with these people I don't have to build a polyglot messenger which speaks what's up Facebook
messenger Google blah blah blah all of them in parallel please please please so so far we don't
really know what this is going to be the in the law of the DMA in the text it doesn't say you have
to use an open standard but like basically we continue working with everyone the European
Commission's and the gatekeepers and all the big corporations trying to convince everyone that
that's the best way to go. I think the US equivalent of DMA is American Innovation and Choice
Online Act maybe one of Senator Wyden's initiatives perhaps but you don't need to go and look it up
and there was something I wanted to add to that but I've gotten what it was just the lobbying
try to get everybody on a single it was it would have been amazing if we'd persuaded the Commission
to basically put into law you have to speak an open standard the reason that they didn't is first
of all it's not really their job as politicians to dictate the actual technical implementation
they need to say what the outcome should be like a more competitive environment without massive
anti-trust behavior but it's up to us literally up to us a lot in this room to figure out what that
should really look like and the other big problem is that there isn't a standard that is suitable
like matrix is great it's looked after the matrix org standard foundation but it's not an internationally
recognized standards body so if we're gone through ITF already then perhaps that would have worked
it's not like they can just tap on the wall and say use matrix even though it has some traction
I would say this is an amazing segue to travestalk right now unless there are any other questions
which I can cram in but I'm not allowed to no thank you very much