The Path to Peer-to-Peer Matrix
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Hodgson, Matthew
Matrix is an open source project run by the non-profit Matrix.org Foundation dedicated to building an open protocol and communication network for decentralised, encrypted communication - providing a viable open alternative to WhatsApp, Slack, Discord an other proprietary communication silos. In this talk we will show of the work we've been doing over the last year to shift Matrix from being a decentralised-server architecture to a fully decentralised-client p2p architecture, through running clientside homeservers and experimenting with libp2p and friends as a p2p transport. We'll also show the route we'll be following over the year to go from proof-of-concept to the live Matrix network.
 Okay. Am I allowed to start? Okay. Hi, everybody. I'm Matthew. I'm the totally in for the matrix and projects. I'm here to talk to you about the path both in past and future of peer-to-peer matrix. I guess that most people know what matrix is by now. Anybody not know what matrix is? Free people. That is absolutely hilarious and bed-fying. Very, very quickly, I'll try to rattle through basically the normal slides that we have to try to explain what matrix is. If you haven't seen this QR code, scan it now if you want to try to do a live demo that will go catastrophically wrong. Question? Well, that's a good start. Well, what about that URL? Matrix.org-Matthewp-demo.html. Does that work? Oh, yeah. Remove the demo. Yeah. I remember now. I was trying to make it shorter and then forgot to update the thing. Okay. Well, remove the dash demo, everybody, and go and try grabbing some Docker stuff slowly over the next 10 minutes. So, we'll speed through that. Matrix, open network, real-time, communication, decentralized. Use it for chat, VoIP, VR, AR, IoT. The whole point is to be the missing communication layer of the web. It sucks that the web never got real-time communication. You have amazing things like WebRTC for doing all of your VoIP and media, but there's no signaling plane. There's no way to actually set up those communications in a decentralized manner. Matrix tries to be it. You can have a bunch of existing solos. Matrix sits in the middle, gluing them together using the bridges. If people saw half-shots, bridging talk earlier in the real-time comms room, you can see that's a real thing. And the big differentiator is that there's just no single point where your conversations gather. It's replicated over all of the servers. There is never a choke point unless everybody's on the same server, but we'll ignore that. Architecturally, you have your servers, application servers, clients, and we still have these wretched identity servers, which are the bane of our life, which go and map email addresses and phone numbers through to Matrix IDs, which we're going to somehow find a way to decentralize in the near future, I hope. And then the spec gives you conversation history, group messaging, E2E encryption, VoIP, all sorts of server side fun and stuff, decentralized content, repository, unread counts, account data, et cetera. The ecosystem is the spec, the clients, the servers. Third-party clients contributed from the wider community, third-party bridges, bots, integrations, and other server implementations, and it's growing all the time. RiotX is coming very soon on Android, out of beta, as a complete rewrite of the Android app. If you aren't using RiotX already, please start playing with it. We'll basically be announcing the path to 1.0 in the big K auditorium talk at 4 p.m. today, and that's probably what you want to know about Matrix. In terms of uptake, this is the daily active users that we're seeing on the Matrix.org server over the last couple of years. You can see it started off in 2016. Pretty miserable, grew relatively well. We had a bunch of problems in the summer of 2018, but since then, it's been hockey-sticking its way up in a way which is quite scary in terms of handling the scalability, fun and games on Synapse that results in that. Community-wise, if you literally look for distinct MX IDs, you get $13.5 million. On the Matrix.org, in fact, across the network that phones home, we see about 5 million messages a day. If you select Count Star of Rooms on the Matrix.org server, it's about 4.5 million chat rooms. We can see about 20,000 servers phoning home, so we roughly double that. There's a bunch don't phone home, which gives us 40,000 total. On the Matrix.org home server, it's about 35 in a second, and about 3,500 a second messages out. Loads of projects building on Matrix. Lots of companies, including some really big, scary ones, building on Matrix. Lovely ones who we appreciate building on top of Matrix. And also, some even bigger, lovely governments, building on top of Matrix, too. I'm sure you know that France deployed Matrix across the entire public sector over the course of the last year and a half. Also, Germany has announced at Christmas that the Bundeswehr, the Ministry of Defence, has a Matrix trial running. Also, there are dot-gov servers in the US running on Matrix, and the fourth one is possibly, hopefully, the UK. So, that's enough boring stuff about Generic Matrix. Can everyone hear me, by the way? Good. I should check. Here's the Matrix. Why? Matrix is great, but obviously, today, we have home servers. And if you want to have full autonomy of your conversations, you have to run your own server, which is great for us a lot, because probably at least two-thirds of the room are professional citizens. We're all feeling a bit smug and elitist and thinking, wow, this is amazing. I can run my own server. For everybody else, they're screwed. They have to go and trust somebody else to run the server for them. The whole point was to give autonomy for people to give back control of communication to the users, and yet, the users are a bit lost. So, that's one massive reason to look at P2P. Separately, we also get some better onboarding, frankly, from a selfish, riot-style app developer perspective. It's a pain in the ass to have to tell people, hey, which server do you want to connect to? Do you want to use Matrix.org or something from modular.im or use something else? And instead, how about you just start off P2P and just join the big global P2P network? Really cool. It would also be internet less operation. If you're going hiking, if you're on a plane, if you're hypothetically in the Euro-style coming over from London, wouldn't it be nice to be able to communicate locally? Also, big deal is metadata, because at the moment, metadata pools on your home server. You can see every message in terms of who sent it and when they sent it, where they sent it from. And as I'm sure we all know, metadata in as many ways, more intelligence, more useful from an intelligence perspective than the actual data itself. Also, let's just run client-side bridges a bit more easily. That would be really nice for protocols not thinking of, I know what's that by name, which don't like you bridging in from a kind of bridge service provider. If you can run it yourself as a user, if you want to give your WhatsApp credentials to a third-party client like a Matrix bridge, that's kind of your problem. And then finally, honestly, it raises the bar for development. So it forces us to do multi-homed accounts, because if I'm going peer-to-peer and I've got my account running on my phone and on my laptop, I obviously want my same identity to be on two devices. But if each device is a home server, that also gives us the ability to port accounts between home servers and account migration. It also forces the home servers to get smaller. It also forces us to do smarter routing algorithms, because if everybody here was running their own home server on their phone right now, the idea of having to do 296 concurrent HTTP hits every time you send a message is going to be nuts. Also forces us to do low bandwidth transport, because you don't want to be chashering away on HTTP all the time, because that's just going to chew your battery and bandwidth. The nicest thing, though, and the really, really, really exciting thing about this, is that the clients and the bots and the bridges stay absolutely identical. Now, the single line of code needed to be changed, because we're just swabbing out a home server that runs somewhere in the cloud and putting it on your phone. So very quick flashback to 2015. You can tell we're in 2015, because we've gone 43 aspect ratio. And I gave a talk at the conference called Gena Entrepique in Ren, where we announced our end-to-end encryption for the very first time. It was pre-Megome, it was home, and it was a really cool cryptography conference. And as part of announcing home, I wrote a bunch of slides to try to avoid getting crucified for the fact that we weren't protecting metadata at that point. And it went like this. Matrix is all about creating bridges. You can't bridge without having metadata leaks, because the bridge is an unavoidable metadata leak. However, we also expose metadata on home servers. You have the same DAG that builds up on all the different servers for the same room. Can we do better? And we looked at Pond. Who knows Pond? Interestingly few people, about 10 people. Pond was a really cool, really early metadata-resistant messaging app written by Adam Langley. I think from Google is the guy who, I think, ran the TLS team at Google at the time. And basically, it works like this. You had Tor, you had hidden services in Tor, and you had a messaging client that would route messages via store-and-forward hidden services, as well as doing traffic pattern resistance so you couldn't see who was talking to who when. The disadvantage is that it took about 15 minutes to send a message, because it was deliberately delaying things in order to avoid the traffic patterns being visible. But we did the same thought experiment, back in 2015, saying, well, Matrix is meant to evolve to support this, and I'm sure it will happen real soon now. Could we do something similar? Yeah, put home servers on the client, use hidden services of some kind to store-and-forward, and incrementally migrate or bridge from Classic DAG. So it would look like that. You'd have a home server running on your little gray client, and the client is still sitting there connecting to it, and then it's bouncing off Tor. So back to the present day, almost. 2018 came up with this diagram, was noodling about where Matrix could go in future. Today, you end up with server-server client end-to-end encryption setting in the client. What if in the future you had the kind of Matrix daemon that ran on your kind of operating system as a Unix daemon that would offload your own, and then you don't have to keep implementing E2E everywhere? Then what would happen if you actually put that guy as a reusable module into your client's cross-platform? And perhaps that guy could evolve into a full peer-to-peer home server if you want to put a home server in it, and you could even offload the E2E into it, and how in a second, why not actually embed that in the client too, to the extent that you end up with a client with a home server in it? And the fun thing is that we've actually built this now. It's called pantalime-in. Talk about it later on today. We're also just in the process of rewriting pantalime-in from Python into Rust and using it as an embeddable engine that can go into arbitrary clients. But meanwhile, what's happening over here? Back to the actual present day. We've got three experiments where we've been playing in PCB. The first one isn't actually we. It's done by Tim and Terry. You aren't here anywhere. There he is, who reached out probably, well, nine months ago or something or a year ago to say, hey, I'm doing a bachelor's project. Can I do something with peer-to-peer matrix? At which point, I think I basically gave this talk to him very quickly and said, hey, go wild. And he looked at putting matrix over co-app as low bandwidth protocol using IgDRABZIL as the overall PCB overlay network. Who knows what IgDRABZIL is? Okay, about 20% of the room. It's really cool. Experimental peer-to-peer, end-to-end encrypted overlay network basically creates a big spanning tree over a set of manually defined pairings. But once the spanning tree is up and running, you can go and root to people end-to-end encrypted across this thing. So what he did was to go and take Synapse, originally Dendrite, but Dendrite wasn't working at the time. So it took Synapse and did a proxy that would take HTTP and do co-app over IgDRABZIL. Very similar to the talk that we gave last year of co-app proxy. And basically has a similar architecture. That's literally a picture of IgDRABZIL, the spanning tree of life. And it's literally a spanning tree with a root in the middle. And you can do shortcuts once you've discovered nodes in different places, but you go and discover them via a HT and then you go and navigate the spanning tree to actually have the routing to go and find somebody. So if you're familiar with spanning tree protocol, this is just it for the internet. Architecturally, home server sitting locally, a client isn't shown, but you have a client talking to a home server, goes through a proxy, goes over IgDRABZIL and out the other side. And I'm assured, I haven't tried it myself, but I'm hoping this is a real screenshot of actually working of two riots on different ports going over IgDRABZIL. So that was a very, very cool project. And we started to build a little peer-to-peer secret community of different people like Neil Alexander, who is the maintainer, the maintainer, one of the maintainers of IgDRABZIL and myself and also Keegan, who wrote Dendrite originally and one of the guys who wrote Dendrite originally, going and sort of playing in the space. So the next experiment we did started in, I think, November or December also, where we took Dendrite, which many will know as our next generation go-lang home server that has been stuck in development hell of various flavors for about two years. And what if you just swapped out its HTTP transport for its go and libP2P equivalent? Who knows what libP2P is? Surprisingly few people. Okay, libP2P is the networking layer for IPFS, and it's a separate module. It's pretty cool. It gives you a whole bunch of different mechanisms for discovery, different transports, different implementations in different languages. Theoretically, it's very, very nice. In practice, it's still a little rough around the edges, but it's enough to play with. And what you can do is to basically say, hey, libP2P, give me a DHT, and you then go and, sorry, I've got my 10-minute warning, which means I have to totally change the tempo of this talk. So you go and say, libP2P, give me a condom, DHT, and then I want a connection between some public keys. So everything is identified by their public keys. So first of all, we had to make Dendrite work again, which it does now for Federation, at least for the subset we need for this demo. But then we needed to make it work a bit more with, say, messages support to actually have scroll back. So ironically, despite being the maintainer for Iqrabsil, Neil Alexander has come along to work on libP2P stuff for now. So sorry for that. But either way, we've now got Dendrite talking slash messages. You then just swap out one listener for the other listener. And the step four is to try to demo this live with everybody in the room, what can possibly go wrong. So if anybody actually followed, oh, that had the right QR code on it, or something, I'm going to go and create an Antaht network right now. It's going to be called Bicephalus, which is the name of this laptop, on channel 11. And don't all try to join it at once, because it will probably just fall over. In fact, I haven't even been able to create it. Oh, no, I have created it. So Neil, are you able to jump on this? Okay. So I'm going to run the commands which are on that URL. I'm going to start off a docker, which is setting up a Postgres install instance. And then I'm going to send a Dendrite running locally on swap somewhere. There it is. So this is invisible on the screen. It's invisible. No. Thank you for telling me. Okay. This is what you may be seeing. In fact, let me just mirror quickly. I thought this is going to be very painful. Okay. So that was, I went and set a Postgres running in a docker there. I've got Dendrite running here. It's found zero rooms. It's advertised zero rooms. But critically, you can see that it has gone and fired up a public key with a node ID on a whole bunch of different listeners, one of which is 169 network on the ad hoc Wi-Fi just created. The reason we're using ad hoc Wi-Fi is that Fozdom is filtering multicast this year. Otherwise, we would just be on multicast on the normal Fozdom network. I've now discovered one other libp2p pair, which is very real. So I'm going to go ahead and run it. I'm going to fire up a port in 8082. Okay. Don't make eye contact with my desktop. So here I am on a brand new Dendrite running locally. I'm going to create an account for myself. They're predictable. Right. Fine. And here I am. So I'm going to explore. No. Okay. So that's what happens if there are no rooms on the network. Okay. I shall create a room quickly. There's a room, testing, public room. And you can see it's got a beautiful host name here, which is the public key on libp2p. And I've gotten created it as such. I think I even remembered to advertise it on the directory. Now, can you see that now? I might not. If I'm not, I'm an idiot. Let's see. And now I haven't published the room in the directory. So the directory here is publishing it into the libp2p Mdht. So that in theory, other people on the network can see it. Come on, demo gods. Everybody pray. Found zero rooms. It's advertised a room. It's found a PS. So you should be able to see it now. Well, I can see it here. So if I go to my new home server with this amazing name, I can see testing there with one person in it, except I'm already in it. Anybody else want to try? Anybody else actually installed the Docker egg and able to roam onto my network? No, it might be. But I'm really surprised. Given I can see you. On the plus side, I can send messages in dendrite. Isn't that amazing, right? Perhaps I can fake it and try to log onto it locally. So I'll go and spin up another riot. Except it's not going to be, it's just going to be talking to my local one. So that's what shows the CS API. Restart the dendrite, you reckon? Five minutes, right? No pressure. Okay, it's restarted. Any better? It's found one room. It's advertised one room. It's discovered one room. How about you restart your dendrite? Oh, you made a room? I only see one room directory here. So you managed to connect to the Antop. Cool. Yeah. No joy, Neil? We tested this like six times in a row and obviously. The weird thing is that I'm seeing the peer, okay? Okay, I'm going to leave this running in the background and go to another demo even though I've only just left, which is the third experiment. So experiment two was this today, going through this in future. However, what would be really awesome would be if we actually put dendrite in the browser and compiled it to Wazim, swapped out the HTTP transport and go for a JavaScript app one, switched out the SQL driver layer for one in JavaScript two, and you put it in a service worker and you can just intercept the outbound traffic from riot and say actually put it into Wazim instead. And that architecture looks like this. So we've gone and written a SQLite driver for go, which talks through to JavaScript. We've written an HTTP driver for go that talks through to JavaScript also. And it looks a bit like this, hopefully. So ironically, this is going to clash with my dendrite demo. I assume the other one isn't working, right? Anoying. Okay, let me get rid of that. And instead, kill that off and quickly go to vector web, p2p and do yarn, run, start. And what this is, is riot web, which has literally got the architecture that we're looking at here. It goes and pulls in dendrite as a Wazim dependency. It goes and links through to the other members here. And in theory, it should be the best demo in the world. However, this one deliberately doesn't work because once we got this bit to work and that bit to work, dendrite is still talking to Postgres rather than SQLite. And it turns out that rewriting dendrite's entire schema to SQLite rather than Postgres is non-trivial. So at the moment, this has got a placeholder in the middle. And so that's fired up now. And I should be able to go, I think, to this guy here. And if the demo gods were at all smiling at me, let me just check what port this is and gone. Then we have the service worker here running in the background, although it's actually not the one that was just deployed. In fact, this is going horribly wrong. Let me try on a totally different IP address. I'm almost out of time here, presumably. This is looking a bit more promising. Perhaps lots and lots of stuff in here. God, the demo gods are not with me today. So what it should be doing, I wonder if I'm just on the wrong port note, is showing a dendrite. Hang on. What port is this on? This is when you demo's go, oh, 8083, obviously. Okay. So here, hopefully, service worker, oh, thank God for that. So demo gods are working, everybody. So service worker registration, so it's gone and registered service worker, it's opened up a DB, it's created a table called ping, which is our fake dendrite because we don't have a real dendrite. Then it's firing up a libp to p thing, which is talking through to a web socket start rendezvous service, is then looking for other people on it. And if I went and set this going in another tab on another port, then the two nodes will hopefully discover one another, over libp to p, and even start, there we go, pinging each other by HTTP. So the two dendrites are now sending pings back and forth, and they're even going to be updating the ping table in the database in order to have storage. So that's basically where we're at right now. It's not fully working as a full dendrite, but it's pretty close. And there we go. The two right hand things, a dendrite in go or dendrite in js, going forward, libp to p or a grad tool or something else, who knows, lots of other questions. What's next in general? E2E encryption going on for DMs today, cross-signing support today, RiotX, and a whole bunch of other stuff too. Thank you very much. Unfortunately, we have no time for questions. Please exit through that door. A lot of people are waiting. And after the dust has settled, please push into the middle. Thank you.
 Okay. Am I allowed to start? Okay. Hi, everybody. I'm Matthew. I'm the totally in for the matrix  and projects. I'm here to talk to you about the path both in past and future of peer-to-peer  matrix. I guess that most people know what matrix is by now. Anybody not know what matrix  is? Free people. That is absolutely hilarious and bed-fying. Very, very quickly, I'll try  to rattle through basically the normal slides that we have to try to explain what matrix  is. If you haven't seen this QR code, scan it now if you want to try to do a live demo  that will go catastrophically wrong. Question? Well, that's a good start. Well, what about  that URL? Matrix.org-Matthewp-demo.html. Does that work? Oh, yeah. Remove the demo. Yeah.  I remember now. I was trying to make it shorter and then forgot to update the thing. Okay.  Well, remove the dash demo, everybody, and go and try grabbing some Docker stuff slowly  over the next 10 minutes. So, we'll speed through that. Matrix, open network, real-time,  communication, decentralized. Use it for chat, VoIP, VR, AR, IoT. The whole point is to be  the missing communication layer of the web. It sucks that the web never got real-time  communication. You have amazing things like WebRTC for doing all of your VoIP and media,  but there's no signaling plane. There's no way to actually set up those communications  in a decentralized manner. Matrix tries to be it. You can have a bunch of existing solos.  Matrix sits in the middle, gluing them together using the bridges. If people saw half-shots,  bridging talk earlier in the real-time comms room, you can see that's a real thing. And  the big differentiator is that there's just no single point where your conversations gather.  It's replicated over all of the servers. There is never a choke point unless everybody's  on the same server, but we'll ignore that. Architecturally, you have your servers, application  servers, clients, and we still have these wretched identity servers, which are the bane  of our life, which go and map email addresses and phone numbers through to Matrix IDs, which  we're going to somehow find a way to decentralize in the near future, I hope. And then the spec  gives you conversation history, group messaging, E2E encryption, VoIP, all sorts of server  side fun and stuff, decentralized content, repository, unread counts, account data, et  cetera. The ecosystem is the spec, the clients, the servers. Third-party clients contributed  from the wider community, third-party bridges, bots, integrations, and other server implementations,  and it's growing all the time. RiotX is coming very soon on Android, out of beta, as a complete  rewrite of the Android app. If you aren't using RiotX already, please start playing  with it. We'll basically be announcing the path to 1.0 in the big K auditorium talk at  4 p.m. today, and that's probably what you want to know about Matrix. In terms of uptake,  this is the daily active users that we're seeing on the Matrix.org server over the last  couple of years. You can see it started off in 2016. Pretty miserable, grew relatively  well. We had a bunch of problems in the summer of 2018, but since then, it's been hockey-sticking  its way up in a way which is quite scary in terms of handling the scalability, fun and  games on Synapse that results in that. Community-wise, if you literally look for distinct MX IDs, you  get $13.5 million. On the Matrix.org, in fact, across the network that phones home, we see  about 5 million messages a day. If you select Count Star of Rooms on the Matrix.org server,  it's about 4.5 million chat rooms. We can see about 20,000 servers phoning home, so  we roughly double that. There's a bunch don't phone home, which gives us 40,000 total. On  the Matrix.org home server, it's about 35 in a second, and about 3,500 a second messages  out. Loads of projects building on Matrix. Lots of companies, including some really big,  scary ones, building on Matrix. Lovely ones who we appreciate building on top of Matrix.  And also, some even bigger, lovely governments, building on top of Matrix, too. I'm sure  you know that France deployed Matrix across the entire public sector over the course of  the last year and a half. Also, Germany has announced at Christmas that the Bundeswehr,  the Ministry of Defence, has a Matrix trial running. Also, there are dot-gov servers  in the US running on Matrix, and the fourth one is possibly, hopefully, the UK. So, that's  enough boring stuff about Generic Matrix. Can everyone hear me, by the way? Good. I  should check. Here's the Matrix. Why? Matrix is great, but obviously, today, we have home  servers. And if you want to have full autonomy of your conversations, you have to run your  own server, which is great for us a lot, because probably at least two-thirds of the room are  professional citizens. We're all feeling a bit smug and elitist and thinking, wow, this  is amazing. I can run my own server. For everybody else, they're screwed. They have to go and  trust somebody else to run the server for them. The whole point was to give autonomy  for people to give back control of communication to the users, and yet, the users are a bit  lost. So, that's one massive reason to look at P2P. Separately, we also get some better  onboarding, frankly, from a selfish, riot-style app developer perspective. It's a pain in  the ass to have to tell people, hey, which server do you want to connect to? Do you want  to use Matrix.org or something from modular.im or use something else? And instead, how about  you just start off P2P and just join the big global P2P network? Really cool. It would  also be internet less operation. If you're going hiking, if you're on a plane, if you're  hypothetically in the Euro-style coming over from London, wouldn't it be nice to be able  to communicate locally? Also, big deal is metadata, because at the moment, metadata  pools on your home server. You can see every message in terms of who sent it and when they  sent it, where they sent it from. And as I'm sure we all know, metadata in as many ways,  more intelligence, more useful from an intelligence perspective than the actual data itself.  Also, let's just run client-side bridges a bit more easily. That would be really nice  for protocols not thinking of, I know what's that by name, which don't like you bridging  in from a kind of bridge service provider. If you can run it yourself as a user, if you  want to give your WhatsApp credentials to a third-party client like a Matrix bridge,  that's kind of your problem. And then finally, honestly, it raises the bar for development.  So it forces us to do multi-homed accounts, because if I'm going peer-to-peer and I've  got my account running on my phone and on my laptop, I obviously want my same identity  to be on two devices. But if each device is a home server, that also gives us the ability  to port accounts between home servers and account migration. It also forces the home  servers to get smaller. It also forces us to do smarter routing algorithms, because  if everybody here was running their own home server on their phone right now, the idea  of having to do 296 concurrent HTTP hits every time you send a message is going to be nuts.  Also forces us to do low bandwidth transport, because you don't want to be chashering away  on HTTP all the time, because that's just going to chew your battery and bandwidth.  The nicest thing, though, and the really, really, really exciting thing about this,  is that the clients and the bots and the bridges stay absolutely identical. Now, the single  line of code needed to be changed, because we're just swabbing out a home server that  runs somewhere in the cloud and putting it on your phone. So very quick flashback to  2015. You can tell we're in 2015, because we've gone 43 aspect ratio. And I gave a talk at  the conference called Gena Entrepique in Ren, where we announced our end-to-end encryption  for the very first time. It was pre-Megome, it was home, and it was a really cool cryptography  conference. And as part of announcing home, I wrote a bunch of slides to try to avoid  getting crucified for the fact that we weren't protecting metadata at that point. And it  went like this. Matrix is all about creating bridges. You can't bridge without having  metadata leaks, because the bridge is an unavoidable metadata leak. However, we also  expose metadata on home servers. You have the same DAG that builds up on all the different  servers for the same room. Can we do better? And we looked at Pond. Who knows Pond? Interestingly  few people, about 10 people. Pond was a really cool, really early metadata-resistant messaging  app written by Adam Langley. I think from Google is the guy who, I think, ran the TLS  team at Google at the time. And basically, it works like this. You had Tor, you had hidden  services in Tor, and you had a messaging client that would route messages via store-and-forward  hidden services, as well as doing traffic pattern resistance so you couldn't see who  was talking to who when. The disadvantage is that it took about 15 minutes to send a message,  because it was deliberately delaying things in order to avoid the traffic patterns being  visible. But we did the same thought experiment, back in 2015, saying, well, Matrix is meant  to evolve to support this, and I'm sure it will happen real soon now. Could we do something  similar? Yeah, put home servers on the client, use hidden services of some kind to store-and-forward,  and incrementally migrate or bridge from Classic DAG. So it would look like that. You'd have  a home server running on your little gray client, and the client is still sitting there  connecting to it, and then it's bouncing off Tor. So back to the present day, almost.  2018 came up with this diagram, was noodling about where Matrix could go in future. Today,  you end up with server-server client end-to-end encryption setting in the client. What if  in the future you had the kind of Matrix daemon that ran on your kind of operating system  as a Unix daemon that would offload your own, and then you don't have to keep implementing  E2E everywhere? Then what would happen if you actually put that guy as a reusable module  into your client's cross-platform? And perhaps that guy could evolve into a full peer-to-peer  home server if you want to put a home server in it, and you could even offload the E2E  into it, and how in a second, why not actually embed that in the client too, to the extent  that you end up with a client with a home server in it? And the fun thing is that we've  actually built this now. It's called pantalime-in. Talk about it later on today. We're also just  in the process of rewriting pantalime-in from Python into Rust and using it as an embeddable  engine that can go into arbitrary clients. But meanwhile, what's happening over here?  Back to the actual present day. We've got three experiments where we've been playing  in PCB. The first one isn't actually we. It's done by Tim and Terry. You aren't here anywhere.  There he is, who reached out probably, well, nine months ago or something or a year ago  to say, hey, I'm doing a bachelor's project. Can I do something with peer-to-peer matrix?  At which point, I think I basically gave this talk to him very quickly and said, hey, go  wild. And he looked at putting matrix over co-app as low bandwidth protocol using IgDRABZIL  as the overall PCB overlay network. Who knows what IgDRABZIL is? Okay, about 20% of the  room. It's really cool. Experimental peer-to-peer, end-to-end encrypted overlay network basically  creates a big spanning tree over a set of manually defined pairings. But once the spanning  tree is up and running, you can go and root to people end-to-end encrypted across this  thing. So what he did was to go and take Synapse, originally Dendrite, but Dendrite wasn't  working at the time. So it took Synapse and did a proxy that would take HTTP and do co-app  over IgDRABZIL. Very similar to the talk that we gave last year of co-app proxy. And basically  has a similar architecture. That's literally a picture of IgDRABZIL, the spanning tree of  life. And it's literally a spanning tree with a root in the middle. And you can do shortcuts  once you've discovered nodes in different places, but you go and discover them via a  HT and then you go and navigate the spanning tree to actually have the routing to go and  find somebody. So if you're familiar with spanning tree protocol, this is just it for  the internet. Architecturally, home server sitting locally, a client isn't shown, but  you have a client talking to a home server, goes through a proxy, goes over IgDRABZIL and  out the other side. And I'm assured, I haven't tried it myself, but I'm hoping this is a  real screenshot of actually working of two riots on different ports going over IgDRABZIL.  So that was a very, very cool project. And we started to build a little peer-to-peer  secret community of different people like Neil Alexander, who is the maintainer, the  maintainer, one of the maintainers of IgDRABZIL and myself and also Keegan, who wrote Dendrite  originally and one of the guys who wrote Dendrite originally, going and sort of playing in the  space. So the next experiment we did started in, I think, November or December also, where  we took Dendrite, which many will know as our next generation go-lang home server that  has been stuck in development hell of various flavors for about two years. And what if you  just swapped out its HTTP transport for its go and libP2P equivalent? Who knows what libP2P  is? Surprisingly few people. Okay, libP2P is the networking layer for IPFS, and it's  a separate module. It's pretty cool. It gives you a whole bunch of different mechanisms  for discovery, different transports, different implementations in different languages. Theoretically,  it's very, very nice. In practice, it's still a little rough around the edges, but it's  enough to play with. And what you can do is to basically say, hey, libP2P, give me a DHT,  and you then go and, sorry, I've got my 10-minute warning, which means I have to totally change  the tempo of this talk. So you go and say, libP2P, give me a condom, DHT, and then I  want a connection between some public keys. So everything is identified by their public  keys. So first of all, we had to make Dendrite work again, which it does now for Federation,  at least for the subset we need for this demo. But then we needed to make it work a bit more  with, say, messages support to actually have scroll back. So ironically, despite being the  maintainer for Iqrabsil, Neil Alexander has come along to work on libP2P stuff for now.  So sorry for that. But either way, we've now got Dendrite talking slash messages. You then  just swap out one listener for the other listener. And the step four is to try to demo this live  with everybody in the room, what can possibly go wrong. So if anybody actually followed,  oh, that had the right QR code on it, or something, I'm going to go and create an Antaht network  right now. It's going to be called Bicephalus, which is the name of this laptop, on channel  11. And don't all try to join it at once, because it will probably just fall over. In fact, I  haven't even been able to create it. Oh, no, I have created it. So Neil, are you able to  jump on this? Okay. So I'm going to run the commands which are on that URL. I'm going  to start off a docker, which is setting up a Postgres install instance. And then I'm going  to send a Dendrite running locally on swap somewhere. There it is.  So this is invisible on the screen. It's invisible. No. Thank you for telling me. Okay. This is  what you may be seeing. In fact, let me just mirror quickly. I thought this is going to  be very painful. Okay. So that was, I went and set a Postgres running in a docker there.  I've got Dendrite running here. It's found zero rooms. It's advertised zero rooms. But  critically, you can see that it has gone and fired up a public key with a node ID on a  whole bunch of different listeners, one of which is 169 network on the ad hoc Wi-Fi just  created. The reason we're using ad hoc Wi-Fi is that Fozdom is filtering multicast this  year. Otherwise, we would just be on multicast on the normal Fozdom network. I've now discovered  one other libp2p pair, which is very real. So I'm going to go ahead and run it. I'm going  to fire up a port in 8082. Okay. Don't make eye contact with my desktop. So here I am  on a brand new Dendrite running locally. I'm going to create an account for myself. They're  predictable. Right. Fine. And here I am. So I'm going to explore. No. Okay. So that's what  happens if there are no rooms on the network. Okay. I shall create a room quickly. There's  a room, testing, public room. And you can see it's got a beautiful host name here, which  is the public key on libp2p. And I've gotten created it as such. I think I even remembered  to advertise it on the directory. Now, can you see that now? I might not. If I'm not,  I'm an idiot. Let's see. And now I haven't published the room in the directory. So the  directory here is publishing it into the libp2p Mdht. So that in theory, other people on the  network can see it. Come on, demo gods. Everybody pray. Found zero rooms. It's advertised a room.  It's found a PS. So you should be able to see it now. Well, I can see it here. So if I go  to my new home server with this amazing name, I can see testing there with one person in  it, except I'm already in it. Anybody else want to try? Anybody else actually installed  the Docker egg and able to roam onto my network? No, it might be. But I'm really surprised.  Given I can see you. On the plus side, I can send messages in dendrite. Isn't that amazing,  right? Perhaps I can fake it and try to log onto it locally. So I'll go and spin up another  riot. Except it's not going to be, it's just going to be talking to my local one. So that's  what shows the CS API. Restart the dendrite, you reckon? Five minutes, right? No pressure.  Okay, it's restarted. Any better? It's found one room. It's advertised one room. It's discovered  one room. How about you restart your dendrite? Oh, you made a room? I only see one room directory  here. So you managed to connect to the Antop. Cool. Yeah. No joy, Neil? We tested this like  six times in a row and obviously. The weird thing is that I'm seeing the peer, okay? Okay,  I'm going to leave this running in the background and go to another demo even though I've only  just left, which is the third experiment. So experiment two was this today, going through  this in future. However, what would be really awesome would be if we actually put dendrite  in the browser and compiled it to Wazim, swapped out the HTTP transport and go for a JavaScript  app one, switched out the SQL driver layer for one in JavaScript two, and you put it  in a service worker and you can just intercept the outbound traffic from riot and say actually  put it into Wazim instead. And that architecture looks like this. So we've gone and written  a SQLite driver for go, which talks through to JavaScript. We've written an HTTP driver  for go that talks through to JavaScript also. And it looks a bit like this, hopefully. So  ironically, this is going to clash with my dendrite demo. I assume the other one isn't  working, right? Anoying. Okay, let me get rid of that. And instead, kill that off and quickly  go to vector web, p2p and do yarn, run, start. And what this is, is riot web, which has literally  got the architecture that we're looking at here. It goes and pulls in dendrite as a Wazim  dependency. It goes and links through to the other members here. And in theory, it should  be the best demo in the world. However, this one deliberately doesn't work because once  we got this bit to work and that bit to work, dendrite is still talking to Postgres rather  than SQLite. And it turns out that rewriting dendrite's entire schema to SQLite rather  than Postgres is non-trivial. So at the moment, this has got a placeholder in the middle. And  so that's fired up now. And I should be able to go, I think, to this guy here. And if the  demo gods were at all smiling at me, let me just check what port this is and gone. Then  we have the service worker here running in the background, although it's actually not  the one that was just deployed. In fact, this is going horribly wrong. Let me try on a totally  different IP address. I'm almost out of time here, presumably. This is looking a bit more  promising. Perhaps lots and lots of stuff in here. God, the demo gods are not with me  today. So what it should be doing, I wonder if I'm just on the wrong port note, is showing  a dendrite. Hang on. What port is this on? This is when you demo's go, oh, 8083, obviously.  Okay. So here, hopefully, service worker, oh, thank God for that. So demo gods are working,  everybody. So service worker registration, so it's gone and registered service worker,  it's opened up a DB, it's created a table called ping, which is our fake dendrite because we  don't have a real dendrite. Then it's firing up a libp to p thing, which is talking through  to a web socket start rendezvous service, is then looking for other people on it. And if  I went and set this going in another tab on another port, then the two nodes will hopefully  discover one another, over libp to p, and even start, there we go, pinging each other by  HTTP. So the two dendrites are now sending pings back and forth, and they're even going  to be updating the ping table in the database in order to have storage. So that's basically  where we're at right now. It's not fully working as a full dendrite, but it's pretty close. And  there we go. The two right hand things, a dendrite in go or dendrite in js, going forward, libp  to p or a grad tool or something else, who knows, lots of other questions. What's next  in general? E2E encryption going on for DMs today, cross-signing support today, RiotX,  and a whole bunch of other stuff too. Thank you very much.  Unfortunately, we have no time for questions. Please exit through that door. A lot of people  are waiting. And after the dust has settled, please push into the middle. Thank you.
dip_p2p_matrix_0_143.png dip_p2p_matrix_144_2627.png dip_p2p_matrix_2628_7619.png dip_p2p_matrix_7620_18263.png dip_p2p_matrix_18264_18863.png dip_p2p_matrix_18864_24827.png dip_p2p_matrix_24828_25055.png dip_p2p_matrix_25056_25643.png dip_p2p_matrix_25644_26507.png dip_p2p_matrix_26508_26795.png dip_p2p_matrix_26796_26879.png dip_p2p_matrix_26880_27395.png dip_p2p_matrix_27396_27611.png dip_p2p_matrix_27612_28043.png dip_p2p_matrix_28044_28307.png dip_p2p_matrix_28308_28439.png dip_p2p_matrix_28440_28595.png dip_p2p_matrix_28596_29387.png dip_p2p_matrix_29388_29615.png dip_p2p_matrix_29616_31283.png dip_p2p_matrix_31284_31331.png dip_p2p_matrix_31332_31511.png dip_p2p_matrix_31512_32447.png dip_p2p_matrix_32448_32687.png dip_p2p_matrix_32688_32831.png dip_p2p_matrix_32832_33275.png dip_p2p_matrix_33276_33731.png dip_p2p_matrix_33732_34787.png dip_p2p_matrix_34788_35339.png dip_p2p_matrix_35340_35471.png dip_p2p_matrix_35472_35735.png dip_p2p_matrix_35736_36580.png