Decentralized Collaborative Annotations using Matrix
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Fedorin, Daniel
This talk showcases Matrix Highlight, a tool built on top of the Matrix protocol to collaboratively annotate and comment on pages on the internet. By building on top of Matrix, we get decentralized, federated, and open web (or more) annotation. This talk will cover a demonstration, using Matrix events for non-messaging purposes, and the benefits of building on Matrix.
 Hi there. Today I'm going to be talking about decentralized collaborative annotations using Matrix. So on the agenda today is a demonstration of a tool, Matrix Highlight, that I've built, which lets you highlight the web collaboratively using Matrix. Then a discussion of why I liked building on Matrix and why I think you might want to consider doing the same. And finally, I'm going to talk about using Matrix for more than messaging. So first of all, a quick demonstration. Alright, time for a quick demonstration. So what I have here are two Chrome windows, both of which are logged into Matrix Highlight but with different accounts. That way I can give you an overview of what it looks like when two users are going at it at the same time. And so what the extension gives you is the ability to right click any page and click highlight with Matrix, which brings up a tooltip that hovers over anything on the page. And this tooltip, actually let me do this first on both sides. It has a bunch of things, but the first thing you need to do to start using the tool is to create a Matrix room that corresponds to your page. I will do that really quickly. And what I'm also going to do is click users here and invite the user on the left so that we can collaborate on this page together. I think this is it. This is their Matrix username. And I will join it on the other side. Right. So now I have two pages logged into the same room. And I can start highlighting. So I can select some text, which gives me a tooltip, and I can clear color and get it. And it appears on both screens. In fact, I can refresh the page, highlight using Matrix again, and it's still there. This is persistent and it's stored on Matrix. I can do this a couple more times. And I like to use different colors because it's prettier that way. But you can definitely stick to whatever you prefer. And of course, what I can also do is click one of these things and make a comment. Fascinating. These comments show up in real time on the other person's end. I just didn't have them open. You're right. And then of course, these two things are synchronized. There's some minor support for text editing. Bold, italics, code. And of course, you can also look through a list of all the quotes that have been highlighted so far. This term, I'm a TA for Organic State Universities. That's the first sentence of this article, and so on and so forth. So why did I choose to build my tool on top of Matrix? There's many good reasons. For one, Synapse is an established piece of tooling for decentralized federated communication. Users could right now install this piece of software and communicate with everyone else on the Federation if they so chose. And you could self-host it. That's just fine. Furthermore, Matrix as a protocol has support for many different things that come up frequently in various use cases. You can send and receive files. You can relate events to each other. One use case for which is threads. You can organize rooms into spaces and spaces into spaces. And it's all very deep. And there's also support for Android encryption. I think we can all agree that providing users privacy is a good thing. And of course, Element itself actually has debugging tools. So I can send custom events without implementing a client myself, and I can view event sources that I receive, and I can browse room state and do all sorts of things that I as a developer can really appreciate. And now we come to how I did it, or Matrix for more than messaging. So take a look at this regular old Matrix event I plucked from one of my rooms. There's a lot of things here, and only two of them matter for us, the type of the event and the content. So the type denotes basically the sort of thing the event represents. In this case, it's a message. And the content, the contents of which usually depend on the type, in this case is the text from the message. I think that's a good idea. The beauty of Matrix though is that you can make your own message types and fill their contents with pretty much whatever you want. And that's exactly what I do. I send a custom highlight event whenever a user selects a piece of text and clicks a color. And this event has five keys, highlight text, which is the text, computer science student in this case. The beginning and end position of the highlight and the color. And replies to these highlights are actually just plain old regular Matrix messages that have an M dot relates to thing in their content, which is actually built into Matrix and it's part of the threading API. So these are actually just threads. And I think the beauty of Matrix is that you can pretty much represent anything in Matrix that can be represented using a graph because events are just nodes and various relations are arrows between them. So if your piece of data can be represented with this and most likely can, you can leverage Matrix to create a cool piece of software like Matrix Highlight. Thanks for watching.
 Hi there.  Today I'm going to be talking about decentralized collaborative annotations using Matrix.  So on the agenda today is a demonstration of a tool, Matrix Highlight, that I've built,  which lets you highlight the web collaboratively using Matrix.  Then a discussion of why I liked building on Matrix and why I think you might want to  consider doing the same.  And finally, I'm going to talk about using Matrix for more than messaging.  So first of all, a quick demonstration.  Alright, time for a quick demonstration.  So what I have here are two Chrome windows, both of which are logged into Matrix Highlight  but with different accounts.  That way I can give you an overview of what it looks like when two users are going at  it at the same time.  And so what the extension gives you is the ability to right click any page and click  highlight with Matrix, which brings up a tooltip that hovers over anything on the page.  And this tooltip, actually let me do this first on both sides.  It has a bunch of things, but the first thing you need to do to start using the tool is  to create a Matrix room that corresponds to your page.  I will do that really quickly.  And what I'm also going to do is click users here and invite the user on the left so that  we can collaborate on this page together.  I think this is it.  This is their Matrix username.  And I will join it on the other side.  Right.  So now I have two pages logged into the same room.  And I can start highlighting.  So I can select some text, which gives me a tooltip, and I can clear color and get it.  And it appears on both screens.  In fact, I can refresh the page, highlight using Matrix again, and it's still there.  This is persistent and it's stored on Matrix.  I can do this a couple more times.  And I like to use different colors because it's prettier that way.  But you can definitely stick to whatever you prefer.  And of course, what I can also do is click one of these things and make a comment.  Fascinating.  These comments show up in real time on the other person's end.  I just didn't have them open.  You're right.  And then of course, these two things are synchronized.  There's some minor support for text editing.  Bold, italics, code.  And of course, you can also look through a list of all the quotes that have been highlighted  so far.  This term, I'm a TA for Organic State Universities.  That's the first sentence of this article, and so on and so forth.  So why did I choose to build my tool on top of Matrix?  There's many good reasons.  For one, Synapse is an established piece of tooling for decentralized federated communication.  Users could right now install this piece of software and communicate with everyone else  on the Federation if they so chose.  And you could self-host it.  That's just fine.  Furthermore, Matrix as a protocol has support for many different things that come up frequently  in various use cases.  You can send and receive files.  You can relate events to each other.  One use case for which is threads.  You can organize rooms into spaces and spaces into spaces.  And it's all very deep.  And there's also support for Android encryption.  I think we can all agree that providing users privacy is a good thing.  And of course, Element itself actually has debugging tools.  So I can send custom events without implementing a client myself, and I can view event sources  that I receive, and I can browse room state and do all sorts of things that I as a developer  can really appreciate.  And now we come to how I did it, or Matrix for more than messaging.  So take a look at this regular old Matrix event I plucked from one of my rooms.  There's a lot of things here, and only two of them matter for us, the type of the event  and the content.  So the type denotes basically the sort of thing the event represents.  In this case, it's a message.  And the content, the contents of which usually depend on the type, in this case is the text  from the message.  I think that's a good idea.  The beauty of Matrix though is that you can make your own message types and fill their  contents with pretty much whatever you want.  And that's exactly what I do.  I send a custom highlight event whenever a user selects a piece of text and clicks a  color.  And this event has five keys, highlight text, which is the text, computer science student  in this case.  The beginning and end position of the highlight and the color.  And replies to these highlights are actually just plain old regular Matrix messages that  have an M dot relates to thing in their content, which is actually built into Matrix and it's  part of the threading API.  So these are actually just threads.  And I think the beauty of Matrix is that you can pretty much represent anything in Matrix  that can be represented using a graph because events are just nodes and various relations  are arrows between them.  So if your piece of data can be represented with this and most likely can, you can leverage  Matrix to create a cool piece of software like Matrix Highlight.  Thanks for watching.
matrix_decentralized_annotations_0_3983.png matrix_decentralized_annotations_3984_5423.png matrix_decentralized_annotations_5424_5735.png matrix_decentralized_annotations_5736_6887.png matrix_decentralized_annotations_6888_7007.png matrix_decentralized_annotations_7008_7400.png