WebMIDI
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Goodwin, Steven
The connection between Web and MIDI seems as likely as the joining of Garlic with Bread! Yet, we now have the power to create music from the web browser! Either by generating MIDI files for later manipulation, or as live instruments, WebMIDI provides us with the power to build some amazing online music applications. In this short form talk, Steven will cover the internal details of the MIDI protocol, and its related file format, to show how music is represented by computers. This details both the recording and playback of music within the browser, and external application. It comes complete with live demos, synthesized sounds, and occasional music jokes!
 Good morning everyone. As that part of the slide says, my name is Steve and as that part of the slide says, I'm a little bit of a geek. If we've all turned up to the right room, I'm going to be talking about Web MIDI for the next 20 minutes or so. We're going to go through who I am, what I've done to be on the stage, what is MIDI in itself, so we can talk about what is Web MIDI, and we'll talk about the protocol of MIDI and some examples, which means I may be playing some music, which is why I have a keyboard and a microphone. It's just a excuse to show off really, isn't it? So, who am I? Well, I've had to deserve to be on this stage, or it should probably be called the ego slide where the speaker brags about its achievements for 20 minutes. Not going to do that because I haven't done anything. Industry veteran, by the way, just means I'm old. Developer and entrepreneur means absolutely nothing. It means no one would give me a job, so I had to create my own. But the important thing is I have a tournament. This is the only musical instrument you can play without actually touching it. Even though if you go to the Museum of Music in Brussels, there is a sign on the tournament that says, do not touch. Go figure. The important thing, I guess, for the purpose of this is that I do like composing music. I'm just not very good at it. So, I thought, okay, I will do what I can, which is code. So, for many, many years, MIDI has been the standard for musicians to store their music into sequences and to playback. Now, MIDI means multiple things to multiple people. When you say MIDI, that's not enough. MIDI sometimes will mean the serial protocol that goes down the cable between synthesizers, between computers, and synthesizers, or drum machines, or other equipment like that. MIDI can also mean the file format. Now, there were three forms of file format. They have been interestingly labeled format zero, format one, and format two. Zero is pretty useless. Everything is on the one track, but it is very easy to parse. MIDI file format one has got every piece of music on a separate track, which makes it very easy, and MIDI file format two used by almost nobody. But you want to know it exists. Some people mean that when they say MIDI. And MIDI is also the hardware specification. There is a chunk of the spec which says exactly how much current has to go through the cable, what sort of tolerances it has to be, how fast it's running, and all of that stuff we don't need to worry about. We're in the safety of the sound box of a web browser, so we don't need to worry about any of them, which is lucky, because web MIDI is an API for that protocol and nothing else. All the bits that would be useful that are necessary in a sequencer are not available in web MIDI. Thank you for coming on to my TED Talk. Goodbye. It's just almost nothing in there. So if there's nothing in there, it will at least be easy to explain. Does anyone decide for the message yet? That is a MIDI message. If I present it in hex, does that help? No? At least one person is nodding. Well, the first part of this message is the command, 1 for 4, 9, and the second one is the channel. Now, a couple of things you can work out from this. Hex numbers, 4 bits long, from 0 to 15. That means there are only 16 channels. That's part of the spec. It's been there since the year. It's not going to change. Unfortunately, however, the MIDI spec version 2 has just been announced, so this talk will be completely redundant by the time I've finished giving it. The first bit is the command, and again, you'll notice it's a single nibble in hex, which means there are only 16 possible commands. That's another caveat. Two possible things. This command dictates these bytes. So serial protocol, they're coming one after the other, and your program will see that first byte. It says, what is this? It's a note on message, at which point it understands that the following two bytes will be a note value and a volume. Maybe not. Never mind. So the obligatory hello world. Apologies for the use of the word navigator in there. It's probably not really necessary. But this is the thing that will take in a message, parses it out and says, well, we know note on is this 90, and we can track it out to the console. So to prove that works. Probably shouldn't say that and then do a live demo. I should do the live demo and then so let me clear that. If you look onto the bottom window, one message for on, one message for off. Still only three bytes each time. 64 volume one. Where's the time? Where's the time of information in that message? Can anyone tell me where the time is gone? The time is in real life. All the time in Web MIDI is controlled by how much time you wait until you send the next message. There is no other way around it. So you are forced to rely on JavaScript, giving you very precise timing in order to create very precise music. This is why I titled it garlic bread. Is anyone aware of the Peter K sketch? The stand up he does where he goes garlic bread. Because it's just incongruous. You have a MIDI protocol that is for music that is real time that needs to have very precise timings. And then you go and stick it in a web browser, which is sandbox and is not very good at precise timing. Which makes it a little bit painful at times if you try to do anything live. So let's continue with that. So if it's a serial protocol, there is one problem with handling it. How do you know if you're in the middle of a message or not? We've established this is the note on command. This is the note on message. What happens if the first byte we read is 60? What command is that? What is it suddenly going to start doing at that point? What if we come in here? What about 100? Well the clue is by writing it down in binary. So if this most significant bit is 1, it means this is a command. Something is happening now. You should be paying attention. And if the most significant bit is 0, it means this is the data for a previous command. Very simple to parse. It's one operation. Ampersand 0x80. But I just said there are 16 possibilities for this number. If we're using the most significant bit, then actually we only have eight possible commands. And that's it. The entire specification is just eight commands. And that's these ones. So note off is the most important one because it's the first one. No idea why. Note on is the second one. Although you will find. And this is where it gets fun. That 100, if the first byte is the command, the second two bytes are the data bytes. Now you can have any number of data bytes you like. They're controlled by the command. This is the volume. Some keyboard manufacturers have decided that if you put 0 as the volume on a note on message, that means note off. Because obviously it's logical, right? So you've got note on and note off, the key pressure just for hitting it harder, essentially. Parameters and programs for changing the sound. Because although there are no sounds in the MIDI spec and there are no mid sounds in the MIDI files, there is still the indication that there would be a sound. So you say, I'm just telling you the note. I'm saying this note goes on. This note goes off. I'm not telling you any more about it. But I will tell you I would like the synthesizer to play a piano or a string sound. It doesn't tell you what the string sound is or how the string sound is formed. Just something in the general area of strings. Changing the pressure more after touch thing. The pitch wheel, obviously, this expensive keyboard does not have pitch wheel. But if you do, then there is a special message for it. And then there's the meta events for everything else. It's like they started designing the keyboard and they thought, yeah, we need notes on. We need the ability to change the program. We need the pitch wheel. Oh, dear, we've run out. So everything else is lumbered in this bottom stage. And there are just hundreds of these meta events, system-exclusive messages that send patches. It's just full of them. This one, I know it's.h file because this list comes from my original C version of this library. And it just lists them all in there. So what is message? What am I missing? In Web MIDI. We've established a timing message is missing. That is just something you have to time yourself. When you deal with MIDI files, and I'll show you some bits later, you have to specify that time. So you have to count the time yourself. So you say, this note comes on, then this note comes on. And there is half a second between it. You then write that in. The duration, again, if you don't have timing messages, you can't have duration. So you have to handle that yourself. Think about how long each of these notes are going to be. Chords. Chords are kind of fundamental in music. This is harmony. Every single piece of reasonably good music will have harmony. And quite a lot of bad music will have harmony as well. But there is no way of describing a chord. You have to say, this note on, then this note on, and then this note on, at the same time. Or as close to the same time as you can possibly manage in your code. Ideally, you'd want that time to be zero, or at least shorter than 10 milliseconds. And there are no conventions. So if you use one of the system exclusive messages, it's up to you how you decide to handle that. Every keyboard manufacturer has their own set of things that they put into those messages. And if you want to process them or understand what that keyboard is sending, you're on your own. So we've got the protocol as a stream of messages. And I was going to give some examples, but I will have to go out of here for that. Let me shut that down. This is the one that I've been fiddling with for the time. Is it this one? Okay. I'll put this up as loud as I can and then flip this on. I switched to bass. Now you can tell that there is a little bit of a latency on me pressing the keys and hearing the notes, which is unfortunate. But that's unfortunately nothing of my doing. I'm not that bad a coder. It is just the time that it takes for the message to come through and the browser to react and the sound engine inside the web audio engine to play those sounds. Because there isn't anything much is really doing there. That's a web audio backend being triggered from the JavaScript in the browser. This is just an example. This has got nothing to do with it. This is an example I was playing with earlier. I was curious about waveforms and how waveforms are created. So I built this web audio thing to create waves using different harmonics. I just needed time to hook it up to the setup today. But if I show you another one like this soundboard for example. If I play notes on here now, I didn't get the looping quite right on it. But all I've done here is I've listened for the note down message. When I get that note on message, I trigger the sample loop. I don't do anything else. When the note up comes, I don't stop it. When the second note down comes, I stop it. That's logic. That's one variable. That's one check. Which you can have hours of fun with. Now while I'm doing that, you might hear something else going on. There's a bass sound going on underneath each of those samples. That bass sound. The bass sound from that window. When you open WebMiddy and you say, I would like to do some WebMiddy, please, Mr. Webbrowser. And the Web browser goes, sure, which would you like? And it will allow you to enumerate every MIDI device plugged into your machine. Which is great. It means if you've got multiple keyboards, you can have one keyboard doing this thing and one kid doing that thing. Works brilliantly. Unfortunately, it works brilliantly for everyone else as well. So if you happen to have two apps that are running in different browsers, it means you can see the other person's traffic. Now I say this because I'm using WebMiddy for music stuff, mostly. Well, as much as you can call that music. It's just a loop. But because I'm using it for music, I think very much in the music sense. But these things are cheap, right? It's a tenner of anyone's money. There is no reason why you can't take that apart, use the switches and hook them up to something else. You might like to do a game controller, which is more accessible than traditional game pads. That would make a very cheap and easy accessible game pad controller. And it's controllable through the web browser with WebMiddy. Unfortunately, if you were doing anything sensitive, like you were requiring someone to put in a pass code using those controllers, it would also be visible in another window. And currently, as it stands, there is nothing to stop you. It will be visible in every window and you can't stop. You can't say, oh, I want this for myself. That controller is available to everyone. And you can start that up on your web page without a click, without the user having to perform things. If you've done any kind of web audio things in the browser, you will probably notice that it doesn't work until the user has actually clicked on something. It requires the user to interact with the page before it gives you control of the web audio device. Not the same for WebMiddy. You can just look at that traffic at any time. Another example we had, which is probably the most interesting one, I suppose, is that one, I think. And of course, it's still running in my other window, so let's shut that down. So again, see if you can spot any latency on this one. This is about the closest I'm going to get. I'll leave that there. I'll quit while I'm ahead. Look at that back on. Now, this is obviously a proper example of a sense, which I didn't write, but it is quite fun and it is again using web audio. And web audio, as you say, you've got to click here before you can actually make sounds. But it's a nice example and you can see that it is possible to build quite big and fun things. And at this point, I realized the person who's written this might be in the room. Nope, good. That would be awful. It's happened before, actually, sort of mid-lectroactivity thing. I was at a computer thing, funnily enough, me computer thing, who'd have thunk it? I was at a computer thing and we're talking about old computers, as my friends often do. And I happen to mention this trick on the ZX81 about how you could put components across the chip to fix the square root bug that existed on the ZX81 at the time. And I was explaining how it worked and why it only worked in that case. And then I found out the guy next to me built it. And the mansplaining level was a 10 at that point. I mean, Fanboy was off the charts. But it's one of those things. I'm sure I'm going to give this talk and the person who wrote it is going to be here. It's really nice. And it works real time. I remember back in the 1990s, you could make a whole album with just that. I don't think I should have done that. Let's do that instead. Oh, so we've got some library links. So let's come back to them. So as I was saying, I used this for music, but you could create a game controller by ripping out the innards and putting that into something else. As we say, it's not keyboards, drum pads. They're very fun. They're very easy. I like those sort of things. The Navation Lump pad, which I can't quite afford, but they are very pretty and they have lots of lights on them. And you can control lighting rigs. When I did some theater stuff a while back, the lighting rig that was a big desk, but controlled by MIDI. Same protocol, note on, meant light on, and note off, meant light off. But it's doable. It's a standard protocol that seems to have gone beyond what it should have ever been doing. The rate of MIDI is very slow. It's about 33-100 bar rate, essentially. Well, it's slow when you think of how many notes could be playing in a piece of music, even a fairly straightforward symphony with only maybe 50 or 60 players. There's 50, 60 notes in there minimum. You've only got 16 channels, and you've got multiple notes. So you're probably not going to get all of them playing at the same time because you can't play all of them with zero time gap. There will always be a time gap between each note. So it will never be quite right. This is the slide you might want to take pictures of if you want to go and play with that synthy thing later. Also, there is no file handling. Oh, sorry. Did I go? There we go. For those that want to take the picture. I'm ready for my close-up. So file handling is missing. If you want to save the MIDI file, you can't. If you've generated something nice, you've got to save it yourself. Don't worry. I've written a library for that. There will be a link later for as well. This is an abstraction. So any manufacturer such as Korg and Yamaha, they will have their own system exclusive data, which you just have to go and find out for them, and any actual sound, the actual noises themselves, not in the spec. It's just an indication that there is a piano sound here. If you do want to have a look, my MIDI library is the second one. These other things are also perfectly valid MIDI libraries. My one was based on the C++ library that I wrote many, many, many years ago. I think my original MIDI library is actually older than this conference, and there's not many things that are. So those are available. Have you got the pictures now? Yeah. So my basic conclusions is the Web MIDI is really interesting for input. Lots of things support it, not just the keyboards, the drum machines, and all that sort of stuff. Very good for handling. You can build your own, but the output is more interesting because now the browser can control synthesizer rigs and it can control drum machines. But the number of people wanting to do that is probably even fewer than want to do input. So with that, I shall leave it open for the last five minutes for questions. I should update my FOSDEM scorecard. That's now the 12th different talk I've given, and I still haven't got good at it. So we've got a microphone. It says questions. And I love my beer. So just to know the question, and for the question, Mark, how many have about five minutes? So it kind of looks like someone took a naive translation of a wire protocol and dumped it straight into JavaScript without any kind of design process. Was there actually a design process, or did someone manage to persuade the Chrome team just to put in this API that no one had really thought about? I don't know why or how Web MIDI came to be. I don't know the people. I suspect that looking at it, they said it's not so much dumping the wire protocol, but dumping what every other MIDI library does. Every other MIDI library just gives you that raw data. Here is the stream. Partly because you can't process the SysX messages. There's no simple way of abstracting that other than to say, it's a SysX message, and here is the binary blob. There could have been abstractions. They could have created an API which said, with an event, and you say, I want this function called when I get a note on event. But they didn't do that, I think, because they wanted to maintain a direct relationship with all the preceding libraries and the way that people wrote MIDI applications. But that's my conjecture. Yes, my library has attached off the GitHub there, does abstract away a lot of this ugliness, and it handles that odd, stupid case as well. Where if the note is on, but the volume is zero, it actually means note off, which is one of those things that fox me for ages. There's also the odd thing as well in the MIDI spec that if you do a note on message and then another note on message, you don't have to say note on again. You just keep going. So you don't get that most significant bit being one for the next message. So you just see note on is a three byte message. Why am I getting five bytes? That's just because there's that quirk that was in the original MIDI spec, because it will cut down on the bandwidth used. You don't have to send a byte. And back in 1982, when this MIDI was created, that one byte was important. Now it's not, but we're stuck with it. So we've got these little caveats in the original spec that have carried through to WebMIDI. I have a microphone for you. So how stable is the WebMIDI stuff? Could it be used for editor librarians, for older synthesizers? Like I have some old stuff that I need to use an Atari emulator to actually edit stuff on. Could WebMIDI be used to reliably do that? Do you think? Is it stable enough? So is WebMIDI stable enough to use as a librarian tool and so forth? It is stable enough. WebMIDI as a spec isn't going to be changing. The API is not going to be changing. There will be upgrades for Web 2.0, which is coming through the pipeline now. But it's more the case of do you want to spend that amount of time writing a librarian tool when there might be better ways of doing it? It would probably be good for you. It would be good for me some of the time. But because it's coming quite late into the game, I'm concerned that it might only be used for fun projects like this rather than serious projects that have already got librarian tools and other system tools. But I'd like to see it done more. I don't. As soon as someone puts a VR headset down, I know that interest is going there and not here. No, you still have two minutes of interest. Okay. So can we muster two minutes of interest? No, no. Thank you a lot. Thank you very much. Thank you.
 Good morning everyone. As that part of the slide says, my name is Steve and as that part  of the slide says, I'm a little bit of a geek. If we've all turned up to the right room,  I'm going to be talking about Web MIDI for the next 20 minutes or so. We're going to  go through who I am, what I've done to be on the stage, what is MIDI in itself, so we  can talk about what is Web MIDI, and we'll talk about the protocol of MIDI and some examples,  which means I may be playing some music, which is why I have a keyboard and a microphone.  It's just a excuse to show off really, isn't it? So, who am I? Well, I've had to deserve to be  on this stage, or it should probably be called the ego slide where the speaker brags about  its achievements for 20 minutes. Not going to do that because I haven't done anything.  Industry veteran, by the way, just means I'm old. Developer and entrepreneur means absolutely  nothing. It means no one would give me a job, so I had to create my own. But the important thing  is I have a tournament. This is the only musical instrument you can play without actually touching  it. Even though if you go to the Museum of Music in Brussels, there is a sign on the  tournament that says, do not touch. Go figure. The important thing, I guess, for the purpose  of this is that I do like composing music. I'm just not very good at it. So, I thought,  okay, I will do what I can, which is code. So, for many, many years, MIDI has been the  standard for musicians to store their music into sequences and to playback. Now, MIDI  means multiple things to multiple people. When you say MIDI, that's not enough. MIDI  sometimes will mean the serial protocol that goes down the cable between synthesizers,  between computers, and synthesizers, or drum machines, or other equipment like that. MIDI  can also mean the file format. Now, there were three forms of file format. They have  been interestingly labeled format zero, format one, and format two. Zero is pretty useless.  Everything is on the one track, but it is very easy to parse. MIDI file format one has  got every piece of music on a separate track, which makes it very easy, and MIDI file format  two used by almost nobody. But you want to know it exists. Some people mean that when  they say MIDI. And MIDI is also the hardware specification. There is a chunk of the spec  which says exactly how much current has to go through the cable, what sort of tolerances  it has to be, how fast it's running, and all of that stuff we don't need to worry about.  We're in the safety of the sound box of a web browser, so we don't need to worry about  any of them, which is lucky, because web MIDI is an API for that protocol and nothing else.  All the bits that would be useful that are necessary in a sequencer are not available  in web MIDI. Thank you for coming on to my TED Talk. Goodbye. It's just almost nothing  in there. So if there's nothing in there, it will at least be easy to explain. Does  anyone decide for the message yet? That is a MIDI message. If I present it in hex, does  that help? No? At least one person is nodding. Well, the first part of this message is the  command, 1 for 4, 9, and the second one is the channel. Now, a couple of things you can  work out from this. Hex numbers, 4 bits long, from 0 to 15. That means there are only 16  channels. That's part of the spec. It's been there since the year. It's not going to change.  Unfortunately, however, the MIDI spec version 2 has just been announced, so this talk will  be completely redundant by the time I've finished giving it. The first bit is the command, and  again, you'll notice it's a single nibble in hex, which means there are only 16 possible  commands. That's another caveat. Two possible things. This command dictates these bytes.  So serial protocol, they're coming one after the other, and your program will see that  first byte. It says, what is this? It's a note on message, at which point it understands  that the following two bytes will be a note value and a volume. Maybe not. Never mind.  So the obligatory hello world. Apologies for the use of the word navigator in there. It's  probably not really necessary. But this is the thing that will take in a message, parses  it out and says, well, we know note on is this 90, and we can track it out to the console.  So to prove that works. Probably shouldn't say that and then do a live demo. I should  do the live demo and then so let me clear that. If you look onto the bottom window,  one message for on, one message for off. Still only three bytes each time. 64 volume one.  Where's the time? Where's the time of information in that message? Can anyone tell me where  the time is gone? The time is in real life. All the time in Web MIDI is controlled by  how much time you wait until you send the next message. There is no other way around  it. So you are forced to rely on JavaScript, giving you very precise timing in order to  create very precise music. This is why I titled it garlic bread. Is anyone aware of the Peter  K sketch? The stand up he does where he goes garlic bread. Because it's just incongruous.  You have a MIDI protocol that is for music that is real time that needs to have very  precise timings. And then you go and stick it in a web browser, which is sandbox and  is not very good at precise timing. Which makes it a little bit painful at times if  you try to do anything live. So let's continue with that. So if it's a serial protocol, there  is one problem with handling it. How do you know if you're in the middle of a message  or not? We've established this is the note on command. This is the note on message. What  happens if the first byte we read is 60? What command is that? What is it suddenly going  to start doing at that point? What if we come in here? What about 100? Well the clue is  by writing it down in binary. So if this most significant bit is 1, it means this is a  command. Something is happening now. You should be paying attention. And if the most significant  bit is 0, it means this is the data for a previous command. Very simple to parse. It's  one operation. Ampersand 0x80. But I just said there are 16 possibilities for this number.  If we're using the most significant bit, then actually we only have eight possible commands.  And that's it. The entire specification is just eight commands. And that's these ones.  So note off is the most important one because it's the first one. No idea why. Note on is  the second one. Although you will find. And this is where it gets fun. That 100, if the  first byte is the command, the second two bytes are the data bytes. Now you can have  any number of data bytes you like. They're controlled by the command. This is the volume.  Some keyboard manufacturers have decided that if you put 0 as the volume on a note on message,  that means note off. Because obviously it's logical, right? So you've got note on and  note off, the key pressure just for hitting it harder, essentially. Parameters and programs  for changing the sound. Because although there are no sounds in the MIDI spec and there are  no mid sounds in the MIDI files, there is still the indication that there would be a  sound. So you say, I'm just telling you the note. I'm saying this note goes on. This note  goes off. I'm not telling you any more about it. But I will tell you I would like the synthesizer  to play a piano or a string sound. It doesn't tell you what the string sound is or how the  string sound is formed. Just something in the general area of strings. Changing the pressure  more after touch thing. The pitch wheel, obviously, this expensive keyboard does not have pitch  wheel. But if you do, then there is a special message for it. And then there's the meta  events for everything else. It's like they started designing the keyboard and they thought,  yeah, we need notes on. We need the ability to change the program. We need the pitch wheel.  Oh, dear, we've run out. So everything else is lumbered in this bottom stage. And there  are just hundreds of these meta events, system-exclusive messages that send patches. It's just full  of them. This one, I know it's.h file because this list comes from my original C version  of this library. And it just lists them all in there. So what is message? What am I missing?  In Web MIDI. We've established a timing message is missing. That is just something you have  to time yourself. When you deal with MIDI files, and I'll show you some bits later, you have  to specify that time. So you have to count the time yourself. So you say, this note comes  on, then this note comes on. And there is half a second between it. You then write that  in. The duration, again, if you don't have timing messages, you can't have duration.  So you have to handle that yourself. Think about how long each of these notes are going  to be. Chords. Chords are kind of fundamental in music. This is harmony. Every single piece  of reasonably good music will have harmony. And quite a lot of bad music will have harmony  as well. But there is no way of describing a chord. You have to say, this note on, then  this note on, and then this note on, at the same time. Or as close to the same time as  you can possibly manage in your code. Ideally, you'd want that time to be zero, or at least  shorter than 10 milliseconds. And there are no conventions. So if you use one of the system  exclusive messages, it's up to you how you decide to handle that. Every keyboard manufacturer  has their own set of things that they put into those messages. And if you want to process  them or understand what that keyboard is sending, you're on your own.  So we've got the protocol as a stream of messages. And I was going to give some examples, but  I will have to go out of here for that. Let me shut that down. This is the one that I've  been fiddling with for the time. Is it this one? Okay. I'll put this up as loud as I can  and then flip this on. I switched to bass. Now you can tell that there is a little bit  of a latency on me pressing the keys and hearing the notes, which is unfortunate. But that's  unfortunately nothing of my doing. I'm not that bad a coder. It is just the time that  it takes for the message to come through and the browser to react and the sound engine  inside the web audio engine to play those sounds. Because there isn't anything much  is really doing there. That's a web audio backend being triggered from the JavaScript  in the browser. This is just an example. This has got nothing to do with it. This is an  example I was playing with earlier. I was curious about waveforms and how waveforms  are created. So I built this web audio thing to create waves using different harmonics.  I just needed time to hook it up to the setup today. But if I show you another one like  this soundboard for example. If I play notes on here now, I didn't get the looping quite  right on it. But all I've done here is I've listened for the note down message. When I  get that note on message, I trigger the sample loop. I don't do anything else. When the note  up comes, I don't stop it. When the second note down comes, I stop it. That's logic.  That's one variable. That's one check. Which you can have hours of fun with. Now while  I'm doing that, you might hear something else going on. There's a bass sound going on underneath  each of those samples. That bass sound. The bass sound from that window. When you open  WebMiddy and you say, I would like to do some WebMiddy, please, Mr. Webbrowser. And the  Web browser goes, sure, which would you like? And it will allow you to enumerate every MIDI  device plugged into your machine. Which is great. It means if you've got multiple keyboards,  you can have one keyboard doing this thing and one kid doing that thing. Works brilliantly.  Unfortunately, it works brilliantly for everyone else as well. So if you happen to have two  apps that are running in different browsers, it means you can see the other person's traffic.  Now I say this because I'm using WebMiddy for music stuff, mostly. Well, as much as you  can call that music. It's just a loop. But because I'm using it for music, I think very  much in the music sense. But these things are cheap, right? It's a tenner of anyone's  money. There is no reason why you can't take that apart, use the switches and hook them  up to something else. You might like to do a game controller, which is more accessible  than traditional game pads. That would make a very cheap and easy accessible game pad  controller. And it's controllable through the web browser with WebMiddy. Unfortunately,  if you were doing anything sensitive, like you were requiring someone to put in a pass  code using those controllers, it would also be visible in another window. And currently,  as it stands, there is nothing to stop you. It will be visible in every window and you  can't stop. You can't say, oh, I want this for myself. That controller is available to  everyone. And you can start that up on your web page without a click, without the user  having to perform things. If you've done any kind of web audio things in the browser, you  will probably notice that it doesn't work until the user has actually clicked on something.  It requires the user to interact with the page before it gives you control of the web  audio device. Not the same for WebMiddy. You can just look at that traffic at any time.  Another example we had, which is probably the most interesting one, I suppose, is that  one, I think. And of course, it's still running in my other window, so let's shut that down.  So again, see if you can spot any latency on this one.  This is about the closest I'm going to get. I'll leave that there. I'll quit while I'm  ahead. Look at that back on. Now, this is obviously a proper example of a sense, which  I didn't write, but it is quite fun and it is again using web audio. And web audio, as  you say, you've got to click here before you can actually make sounds. But it's a nice  example and you can see that it is possible to build quite big and fun things. And at  this point, I realized the person who's written this might be in the room. Nope, good. That  would be awful. It's happened before, actually, sort of mid-lectroactivity thing. I was at  a computer thing, funnily enough, me computer thing, who'd have thunk it? I was at a computer  thing and we're talking about old computers, as my friends often do. And I happen to mention  this trick on the ZX81 about how you could put components across the chip to fix the  square root bug that existed on the ZX81 at the time. And I was explaining how it worked  and why it only worked in that case. And then I found out the guy next to me built it. And  the mansplaining level was a 10 at that point. I mean, Fanboy was off the charts. But it's  one of those things. I'm sure I'm going to give this talk and the person who wrote it  is going to be here. It's really nice. And it works real time. I remember back in the  1990s, you could make a whole album with just that. I don't think I should have done that.  Let's do that instead. Oh, so we've got some library links. So let's come back to them.  So as I was saying, I used this for music, but you could create a game controller by  ripping out the innards and putting that into something else. As we say, it's not keyboards,  drum pads. They're very fun. They're very easy. I like those sort of things. The Navation  Lump pad, which I can't quite afford, but they are very pretty and they have lots of  lights on them. And you can control lighting rigs. When I did some theater stuff a while  back, the lighting rig that was a big desk, but controlled by MIDI. Same protocol, note  on, meant light on, and note off, meant light off. But it's doable. It's a standard protocol  that seems to have gone beyond what it should have ever been doing. The rate of MIDI is  very slow. It's about 33-100 bar rate, essentially. Well, it's slow when you think of how many  notes could be playing in a piece of music, even a fairly straightforward symphony with  only maybe 50 or 60 players. There's 50, 60 notes in there minimum. You've only got 16  channels, and you've got multiple notes. So you're probably not going to get all of  them playing at the same time because you can't play all of them with zero time gap.  There will always be a time gap between each note. So it will never be quite right. This  is the slide you might want to take pictures of if you want to go and play with that synthy  thing later. Also, there is no file handling. Oh, sorry. Did I go? There we go. For those  that want to take the picture. I'm ready for my close-up. So file handling is missing.  If you want to save the MIDI file, you can't. If you've generated something nice, you've  got to save it yourself. Don't worry. I've written a library for that. There will be  a link later for as well. This is an abstraction. So any manufacturer such as Korg and Yamaha,  they will have their own system exclusive data, which you just have to go and find out  for them, and any actual sound, the actual noises themselves, not in the spec. It's just  an indication that there is a piano sound here. If you do want to have a look, my MIDI  library is the second one. These other things are also perfectly valid MIDI libraries. My  one was based on the C++ library that I wrote many, many, many years ago. I think my original  MIDI library is actually older than this conference, and there's not many things that are. So those  are available. Have you got the pictures now? Yeah. So my basic conclusions is the Web  MIDI is really interesting for input. Lots of things support it, not just the keyboards,  the drum machines, and all that sort of stuff. Very good for handling. You can build your  own, but the output is more interesting because now the browser can control synthesizer rigs  and it can control drum machines. But the number of people wanting to do that is probably  even fewer than want to do input. So with that, I shall leave it open for the last five minutes  for questions. I should update my FOSDEM scorecard. That's now the 12th different talk I've given,  and I still haven't got good at it. So we've got a microphone. It says questions. And I  love my beer. So just to know the question, and for the question, Mark, how many have  about five minutes? So it kind of looks like someone took a naive translation of a wire  protocol and dumped it straight into JavaScript without any kind of design process. Was there  actually a design process, or did someone manage to persuade the Chrome team just to  put in this API that no one had really thought about? I don't know why or how Web MIDI came  to be. I don't know the people. I suspect that looking at it, they said it's not so much  dumping the wire protocol, but dumping what every other MIDI library does. Every other  MIDI library just gives you that raw data. Here is the stream. Partly because you can't  process the SysX messages. There's no simple way of abstracting that other than to say,  it's a SysX message, and here is the binary blob. There could have been abstractions.  They could have created an API which said, with an event, and you say, I want this function  called when I get a note on event. But they didn't do that, I think, because they wanted  to maintain a direct relationship with all the preceding libraries and the way that people  wrote MIDI applications. But that's my conjecture. Yes, my library has attached off the GitHub  there, does abstract away a lot of this ugliness, and it handles that odd, stupid case as well.  Where if the note is on, but the volume is zero, it actually means note off, which is  one of those things that fox me for ages. There's also the odd thing as well in the  MIDI spec that if you do a note on message and then another note on message, you don't  have to say note on again. You just keep going. So you don't get that most significant bit  being one for the next message. So you just see note on is a three byte message. Why am  I getting five bytes? That's just because there's that quirk that was in the original  MIDI spec, because it will cut down on the bandwidth used. You don't have to send a byte.  And back in 1982, when this MIDI was created, that one byte was important. Now it's not,  but we're stuck with it. So we've got these little caveats in the original spec that have  carried through to WebMIDI. I have a microphone for you.  So how stable is the WebMIDI stuff? Could it be used for editor librarians, for older  synthesizers? Like I have some old stuff that I need to use an Atari emulator to actually  edit stuff on. Could WebMIDI be used to reliably do that? Do you think? Is it stable enough?  So is WebMIDI stable enough to use as a librarian tool and so forth? It is stable enough. WebMIDI  as a spec isn't going to be changing. The API is not going to be changing. There will  be upgrades for Web 2.0, which is coming through the pipeline now. But it's more the case  of do you want to spend that amount of time writing a librarian tool when there might  be better ways of doing it? It would probably be good for you. It would be good for me some  of the time. But because it's coming quite late into the game, I'm concerned that it  might only be used for fun projects like this rather than serious projects that have already  got librarian tools and other system tools. But I'd like to see it done more. I don't.  As soon as someone puts a VR headset down, I know that interest is going there and not  here.  No, you still have two minutes of interest.  Okay. So can we muster two minutes of interest? No, no.  Thank you a lot.  Thank you very much. Thank you.
webmidi_0_167.png webmidi_168_431.png webmidi_432_947.png webmidi_948_1067.png webmidi_1068_1223.png webmidi_1224_2123.png webmidi_2124_4127.png webmidi_4128_4595.png webmidi_4596_6935.png webmidi_6936_7439.png webmidi_7440_7499.png webmidi_7500_9491.png webmidi_9492_9587.png webmidi_9588_14051.png webmidi_14052_16103.png webmidi_16104_16247.png webmidi_16248_16307.png webmidi_16308_16355.png webmidi_16356_16415.png webmidi_16416_16487.png webmidi_16488_23639.png webmidi_23640_23867.png webmidi_23868_26999.png webmidi_27000_27095.png webmidi_27096_27335.png webmidi_27336_27407.png webmidi_27408_29447.png webmidi_29448_29759.png webmidi_29760_30803.png webmidi_30804_31391.png webmidi_31392_32051.png webmidi_32052_38447.png webmidi_38448_38608.png