Having fun with music and Keras
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Raas, Marcel
This talk is about applying deep learning to music. We will look at the raw music data and discover the following: How to detect instruments from a piece of music How to detect what is being played by what instrument How to isolate instruments in multi-instrument (polyphonic) music Instead of applying it to existing music we will generate our own music using some simple musical rules. The benefit of this is that we are in control of the complexity and we know exactly what is being played. We start out simple and then start adding more instruments, different timbres, etc. As we go up in complexity, we shall see how to adapt our models to be able to deal with it. This gives interesting insights in what structures in deep nets work well. I will show: How to build a simple synthesizer using numpy How to create an unlimited data set of improvisation that sounds musical How to use this data set for detecting instruments using deep learning How to filter out one instrument when multiple synthesizers are playing at once
 I actually changed my title of the talk to having fun with music and Keras because I think that's what it's all about. Sorry? Oh. So, what's my personal motivation for going to music and doing stuff with it? Well, I play guitar and when I was studying playing guitar 20 years ago, I was listening to a CD trying to figure out what notes were being played, push the rewind button, listening over again, try it, try it, keep repeating, repeating. It took so much effort. I was like, isn't there a computer program that actually can do this for me? Why is there no computer program that can do this for me? But apparently it's very difficult and right now with deep learning, we are actually getting close to tackling these kind of problems. And so that's my personal motivation for looking into it. So today, what are we going to do? First we are going to build a simple synthesizer using just NumPy and some Jupyter notebook enhancements. Then I will explain it just a little bit about music theory and come up with a very simple way of generating, well, it sounds musical. It's not the biggest new summer hit of 2018. Let's be fair about it. It's just some music. And then use this generated, we are going to generate data and we can use it for deep learning. So there's two benefits. First of all, we are in control of our own complexity. So we can make the sounds more distinct or more similar. But also we have also an unlimited data set. We can just keep on iterating and we know what the ground truth is. So we don't have the effort of labeling our data or that's all being taken care of. So that's why we take this approach and also it's way more fun this way because then we can build some synthesizers in NumPy. So what are we going to do on the deep learning part? So on synthesized music, I will single out a single instrument when multiple synthesizers are playing together. So just filtering it out. Then the second thing that I will be covering is detect when what notes are being played. First for a single instrument and then when instruments are playing together and when there are multiple notes being played by the same instrument. And then finally I will extend that model with multiple notes and I also have a, I couldn't resist doing one real world example where I've taken a backing track, played some guitar on it and tried to find, to have a detector when I'm playing or not. So let's first make some with this, this is readable. I'll show you to make it a bit bigger. So first let's have a look at what is music actually. So let's, I've loaded here some data and if you look at the data it just consists of raw, the raw data is just integers. What do these integers represent? Integers actually represent how far the speaker is going to move left or right. So since the integers have a specific resolution, so right now they're 16 bit integers and I'm going to scale them between minus one and one being the all the way to one end and all the way to the other end for the speaker. Well if you look at the shape it's just one big array, one dimensional array because it's mono if it was stereo you have like multiple channels over here. Then we also get another quantity back and that's called the sample rate and that basically tells you how many samples are there per second. So every say there's this like normal CD quality music where 44 kilohertz and that means you have 44,000 samples per second. So we have this, so if we would have 44,000 of these numbers we would have one second of music. So we can plot them. It looks like this. This is for a whole piece of music on the X-axis you have the time and seconds. So there's an intro over here, some bunch of music over here and then finally some intermediate part in the outro. Now let's just zoom in a bit and there's this very nice iPython display widget where you can just put in any array and just listen to it. It automatically also scales it to minus one one so you don't necessarily have to do it. And then let's hope this all works. So this is now what you find here below. That's it. If we're going to zoom in a bit I'm just going to explain you how difficult music is. And we find something like this. That's all encoded in this part of music. If we zoom in slightly more then this you should reconstruct from this mess of stuff over there. That's really a difficult signal basically. If you look at, so now we are going to make a simple synthesizer. We're going to make some own music. So let's start with sign generator. Sign is just the simplest periodic signal that we can think of. So we start doing it in the data frame. We have some time. We sample it with the sample rate. And then we're going to compute the sign of it every time step. Let's do it. Then we get this kind of big blob. Then if we zoom in we see actually the sign back. And there's also a nice thing. There's a method called free-y. And actually what free-y does is it takes this signal. This signal and it will determine what frequencies are present in the signal. So in this case I've made a sign with 440 hertz. You can actually see if we make a spectral representation for it we see actually one big line of this 430 hertz. We can also listen to it. So let's do it. Not the most pleasant sound. But that's a sign. It's not very musical at this point. So let's make a, let's put an envelope on it. So let's have an exponential decay of this sine wave. How does that sound? Already slightly better. Then let's add some multiple signs together. Last thing about tones is I can, if I have a sign of some frequency I can add another sign of twice the frequency or three times the frequency. And it will still sound as the original sign that I had. The same tone will still be perceived. But it sounds different. It has a different sort of timbre or different feel. It's more bright or, but it doesn't change the perception of the pitch. So let's try that out. So I've now again added, so here I'm just adding a couple of signs together. Like 440 hertz, two times, three times, four times with some different random amplitudes that I've put in. And we're going to, well this is down the thing that we generate. In spectral we see actually the lines coming back. And then how does it sound? So it's still, and if you look at the, so this is still the same pitch, right? So if we go back, the same pitch, right? Although we have added a lot of extra frequencies to it. So what we also can do, I've added over here, what we also can do is we can also just give the signs random phases. So what I've done over here is here I've added the signs together with all phases zero. And then I did some random phases. So the shape is completely different, right? Will they still sound the same, do you think? Sorry? Sort of. Sort of, well, it's a bit late this thing. It's one. Do you hear a difference? It's exactly the same. So your ear is completely, cannot hear the difference between this part and this part. So already when you're doing deep learning things and you're looking at RMSEs and stuff like that, you have to take into account that the RMSE might not be a very good quantity for comparing signals with each other. Well, let's add some decay on the additive sound. Let's do that. We have something that could, it's a very simple electrical piano or maybe a harp or it's not very beautiful, but I can play something with it. Now let's have just as a sidetrack, let's do some drums. So here I'm taking, I'm slowly going to, doing one over the inverse of the, so the frequencies gets much lower as we go on. And also I'm going to add some exponential decay. Let's listen to that. This one. I have a drum kick. Then for the snare, I'm going to build a snare from a drum kick plus a bit of noise. So noise, I'm sorry, but we're going to listen to noise. That's noise. And if I add some, so the kick and a very short bit of the noise together, we have something that's like a snare drum. So you can do a lot of cool things with just simple things, combining simple things. There's another thing that's called subtractive synthesis. It actually starts with building a very rich waveform, like a square wave has a lot of frequencies in it. It has a, so it's very frequency rich as, and then you actually cut out some frequencies in order to make it, again, nice sounding. So this is like a Mario computer game. That's without filtering and then with the filtering, it sounds like this. So there's a lot of different ways of generating sound. There's also FM synthesis, which is different way. You can also try to model a complete piano with all the resonances and stuff like that. It goes on and on, but this is just a very simple way of building it. Okay, let's start, make some music. So who knows a bit about music? I'm going to explain it anyway. Okay, that's more, luckily, quite a lot. So everyone knows this picture, right, of a piano. Actually, there's, the nice thing about the white notes, the white notes are all in the same scale. But let's, I will talk about it a little bit later. So if you look at the notes, every note in modern western tuning is very easily calculated. So you have just the 12th root of two to the power of N, where N is the number of half steps away from some root note. And a half step is like going from this to that, to this to that, that's the half step. So if you want to know the frequency of this note, I'll just count back how many I'm away from A4, say that's this note, now one, two, three, four, add that into this formula, and I can compute all the frequencies of the piano that are there. There are different ways of tuning, but this is the simplest one. So, well, I've created here now a small instrument from the things that we did in the previous notebook, and I can just generate now a note. That's a note. But now let's have a look at scale. So the scale is basically groups of notes that fit well together. So Doremi Fasola Tito, everyone knows it, right? Well, you can easily... So that's Doremi Fasola Tito, and then we can also make for every note that is in a scale, you can also make a chord out of it. So you can also do that mathematically. So now we have this kind of rules that govern together, which note sounds musical together. But we can just make some random walk in this scale, and then mix it together and try to find out something that fits well together. But first I will put a chord progression below it, just to listen to... I should first say this. I took this chord progression because it's the most common chord progression in pop music, and I just decided to go for that one again. So you probably all know a song that fits with this, I suppose. So now the notes are just... I'm going to generate some notes on top of this, and this is just going to be a random walk. So, of course, I don't want my notes to sound too high, and I don't want to sound them too low, so I'm going to have a bounded random walk. So whenever I'm going to draw a new note that is above the limit, I'm going to bounce back and go back again. So that's the way I do it, and I'm also going to... So there's a lot of... Some probabilities of making the random walk. So I will never stay on the same note, and there's quite a big chance of going to a note that is close to the other note, and there's a smaller note, the smaller chance to making a big step, and so on. I just tuned this a little bit by hand until I thought, okay, this is sort of acceptable. Well, if you take the random walk, and if I just index it with the scale that I put in, I just get a lot of... Well, notes. And then I can just generate those notes and mix it together, and then we get something like this. This can go on like infinite time. Sounds like better than the summer hit. Sounds even better. So, well, let's go... So now that we have this infinite data set, I'm going to the deep learning applications. And the first thing I will do is to filter out the lead instrument. So in the previous sound fragment, I had the chords and the melody. Now I'm going to learn the algorithm to get this melody and regenerate only the melody again, the chords plus melody, and go to only the melody. So the model is input wave with three instruments, output wave with just one instrument. And actually, one way of doing it is to do it with Outrancoder-like. Who knows what an Outrancoder is? Okay, I will just go quickly over it. An Outrancoder is a neural network which goes from an input, like for instance, it took in this case a picture, and the picture will go through a set of layers. Then it will go typically to a layer that has less dimensions than the original layer, and then it will scale back up to the original input size. So you get this picture in, and it has to, so you train it by giving this picture in and the same picture as a result that it has to produce again. So then it will learn ways to actually come up with a way of compressing it, such that it makes it also easy to generate it again, back again. So this is the Outrancoder. One of the things with Outrancoder is like an architecture that has the same input nodes as output nodes, and you train it with the same input as the output. We can actually do something similar, but instead of using the original and the input, the same as the output, we can say, well, I will take the input, the complete wave with combined instruments, and as an output, I will just only give it the melody. So this is the input. And this is the output. And this is how we build it. So we start with a model, like a Keras model. Then I'm going to add a dance layer, which is, I'm going to, I should have mentioned that before. So the music, I cannot, music is like a continuous signal, right? It goes on, it goes on, it goes on. So we have to cut it up in some way. So what I did is I cut the music up into small fragments of 1,000 samples, 1,024 actually, and then I would, on this fragment, I would just filter out the lead instrument. So that's what I'm doing, putting in here. So there's 1,024 samples going in. Then I have a prelude layer, just try out different things, but this worked best. Then we have another dance layer, so this one is smaller, I took the half of it, and then we're going to scale up again to the original output shape. And then we're going to put it on the target and the outputs. And how well, how does it sound before? It's like this. Should maybe have a bit shorter fragments here. I see people sitting like this. But not so much. But network still has to learn a lot. I must say that actually in generating this part of the music, I'm still putting in the original waveform but then chopped up in batches of 1,024. Sorry, question. How do you deal with when the phase is not there at your ear, please? So actually what I'm doing is I'm taking the, I'm doing the batched ways, batched way, and then I'm shifting also half by half. And then I'm counting, adding them together with like cross fading. So that's, indeed that solves it. So I also will fit, well, this is why the original title. So actually you can fit a model into epochs. That's it. That's how it looked like. And let's see how it sounds. And this is. Can even, that's it. Is it overfitted? It would probably, if you just have this data set, it's very likely to be overfitted, right? So what I did here is I generated the data set again. So just a totally new data set. I also put in a different, a different type of scale here. I trained it on a minor scale. And this is a, this major, no it's a Frigian harmonic, I believe. Frigian dominant. So it's a different scale. And let's listen to it. Still a bit of noise left in there. It's sort of, that still needs to, if you train it for longer than two epochs you get more, you get better results. And also I can also make the epochs really, really long, of course. Infinite amounts of data. Let's go to another part. And that is the node detection. So right now we're going to detect what nodes are being played. So here the model is going to be wave in sequence of detected nodes out. So the method will here be, so generate some random music, drop the wave data again in small batches of one tenth of a second, then use Fourier. Fourier is the way of getting to the frequencies that we saw earlier. And then for each of these batches, we're going to predict what nodes being played. And we're going to write this as a multi-class classification problem. I will show you in more detail later. And then each class is going to be one node that's being played. And I will train it as a gait and recurrent unit as a classifier. So for generating and rendering the data, so what I get is a data frame where I take the original data that I did random walk. From the random walk, I will basically make a data frame that looks like this. I have an offset, an end, and between those two, I have one-hut encoded. So basically, well, it's not necessarily one-hut, but I've listed what nodes are being played in this sequence. Then finally, I'm going to match my batches to one of this offset's ends. And then I know, OK, this node's being played or not. So if you look at the Fourier analysis of a waveform, it looks like this. So actually, you can already see here the individual nodes. You can see it back for the one instrument. You also see, I don't know if it's very visible, but there's like the same pattern. It's also repeated above here. And it's like at three times the frequency. And that has to do with the extra nodes that I showed at the beginning for adding the additional signs to the original nodes. So setting up the group. So who has ever used a gated recurrent unit before? Who knows a bit about recurrent neural nets? So I'll just quickly go over maybe some short introduction to it. So I will only keep it at recurrent neural nets. So what is recurrent neural nets? So starting with some input x, then that will go to a hidden state h. And that will produce an output o. So what you can see is like if you have a sequence of x's, then you will basically update every time the hidden state here. So this one will update it over here. This one will update it over there. And then this will is a very good way of accumulating evidence for a specific feature. You don't, and you can also do this on randomly sized sequences and stuff like that. Yeah, that's what I want to say about it. So I am, since I can, I know that I've generated data with only 14 different nodes, I'm using a 14 dimensional state. So that's also the same number of nodes that can be played. So I have put the number of time steps to one. And I put the RNN stateful, which means that it will keep on remembering what it has seen before. And then finally, I will apply the Fourier before going into it. And I'll show you the dimension, the modeling building over here. So this is the number of channels that I, there's for a great, great recurrent unit, you basically build it up with a number of steps per batch. So you basically chop up your original signal into batches. Then in that batch, you have a number of time steps per batch. And then you have a number of channels, which for typically raw WAV files, it's only one. And since we are now giving it a Fourier input, it will be actually a lot of frequencies per time step in this case. So we have a slightly more complicated input over here. Then we have the gate to current units. We put at the end a dense layer just for compensating, being able to fit it a bit easier. Then we have input over here. At 32 is a batch size, just one step per, so one Fourier spectrum per time step. So 2049 Fourier dimensions. And then our gate to current units. And finally, we have our network predicting which one of these 14 nodes are being played. And we use the categorical cross entropy for fitting it. So this is fitting it, typically Kira's output. And then if we look at the prediction, so if we, this on the right is what we put in. As a function of time over here, we see what node is being played as a yellow blob, as a punch card basically. And this is what it predicts now. So this is only after just a little bit of fitting. Okay, so this is actually, was the easy one. If we, by the way, if we test it, we see that it's also still working. I must say that if you train this a lot longer, it starts to look a lot better. And actually, I will do that for this one. So we're going to make it a bit harder now. Instead of looking at a single instrument, we're now going to look at the combined, the whole sequence of the music that I displayed earlier. So we're going to add a chords and bass instrument. So this is how it sounds. And we're also going to add harmony. So the lead, the melody instrument is going to play two notes at the same time. So that's the music. This is how the free spectrum now looks like. So this is also what I'm going to offer to my network to fit on. And you can see over here, it's messy, right? It's really messy. It's hard for me to find out what's being played. So if I look a little bit higher, I can still find the dots there. So the information is still in a way it's still there. But it's really hard to find, for instance, to come up with business rules that capture this problem very well. So this case, I'm still using the same network topology. I think I've did. First look here, split, did it half, the time frequency is twice as big. And then let's fit it, get some coffee, and it looks like this. And so this is after a lot of iterations. It's actually, it finds it, as you can see, it finds it back quite well. Every now and then, they're still like very close to the notes. It's not completely confident. You see over a year sometimes, so when it's close to the part where it's really difficult. So if you remember this part, right, that corresponds to this part. So it really has done quite well at this part. Well, okay, if we get a bit of coffee and it looks already this good, what will happen if we train this for one night? It actually does it almost perfectly, I would say. So that's that. Now the final thing is, let's do it on real music now. So and the cases, can we detect when an instrument is playing? So we're going to load a bit of data. And this is, I played some music over a backing track. And there's a full mix. And there's, so because I played over it, I can also just mix down only the guitar. And I have the two separately. Now what I'm going to do, first of all, we have a big issue of detecting when is the guitar playing solo. So here, this is only the guitar part. And first of all, I cannot say if the guitar is below zero, I'm not playing. Because it means the guitar goes, the waveform goes through zero all the time. So I cannot just say below some threshold, it's not working. So instead of that, I'm calculating the rolling RMC. And when the rolling RMC is dropping below some level, I'm going to gate it. So I'm going to stop saying the guitar is still playing. So if you just, so if you look in this graph, at the time, the guitar is in blue. And you see the original, so the mixed waveform in gray. And you see the rolling RMC in the orange. And whenever I'm, the green line is just one or zero, whether I'm playing or not playing. So I can basically mask the signal with the zeros and the ones. So I'm going to multiply the signal with zero when I'm not playing or with one if I'm playing. And then we can actually hear the effect of the gate. So that detects when it's not playing or not playing. Sometimes it's a little bit of a jitter, but that happens when the, yeah, it just happens. Now we're going to build a model. And we're again going to chop up in fragments of 0.1 seconds. And that's going to be time resolution. So I can either know I'm going to be playing the instruments or not within this time resolution. And for each fragment, we are going to detect whether the guitar is playing or not. So we have, yeah, so we have a short fragment, which is 0.1 seconds. Then we have the model and we are going to predict playing or not playing a Boolean. So this is how it looks like. So I'm taking an input over here with the fragment length, which is 0.1 seconds. And I'm actually one very nice thing about TensorFlow and Keras is that you can still apply some specific functions within Keras. So instead of first doing Fourier by hand before, I can also say, well, have applied a Fourier somewhere in the middle of my network and calculate even the gradients of the dead operation. So you have a look at what kind of transformations are still in TensorFlow or that you can actually just do some nice things inside your model instead of in the pre-processing phase. So here I'm actually doing the Fourier inside the model with a lambda. And this is the output shape will be then the fragment, will still be the fragment length. And finally, I'm going to put on that dense layer that's just going to have a sigmoid activation. So it's either a 0 or a 1. And I'm going to predict whether I'm playing or not. So the model then, you see, this is the time domain, the wave function. This is the frequency domain, like the stripes. And I'm finally, yes or no. So actually, the nice thing about this, we can actually listen to it and watch at the same time. So I made, does anyone know, movie pie? It's definitely a package that you want to check out because there's a very nice function where you can just make a frame given some time that you, some value, some x. And then you can render it to a movie very easily. So I use this for these movies. What we're going to see is we have the waveform and we are going to have a sliding window going across the waveform. And as the waveform progresses, I'm going to light up blue whenever I detected the note I was playing. Then we have also the green line with the gating, whether I'm playing or not, as well as the prediction in orange. Okay, I'll open it differently. Um. Take my hand, von Neufsak. Hard to sew.era Joony Strong So there's a slight delay between the Bluetooth speaker and the......the......you can also see that music, not totally silent. But still we have......it's lining up great. And then I'll just... because otherwise you'll hear my... me fucking up. That's it. Okay, I think we've got a few minutes for questions. Thanks for your talk. I'm really curious how fast can a network figure this stuff out. Like, would it be applicable for music visualization? Because with electronic music it's fairly easy to do stuff. BTT action, et cetera, but classic music or acoustic is kind of hard. So would this be useful for that case? Well, with music, people have a very good internal clock for when it comes to......they have a very good internal time resolution when it comes to music perception. So you need to be really, really, really......you have to have a very short latency. So for instance, already the rendering, you need to be able to do it within......like order of 16th milliseconds. So I don't know if a lot of people probably did see a little bit of a desync between the USB. Well, already there, humans are very perceptible to small latencies. So I think it, in principle, would be possible, but you......it's not going to be easy. Well, for what we see right now, I don't know actually. But I know that latencies of 20 milliseconds can become noticeable already. Especially when you're making the music. So if you're playing an instrument, and you have a latency of 20 milliseconds, you'll already start noticing it. But that's when you're playing it. If you're just listening to it and seeing it. Of course, with electronic music, you can already pre-process it probably, right? I think in pre-renders, I suppose. But for real live music, where there's people playing live on stage with no backing track or whatever, then it's going to be hard, I suppose. Okay, any other questions? So how do you deal with more complex music like jazz, key changes? Well, key changes is... I'm going to repeat that for the people in the audience. So I deal with more complex music like jazz, where there's key changes. In principle, so now I've really tuned my model to the complexity that I have right now. So I've only...my hidden state has only 14 neurons, which is exactly the number of notes that I can draw from the random walk. In principle, you can still enlarge it to the complete spectrum of all different possible notes on a piano, for instance. But then, okay, jazz music has still a lot of... The note gets banded up, it goes higher slowly, so it's in between or vibrato, vibrato, stuff like that. So there's a lot of, I would say, additional things that color the music that you still need to be able to write down, maybe, or not. Or maybe can even harm you. For instance, vibrato could maybe be transcribed as higher, lower, higher, lower, and stuff like that. So it's going to be difficult. But I suppose that...actually, the thing that I tried here right now is actually to make it simple and then build it up. So I'm actually not intending to stop it at this point and slowly maybe get some jazz going on. I think we have time for one more question. I'm going to publish these notebooks. So they are published. So if you look at my GitHub and my name, Marcel Raas, you'll find it. There's also...the whole synthesizer thing is also there. It could be cleaned up a bit. Okay, are there any other questions? If not, I'd like to thank Marcel. Thank you very much.
 I actually changed my title of the talk to having fun with music and Keras because I  think that's what it's all about.  Sorry?  Oh.  So, what's my personal motivation for going to music and doing stuff with it?  Well, I play guitar and when I was studying playing guitar 20 years ago, I was listening  to a CD trying to figure out what notes were being played, push the rewind button, listening  over again, try it, try it, keep repeating, repeating.  It took so much effort.  I was like, isn't there a computer program that actually can do this for me?  Why is there no computer program that can do this for me?  But apparently it's very difficult and right now with deep learning, we are actually getting  close to tackling these kind of problems.  And so that's my personal motivation for looking into it.  So today, what are we going to do?  First we are going to build a simple synthesizer using just NumPy and some Jupyter notebook  enhancements.  Then I will explain it just a little bit about music theory and come up with a very simple  way of generating, well, it sounds musical.  It's not the biggest new summer hit of 2018.  Let's be fair about it.  It's just some music.  And then use this generated, we are going to generate data and we can use it for deep  learning.  So there's two benefits.  First of all, we are in control of our own complexity.  So we can make the sounds more distinct or more similar.  But also we have also an unlimited data set.  We can just keep on iterating and we know what the ground truth is.  So we don't have the effort of labeling our data or that's all being taken care of.  So that's why we take this approach and also it's way more fun this way because then we  can build some synthesizers in NumPy.  So what are we going to do on the deep learning part?  So on synthesized music, I will single out a single instrument when multiple synthesizers  are playing together.  So just filtering it out.  Then the second thing that I will be covering is detect when what notes are being played.  First for a single instrument and then when instruments are playing together and when  there are multiple notes being played by the same instrument.  And then finally I will extend that model with multiple notes and I also have a, I couldn't  resist doing one real world example where I've taken a backing track, played some guitar  on it and tried to find, to have a detector when I'm playing or not.  So let's first make some with this, this is readable.  I'll show you to make it a bit bigger.  So first let's have a look at what is music actually.  So let's, I've loaded here some data and if you look at the data it just consists of raw,  the raw data is just integers.  What do these integers represent?  Integers actually represent how far the speaker is going to move left or right.  So since the integers have a specific resolution, so right now they're 16 bit integers and I'm  going to scale them between minus one and one being the all the way to one end and all  the way to the other end for the speaker.  Well if you look at the shape it's just one big array, one dimensional array because it's  mono if it was stereo you have like multiple channels over here.  Then we also get another quantity back and that's called the sample rate and that basically  tells you how many samples are there per second.  So every say there's this like normal CD quality music where 44 kilohertz and that means you  have 44,000 samples per second.  So we have this, so if we would have 44,000 of these numbers we would have one second  of music.  So we can plot them.  It looks like this.  This is for a whole piece of music on the X-axis you have the time and seconds.  So there's an intro over here, some bunch of music over here and then finally some intermediate  part in the outro.  Now let's just zoom in a bit and there's this very nice iPython display widget where  you can just put in any array and just listen to it.  It automatically also scales it to minus one one so you don't necessarily have to do it.  And then let's hope this all works.  So this is now what you find here below.  That's it.  If we're going to zoom in a bit I'm just going to explain you how difficult music is.  And we find something like this.  That's all encoded in this part of music.  If we zoom in slightly more then this you should reconstruct from this mess of stuff  over there.  That's really a difficult signal basically.  If you look at, so now we are going to make a simple synthesizer.  We're going to make some own music.  So let's start with sign generator.  Sign is just the simplest periodic signal that we can think of.  So we start doing it in the data frame.  We have some time.  We sample it with the sample rate.  And then we're going to compute the sign of it every time step.  Let's do it.  Then we get this kind of big blob.  Then if we zoom in we see actually the sign back.  And there's also a nice thing.  There's a method called free-y.  And actually what free-y does is it takes this signal.  This signal and it will determine what frequencies are present in the signal.  So in this case I've made a sign with 440 hertz.  You can actually see if we make a spectral representation for it we see actually one  big line of this 430 hertz.  We can also listen to it.  So let's do it.  Not the most pleasant sound.  But that's a sign.  It's not very musical at this point.  So let's make a, let's put an envelope on it.  So let's have an exponential decay of this sine wave.  How does that sound?  Already slightly better.  Then let's add some multiple signs together.  Last thing about tones is I can, if I have a sign of some frequency I can add another  sign of twice the frequency or three times the frequency.  And it will still sound as the original sign that I had.  The same tone will still be perceived.  But it sounds different.  It has a different sort of timbre or different feel.  It's more bright or, but it doesn't change the perception of the pitch.  So let's try that out.  So I've now again added, so here I'm just adding a couple of signs together.  Like 440 hertz, two times, three times, four times with some different random amplitudes  that I've put in.  And we're going to, well this is down the thing that we generate.  In spectral we see actually the lines coming back.  And then how does it sound?  So it's still, and if you look at the, so this is still the same pitch, right?  So if we go back, the same pitch, right?  Although we have added a lot of extra frequencies to it.  So what we also can do, I've added over here, what we also can do is we can also just give  the signs random phases.  So what I've done over here is here I've added the signs together with all phases  zero.  And then I did some random phases.  So the shape is completely different, right?  Will they still sound the same, do you think?  Sorry?  Sort of.  Sort of, well, it's a bit late this thing.  It's one.  Do you hear a difference?  It's exactly the same.  So your ear is completely, cannot hear the difference between this part and this part.  So already when you're doing deep learning things and you're looking at RMSEs and stuff  like that, you have to take into account that the RMSE might not be a very good quantity  for comparing signals with each other.  Well, let's add some decay on the additive sound.  Let's do that.  We have something that could, it's a very simple electrical piano or maybe a harp or  it's not very beautiful, but I can play something with it.  Now let's have just as a sidetrack, let's do some drums.  So here I'm taking, I'm slowly going to, doing one over the inverse of the, so the  frequencies gets much lower as we go on.  And also I'm going to add some exponential decay.  Let's listen to that.  This one.  I have a drum kick.  Then for the snare, I'm going to build a snare from a drum kick plus a bit of noise.  So noise, I'm sorry, but we're going to listen to noise.  That's noise.  And if I add some, so the kick and a very short bit of the noise together, we have something  that's like a snare drum.  So you can do a lot of cool things with just simple things, combining simple things.  There's another thing that's called subtractive synthesis.  It actually starts with building a very rich waveform, like a square wave has a lot of frequencies  in it.  It has a, so it's very frequency rich as, and then you actually cut out some frequencies  in order to make it, again, nice sounding.  So this is like a Mario computer game.  That's without filtering and then with the filtering, it sounds like this.  So there's a lot of different ways of generating sound.  There's also FM synthesis, which is different way.  You can also try to model a complete piano with all the resonances and stuff like that.  It goes on and on, but this is just a very simple way of building it.  Okay, let's start, make some music.  So who knows a bit about music?  I'm going to explain it anyway.  Okay, that's more, luckily, quite a lot.  So everyone knows this picture, right, of a piano.  Actually, there's, the nice thing about the white notes, the white notes are all in the  same scale.  But let's, I will talk about it a little bit later.  So if you look at the notes, every note in modern western tuning is very easily calculated.  So you have just the 12th root of two to the power of N, where N is the number of half  steps away from some root note.  And a half step is like going from this to that, to this to that, that's the half step.  So if you want to know the frequency of this note, I'll just count back how many I'm away  from A4, say that's this note, now one, two, three, four, add that into this formula, and  I can compute all the frequencies of the piano that are there.  There are different ways of tuning, but this is the simplest one.  So, well, I've created here now a small instrument from the things that we did in the previous  notebook, and I can just generate now a note.  That's a note.  But now let's have a look at scale.  So the scale is basically groups of notes that fit well together.  So Doremi Fasola Tito, everyone knows it, right?  Well, you can easily...  So that's Doremi Fasola Tito, and then we can also make for every note that is in a  scale, you can also make a chord out of it.  So you can also do that mathematically.  So now we have this kind of rules that govern together, which note sounds musical together.  But we can just make some random walk in this scale, and then mix it together and try to  find out something that fits well together.  But first I will put a chord progression below it, just to listen to...  I should first say this.  I took this chord progression because it's the most common chord progression in pop music,  and I just decided to go for that one again.  So you probably all know a song that fits with this, I suppose.  So now the notes are just...  I'm going to generate some notes on top of this, and this is just going to be a random  walk.  So, of course, I don't want my notes to sound too high, and I don't want to sound them too  low, so I'm going to have a bounded random walk.  So whenever I'm going to draw a new note that is above the limit, I'm going to bounce back  and go back again.  So that's the way I do it, and I'm also going to...  So there's a lot of...  Some probabilities of making the random walk.  So I will never stay on the same note, and there's quite a big chance of going to a note  that is close to the other note, and there's a smaller note, the smaller chance to making  a big step, and so on.  I just tuned this a little bit by hand until I thought, okay, this is sort of acceptable.  Well, if you take the random walk, and if I just index it with the scale that I put in,  I just get a lot of...  Well, notes.  And then I can just generate those notes and mix it together, and then we get something  like this.  This can go on like infinite time.  Sounds like better than the summer hit.  Sounds even better.  So, well, let's go...  So now that we have this infinite data set, I'm going to the deep learning applications.  And the first thing I will do is to filter out the lead instrument.  So in the previous sound fragment, I had the chords and the melody.  Now I'm going to learn the algorithm to get this melody and regenerate only the melody  again, the chords plus melody, and go to only the melody.  So the model is input wave with three instruments, output wave with just one instrument.  And actually, one way of doing it is to do it with Outrancoder-like.  Who knows what an Outrancoder is?  Okay, I will just go quickly over it.  An Outrancoder is a neural network which goes from an input, like for instance, it took  in this case a picture, and the picture will go through a set of layers.  Then it will go typically to a layer that has less dimensions than the original layer,  and then it will scale back up to the original input size.  So you get this picture in, and it has to, so you train it by giving this picture in  and the same picture as a result that it has to produce again.  So then it will learn ways to actually come up with a way of compressing it, such that  it makes it also easy to generate it again, back again.  So this is the Outrancoder.  One of the things with Outrancoder is like an architecture that has the same input nodes  as output nodes, and you train it with the same input as the output.  We can actually do something similar, but instead of using the original and the input,  the same as the output, we can say, well, I will take the input, the complete wave with  combined instruments, and as an output, I will just only give it the melody.  So this is the input.  And this is the output.  And this is how we build it.  So we start with a model, like a Keras model.  Then I'm going to add a dance layer, which is, I'm going to, I should have mentioned  that before.  So the music, I cannot, music is like a continuous signal, right?  It goes on, it goes on, it goes on.  So we have to cut it up in some way.  So what I did is I cut the music up into small fragments of 1,000 samples, 1,024 actually,  and then I would, on this fragment, I would just filter out the lead instrument.  So that's what I'm doing, putting in here.  So there's 1,024 samples going in.  Then I have a prelude layer, just try out different things, but this worked best.  Then we have another dance layer, so this one is smaller, I took the half of it, and  then we're going to scale up again to the original output shape.  And then we're going to put it on the target and the outputs.  And how well, how does it sound before?  It's like this.  Should maybe have a bit shorter fragments here.  I see people sitting like this.  But not so much.  But network still has to learn a lot.  I must say that actually in generating this part of the music, I'm still putting in the  original waveform but then chopped up in batches of 1,024.  Sorry, question.  How do you deal with when the phase is not there at your ear, please?  So actually what I'm doing is I'm taking the, I'm doing the batched ways, batched way, and  then I'm shifting also half by half.  And then I'm counting, adding them together with like cross fading.  So that's, indeed that solves it.  So I also will fit, well, this is why the original title.  So actually you can fit a model into epochs.  That's it.  That's how it looked like.  And let's see how it sounds.  And this is.  Can even, that's it.  Is it overfitted?  It would probably, if you just have this data set, it's very likely to be overfitted, right?  So what I did here is I generated the data set again.  So just a totally new data set.  I also put in a different, a different type of scale here.  I trained it on a minor scale.  And this is a, this major, no it's a Frigian harmonic, I believe.  Frigian dominant.  So it's a different scale.  And let's listen to it.  Still a bit of noise left in there.  It's sort of, that still needs to, if you train it for longer than two epochs you get  more, you get better results.  And also I can also make the epochs really, really long, of course.  Infinite amounts of data.  Let's go to another part.  And that is the node detection.  So right now we're going to detect what nodes are being played.  So here the model is going to be wave in sequence of detected nodes out.  So the method will here be, so generate some random music, drop the wave data again in  small batches of one tenth of a second, then use Fourier.  Fourier is the way of getting to the frequencies that we saw earlier.  And then for each of these batches, we're going to predict what nodes being played.  And we're going to write this as a multi-class classification problem.  I will show you in more detail later.  And then each class is going to be one node that's being played.  And I will train it as a gait and recurrent unit as a classifier.  So for generating and rendering the data, so what I get is a data frame where I take  the original data that I did random walk.  From the random walk, I will basically make a data frame that looks like this.  I have an offset, an end, and between those two, I have one-hut encoded.  So basically, well, it's not necessarily one-hut, but I've listed what nodes are being played  in this sequence.  Then finally, I'm going to match my batches to one of this offset's ends.  And then I know, OK, this node's being played or not.  So if you look at the Fourier analysis of a waveform, it looks like this.  So actually, you can already see here the individual nodes.  You can see it back for the one instrument.  You also see, I don't know if it's very visible, but there's like the same pattern.  It's also repeated above here.  And it's like at three times the frequency.  And that has to do with the extra nodes that I showed at the beginning for adding the additional  signs to the original nodes.  So setting up the group.  So who has ever used a gated recurrent unit before?  Who knows a bit about recurrent neural nets?  So I'll just quickly go over maybe some short introduction to it.  So I will only keep it at recurrent neural nets.  So what is recurrent neural nets?  So starting with some input x, then that will go to a hidden state h.  And that will produce an output o.  So what you can see is like if you have a sequence of x's, then you will basically update  every time the hidden state here.  So this one will update it over here.  This one will update it over there.  And then this will is a very good way of accumulating evidence for a specific feature.  You don't, and you can also do this on randomly sized sequences and stuff like that.  Yeah, that's what I want to say about it.  So I am, since I can, I know that I've generated data with only 14 different nodes, I'm using  a 14 dimensional state.  So that's also the same number of nodes that can be played.  So I have put the number of time steps to one.  And I put the RNN stateful, which means that it will keep on remembering what it has seen  before.  And then finally, I will apply the Fourier before going into it.  And I'll show you the dimension, the modeling building over here.  So this is the number of channels that I, there's for a great, great recurrent unit,  you basically build it up with a number of steps per batch.  So you basically chop up your original signal into batches.  Then in that batch, you have a number of time steps per batch.  And then you have a number of channels, which for typically raw WAV files, it's only one.  And since we are now giving it a Fourier input, it will be actually a lot of frequencies  per time step in this case.  So we have a slightly more complicated input over here.  Then we have the gate to current units.  We put at the end a dense layer just for compensating, being able to fit it a bit easier.  Then we have input over here.  At 32 is a batch size, just one step per, so one Fourier spectrum per time step.  So 2049 Fourier dimensions.  And then our gate to current units.  And finally, we have our network predicting which one of these 14 nodes are being played.  And we use the categorical cross entropy for fitting it.  So this is fitting it, typically Kira's output.  And then if we look at the prediction, so if we, this on the right is what we put in.  As a function of time over here, we see what node is being played as a yellow blob, as  a punch card basically.  And this is what it predicts now.  So this is only after just a little bit of fitting.  Okay, so this is actually, was the easy one.  If we, by the way, if we test it, we see that it's also still working.  I must say that if you train this a lot longer, it starts to look a lot better.  And actually, I will do that for this one.  So we're going to make it a bit harder now.  Instead of looking at a single instrument, we're now going to look at the combined, the  whole sequence of the music that I displayed earlier.  So we're going to add a chords and bass instrument.  So this is how it sounds.  And we're also going to add harmony.  So the lead, the melody instrument is going to play two notes at the same time.  So that's the music.  This is how the free spectrum now looks like.  So this is also what I'm going to offer to my network to fit on.  And you can see over here, it's messy, right?  It's really messy.  It's hard for me to find out what's being played.  So if I look a little bit higher, I can still find the dots there.  So the information is still in a way it's still there.  But it's really hard to find, for instance, to come up with business rules that capture  this problem very well.  So this case, I'm still using the same network topology.  I think I've did.  First look here, split, did it half, the time frequency is twice as big.  And then let's fit it, get some coffee, and it looks like this.  And so this is after a lot of iterations.  It's actually, it finds it, as you can see, it finds it back quite well.  Every now and then, they're still like very close to the notes.  It's not completely confident.  You see over a year sometimes, so when it's close to the part where it's really difficult.  So if you remember this part, right, that corresponds to this part.  So it really has done quite well at this part.  Well, okay, if we get a bit of coffee and it looks already this good, what will happen  if we train this for one night?  It actually does it almost perfectly, I would say.  So that's that.  Now the final thing is, let's do it on real music now.  So and the cases, can we detect when an instrument is playing?  So we're going to load a bit of data.  And this is, I played some music over a backing track.  And there's a full mix.  And there's, so because I played over it, I can also just mix down only the guitar.  And I have the two separately.  Now what I'm going to do, first of all, we have a big issue of detecting when is the  guitar playing solo.  So here, this is only the guitar part.  And first of all, I cannot say if the guitar is below zero, I'm not playing.  Because it means the guitar goes, the waveform goes through zero all the time.  So I cannot just say below some threshold, it's not working.  So instead of that, I'm calculating the rolling RMC.  And when the rolling RMC is dropping below some level, I'm going to gate it.  So I'm going to stop saying the guitar is still playing.  So if you just, so if you look in this graph, at the time, the guitar is in blue.  And you see the original, so the mixed waveform in gray.  And you see the rolling RMC in the orange.  And whenever I'm, the green line is just one or zero, whether I'm playing or not playing.  So I can basically mask the signal with the zeros and the ones.  So I'm going to multiply the signal with zero when I'm not playing or with one if I'm playing.  And then we can actually hear the effect of the gate.  So that detects when it's not playing or not playing.  Sometimes it's a little bit of a jitter, but that happens when the, yeah, it just happens.  Now we're going to build a model.  And we're again going to chop up in fragments of 0.1 seconds.  And that's going to be time resolution.  So I can either know I'm going to be playing the instruments or not within this time resolution.  And for each fragment, we are going to detect whether the guitar is playing or not.  So we have, yeah, so we have a short fragment, which is 0.1 seconds.  Then we have the model and we are going to predict playing or not playing a Boolean.  So this is how it looks like.  So I'm taking an input over here with the fragment length, which is 0.1 seconds.  And I'm actually one very nice thing about TensorFlow and Keras is that you can still  apply some specific functions within Keras.  So instead of first doing Fourier by hand before, I can also say, well, have applied  a Fourier somewhere in the middle of my network and calculate even the gradients of the dead  operation.  So you have a look at what kind of transformations are still in TensorFlow or that you can actually  just do some nice things inside your model instead of in the pre-processing phase.  So here I'm actually doing the Fourier inside the model with a lambda.  And this is the output shape will be then the fragment, will still be the fragment length.  And finally, I'm going to put on that dense layer that's just going to have a sigmoid  activation.  So it's either a 0 or a 1.  And I'm going to predict whether I'm playing or not.  So the model then, you see, this is the time domain, the wave function.  This is the frequency domain, like the stripes.  And I'm finally, yes or no.  So actually, the nice thing about this, we can actually listen to it and watch at the  same time.  So I made, does anyone know, movie pie?  It's definitely a package that you want to check out because there's a very nice function  where you can just make a frame given some time that you, some value, some x.  And then you can render it to a movie very easily.  So I use this for these movies.  What we're going to see is we have the waveform and we are going to have a sliding window  going across the waveform.  And as the waveform progresses, I'm going to light up blue whenever I detected the note  I was playing.  Then we have also the green line with the gating, whether I'm playing or not, as well  as the prediction in orange.  Okay, I'll open it differently.  Um.  Take my hand, von Neufsak.  Hard to sew. era Joony Strong  So there's a slight delay between the Bluetooth speaker and the... ...the... ...you can also see that music, not totally silent.  But still we have... ...it's lining up great.  And then I'll just... because otherwise you'll hear my... me fucking up.  That's it.  Okay, I think we've got a few minutes for questions.  Thanks for your talk.  I'm really curious how fast can a network figure this stuff out.  Like, would it be applicable for music visualization?  Because with electronic music it's fairly easy to do stuff.  BTT action, et cetera, but classic music or acoustic is kind of hard.  So would this be useful for that case?  Well, with music, people have a very good internal clock for when it comes to... ...they have a very good internal time resolution when it comes to music perception.  So you need to be really, really, really... ...you have to have a very short latency.  So for instance, already the rendering, you need to be able to do it within... ...like order of 16th milliseconds.  So I don't know if a lot of people probably did see a little bit of a desync between the USB.  Well, already there, humans are very perceptible to small latencies.  So I think it, in principle, would be possible, but you... ...it's not going to be easy.  Well, for what we see right now, I don't know actually.  But I know that latencies of 20 milliseconds can become noticeable already.  Especially when you're making the music.  So if you're playing an instrument, and you have a latency of 20 milliseconds, you'll already start noticing it.  But that's when you're playing it.  If you're just listening to it and seeing it.  Of course, with electronic music, you can already pre-process it probably, right?  I think in pre-renders, I suppose.  But for real live music, where there's people playing live on stage with no backing track or whatever,  then it's going to be hard, I suppose.  Okay, any other questions?  So how do you deal with more complex music like jazz, key changes?  Well, key changes is...  I'm going to repeat that for the people in the audience.  So I deal with more complex music like jazz, where there's key changes.  In principle, so now I've really tuned my model to the complexity that I have right now.  So I've only...my hidden state has only 14 neurons,  which is exactly the number of notes that I can draw from the random walk.  In principle, you can still enlarge it to the complete spectrum of all different possible notes on a piano, for instance.  But then, okay, jazz music has still a lot of...  The note gets banded up, it goes higher slowly,  so it's in between or vibrato, vibrato, stuff like that.  So there's a lot of, I would say, additional things that color the music that you still need to be able to write down, maybe, or not.  Or maybe can even harm you.  For instance, vibrato could maybe be transcribed as higher, lower, higher, lower, and stuff like that.  So it's going to be difficult.  But I suppose that...actually, the thing that I tried here right now is actually to make it simple and then build it up.  So I'm actually not intending to stop it at this point and slowly maybe get some jazz going on.  I think we have time for one more question.  I'm going to publish these notebooks.  So they are published.  So if you look at my GitHub and my name, Marcel Raas, you'll find it.  There's also...the whole synthesizer thing is also there.  It could be cleaned up a bit.  Okay, are there any other questions?  If not, I'd like to thank Marcel.  Thank you very much.
Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_0_143.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_144_6707.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_6708_7211.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_7212_7931.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_7932_8195.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_8196_9347.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_9348_9707.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_9708_9911.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_9912_10163.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_10164_10391.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_10392_10607.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_10608_10811.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_10812_11963.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_11964_12059.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_12060_12131.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_12132_12479.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_12480_12587.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_12588_12647.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_12648_12791.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_12792_14363.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_14364_14471.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_14472_14867.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_14868_15431.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_15432_15635.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_15636_15779.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_15780_15983.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_15984_16055.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_16056_16175.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_16176_16367.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_16368_16979.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_16980_17255.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_17256_17327.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_17328_17867.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_17868_17963.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_17964_18179.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_18180_18431.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_18432_37511.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_37512_38459.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_38460_38903.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_38904_39215.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_39216_40763.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_40764_43895.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_43896_44399.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_44400_44483.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_44484_44747.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_44748_47027.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_47028_47651.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_47652_47819.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_47820_47999.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_48000_48263.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_48264_56075.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_56076_56315.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_56316_56423.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_56424_57131.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_57132_65747.png Marcel_Raas_-_Change_music_in_two_epochs_epPFCdEgHxM_65748_65850.png