[User] Please transcribe this text

[Assistant] Very good, everyone! That's a little fast. You may have to practice a couple times, but that's okay. This has been ELF Learning. We'll see you next time. Goodbye!


[User] Please transcribe this text

[Assistant] If you want to swim again, you have to stop smoking now.


[User] Please transcribe this text

[Assistant] It's called a nicotine patch. You place it on your bare skin. You get the patch for free. For free? For free.


[User] Please transcribe this text

[Assistant] Mr. Basikoff? You can quit smoking. You have to.


[User] Please transcribe this text

[Assistant] I found the U-net architecture useful for many applications I've worked on and this is one of the most important and foundational neural network architectures of computer vision today. So in this video, let's dig into the details of exactly how the U-net works.


[User] Please transcribe this text

[Assistant] This is what a U-net looks like. I know there's a lot going on in this picture. It's okay. We'll break it down and step it through one step at a time. But I want to quickly show you the final outcome of what a U-net architecture looks like.


[User] Please transcribe this text

[Assistant] And this, by the way, is also why it's called a U-net, because when you draw it like this, it looks a lot like a U.


[User] Please transcribe this text

[Assistant] Let's break it down and build this back up one piece at a time so we can make sure we know what


[User] Please transcribe this text

[Assistant] All of the things going on in this diagram actually are doing.


[User] Please transcribe this text

[Assistant] These ideas were due to Olaf Ronneberger, Philip Fisher, and Thomas Bronx.


[User] Please transcribe this text

[Assistant] Fun fact, when they wrote their original UNet paper, they were thinking of the application of biomedical image segmentation, really segmenting medical images. But these ideas turned out to be useful for many other computer vision semantic segmentation applications as well.


[User] Please transcribe this text

[Assistant] So the input to the unit is an image, let's say as h by w.


[User] Please transcribe this text

[Assistant] By three, for three channels, uh, RGB channels.


[User] Please transcribe this text

[Assistant] I'm going to visualize this image as a thin layer like that.


[User] Please transcribe this text

[Assistant] I know that previously we had taken neural network layers and drawn them, you know, as 3D blocks like this, where this might be, right, sort of H by W.


[User] Please transcribe this text

[Assistant] By three, but in order to make the unit diagram look simpler, I'm going to imagine looking at this edge on. So all you see is this.


[User] Please transcribe this text

[Assistant] And uh, the rest of, you know, this is hidden behind this.


[User] Please transcribe this text

[Assistant] Dark blue rectangle that I've drawn and so that's what this is. Uh, the height of this is, you know, the height h and the width of this is


[User] Please transcribe this text

[Assistant] The number of channels, the depth is 3, so it looks very thin.


[User] Please transcribe this text

[Assistant] Now the first part of the unit uses normal feed-forward neural network convolutional layers. So I'm going to use a black arrow to denote a convolutional layer followed by a ReLU activation function. So the next layer,


[User] Please transcribe this text

[Assistant] Maybe they'll say that, uh, we've increased the number of channels a little bit, but the dimension is still high by width by a little bit more channels.


[User] Please transcribe this text

[Assistant] And then another convolutional layer with a regular activation function.


[User] Please transcribe this text

[Assistant] Now we're still in the first half of the neural network. We're going to use max pooling to reduce the height and width. So maybe you end up with a set of activations where the height and width is lower, but maybe a sticker. So the number of channels is increasing.


[User] Please transcribe this text

[Assistant] Then we have two more layers of normal feed forward convolutions.


[User] Please transcribe this text

[Assistant] With a ReLU activation function, and then let's apply max pool linear gain, and so you get that.


[User] Please transcribe this text

[Assistant] Right, and then, you know, repeat again, and so you end up with this.


[User] Please transcribe this text

[Assistant] So so far, this is the normal convolution layers with activation functions that you've been used to from earlier videos with occasional max pooling layers. So notice that the height


[User] Please transcribe this text

[Assistant] Of this layer, height and width is now very small.


[User] Please transcribe this text

[Assistant] So we're going to start to apply transpose convolution layers, which I'm going to denote by the green arrow in order to build the dimension of this neural network backup.


[User] Please transcribe this text

[Assistant] So with the first, uh, transpose convolutional layer, or transconv layer,


[User] Please transcribe this text

[Assistant] You are going to get a set of activations that looks like that. In this example, we did not increase the height and width, but we did decrease the number of channels.


[User] Please transcribe this text

[Assistant] But there's one more thing, uh, you need to do to build a unit, which is to add in that skip connection, which I'm going to denote with this gray arrow. What the skip connection does is it takes


[User] Please transcribe this text

[Assistant] This set of activations, and just copies it over to the right, and so the set of activations you end up with is like this. The light blue part comes from the transpose convolution, and the dark blue part is just copied over from the left.


[User] Please transcribe this text

[Assistant] To keep on building up the unit, we are going to then


[User] Please transcribe this text

[Assistant] Apply a couple more layers of the regular convolutions, followed by a value activation function, so denoted by the black arrows like so.


[User] Please transcribe this text

[Assistant] And then we'll apply another transpose convolution layer. So green arrow. And here we're going to start to increase the dimension, increase the height and width of this image. And so now the height is getting bigger.


[User] Please transcribe this text

[Assistant] But here too, we're going to apply a skip connection. So there's a gray arrow again. We're going to take this set of activations and just copy it right there over to the right.


[User] Please transcribe this text

[Assistant] More convolutional layers, another transposed convolution, skip connection. Once again, we're going to take this set of activations and copy it over to the right. And then more convolutional layers followed by another transposed convolution, skip connection, copy that over. And now we're back.


[User] Please transcribe this text

[Assistant] To a set of activations that is the original input image's height and width.


[User] Please transcribe this text

[Assistant] We're going to have a couple more layers of a normal feedforward convolutions.


[User] Please transcribe this text

[Assistant] And then finally to take this and map this to our segmentation map, we're going to use a one by one convolution, which I'm going to denote with that magenta arrow to finally give us this, which is going to be our output.


[User] Please transcribe this text

[Assistant] The dimensions of this output layer is going to be h.


[User] Please transcribe this text

[Assistant] By w, so the same dimensions as our original input, by num classes.


[User] Please transcribe this text

[Assistant] So if you have three classes you're trying to recognize, this will be three. If you have ten different classes you're trying to recognize in your segmentation map, then that last number will be ten.


[User] Please transcribe this text

[Assistant] And so what this does is for every one of your pixels, you have h by w pixels.


[User] Please transcribe this text

[Assistant] You have, you know, an array or a vector essentially of n classes numbers that tells you for one pixel how likely is that pixel to come from each of these different classes. And if you take a argmax over these n classes, then that's how you classify


[User] Please transcribe this text

[Assistant] Each of the pixels into one of the clauses and you can visualize it like the segmentation map shown on the right.


[User] Please transcribe this text

[Assistant] So that's it. You've learned about the transpose convolution and the unit architecture. Congrats on getting to the end of this week's videos. I hope you also enjoy working through these ideas in the program exercise.


[User] Please transcribe this text

[Assistant] Next week we'll come back and talk about some specialized architectures for face recognition and for neural style transfer where you get to create some really interesting artwork using neural networks. Have fun with the program exercise and I look forward to seeing you next week.


[User] Please transcribe this text

[Assistant] This is Uncle Sasha. I have good news and bad news.


[User] Please transcribe this text

[Assistant] First, the bad news. My temperature is 102. I have a fever.


[User] Please transcribe this text

[Assistant] No, no, no. Don't worry. I will be fine in a few days.


[User] Please transcribe this text

[Assistant] Do you have your list of English sentences? Take it out and practice.


[User] Please transcribe this text

[Assistant] Mommy, don't forget, you have a doctor's appointment today. Of course, I'm very busy today. Mommy, this is important.


[User] Please transcribe this text

[Assistant] Eddy, don't you have something to do?


[User] Please transcribe this text

[Assistant] Little by little, one day at a time.


[User] Please transcribe this text

[Assistant] A notebook. I take notes when the person is talking.


[User] Please transcribe this text

[Assistant] Jean-Claude didn't tell me. You won the poetry contest.


[User] Please transcribe this text

[Assistant] He takes after his father.


[User] Please transcribe this text

[Assistant] People never lie so much as before an election, during a war, or after a hunt.


[User] Please transcribe this text

[Assistant] Because of their scruples and the weak grow strong because of their audacity.


[User] Please transcribe this text

[Assistant] Man cannot control the current of events. He can only float with them and steer.


[User] Please transcribe this text

[Assistant] When you want to fool the world, tell the truth.


[User] Please transcribe this text

[Assistant] Be polite. Write diplomatically. Even in a declaration of war, one observes the rules of politeness.


[User] Please transcribe this text

[Assistant] Politics is the art of the possible, the attainable, the art of the next best.


[User] Please transcribe this text

[Assistant] Politics is not an exact science.


[User] Please transcribe this text

[Assistant] The main thing is to make history, not to write it.


[User] Please transcribe this text

[Assistant] Political judgment is the ability to hear the distant hoofbeats of the horse of history.


[User] Please transcribe this text

[Assistant] You always think that the worst is still to come, and yet it is over already.


[User] Please transcribe this text

[Assistant] Woe to the leader whose arguments at the end of a war are not as plausible as they were at the beginning.


[User] Please transcribe this text

[Assistant] If you want to build socialism, choose a country that you don't care for.


[User] Please transcribe this text

[Assistant] Beer makes you lazy, stupid, and powerless.


[User] Please transcribe this text

[Assistant] With bad laws and good officials, it is quite possible to rule the country. But if the officials are bad, even the best laws will not help.


[User] Please transcribe this text

[Assistant] Life has taught me a lot to forgive, but even more to seek forgiveness.


[User] Please transcribe this text

[Assistant] The attitude of the state towards the teacher is a state policy that testifies either to the strength of the state or to its weakness.


[User] Please transcribe this text

[Assistant] When the arguments end, the guns begin to speak. Strength is the last argument of a dullard.


[User] Please transcribe this text

[Assistant] We are all a people, and so is the government.


[User] Please transcribe this text

[Assistant] A wise person is only one who knows not much, but the necessary.


[User] Please transcribe this text

[Assistant] Avoid people who try to diminish your ambitions.


[User] Please transcribe this text

[Assistant] Always beware of building castles in the air. These buildings are the easiest to build, but the hardest to destroy.


[User] Please transcribe this text

[Assistant] All treaties between great states cease to be binding when they come in conflict with the struggle for existence.


[User] Please transcribe this text

[Assistant] Fools say that they learn by experience. I prefer to profit from others' experience.


[User] Please transcribe this text

[Assistant] The statesman's task is to hear God's footsteps marching through history and to try to catch on to his coattails as he marches past.


[User] Please transcribe this text

[Assistant] A really great man is known by three signs, generosity in the design, humanity in the execution, moderation in success.


[User] Please transcribe this text

[Assistant] A little caution outflanks a large cavalry.


[User] Please transcribe this text

[Assistant] Better pointed bullets than pointed speeches.


[User] Please transcribe this text

[Assistant] Beware of sentimental alliances, where the consciousness of good deeds is the only compensation for noble sacrifices.


[User] Please transcribe this text

[Assistant] Appreciate your traitor, because a traitor is the best teacher.


[User] Please transcribe this text

[Assistant] Hello. Do you know your colors? Can you say them with me? Here we go. Red Red Blue Blue Yellow Yellow Green Green Orange Orange Pink


[User] Please transcribe this text

[Assistant] Pink. Purple. Purple. Brown. Brown. Black. Black. White. White. And there's one more. What is it? Gray. Gray. Very good.


[User] Please transcribe this text

[Assistant] Let's try that one more time, just a little faster.


[User] Please transcribe this text

[Assistant] Red, red. Blue, blue. Yellow, yellow. Green, green. Orange, orange. Pink, pink.


[User] Please transcribe this text

[Assistant] Purple, purple. Brown, brown. Black, black. White, white. And one more. Gray, gray.


[User] Please transcribe this text

[Assistant] Very good. Now let's try to say the colors backwards.


[User] Please transcribe this text

[Assistant] Here we go. Grey. Grey. White. White. Black. Black.


[User] Please transcribe this text

[Assistant] Brown, brown. Purple, purple. Pink, pink.


[User] Please transcribe this text

[Assistant] Orange, orange, green, green. Yellow, yellow, blue, blue. And one more, red, red. Nice.


[User] Please transcribe this text

[Assistant] Last time, let's say them all together. Here we go. Red. Blue. Yellow. Green. Orange. Pink. Purple. Brown. Black. White. And one more. Gray.


[User] Please transcribe this text

[Assistant] This has been ELF Learning with Colors. Thank you very much for watching.


[User] Please transcribe this text

[Assistant] Good morning everybody, welcome to today's webinar, Landscape Photography with Drew J. Altdorfer, our very special guest today. Uh, so Drew's gonna be talking us through his workflow and capture one and a few other bits. Drew, are you there? I am, I am here.


[User] Please transcribe this text

[Assistant] Fantastic. So if you don't know who you're listening to, I'm the guy at the top, uh, David, part of the software sales team based out of Copenhagen and Denmark. And down below is Drew, uh, who's product manager and CBO of Capture One, also Copenhagen and Denmark. So we've got, uh, 60 minutes.


[User] Please transcribe this text

[Assistant] With each other today, you're very welcome and encouraged to ask questions throughout. I'll interrupt Drew at sort of pertinent times. If you could try to keep your questions on what Drew is talking about that not only helps us but also anyone else.


[User] Please transcribe this text

[Assistant] Who's listening as well, so we don't go off on any wild tangents. I am recording the webinar too, so you can always watch us again at your leisure, if you wish.


[User] Please transcribe this text

[Assistant] Drew's already taken charge of presentation, how about that? Ta-da! Ta-da!


[User] Please transcribe this text

[Assistant] Fantastic. So uhm, we should be able to see your Drew's screen right now.


[User] Please transcribe this text

[Assistant] Yes, we can. Great. Uhm, and first of all, Drew, we were just going to talk a little bit about your workspace.


[User] Please transcribe this text

[Assistant] Oh, I'm just gonna stop you there, Drew. We can't see the pop-up windows, I think, because of, are you sharing Capture One and not the whole monitor, maybe?


[User] Please transcribe this text

[Assistant] And just uhm, explain how you got your extra margin drew, just in case anyone missed that.


[User] Please transcribe this text

[Assistant] Yep, and important that you've got proof margin turned on.


[User] Please transcribe this text

[Assistant] But I guess once you're in the processing phase, you don't, you know, you don't care so much about what the aperture was or whatever, not much you can do about it then.


[User] Please transcribe this text

[Assistant] So what's the disadvantage then of just hitting linear response on everything?


[User] Please transcribe this text

[Assistant] Right, so colour is something you leave for sort of second stage.


[User] Please transcribe this text

[Assistant] Yeah, I think that's a little, little known, um, uh, feature that you can duplicate tools, which we probably see with your curve tool, I'm guessing a bit later as well.


[User] Please transcribe this text

[Assistant] That's a, that's a good way to summarize it. I think there's no, there's nothing to be ashamed of using sliders, but uhm, with curves, you know, you get exactly much more control of your set, which may of course will lead to a better image or uhm,


[User] Please transcribe this text

[Assistant] An image which you might not have been able to get with purely sliders alone.


[User] Please transcribe this text

[Assistant] Uhm, before we go to the next image, I'm just gonna throw a couple of questions at you. Uh, Paul was asking that he finds the white border more distracting. He uses the grey border. Do you think that affects his judgement of colour?


[User] Please transcribe this text

[Assistant] Do you ever use the five point preset on the curve tool? I don't know if you want to zip into that and just show. I mean, that's just a useful kind of, uh,


[User] Please transcribe this text

[Assistant] Uhm, a preset's a good finishing starting tool or should they be avoided altogether? I mean, my feeling is I don't use presets just because uhm,


[User] Please transcribe this text

[Assistant] I've yet to sort of, I suppose, discover presets which have the same kind of effect over all images, but there also can be a good springboard jumping off point. So I think personally that's a good way to use them. I don't know if you ever dip into them, Drew.


[User] Please transcribe this text

[Assistant] And I think, you know, it's not necessarily the magic bullet to instantly make any image look fantastic, but as a springboard to save you time, uh, I think they're perfect for that. But as you said, you know, the reason also why I don't use them is because we're generally showing people how the software works and just applying a preset doesn't really teach anybody anything.


[User] Please transcribe this text

[Assistant] You know, making us jealous. I've always liked the penguin shot, you know.


[User] Please transcribe this text

[Assistant] For those of you who don't know how to use local adjustments, uhm, I will put a YouTube link for the basic tutorial and also in April, there's gonna be a webinar purely on local adjustments. So if that's something new to you, then it's not difficult, uhm, but just, you know, a few starting points will get you on the road much faster.


[User] Please transcribe this text

[Assistant] It's kind of crucial to your work, I think, isn't it? Isn't it, Drew?


[User] Please transcribe this text

[Assistant] Uh, Johnny was asking, uh, well first from Johnny, how do you adjust fog for the best rendition? And John said, can we see the global levels and curve adjustment for your penguins?


[User] Please transcribe this text

[Assistant] And you've dumped a bit of uh, clarity on there as well, I see.


[User] Please transcribe this text

[Assistant] Just, just, just point to the clarity tool just in case anyone missed it.


[User] Please transcribe this text

[Assistant] Uhm, okay, Elizabeth says she hates you, because she was just looking at Antarctica Pix the other day, and I've gone there too soon.


[User] Please transcribe this text

[Assistant] I'm beginning to realize that the material is all there, but I could conceivably vastly improve them by revisiting the processing. I can feel years of work coming on. There you go.


[User] Please transcribe this text

[Assistant] Gotcha. And a few people are jumping up and down and say, where can I get this overlaying? So how much do you want to sell it for, Drew? Ooh, that's a good question.


[User] Please transcribe this text

[Assistant] I'm sure we can, uh, we can maybe, uh, point a few people in the direction to find something like that as well.


[User] Please transcribe this text

[Assistant] Do you apply clarity, uh, locally only?


[User] Please transcribe this text

[Assistant] Uhm, or, uhm, to global adjustments too. He's asking because, sorry, I'm just seeing, wrote the second part, he's had mixed feelings on clarity not giving enough control over mid-shadow tones.


[User] Please transcribe this text

[Assistant] Gotcha. Cool. This is a great shot, especially when you see the raw file.


[User] Please transcribe this text

[Assistant] And I think, I think, uhm, Neil's, our image professor, proved it actually on a blog post that he was using it on other cameras and it had, did have a good beneficial effect as well.


[User] Please transcribe this text

[Assistant] And we've got, we've got five minutes so we could see, we could see your speedy steps and what you did for adjustment for this one.


[User] Please transcribe this text

[Assistant] And I guess you did a colour, did you do a colour pick within the mask or was it just a...


[User] Please transcribe this text

[Assistant] Yeah, Jason was asking about the, the plexiglass used for LCC, LCC. A reply to him is just basically anything you can get your hands on that is, uh,


[User] Please transcribe this text

[Assistant] Plastic, you know, I've used tracing paper. I think Neil's used a shopping bag once, I think. So as long as it's a heavily diffuse, singular colored piece of material, you don't have to get too specific about it.


[User] Please transcribe this text

[Assistant] Yeah, exactly. So, I think people always expect there's some $500 fancy item out there, but, you know, in an emergency, as I said, Neil's used a shopping bag once, so, you know, it's...


[User] Please transcribe this text

[Assistant] Yeah, there are 500 other solutions that don't necessarily need such, so.


[User] Please transcribe this text

[Assistant] Uhm, and John was asking actually why we're on this image, as it's pretty monochromatic anyway. If you go to your, not the one you're on, the, uhm,


[User] Please transcribe this text

[Assistant] Uh, Stuttgart Mercedes here, if you go to the final image. Uh, he said, why use an RGB curve when there's very little color in there, but I guess that was going to give you a boost of what small tones were in there.


[User] Please transcribe this text

[Assistant] Yeah, and I think if you'd done that purely on Luma, then we would have lost out on just boosting it a little bit.


[User] Please transcribe this text

[Assistant] That takes us up to bang on 60 minutes, pretty much.


[User] Please transcribe this text

[Assistant] Perfect. That was good timing. Uh, Paul says, thanks for a great webinar. We'll have to watch the replay to take it all in. Yep, me too as well.


[User] Please transcribe this text

[Assistant] Uhm, so, do you mind if I steal the screen back off you for a second, Drew?


[User] Please transcribe this text

[Assistant] That's your picture again, Drew. That's not what I want. There we go. So just to let you know, next webinar, another guest, 30th of March, so next week, a good friend of ours, Pratik, who's a re-toucher.


[User] Please transcribe this text

[Assistant] Uh, works, uh, quite heavily in Capture One just for the initial, uh, color grade and effect. So he's basically gonna sit with us and, uhm,


[User] Please transcribe this text

[Assistant] And show you what he doesn't capture on, colour-wise.


[User] Please transcribe this text

[Assistant] So he's based out of the US, but he's actually in the UK at the moment, so I'm meeting up with him in Oxford. And we're also going to have, after that, just kind of an extended webinar. We're going to have a discussion with both Pratik himself and his partner, Bella, who's also a fine art photographer who work closely together as well. So a fun pact.


[User] Please transcribe this text

[Assistant] Uhm, and look out for roughly in one hour's time there'll be a follow-up email which has got a link to a YouTube channel where you'll find lots of other resources and everything else as well. So.


[User] Please transcribe this text

[Assistant] Thanks man, that was a pleasure Drew, if you want to watch it all over again, you can always come back this afternoon as well.


[User] Please transcribe this text

[Assistant] Zebra. Thank you very much for watching everyone. This has been Elf Learning with Animals. We will see you next time.


[User] Please transcribe this text

[Assistant] That was a while ago, before the storm.


[User] Please transcribe this text

[Assistant] I tried to call, but there is no phone service.


[User] Please transcribe this text

[Assistant] Anderson's not, uh, cursed, is he? Cursed? No, no, no, no. He's fine. Mental? Absolutely.


[User] Please transcribe this text

[Assistant] Now the tightrope is our dimension. And our dimension has rules. You can move forwards or backwards.


[User] Please transcribe this text

[Assistant] In the last video you saw the notation we'll use to define sequence learning problems. Now let's talk about how you can build a model, build a neural network to learn the mapping from X to Y.


[User] Please transcribe this text

[Assistant] Now one thing you could do is try to use a standard neural network for this task. So in our previous example we had nine input words, so you could imagine trying to take these nine input words, maybe the nine one-hot vectors, and feeding them into a standard neural network


[User] Please transcribe this text

[Assistant] Maybe a few hidden layers and then eventually have this output denying values 0 or 1 that tell you whether each word is part of a person's name.


[User] Please transcribe this text

[Assistant] But this turns out not to work well and there are really two main problems with this. The first is that the


[User] Please transcribe this text

[Assistant] Inputs and outputs can be different lengths and different examples.


[User] Please transcribe this text

[Assistant] So it's not as if every single example had the same input length tx or the same output length ty.


[User] Please transcribe this text

[Assistant] And maybe if every sentence had a maximum length, maybe you could pad or zero pad every input up to that maximum length, but this still doesn't seem like a good representation.


[User] Please transcribe this text

[Assistant] And then the second and maybe more serious problem is that a naive neural network architecture like this, it doesn't share features learned across different positions of text.


[User] Please transcribe this text

[Assistant] In particular, if the neural network has learned that maybe the word Harry appearing in position 1, um, gives a sign that that's part of a person's name, then won't it be nice if it automatically figures out that Harry appearing in some other position, XT, also means that that might be a person's name.


[User] Please transcribe this text

[Assistant] And this is maybe similar to what you saw in convolutional neural networks where you want things learned for one part of the image to generalize quickly to other parts of the image and we like similar effects for sequence data as well.


[User] Please transcribe this text

[Assistant] And similar to what you saw with confidence, using a better representation will also let you reduce the number of parameters in your model.


[User] Please transcribe this text

[Assistant] So previously we said that each of these is a 10,000 dimensional one-hot vector and so this is just a very large input layer if the total input size was, you know, maximum number of words times 10,000.


[User] Please transcribe this text

[Assistant] Uhm, and the weight matrix of this first layer would end up having an enormous number of parameters. So a recurrent neural network, which we'll start to describe in the next slide, does not have either of these disadvantages. So what is a recurrent neural network? Let's build one up.


[User] Please transcribe this text

[Assistant] So if you are reading the sentence from left to right, uhm, the first word you would read is the, some first word say x1.


[User] Please transcribe this text

[Assistant] And what we're going to do is take the first word and feed it into a neural network layer. I'm gonna draw it like this. So that's a


[User] Please transcribe this text

[Assistant] What a recurrent neural network does is when it then goes on to read the second word in a sentence, say x2, instead of just predicting


[User] Please transcribe this text

[Assistant] Y2 using only X2, it also gets


[User] Please transcribe this text

[Assistant] To input some information from


[User] Please transcribe this text

[Assistant] Inputs the third word x3 and it tries to predict, outputs some prediction y hat 3.


[User] Please transcribe this text

[Assistant] Up until the last time step where inputs x, t, x.


[User] Please transcribe this text

[Assistant] At least in this example, Tx is equal to Ty, and the architecture will change a bit if Tx and Ty are not identical. And so at each time step,


[User] Please transcribe this text

[Assistant] We'll also have some, you know, made up activation at time zero. This is usually the vector of zeros. Some researchers will initialize a zero randomly. You have other ways to, uh, initialize a zero, but really having a vector of zeros as the fake.


[User] Please transcribe this text

[Assistant] Time zero activation is the most common choice.


[User] Please transcribe this text

[Assistant] And so that gets input to the neural network. In some research papers or in some books, you see this type of neural network drawn with the following diagram in which at every time step you input x and output y hat, uhm, maybe sometimes there'll be a


[User] Please transcribe this text

[Assistant] This is, uh, the shaded box here denotes a time delay of one step. Uhm, I personally find these recurrent diagrams much harder to interpret and so throughout this course I'll tend to draw the unrolled diagram like the one you have on the left.


[User] Please transcribe this text

[Assistant] But if you see something like the diagram on the right in a textbook or in a research paper, what it really means or the way I tend to think about it is to mentally unrow it into the diagram you have on the left instead.


[User] Please transcribe this text

[Assistant] The Recurrent Neural Network scans through the data from left to right and


[User] Please transcribe this text

[Assistant] The parameters used for each time step are shared. So there will be a set of parameters, which we'll describe in greater detail on the next slide.


[User] Please transcribe this text

[Assistant] But the parameters governing the connection from x1 to the hidden layer will be some set of parameters we're going to write as w a x and it's the same parameters w a x that it uses for every time step. I guess we could write w a x there as well.


[User] Please transcribe this text

[Assistant] The activations, the horizontal connections, will be governed by some set of parameters WAA. And it's the same parameters


[User] Please transcribe this text

[Assistant] WAA used on every time step, and similarly, there's some WYA that governs the, uh, output predictions.


[User] Please transcribe this text

[Assistant] So in this recurrent neural network, what this means is that when making the prediction for y3, it gets the information not only from x3, but also the information from x1 and x2, because the information from x1


[User] Please transcribe this text

[Assistant] information about the words x4, x5, xx, and so on. And so this is a problem, because uhm, if you're given a sentence,


[User] Please transcribe this text

[Assistant] He said Teddy Roosevelt was a great president. In order to decide whether or not the word Teddy is part of a person's name, it'd be really useful to know not just information from the first two words, but to know information from the later words in the sentence as well. Because the sentence could also happen, he said, Teddy bears are on sale.


[User] Please transcribe this text

[Assistant] And so given just the first three words, it's not possible to know for sure whether the word Teddy is part of a person's name. In the first example it is, in the second example it's not, but you can't tell the difference if you look only at the first three words.


[User] Please transcribe this text

[Assistant] So one limitation of this particular neural network structure is that the prediction at a certain time uses


[User] Please transcribe this text

[Assistant] Inputs uses information from the inputs earlier in the sequence, but not information later in the sequence. We will address this in a later video where we talk about bi-directional recurrent neural networks or BRNNs. But for now, this simpler unidirectional


[User] Please transcribe this text

[Assistant] Neural network architecture will suffice to explain the key concepts. And we'll just have to make a quick modification to these ideas later to enable, say, the prediction of y hat 3 to use both information earlier in the sequence as well as information later in the sequence. We'll get to that in a later video.


[User] Please transcribe this text

[Assistant] So let's not write explicitly what are the calculations that this neural network does.


[User] Please transcribe this text

[Assistant] Here's a cleaned up version of the picture of the neural network.


[User] Please transcribe this text

[Assistant] As I mentioned previously, typically you start it off with the input a0 equals the vector of all zeros. Next, this is what forward propagation looks like.


[User] Please transcribe this text

[Assistant] 1, the prediction times that 1, that would be some activation function, maybe a different activation function, uhm, than the one above, but apply to w, y, a, times a1, plus b, y.


[User] Please transcribe this text

[Assistant] And the notation convention I'm going to use for the substitute of these matrices like that example.


[User] Please transcribe this text

[Assistant] W-A-X, uhm, the second index means that this W-A-X is going to be multiplied by some X-like quantity, and this A means that this is used to compute some A-like quantity, like, like so. And similarly you notice that here, W-Y-A,


[User] Please transcribe this text

[Assistant] It's multiplied by some a-like quantity to compute, you know, a y-type quantity.


[User] Please transcribe this text

[Assistant] The activation function used to compute the


[User] Please transcribe this text

[Assistant] Activations will often be a tannic in the choice of an RNN and sometimes red lues are also used although there's a


[User] Please transcribe this text

[Assistant] Uhm, although, uh, but the 10H is actually a pretty common choice.


[User] Please transcribe this text

[Assistant] And uh, we have other ways of preventing the vanishing gradient problem, which we'll talk about later this week.


[User] Please transcribe this text

[Assistant] And depending on what your output y is, if it is a binary classification problem, then I guess you would use a sigmoid activation function, or it could be a softmax if you have a k-way classification problem, but the choice of activation function here would depend on what type of output y you have.


[User] Please transcribe this text

[Assistant] So for the new entity recognition task, where y was either 0 or 1, I guess the second g, uhm, could be a sigmoid activation function.


[User] Please transcribe this text

[Assistant] Uh, and, and I guess you could write, you know, g2 if you want to distinguish that this is a, this could be different activation functions, but I usually won't do that. And then more generally, at time t, a t will be g of w a a times a from the previous time step plus w a x of x from the current time step plus b a and, uhm, y hat


[User] Please transcribe this text

[Assistant] t is equal to g, again, could be different activation functions, but g of w y a times a t plus b y.


[User] Please transcribe this text

[Assistant] So these equations define forward propagation in a neural network where you would


[User] Please transcribe this text

[Assistant] Start off with a0 as a vector of all zeros and then using a0 and x1 you would compute a1 and y hat 1 and then you


[User] Please transcribe this text

[Assistant] Take x2, uhm, and use x2 and a1 to compute a2 and y hat 2, uhm, and so on, and you carry out four propagation going from the left to the right of this picture. Now, in order to help us develop the more complex neural networks, I'm just gonna take this notation and simplify it a little bit. So let me copy these two equations to the next slide. Right, here they are.


[User] Please transcribe this text

[Assistant] And what I'm going to do is actually take, so to simplify the notation a bit, I'm actually going to take that and write in a slightly simpler way.


[User] Please transcribe this text

[Assistant] That underlying quantity on the left and right are supposed to be equivalent. So the way we define W A is we'll take


[User] Please transcribe this text

[Assistant] This matrix WAA and this matrix WAX.


[User] Please transcribe this text

[Assistant] And uhm, put them side by side, kind of stack them horizontally as follows. And this will be the matrix W A.


[User] Please transcribe this text

[Assistant] So for example, if A was a 100 dimensional, and in our running example X was 10,000 dimensional, then WAA


[User] Please transcribe this text

[Assistant] would have been a 100 by 100 dimensional matrix and W A X would have been a 100 by


[User] Please transcribe this text

[Assistant] 10,000 dimensional matrix. And so stacking these two matrices together, this would be 100 dimensional, this would be 100, and this would be


[User] Please transcribe this text

[Assistant] I guess 10,000 elements, so W A will be a 100 by uhm, 10100 dimensional matrix.


[User] Please transcribe this text

[Assistant] Right, I guess this diagram on the left is not drawn to scale, right, since W A X would be a very wide matrix. And what this notation means is


[User] Please transcribe this text

[Assistant] To just take the two vectors and stack them together. So I'm going to use that notation to denote that we're going to take the vector a t minus one. So that's uh,


[User] Please transcribe this text

[Assistant] 100 dimensional and stack it on top of a T. So this ends up being a uhm,


[User] Please transcribe this text

[Assistant] 1 0 1 0 0 dimensional vector. And so hopefully you can check for yourself that this matrix times this vector, uhm, just gives you back the original quantity, right? Because, uh, now,


[User] Please transcribe this text

[Assistant] So the advantage of this notation is that rather than carrying around two parameter matrices, WAA and WAX, we can compress them into just one parameter matrix, WA. And this will simplify our notation for when we develop more complex models. And then for this, in a similar way, uhm, I'm just gonna rewrite this slightly. I'm gonna write this as WY, AT plus BY.


[User] Please transcribe this text

[Assistant] W-Y and B-Y, it denotes what type of output quantity we're computing. So W-Y indicates that that's a weight matrix computing a Y-like quantity, and here W-A and B-A on top indicates the


[User] Please transcribe this text

[Assistant] Those are the parameters for computing, you know, like an A, an activation output quantity. So that's it. You now know what is a basic recurrent neural network. Next, let's talk about backpropagation and how you learn with these RNNs.


[User] Please transcribe this text

[Assistant] And if you want to keep learning with the best movies and TV series so you can finally understand fast speaking natives without getting lost, without missing the jokes and without subtitles, then just click on the subscribe button and the bell down below.


[User] Please transcribe this text

[Assistant] Ross finds out that when Rachel thought Joey was proposing to her, she said yes.


[User] Please transcribe this text

[Assistant] Ross makes a big deal out of it but doesn't say a single word to Joey as he is extremely disappointed.


[User] Please transcribe this text

[Assistant] When they meet at Central Perk and Joey convinces Ross that they have to talk about it, Ross refuses.


[User] Please transcribe this text

[Assistant] Jerry's way to solve the conflict only makes things worse.


[User] Please transcribe this text

[Assistant] Take a look at the sentences. Which of them is an expression related to anger?


[User] Please transcribe this text

[Assistant] My neighbors are driving me up the wall with their parties. I've had it up to here with your excuses. She was delighted by the results of her exam.


[User] Please transcribe this text

[Assistant] We use the adverb technically when we say something that should be interpreted according to a strict way of understanding the meaning of a rule or a set of facts. Let's take a look at some examples.


[User] Please transcribe this text

[Assistant] Connected speech is used when we speak quickly and casually and it helps your English sound more natural.


[User] Please transcribe this text

[Assistant] Linking is the combination of the sounds from the end of one word with the beginning of the next word.


[User] Please transcribe this text

[Assistant] One type of linking occurs when we have a word ending in a consonant sound and the next one beginning with a vowel sound. This is the case in didn't even.


[User] Please transcribe this text

[Assistant] In the sentence, the T sound is dropped in the negative auxiliary didn't.


[User] Please transcribe this text

[Assistant] The N sound links directly to even as it starts with a vowel sound.


[User] Please transcribe this text

[Assistant] Before we take a look at some examples of consonant plus vowel linking, take your time to listen again and repeat. Keep your ears tuned.


[User] Please transcribe this text

[Assistant] The phrase let's put aside or apart the fact that means disregarding or not taking account of the fact or facts that are going to be mentioned.


[User] Please transcribe this text

[Assistant] It's used when you want to interrupt someone while they are talking. However, this one is used in casual situations.


[User] Please transcribe this text

[Assistant] Despite the fact that Ross was being super emphatic with the speech, Joey didn't understand the use of air quotes. Air quotes are an interesting use of body language. They are imaginary quotation marks that you make in the air with your fingers and are often used to express sarcasm, irony or euphemism.


[User] Please transcribe this text

[Assistant] As we just saw, Joey in fact didn't understand the use of air quotes. Even though he was being honest and the tone of his voice was not sarcastic by saying, I'm sorry, followed by air quotes, Ross understood that Joey was not actually sorry.


[User] Please transcribe this text

[Assistant] You probably know the word scream. It's both a verb and a noun. But what may confuse you are the prepositions used with it. When your scream is directed towards someone, you don't say scream with or to someone but rather at someone.


[User] Please transcribe this text

[Assistant] Some interesting collocations are scream in or with pain and scream for help.


[User] Please transcribe this text

[Assistant] If you curse, you use rude or offensive language, usually because you are angry about something.


[User] Please transcribe this text

[Assistant] As a slang expression, hit me also means, deal me a card and serve me another drink, like in these examples.


[User] Please transcribe this text

[Assistant] If you tell someone, I can see it in your eyes, you suggest that they have some thoughts or intentions but don't express them directly, be it with actions or words.


[User] Please transcribe this text

[Assistant] It's interesting to see that an animal name is also used as a verb here.


[User] Please transcribe this text

[Assistant] As a verb, duck means to move your head or the top part of your body quickly down, especially to avoid being hit.


[User] Please transcribe this text

[Assistant] Fun fact, the word duck gains this meaning as a verb because of duck's movement of suddenly going underwater and disappearing.


[User] Please transcribe this text

[Assistant] Your reflexes are your ability to react quickly with your body when something unexpected happens, for example when you have to dodge to avoid being hurt by someone.


[User] Please transcribe this text

[Assistant] If you can't or couldn't help something such as acting in a particular way or making a particular remark,


[User] Please transcribe this text

[Assistant] You are or were not able to control or stop it. If you want to sound more emphatic, you can even add but after help. Let's take a look at some examples.


[User] Please transcribe this text

[Assistant] Your hand is referred to as your fist when you have bent your fingers in towards the palm in order to hit someone, to make an angry gesture or to hold something.


[User] Please transcribe this text

[Assistant] Some common collocations are clench your fist, fist fight and punch your fist.


[User] Please transcribe this text

[Assistant] As Ross wasn't able to duck when Joey hit him, it suggests that something might be wrong with his reflexes. Reflex tests measure the presence and strength of a number of reflexes. In doing so they help to evaluate the integrity of the nerve circuits involved.


[User] Please transcribe this text

[Assistant] When Ross says accidentally using air quotes, he is implying that what Joey did was an accident


[User] Please transcribe this text

[Assistant] Joey was not thinking straight. What Joey did was on purpose.


[User] Please transcribe this text

[Assistant] Which picture best represents the kind of reflex Joey mentioned?


[User] Please transcribe this text

[Assistant] What did Ross say after and I grieve on a stand, have driven on the sand, could even understand?


[User] Please transcribe this text

[Assistant] Let's see who you are. I take off your disguise. And flee!


[User] Please transcribe this text

[Assistant] Enemies of Asgard assembling, plotting, I'll demise all while you, Odin, the protector of those nine realms, are sitting here in your bathrobe, eating grapes.


[User] Please transcribe this text

[Assistant] GOAT G O A T GOAT


[User] Please transcribe this text

[Assistant] You'll be discharged today, okay? I don't need a surgery? No. I'm not gonna die, I'm fine. As a fiddle, whatever.


[User] Please transcribe this text

[Assistant] Oh, it's all good fun. But as is the last night, I really do need them back.


[User] Please transcribe this text

[Assistant] Oh, it's all good fun. But as is the last night, I really do need them back.


[User] Please transcribe this text

[Assistant] The different versions of RNNs you've seen so far will already work quite well by themselves, but for learning very complex functions, it sometimes is useful to stack multiple layers of RNNs together to build even deeper versions of these models. In this video, you'll see how to build these deeper RNNs. Let's take a look.


[User] Please transcribe this text

[Assistant] So you remember, for a standard neural network, you would have an input X, and then that's fed to some hidden layer, and so that might have activations, uh, say A1 for the first hidden layer,


[User] Please transcribe this text

[Assistant] And then that's fed to the next layer with activations A2, then maybe another layer, activations A3, and then you make a prediction y hat.


[User] Please transcribe this text

[Assistant] So a deep RNN is a bit like this, uh, but taking this network that I just drew by hand and unrolling that in time. So let's take a look.


[User] Please transcribe this text

[Assistant] So here's the standard RNN that you've seen so far, but I've changed the notation a little bit, which is that instead of writing this as a


[User] Please transcribe this text

[Assistant] Zero, for the activation time zero, I've added this square bracket, uh, one to denote that this is for layer one. So the notation we're going to use is A


[User] Please transcribe this text

[Assistant] Square bracket L to denote that there's an activation associated with layer L and then angle bracket T to denote that that's associated with time T.


[User] Please transcribe this text

[Assistant] So this would have an activation, uhm, A, first layer, time one. This would be a first layer, time two. First layer, time three. First layer, time four.


[User] Please transcribe this text

[Assistant] And then we can just stack these things on top.


[User] Please transcribe this text

[Assistant] And so this would be a neural network with, uh, three hidden layers. So let's look at an example of, uhm, how this value is computed. So A, two, three.


[User] Please transcribe this text

[Assistant] So we compute it as an activation function G applied to


[User] Please transcribe this text

[Assistant] A weight matrix, this is going to be W A, because computing an A quantity, an activation quantity, and for the second layer,


[User] Please transcribe this text

[Assistant] And so, I'm gonna, uh, give this a


[User] Please transcribe this text

[Assistant] Two, two is that thing, comma, A, one, three is that thing, plus BA associates the second layer, uhm, and that's how you get that activation value.


[User] Please transcribe this text

[Assistant] Every one of these computations at this layer.


[User] Please transcribe this text

[Assistant] Whereas in contrast, the first layer would have its own parameters, wa1 and, uh, ba1.


[User] Please transcribe this text

[Assistant] So whereas for standard neural nets like the one on the left, you know, we've seen neural networks that are very, very deep, maybe over a hundred layers.


[User] Please transcribe this text

[Assistant] For RNNs, uhm, having three layers is already quite a lot. Because of the temporal dimension, these networks can already get quite big, even if you have just a small handful of layers. And, uh, you don't usually see this stacked up to be, you know, like a hundred layers.


[User] Please transcribe this text

[Assistant] One thing you do see sometimes is that, uh, you have recurrent layers that stacked on top of each other, but then you might take the output here,


[User] Please transcribe this text

[Assistant] Let's get rid of this, and then just have a bunch of deep layers that are not connected horizontally, but to have a deep network here that then finally predicts Y1, and you can have the same, you know, deep network here that, uh,


[User] Please transcribe this text

[Assistant] So this is a type of network architecture that we'll see a little bit more when we have.


[User] Please transcribe this text

[Assistant] And quite often these blocks, uh, don't just have to be standard RNN, the simple RNN model. They can also be GRU blocks or LSTM blocks. And, uh, finally you can also build deep versions of the bi-directional RNN. Uhm, because


[User] Please transcribe this text

[Assistant] Because deep RNNs are quite computationally expensive to train, there's often a large temporal extent already, although you just don't see as many, uh,


[User] Please transcribe this text

[Assistant] Deep recurrent layers, just as it gets three deep recurrent layers that are connected in time. Uh, you don't see as many deep recurrent layers as you would see, you know, number of layers in a deep conventional neural network.


[User] Please transcribe this text

[Assistant] So that's it for deep RNNs. With what you've seen this week, ranging from the basic RNN, the basic current unit, to the GRU, to STM, to the bi-directional RNN, to the deep versions of this that you just saw, you now have a very rich toolbox for constructing very powerful models for learning sequence models.


[User] Please transcribe this text

[Assistant] Uhm, I hope you enjoyed this week's videos. Best of luck with the programming exercises and I look forward to seeing you next week.


[User] Please transcribe this text

[Assistant] My head. My head. My hair. My hair. My face. My face.


[User] Please transcribe this text

[Assistant] My eyes. My eyes. My ears.


[User] Please transcribe this text

[Assistant] My ears. My mouth. My mouth. My teeth. My teeth. My nose. My nose. My tongue. My tongue. My cheeks. My cheeks. My chin. My chin. My neck. My neck.


[User] Please transcribe this text

[Assistant] My shoulders, my shoulders. My back, my back. My belly, my belly. My butt, my butt.


[User] Please transcribe this text

[Assistant] My arms. My arms. My elbow. My elbow. My hand. My hand.


[User] Please transcribe this text

[Assistant] My fingers. My fingers. My legs. My legs.


[User] Please transcribe this text

[Assistant] My knees. My knees. My feet. My feet. My toes. My toes.


[User] Please transcribe this text

[Assistant] You don't develop courage by being happy in your relationships every day. You develop it by surviving difficult times and challenging adversity.


[User] Please transcribe this text

[Assistant] To get great pleasure, you need to limit yourself.


[User] Please transcribe this text

[Assistant] Let's start to formalize the problem of learning a good word embedding. When you implement an algorithm to learn a word embedding, what you end up learning is an embedding matrix. Let's take a look at what that means. Let's say as usual, we're using our 10,000 word vocabulary. So the vocabulary is A, Aaron, Orange, Zulu, uh, maybe also unknown word as a token.


[User] Please transcribe this text

[Assistant] What we're going to do is learn a embedding matrix E, which is going to be a 300 dimensional by 10,000.


[User] Please transcribe this text

[Assistant] Dimensional matrix, if you have 10,000 words in vocabulary, or maybe 10,001 if the unaware token is, uh, one extra token.


[User] Please transcribe this text

[Assistant] And the columns of this matrix would be the different embeddings for the 10,000 different words you have in your vocabulary. So orange was word number 6257 in our vocabulary of 10,000 words. So one piece of notation we'll use is that


[User] Please transcribe this text

[Assistant] With zeros everywhere and a one in position six, two, five, seven. And so this would be a, uh, ten thousand.


[User] Please transcribe this text

[Assistant] Dimensional vector with a one in just one position. So this isn't quite drawn to scale, yes. This should be as tall as the embedding matrix on the left is wide.


[User] Please transcribe this text

[Assistant] And if the embedding matrix is called capital E, then notice that if you take E and multiply it,


[User] Please transcribe this text

[Assistant] Then this will be a three hundred dimensional vector. So E is three hundred by ten thousand and O is ten thousand by one. So the product will be three hundred


[User] Please transcribe this text

[Assistant] So the three-dimensional vector. And, uh, notice that to compute the first element of this vector, of this three-dimensional vector, what you do is you would multiply the first row of the matrix E with this.


[User] Please transcribe this text

[Assistant] But all of these elements are 0 except for element 6, 2, 5, 7. And so you end up with 0 times this, 0 times this, 0 times this, and so on. And then 1 times


[User] Please transcribe this text

[Assistant] Whatever this is, and the zero times this, zero times this, and so on. And so, you end up with the first element as, you know, whatever is that, uhm, element up there.


[User] Please transcribe this text

[Assistant] And then for the second element of this three-dimensional vector we're computing you would take the vector


[User] Please transcribe this text

[Assistant] 0, 6, 5, 7 and multiply it by the second row of the matrix E. So again you have 0 times this plus 0 times this plus 0 times all of these other elements and then 1 times this


[User] Please transcribe this text

[Assistant] And then zero times everything else and add that together so you end up with this and so on as you go down the rest of this column.


[User] Please transcribe this text

[Assistant] So that's why the embedding matrix E times this one hot vector here winds up selecting out


[User] Please transcribe this text

[Assistant] This, uh, three-hundred dimensional column corresponding to the word orange. So this is going to be equal to E, 6, 2,


[User] Please transcribe this text

[Assistant] Five, seven, which is the notation we're going to use to represent the embedding vector, the three hundred by one dimensional vector for the word orange.


[User] Please transcribe this text

[Assistant] Pull the trigger, and it'll be the last thing you do.


[User] Please transcribe this text

[Assistant] It's an easy one. That's our bad guy. Eddie Deacon, he's a low-level drug dealer.


[User] Please transcribe this text

[Assistant] We just looked at the character of each of the strings. Now let's learn to combine them.


[User] Please transcribe this text

[Assistant] The most commonly used way is to play two of them at once, called a double stop.


[User] Please transcribe this text

[Assistant] Double stops are fairly natural to violins, violas, and cellos, but not so much to the double bass, where they tend to be somewhat muddy and unappealing in an orchestral texture.


[User] Please transcribe this text

[Assistant] In an orchestral situation, double stops should be a sign that are more comfortable and natural to play.


[User] Please transcribe this text

[Assistant] In general, intervals that include at least one open string are the easiest of all, because the left hand only has to change positions on one string.


[User] Please transcribe this text

[Assistant] Technically, any harmonic interval from a unison to an octave is possible in any position on the violin and viola, so long as the music is well written for it.


[User] Please transcribe this text

[Assistant] Constant double stops in every part throughout every piece is not good writing, as it suggests that the orchestrator is simply attempting to realize piano-type chords and has no understanding of orchestral texture.


[User] Please transcribe this text

[Assistant] Also, a passage of double stops requiring many different finger positions is more of a solo part than an effective orchestral part.


[User] Please transcribe this text

[Assistant] The maximum reach for double stops on the cello in first position is a sixth, unless one of the strings is open. The thumb may also be used to help negotiate double stops.


[User] Please transcribe this text

[Assistant] Double bass octaves are quite effective if the bottom note is an open string. But this essentially limits the effect to only three or four notes.


[User] Please transcribe this text

[Assistant] Triple stops are essentially broken chords, as the bow can rest on only two strings simultaneously. Some type of roll or grace note is required to make these stops work.


[User] Please transcribe this text

[Assistant] While the triple stop is tremendously effective, for orchestral purposes it's almost always a double stop plus an open string, or two open strings plus one fingered note.


[User] Please transcribe this text

[Assistant] It tends to be used mostly for punctuation, like giving a melodic passage some harmonic reference.


[User] Please transcribe this text

[Assistant] Or it can help bring a cadence to a close with a titanic sound.


[User] Please transcribe this text

[Assistant] Quadruple stops are also possible, and here it's even more important to realize that the chord will be broken, and that open strings will feature heavily.


[User] Please transcribe this text

[Assistant] Some passages of arpeggiation are also essentially triple stops and quadruple stops expressed as single notes in sequence.


[User] Please transcribe this text

[Assistant] The player's natural impulse when seeing an interval without any guide will be to play either the top or bottom note. This is called divisi.


[User] Please transcribe this text

[Assistant] Each desk will divide the interval between them, with the player on the right playing the upper note.


[User] Please transcribe this text

[Assistant] Divisi is actually a preferred way of dealing with multiple pitches in a part.


[User] Please transcribe this text

[Assistant] Because it results in a much more secure sound. It's also much easier on the players.


[User] Please transcribe this text

[Assistant] Divisi parts can also include completely separate lines within one staff.


[User] Please transcribe this text

[Assistant] Or these separate lines may be divided over two or more staves.


[User] Please transcribe this text

[Assistant] One drawback with divisi is that it spreads the projection of the strings, so that fewer players are covering more notes.


[User] Please transcribe this text

[Assistant] This means that the strings tend to lose a little power in a divisi passage. Though not a huge amount.


[User] Please transcribe this text

[Assistant] If it's important for the continuity of a chord or a phrase to maintain a precise level of dynamic,


[User] Please transcribe this text

[Assistant] Then the interval should be played as a double stop, marked as non-divisi.


[User] Please transcribe this text

[Assistant] When the divisi passage returns to one note, and you intend both players to resume playing, then that note should be marked unison.


[User] Please transcribe this text

[Assistant] If the music does this more than once in close proximity, then just mark simile or sim over the next occurrence, and the players will apply the correct approach without any further markings.


[User] Please transcribe this text

[Assistant] Let's change gears in our next video, with a look at one of the most delicate of string effects, the harmonic.


[User] Please transcribe this text

[Assistant] One word of advice, don't fuck this up. I don't know why Gwen trusts you, but if she trusts you, we trust you. Understand?


[User] Please transcribe this text

[Assistant] If you have the ability to love, love yourself first.


[User] Please transcribe this text

[Assistant] You give your life away to a function that doesn't interest you.


[User] Please transcribe this text

[Assistant] Love is alright for those who can handle the psychic overload.


[User] Please transcribe this text

[Assistant] People are strange. They are constantly angered by trivial things.


[User] Please transcribe this text

[Assistant] There is always one woman to save you from another. And as that woman saves you, she makes ready to destroy.


[User] Please transcribe this text

[Assistant] Even the most horrible human being on earth deserves to wipe his ass.


[User] Please transcribe this text

[Assistant] There is loneliness in this world so great that you can see it in the slow movement of a clock's hand.


[User] Please transcribe this text

[Assistant] We must bring our own light to the darkness.


[User] Please transcribe this text

[Assistant] There are worse things than being alone, but it often takes decades to realize this. And most often when you do, it's too late and there's nothing worse than too late.


[User] Please transcribe this text

[Assistant] Then you've still got a soul left to lose.


[User] Please transcribe this text

[Assistant] The free soul is rare, but you know it when you see it, basically because you feel good, very good, when you are near or with them.


[User] Please transcribe this text

[Assistant] If you managed to deceive a person, this does not mean that he is a fool.


[User] Please transcribe this text

[Assistant] Too often the people complain that they have done nothing with their lives, and then they wait for somebody to tell them that this isn't so.


[User] Please transcribe this text

[Assistant] I often carry things to read so that I will not have to look at the people.


[User] Please transcribe this text

[Assistant] Some people like what you do. Some people hate what you do. But most people simply don't give a damn.


[User] Please transcribe this text

[Assistant] Love is a form of prejudice. You love what you need. You love what makes you feel good. You love what is convenient.


[User] Please transcribe this text

[Assistant] Drinking is an emotional thing. It juggles you out of the standardism of everyday life. Out of everything being the same.


[User] Please transcribe this text

[Assistant] Are you incapable of restraining yourself, or do you take pride in being an insufferable know-it-all?


[User] Please transcribe this text

[Assistant] Now it's time to look at each instrument individually, explore its strengths and quirks, and analyze its role in the orchestra. Let's start with the violin, which is in some ways the basic building block of the orchestra.


[User] Please transcribe this text

[Assistant] All the other members of the string section are either constructed along its model or adapted to its parameters.


[User] Please transcribe this text

[Assistant] Additionally, the inclusion of the other instruments in the wind and brass sections are contingent on their ability to sound good in combination and contrast to the violin.


[User] Please transcribe this text

[Assistant] Let's review some of the information we learned while studying the string section as a whole, but isolating the details about the violin. First, let's consider its range, starting at G below middle C, and extending upwards past E above the staff, Ottawa.


[User] Please transcribe this text

[Assistant] In some ways, this is an idealized representation of the human voice in the range of a soprano singer.


[User] Please transcribe this text

[Assistant] The lower notes have a character that can sound forceful and husky, or softly crooning.


[User] Please transcribe this text

[Assistant] The middle register is not so penetrating, but has a gentle lyrical quality. Then the higher notes speak with a certain soaring brilliance.


[User] Please transcribe this text

[Assistant] This is directly related to the weight and length of the strings and the resonance of the violin body.


[User] Please transcribe this text

[Assistant] These factors combine to give the violin certain strengths and ideal ranges of tone.


[User] Please transcribe this text

[Assistant] As you may recall, the tuning of the violin is in fifths, starting with the low G string.


[User] Please transcribe this text

[Assistant] G string notes are particularly strong and can sound much tenser in high positions than notes played on the other strings.


[User] Please transcribe this text

[Assistant] The D string may be the gentlest string dynamically, but that can be particularly useful in stating a phrase with calmness and eloquence. It's a particular favorite of late classical and romantic composers, as this subtle sound is even maintained in higher positions.


[User] Please transcribe this text

[Assistant] The A string has the potential for soulful expression that's on a par with the D string, along with a greater strength of projection similar to the E string, which we'll get to in a moment.


[User] Please transcribe this text

[Assistant] However, it's not quite as brilliant in higher positions, nor as unique sounding as the D. It's better in most cases with lyric phrasing to exploit first through fourth positions on the A string, and then cross over to the E string.


[User] Please transcribe this text

[Assistant] The E string could easily take up an entire hour of samples to demonstrate its possibilities. Let's deal with the most important in what time we've got. Obviously, it's the highest string on the violin, the thinnest in diameter, and the tightest in tension. This gives it enormous powers of projection.


[User] Please transcribe this text

[Assistant] There's a certain precision and incisiveness unique to the E string that allows for a range of extremely well-stated musical expression. This alone would be justification for the dominance of the violin in orchestral scoring. Simply put, it just speaks better than any other bowed string in the whole string section.


[User] Please transcribe this text

[Assistant] The E string is also the ideal tool for focusing the energy of a phrase and taking it to great heights of pitch and expression. It's the most widely used string for stating the peak of a dramatic arc or wrapping up a finale.


[User] Please transcribe this text

[Assistant] The combination of all these expressive ranges is what makes the violin unique.


[User] Please transcribe this text

[Assistant] But that's only the very smallest beginning to what a violin can do. Let's take that further in the next video.


[User] Please transcribe this text

[Assistant] Thirteen. I'll handle it.


[User] Please transcribe this text

[Assistant] I feel exactly the same way. I like to learn new recipes, especially dishes from other countries. Do you enjoy cooking?


[User] Please transcribe this text

[Assistant] I know, right? Maybe next time you come over we can whip up a meal together.


[User] Please transcribe this text

[Assistant] That was absolutely fantastic. Now we'll switch lines. You'll start and I'll respond. Okay. Let's go.


[User] Please transcribe this text

[Assistant] The two terms glissando and portamento are often used to describe the same effect, sliding up or down the string in the space of one bow stroke.


[User] Please transcribe this text

[Assistant] But there is a slight distinction even though most composers ignore it. Glissando may be thought of as the pure musical effect of the slide itself, where the focus is on getting somewhere some distance away.


[User] Please transcribe this text

[Assistant] The first thing to understand about this effect is that a true glissando covers only one string, meaning that every grade of pitch will be heard between its starting and ending points.


[User] Please transcribe this text

[Assistant] Often though, a composer will ask the player to play a glissando that crosses the strings, meaning that the effect will have to be faked. It works when a group of instruments is playing, but isn't so successful as a solo.


[User] Please transcribe this text

[Assistant] Portamento and glissando are marked by a straight line with the abbreviations port or gliss. If there are several indications in a row, only the first of such lines requires these words. But in every case, the composer must also add a slur, or the marking has no meaning.


[User] Please transcribe this text

[Assistant] There are also fingered glissandos, but in essence, this is really just a run of notes and should be fully notated if used.


[User] Please transcribe this text

[Assistant] While the strict definition of glissando is a slide as a distinct musical effect, the textbook meaning of portamento is a bit different.


[User] Please transcribe this text

[Assistant] Portamento isn't a slide for its own sake, but out of necessity.


[User] Please transcribe this text

[Assistant] Consider the limited range of the finger positions and the need to smoothly connect legato passages of great scope.


[User] Please transcribe this text

[Assistant] The fingers will naturally have to slide up or down to reposition themselves, and this tendency has become a bit of a cliché.


[User] Please transcribe this text

[Assistant] Most string players will reign this in the best they can, so that it doesn't test the listener's patience. But if used in the right way, portamento can add meaning and lyricism to the arc of a phrase.


[User] Please transcribe this text

[Assistant] Let's examine one more technique before we move on to the individual instruments. Pizzicato, coming your way in the next video.


[User] Please transcribe this text

[Assistant] No-born girls say, me lord, not my lord. If you're going to pose as a commoner, you should do it properly.


[User] Please transcribe this text

[Assistant] In the last video, I mentioned that the first violins tended to have anywhere from slightly more to greatly more accomplished players than the second violins.


[User] Please transcribe this text

[Assistant] Even though it's important to be mindful of this distinction, an orchestrator shouldn't play it safe to the point of complete boredom for the seconds.


[User] Please transcribe this text

[Assistant] It's always good to give these players some challenges, and by examining their roles and parts from the best orchestral repertoire, build your own approach for satisfying the needs of this very capable section. There's also a bit of a gray area here.


[User] Please transcribe this text

[Assistant] The leader of the second violin should be a player who's quite gifted and capable of a degree of leadership.


[User] Please transcribe this text

[Assistant] This may also account for the co-principal who shares their desk, and for the second or even third desk as well.


[User] Please transcribe this text

[Assistant] These front players can often be better musicians than the last few desks of the first violin section.


[User] Please transcribe this text

[Assistant] So never limit the possibilities of your scoring simply on the presumption that all firsts are better than all seconds.


[User] Please transcribe this text

[Assistant] Be aware though that even with the best possible players in either section, the roles are not the same at all. Seconds are necessary,


[User] Please transcribe this text

[Assistant] Not just to support the firsts, but to help ground them. It's worth examining this relationship in detail.


[User] Please transcribe this text

[Assistant] The first and most obvious is in providing a solid bass upon which the firsts can play an octave higher. This is a very potent melodic sound.


[User] Please transcribe this text

[Assistant] Seconds also commonly play closer harmonic intervals with the firsts in melodic scoring, especially thirds and sixths.


[User] Please transcribe this text

[Assistant] In these cases, the intonation must be flawless and the combined tone seamless.


[User] Please transcribe this text

[Assistant] The seconds also play unison with the firsts quite often. This is an extremely forceful sound.


[User] Please transcribe this text

[Assistant] There's a tendency amongst some composers to feel responsible for the seconds, and to give them parts with more interest.


[User] Please transcribe this text

[Assistant] This is commonly called keeping the seconds busy, and is an admirable goal, so long as the orchestrator doesn't overdo it, and gives seconds far too much music for their own good.


[User] Please transcribe this text

[Assistant] It's better to ease off on the seconds a bit, but allow them to shine here and there.


[User] Please transcribe this text

[Assistant] Quite a few great scores will have the seconds cover many of the melodic duties.


[User] Please transcribe this text

[Assistant] The advantage here is that their seating will give that line a slightly more distant sound and give the firsts a chance to rest.


[User] Please transcribe this text

[Assistant] Also, different groups of musicians play somewhat differently, even within the same orchestra.


[User] Please transcribe this text

[Assistant] When the firsts are handling the melodic duties on their own, the seconds will often lead the rest of the strings in the accompaniment, or combine with the violas. Patterns are often a feature of this support.


[User] Please transcribe this text

[Assistant] The seconds may prove better suited to some types of support playing. They're more confident in the lower range.


[User] Please transcribe this text

[Assistant] And usually seated closer to the violas and cellos than are the first violins.


[User] Please transcribe this text

[Assistant] This allows them to combine their tone and rhythm with more intimacy than the firsts.


[User] Please transcribe this text

[Assistant] But a lot of the time the seconds are simply providing a middle voice in a larger orchestral texture. Here's where the importance of good part writing is at a premium.


[User] Please transcribe this text

[Assistant] Especially as it creates a more compelling relationship between the seconds and the other strings.


[User] Please transcribe this text

[Assistant] We'll return to the issue of orchestral roles later in this string orchestration guide videos. But before we leave the topic of the violin,


[User] Please transcribe this text

[Assistant] Let's look at what it can do all by itself against the whole orchestra. Coming your way in the next video.


[User] Please transcribe this text

[Assistant] If you really want to escape the things that harass you, what you're needing is not to be in a different place, but to be a different person.


[User] Please transcribe this text

[Assistant] Life is like a play. It's not the length, but the excellence of the acting that matters.


[User] Please transcribe this text

[Assistant] Difficulties strengthen the mind as labor does the body.


[User] Please transcribe this text

[Assistant] You act like mortals in all that you fear and like immortals in all that you desire.


[User] Please transcribe this text

[Assistant] Sometimes even to live is an act of courage.


[User] Please transcribe this text

[Assistant] We are not ill provided, but use what we have wastefully.


[User] Please transcribe this text

[Assistant] The greatest blessings of mankind are within us and within our reach. A wise man is content with his lot, whatever it may be, without wishing for what he has not.


[User] Please transcribe this text

[Assistant] What need is there to weep over parts of life? The whole of it calls for tears.


[User] Please transcribe this text

[Assistant] If you live in harmony with nature, you will never be poor. If you live according to what others think, you will never be rich.


[User] Please transcribe this text

[Assistant] As long as you live, keep learning how to live.


[User] Please transcribe this text

[Assistant] People are frugal and guarding their personal property. But as soon as it comes to squandering time,


[User] Please transcribe this text

[Assistant] Leisure without books is death and burial of a man alive.


[User] Please transcribe this text

[Assistant] Often a very old man has no other proof of his long life than his age.


[User] Please transcribe this text

[Assistant] And we suffer more from imagination than from reality.


[User] Please transcribe this text

[Assistant] A gift consists not in what is done or given, but in the intention of the giver or doer.


[User] Please transcribe this text

[Assistant] Nothing is more honorable than a grateful heart.


[User] Please transcribe this text

[Assistant] There is no genius without a touch of madness.


[User] Please transcribe this text

[Assistant] But life is very short and anxious for those who forget the past, neglect the present, and fear the future.


[User] Please transcribe this text

[Assistant] While we wait for life, life passes.


[User] Please transcribe this text

[Assistant] But it is easy to do so by example.


[User] Please transcribe this text

[Assistant] It is a tough road that leads to the heights of greatness.


[User] Please transcribe this text

[Assistant] He who has injured thee was either stronger or weaker than thee.


[User] Please transcribe this text

[Assistant] It is more civilized to make fun of life than to bewail it.


[User] Please transcribe this text

[Assistant] For all the rest of existence is not life, but merely time.


[User] Please transcribe this text

[Assistant] To win true freedom, you must be a slave to philosophy.


[User] Please transcribe this text

[Assistant] Of this one thing, make sure against your dying day, that your faults die before you do.


[User] Please transcribe this text

[Assistant] There is no enjoying the possession of anything valuable, unless one has someone to share it with.


[User] Please transcribe this text

[Assistant] To be always fortunate and to pass through life with a soul that has never known sorrow is to be ignorant of one half of nature.


[User] Please transcribe this text

[Assistant] It is quality rather than quantity that matters.


[User] Please transcribe this text

[Assistant] What really ruins our character is the fact that none of us looks back over his life.


[User] Please transcribe this text

[Assistant] Let's look for the animals. Hmm, giraffe.


[User] Please transcribe this text

[Assistant] No frog. No tortoise.


[User] Please transcribe this text

[Assistant] In the last video you saw how the skip grant model allows you to construct a supervised learning task where you map from context to target words and how that allows you to learn a useful word embedding.


[User] Please transcribe this text

[Assistant] But the downside of that was the softmax objective was slow to compute. In this video you see a modified learning problem called negative sampling that allows you to do something similar to the skipgram model you saw just now, but with a much more efficient learning algorithm. Let's see how you can do that.


[User] Please transcribe this text

[Assistant] So what we're going to do in this algorithm is create a new supervised learning problem. And the problem is, given a pair of words like orange and juice, we're going to predict is this a


[User] Please transcribe this text

[Assistant] So in this example, orange juice was a positive example. And uhm, how about orange juice?


[User] Please transcribe this text

[Assistant] And king, well that's a negative example, so I'm going to write zero for the target.


[User] Please transcribe this text

[Assistant] So what we're going to do is we're actually going to sample a context and a target word. So in this case we have orange and juice and we'll associate that with a label of one. So we'll just put orange.


[User] Please transcribe this text

[Assistant] Word in the middle. And then having generated a positive example, so the positive example is generated exactly how we generated it in the previous videos. Sample a context word, look around a window of say plus minus 10 words and pick a target word. So that's how you generate the first row of this table with orange juice one.


[User] Please transcribe this text

[Assistant] And then to generate the negative examples, I'm going to take the same context word and then just pick a word at random from the dictionary. So in this case, I chose the word king at random and we'll label that as zero. And then let's take orange and let's pick another random word from the dictionary.


[User] Please transcribe this text

[Assistant] Under the assumption that if we pick a random word, it probably won't be associated with the word orange. So orange, book, zero. Uhm, and let's pick a few others. Orange, maybe just by chance we'll pick the zero, and then orange, and maybe just by chance we'll pick the word of, and we'll put a zero there.


[User] Please transcribe this text

[Assistant] And notice that, uh, of is labeled as zero even though the word of actually appears next to orange as well.


[User] Please transcribe this text

[Assistant] So to summarize, the way we generated this dataset is we'll pick a context word and then pick a target word and that is the first row.


[User] Please transcribe this text

[Assistant] of this table. That gives us a positive example. So context, target, and then give that a label of 1.


[User] Please transcribe this text

[Assistant] And then what we do is for some number of times, say k times,


[User] Please transcribe this text

[Assistant] We're going to take the same context word and then pick random words from the dictionary. King, book, d, uh, whatever comes out of random from the dictionary and label all those words.


[User] Please transcribe this text

[Assistant] Zero. And those would be our negative examples.


[User] Please transcribe this text

[Assistant] And it's okay if just by chance one of those words we picked at random from the dictionary happens to appear in a window, in a plus minus ten word window, say, uh, next to the context word orange.


[User] Please transcribe this text

[Assistant] Then we're going to create a supervised learning problem where the learning algorithm inputs x, inputs this pair of words, and then has to predict


[User] Please transcribe this text

[Assistant] The target label would predict the output y.


[User] Please transcribe this text

[Assistant] So the problem is really, given a pair of words, like orange and juice, do you think they appear together? Do you think I got these two words by sampling two words close to each other? Or do you think I got them as, uhm, one word from the text and one word chosen randomly from the dictionary?


[User] Please transcribe this text

[Assistant] Recommend that maybe K is 5 to 20 for smaller datasets.


[User] Please transcribe this text

[Assistant] And if you have a very large data set, then choose k to be, uhm, smaller. So k equals 2 to 5 for larger data sets and, uh, larger values of k for smaller data sets.


[User] Please transcribe this text

[Assistant] And in this example I've just used k equals 4.


[User] Please transcribe this text

[Assistant] Next, let's describe the supervised learning model for learning the mapping from x to y.


[User] Please transcribe this text

[Assistant] So here was the softmax model you saw from the previous video.


[User] Please transcribe this text

[Assistant] And, uh, here is the training set we got from the previous slide where again this is going to be the new input x and this is going to be the value of y you're trying to predict. So to define the model I'm going to use this to


[User] Please transcribe this text

[Assistant] Uh, denote this with c for the context word, this to denote the possible target word, t, and this, I'll use y to denote 0, 1, you know, this is a context-target pair.


[User] Please transcribe this text

[Assistant] So what we're going to do is define a logistic regression model. We say that the chance that y is equal to 1, given the input c comma t pair, we're going to model this as basically a logistic regression model, but the specific formula we use is sigmoid applied to theta transpose, uh, theta t transpose e c.


[User] Please transcribe this text

[Assistant] So the parameters are similar as before. You have one parameter vector theta for each possible target word and a separate parameter vector, really the embedding vector for each possible context word.


[User] Please transcribe this text

[Assistant] And we're going to use this formula to estimate the probability that y is equal to 1. So if you have k examples here, uhm, then you can think of this as having


[User] Please transcribe this text

[Assistant] A k to 1 ratio of negative to positive examples. So for every positive examples, you will have k to 1.


[User] Please transcribe this text

[Assistant] Negative examples would wish to train this logistic regression life model. And so to draw this as a neural network, if the input word is uhm, orange,


[User] Please transcribe this text

[Assistant] Do that multiplication to get the embedding vector 6-5-7. And then what you have is really 10,000 possible logistic regression classification problems, where one of these will be


[User] Please transcribe this text

[Assistant] The, uhm, classifier corresponding to, well, is the


[User] Please transcribe this text

[Assistant] Target word Jews or not. And then there'll be other words. For example, there might be one somewhere down here which is predicting is the word king or not, and so on. For these are possible words in your vocabulary.


[User] Please transcribe this text

[Assistant] Think of this as having ten thousand binary logistic regression classifiers, but instead of training all ten thousand of them on every iteration, we're only going to train five of them. We're going to train the one corresponding to the actual target word we got, and then train four randomly chosen negative examples. And this is for the case where k is equal to four. So instead of having one giant ten thousand way softmax,


[User] Please transcribe this text

[Assistant] Which is very expensive to compute. We've instead turned it into ten thousand binary classification problems. Each of which is quite cheap to compute and on every iteration we're only going to train


[User] Please transcribe this text

[Assistant] Five of them, or more generally, k plus one of them, with k negative examples and one positive examples. And this is why the computational cost of this algorithm is much lower, because you're updating k plus one


[User] Please transcribe this text

[Assistant] Logistic units, K plus 1 binary classification problems, uh, which is relatively cheap to do on every iteration rather than updating a 10,000-way softmax classifier.


[User] Please transcribe this text

[Assistant] So you get to play with this algorithm in the programming exercise for this week as well.


[User] Please transcribe this text

[Assistant] So this technique is called negative sampling, because what you're doing is you had a positive example, the orange and the juice, and then you would go out and deliberately generate a bunch of negative examples, kind of negative sampling, sensitive negative sampling, which wish to train four more of these binary classifiers.


[User] Please transcribe this text

[Assistant] And on every iteration, you choose four different random negative words with which to train your algorithm on. Now before wrapping up,


[User] Please transcribe this text

[Assistant] One more important detail of this algorithm is how do you choose the negative examples?


[User] Please transcribe this text

[Assistant] So after having chosen the context word orange, how do you sample these words to generate the negative examples?


[User] Please transcribe this text

[Assistant] So one thing you could do is, uh, sample the words in the middle, the candidates, uhm,


[User] Please transcribe this text

[Assistant] Target words. One thing you could do is sample it according to the empirical frequency of words in your corpus. So just sample it according to, you know, how often different words appear. But the problem with that is that you end up with a very high representation of words like de, of, and, and so on.


[User] Please transcribe this text

[Assistant] Uhm, one other extreme would be to say you use, you know, one over the vocab size, sample the negative examples, uniform near random, but that's also very non-representative of the distribution of English words. So the authors, uh, Mikhailov et al.,


[User] Please transcribe this text

[Assistant] reported that empirically what they found to work best was to take this heuristic value which is a little bit in between.


[User] Please transcribe this text

[Assistant] The two extremes of sampling from the empirical frequencies from meaning from whatever is the observed distribution in English text to the uniform distribution. And what they did was uhm, they sampled proportional to the frequency of a word to the power of the horse.


[User] Please transcribe this text

[Assistant] So if f of w i is the observed frequency of a particular word in the English language on your training set corpus, then by taking it to the power of three-fourths, uh, this is


[User] Please transcribe this text

[Assistant] Somewhere in between the extreme of taking uniform distribution and the other extreme of just taking whatever was the observed distribution in your trading set.


[User] Please transcribe this text

[Assistant] And so I'm not sure this is very theoretically justified, but multiple researchers are now using this heuristic and it seems to work decently well.


[User] Please transcribe this text

[Assistant] So to summarize, you've seen how you can learn word vectors of a softmax classifier, but that's very computationally expensive. And in this video, you saw how by changing that to a bunch of binary classification problems, you can very efficiently learn word vectors.


[User] Please transcribe this text

[Assistant] And if you run this algorithm, you will be able to learn pretty good word vectors. Now of course, as is the case in other areas,


[User] Please transcribe this text

[Assistant] So that's it for the SkipRAM model.


[User] Please transcribe this text

[Assistant] In the next video, I want to share with you yet another version of a word embedding learning algorithm that is maybe even simpler than what you've seen so far. So in the next video, let's learn about the glove algorithm.


[User] Please transcribe this text

[Assistant] Solo double bass is one of the rarest sounds in an orchestral context.


[User] Please transcribe this text

[Assistant] Part of the reason is mostly its lower strength of projection, and more muted tone compared to its fellow string players.


[User] Please transcribe this text

[Assistant] It's simply impractical for a double bass to emote a solo over a complex texture.


[User] Please transcribe this text

[Assistant] But effective when scored to showcase the sometimes epic character and startling range of the instrument.


[User] Please transcribe this text

[Assistant] A very moving color is the bass section solo, standing out as a voice on its own. This is most effective when lightly accompanied.


[User] Please transcribe this text

[Assistant] At this point, we've covered most large scale issues about the way the strings are scored, both individually and as a section.


[User] Please transcribe this text

[Assistant] That means that you've got a baseline upon which you can build your own arrangements with intelligence and insight.


[User] Please transcribe this text

[Assistant] But before you jump into it, you have to touch minds with a couple of centuries of genius. We'll see how in our final video on studying string orchestration.


[User] Please transcribe this text

[Assistant] Poor is the pupil who does not surpass his master.


[User] Please transcribe this text

[Assistant] He who does not value life does not deserve it.


[User] Please transcribe this text

[Assistant] Beauty perishes in life, but is immortal in art.


[User] Please transcribe this text

[Assistant] Which quote did you like the most? Share your opinion in the comments. Don't miss out on a chance to see the next video. Subscribe.


[User] Please transcribe this text

[Assistant] Hmm, I know where your friend is.


[User] Please transcribe this text

[Assistant] That's because little green worm is hiding in here. In the chrysalis.


[User] Please transcribe this text

[Assistant] Yes, but it was your little green friend.


[User] Please transcribe this text

[Assistant] Well, little green worm wasn't really a worm. Little green worm was a caterpillar.


[User] Please transcribe this text

[Assistant] Yes, you will change into something else.


[User] Please transcribe this text

[Assistant] When you grow up, you won't be a little nut-brown hair anymore.


[User] Please transcribe this text

[Assistant] Sentiment Classification is a task of looking at a piece of text and telling if someone likes or dislikes the thing they are talking about. It is one of the most important building blocks in NLP and is used in many applications. One of the challenges of Sentiment Classification is you might not have a huge labelled training set.


[User] Please transcribe this text

[Assistant] But with word embeddings, you'll be able to build good sentiment classifiers even with only modest size label training sets. Let's see how you can do that.


[User] Please transcribe this text

[Assistant] So here's an example of a sentiment classification problem. The input x is a piece of text and the output y that you want to predict is what is the sentiment.


[User] Please transcribe this text

[Assistant] Such as the star rating of let's say a restaurant review.


[User] Please transcribe this text

[Assistant] So if someone says the dessert is excellent, and they give it a 4-star review, uh, service is quite slow, 2-star review, good for a quick meal, but nothing special, 3-star review, and this is a pretty harsh review, completely lacking good taste, good service, and good ambiance as a 1-star review.


[User] Please transcribe this text

[Assistant] So if you can train a system to map from X to Y based on a label dataset like this, then you could use it to monitor comments that people are saying.


[User] Please transcribe this text

[Assistant] Maybe a restaurant that you run. So people might also post messages about your restaurant on social media, on Twitter or Facebook or Instagram.


[User] Please transcribe this text

[Assistant] So, one of the challenges of sentiment classification is you might not have a huge label data set.


[User] Please transcribe this text

[Assistant] So for sentiment classification tasks, training sets with maybe anywhere from, uh, 10,000 to maybe 100,000 words would not be uncommon. Uh, sometimes even smaller than 10,000 words. And word embeddings that you can take


[User] Please transcribe this text

[Assistant] Can help you do much better on the start, especially when you have a small training set.