[User] Please transcribe this text

[Assistant] So this is standard deviation, this is expectation, and the slope, what's the slope? Well, it's the expectation of whatever you're holding x minus the riskless asset.


[User] Please transcribe this text

[Assistant] Okay, divided by the standard deviation of x. That's the sharp ratio of x.


[User] Please transcribe this text

[Assistant] Every point you could conceivably choose, which is somewhere in this blob, then connect it to the riskless asset.


[User] Please transcribe this text

[Assistant] So every point on this line obviously has the same Sharpe ratio. If you leveraged your position and went to here, you wouldn't change your Sharpe ratio. It's has the same slope. Okay? If you put half your money in the bank and half in the list of assets, you'd be here. Same Sharpe ratio. Anything else you could do has a lower Sharpe ratio. So another thing to say is that, uh, an investor


[User] Please transcribe this text

[Assistant] Should ma- should maximize the sharp ratio. Max sharp ratio.


[User] Please transcribe this text

[Assistant] Okay, so you could in fact tell if somebody had screwed up, instead of doing the global index, they just did the American index, you don't pick something here, you could look at the slope of where they ended up.


[User] Please transcribe this text

[Assistant] And it would be a worse slope and a worse sharp ratio. So you could even judge people how they're doing by looking at their sharp ratio. So let's take the case of the two dice.


[User] Please transcribe this text

[Assistant] Okay, suppose you had one die.


[User] Please transcribe this text

[Assistant] So if you had one die, remember, you invest $100, you could get $101, $102, up through $106. Let's say the interest rate is 3%. So, uhm, so you could get $101,


[User] Please transcribe this text

[Assistant] Okay, so the average of that is obviously 103.5.


[User] Please transcribe this text

[Assistant] Alright, cause one and six is seven and divided by two is three point five. So,


[User] Please transcribe this text

[Assistant] The expected payoff for putting your money into one dice is the average return is 1.035. The riskless return is going to be 1.03.


[User] Please transcribe this text

[Assistant] So it's .035 minus .03, .005, and now what's the standard deviation? What's the variance of this? Well, you could take 101 minus 103.5 squared plus 102 minus 103.5 squared, etc., and take the square root of it, and you get .017. Okay?


[User] Please transcribe this text

[Assistant] So you notice the chance of losing money, by the way, is 50%. 50% of the time you're gonna lose money. And your Sharpe ratio is .05 over .017. So that's about 3.


[User] Please transcribe this text

[Assistant] No, point three. Well, point three. Now that happens to be the Sharpe ratio of the stock market. Just about. Shockingly low, isn't it?


[User] Please transcribe this text

[Assistant] If you look at the history of the stock market, so it depends what year you start and what year you end, you know, if you do it right after a crash or before a crash, but just to pick vague numbers, like 9%, and let's say the interest rate is, you know, 4%, something like that,


[User] Please transcribe this text

[Assistant] You know, it could be 5% or 3%, depends again on the period. Okay, maybe it's 2%, okay, but somewhere like that. And the standard deviation of the stock market is, you know, like 16% or 12%, you know, again, depending on what period, but always around 12 to 18%, let's say 16%. That's the one I like to use, because it's percent a day of, uh, okay, so that, uhm,


[User] Please transcribe this text

[Assistant] That's equal to five percent over sixteen percent, which is about a third. Okay, less, you know, it's about point three.


[User] Please transcribe this text

[Assistant] That's exactly what the Stice had, about .3.


[User] Please transcribe this text

[Assistant] So that's all you can get out of the stock market. Not a very good Sharpe ratio.


[User] Please transcribe this text

[Assistant] So, you could measure, you know, Swenson's Sharpe Ratio. You could measure Ellington's Sharpe Ratio and see what they are. You could see who you think is a better manager. Okay, so now, uhm, uh, uh, uh, uh.


[User] Please transcribe this text

[Assistant] So, I won't tell you those numbers just yet, but now, let's suppose you, uhm,


[User] Please transcribe this text

[Assistant] What else could I do? Okay, so let's suppose you put your money in two dice. That's the whole point of this. Suppose you put half your money in the first dice and half your money in the second dice.


[User] Please transcribe this text

[Assistant] Okay, now what are the chances what's going to happen? Well again, your, your expected return hasn't changed. It's still .035. But, the Sharpe ratio has drastically changed. In fact, you can see that. What are the chances of ending up with $101 now, if you put half of your money in each? The worst case.


[User] Please transcribe this text

[Assistant] It's not 1 in 6. You have to get the worst case in both dice. So to get $101, it's 1 in 36.


[User] Please transcribe this text

[Assistant] Okay, it's vastly less likely that you end up with this extreme bad outcome once you put half the money in each die. So if you compute the standard deviation, it's gonna be a lot smaller number. Okay, it's gonna be .01.


[User] Please transcribe this text

[Assistant] Uh, it's gonna be .012. So the Sharpe ratio's gone from .3 to .4. Okay, just by finding the second die. So what you should be doing is looking, like Swenson does, like everybody does, for those independent risks that the rest of the market is too stupid to see. Even after they got the idea of the index,


[User] Please transcribe this text

[Assistant] 20 years, they didn't realize the index should mean everything, not just in America. So you have a huge opportunity to do better than the rest of the managers, you know? While everybody else is screwing around on this line.


[User] Please transcribe this text

[Assistant] You can be screwing around on that line. So you could run the same risk this guy did. Okay, so if you're Swenson's say, you could leverage.


[User] Please transcribe this text

[Assistant] Not that he does, but if you're Harvard and you're on this thing instead of this thing, you could leverage and go up a here and you could say, look, I've got the same standard deviation everybody else does, and look at my returns. They're so much higher than everybody else's.


[User] Please transcribe this text

[Assistant] Okay, so that's what Harvard is trying to do. They're trying to globally diversify, cut down their standard deviation, and then leverage and compare to everybody else, get a higher return with the same standard deviation.


[User] Please transcribe this text

[Assistant] Okay, so that's the first implication of the theory. Now the second implication is the best part. Let's hope I get to it.


[User] Please transcribe this text

[Assistant] Okay, so I'm going to skip over leverage. So the second story is coming right up. Okay, how are these things priced? Okay, so I have to get to the second part of Shakespeare.


[User] Please transcribe this text

[Assistant] Okay, what can we say about the pricing of individual stock?


[User] Please transcribe this text

[Assistant] Now, it looks like an individual stock is going to be worth more if it contributes no variance, less variance. But that turns out to be completely wrong. And it's shocking that that's wrong. Okay? So, why is this? Well, the crucial thing


[User] Please transcribe this text

[Assistant] Is remember back to our first principle. What is price equal to?


[User] Please transcribe this text

[Assistant] What's the key ingredient to price? What is it always equal to? Basic economics.


[User] Please transcribe this text

[Assistant] Supply and demand, okay, but if an individual's gonna buy so that price is equal to his marginal utility, right?


[User] Please transcribe this text

[Assistant] So, we have to figure out, so, what we've shown so far is that everybody is holding the market. Everybody's, uh, right, holding a market maybe with some riskless assets. Okay, so they're holding the market.


[User] Please transcribe this text

[Assistant] So if I ask, if I'm holding the market and I hold a little bit more of a stock, okay, so if I could pay a little bit more and buy T units of stock X, what good would it do me? Well the expectation, the change in expectation


[User] Please transcribe this text

[Assistant] T dollars of X would be, if I increase T a little bit, the derivative of my expectation is just the expectation of X. Okay, so by holding the new stock, I get expectation of, I get a little more


[User] Please transcribe this text

[Assistant] I get t times the expectation of x. If I came up with t extra dollars, I could increase my expectation by t times the expectation of x, obviously.


[User] Please transcribe this text

[Assistant] But what would happen to my variance? So look at the variance of this. What happens to my variance?


[User] Please transcribe this text

[Assistant] Okay, which is equal, the variance I could write is the covariance of this with itself, Tx plus m.


[User] Please transcribe this text

[Assistant] Okay, so I'm taking, right, I wanna see, everybody's supposed to hold M plus some riskless thing, negative or positive in the bond. Okay, or, but,


[User] Please transcribe this text

[Assistant] The negative or positive in the bond is a riskless asset. That contributes nothing to the covariance. I might as well just look at the covariance of a little bit of money in X with the market.


[User] Please transcribe this text

[Assistant] And what's that covariance going to be? Well, it's covariance of T, Tx,


[User] Please transcribe this text

[Assistant] And tx, covariance of tx and, and, uh, m. Okay, these are definitely not zero anymore. Okay, plus covariance of, uh, of uhm, so it's two.


[User] Please transcribe this text

[Assistant] Okay, so it's the covariance of Tx with itself, Tx with Tx.


[User] Please transcribe this text

[Assistant] Twice the covariance of Tx with M, and then this Tx with M, two covariance Tx with M, and the covariance of M with itself. And I want to differentiate that thing.


[User] Please transcribe this text

[Assistant] With respect to t, if I put just a little bit of money, that being on the margin, what's the marginal effect on my variance, right? I hate variance. But see the mistake people have made in the past, as they said,


[User] Please transcribe this text

[Assistant] Absolutely wrong reasoning. That was Adam Smith's puzzle about water and diamonds. You know, why is water less expensive than diamonds when water is so much more important?


[User] Please transcribe this text

[Assistant] How important the whole thing is, is how important one tiny extra drop of it is. The margin utility. So if you add a tiny bit of x to what you're already holding, which might involve a bunch of x already, so you add a tiny bit more of x to what you're already holding, what's gonna be the change in your variance?


[User] Please transcribe this text

[Assistant] So it's the derivative of this with respect to t when t equals zero. Okay, so this by the way equals, uh, you know, t squared variance of x plus two covariance to t times the covariance of x and m plus the variance of m. Okay, and I'm differentiating this with respect to t.


[User] Please transcribe this text

[Assistant] But if I different, right? Because that's why I've just rewritten the same thing. This is t squared, the variance of x. The t comes out, 2t covariance of x with m, and here's the variance of m.


[User] Please transcribe this text

[Assistant] But if I differentiate with respect to t, I get 2t, variance of x. Remember, it's the derivative when t is zero.


[User] Please transcribe this text

[Assistant] Okay, 2t variance of x plus 2 covariance


[User] Please transcribe this text

[Assistant] of x and m plus zero, because the variance of m doesn't depend on t.


[User] Please transcribe this text

[Assistant] So at t equals zero, this is also zero. You see the change in my variance when I add a little more of x is twice the covariance of x and m.


[User] Please transcribe this text

[Assistant] So what is it that X adds? It adds some expectation, and it also adds some variance, but not according to the variance of X, according to the covariance of X with M.


[User] Please transcribe this text

[Assistant] When I add some more, the price of water is how much extra utility I get given what I've already got. I've already got a huge amount of water. That's why an extra amount of water is not doing much for me. That's why the price is low.


[User] Please transcribe this text

[Assistant] So, x itself might be very dangerous, but if it's independent of all the other stuff, m, that you're holding, in fact, you're only adding a drop of it, it won't change your variance hardly at all. In fact, it'll make, you know, and so it's, uh, this is zero. Okay? So, it's the covariance.


[User] Please transcribe this text

[Assistant] Alright, so that's the crucial idea, that the marginal contribution of every stock depends on its expectation and its covariance, not its variance.


[User] Please transcribe this text

[Assistant] And the expectation is good and the covariance is bad. And all of this is linear because it's differentiable. So it just says that, you know, this is the change in covariance. So you know, remember the guy, the marginal utility of expectation,


[User] Please transcribe this text

[Assistant] So the marginal, how am I doing in time? Okay, one more minute. Margin utility of expectation times the expectation of X. That's the change. When you add a dollars worth, a tiny bit of X, your expectation goes up by the expectation of X.


[User] Please transcribe this text

[Assistant] This is the marginal utility of expectation, which is the good thing, minus the marginal utility of variance, variance, that's the bad thing. And what's the change? The thing that affects the, uh,


[User] Please transcribe this text

[Assistant] The thing that affects, the contribution to variance is the covariance of x with m. Okay, so the punchline is,


[User] Please transcribe this text

[Assistant] Everybody has a linear trade-off between expectation and covariance. Okay, so for any fixed person, it's just a, uh,


[User] Please transcribe this text

[Assistant] A constant times the expectation of X minus a constant times, this doesn't depend on the portfolio. This is the marginal utility at his consumption.


[User] Please transcribe this text

[Assistant] Margin utility of expectation at its consumption. You, you face him with any possible new thing that he could buy, just like a consumer in the first day of class. He could buy apples, he could buy oranges, he could buy pears. The margin utility of each one of those, at the point he's already chosen to consume. This is a number that's fixed, this is a number that's fixed. Whatever X I put in here, that's what I get.


[User] Please transcribe this text

[Assistant] So there's a linear tradeoff between expectation and covariance. Expectation good, covariance bad. So the final picture is that in equilibrium, what you should have is every stock


[User] Please transcribe this text

[Assistant] Okay, this is called the security market line. This is the expectation of X. This is the covariance of X with M.


[User] Please transcribe this text

[Assistant] There's the riskless acid right here, so this is a different diagram.


[User] Please transcribe this text

[Assistant] And with a puzzle, the one we started with on the first day of class, if I have two companies, General Motors and AIDS, anti-AIDS companies, some scientist at Yale discovers a cure for AIDS.


[User] Please transcribe this text

[Assistant] He calls it AIDS. Okay? He discovers a cure for AIDS. If the thing works, he'll make a fortune.


[User] Please transcribe this text

[Assistant] If the thing doesn't work, of course it's gonna go bankrupt and pay anyone no money. Let's say we calculate the expected profits of General Electric, not General Motors is too junky now, General Electric, let's say the expected profits of General Electric equal the expected profits of the anti-age company. Which will sell for a higher price.


[User] Please transcribe this text

[Assistant] Who said that? Who just said that? I don't want to embarrass anyone, but alright, you're anonymous, I didn't see what was. So that's exactly, that's what


[User] Please transcribe this text

[Assistant] You would have thought before this class began, everybody hates variance. The AIDS company is so risky, you know, if you get a fortune or zero, you couldn't get riskier than that.


[User] Please transcribe this text

[Assistant] General Electric is not going to go bankrupt very easily and it's not going to, you know, suddenly multiply its value by 100 times either. It's much more solid. So which would you pay more for? Everyone at the beginning of the class, I think, would have said, like he did, whoever he is, would have said,


[User] Please transcribe this text

[Assistant] General Electric you'd pay more for. But the answer's not General Electric. Why is that? Yes.


[User] Please transcribe this text

[Assistant] Exactly, that's the shocking fact, that's the shocking conclusion. So Shakespeare, yes, nothing ventured, nothing gained. You have to take a risk to expect a higher return. Everyone would have thought


[User] Please transcribe this text

[Assistant] The, the, everyone would have thought that's the AIDS company. Anti-AIDS company. No it's not. And Shakespeare couldn't possibly have figured this out. It's General Electric is the one with the higher risk.


[User] Please transcribe this text

[Assistant] Because it's correlated with the market, okay? It's correlated with the market. Obviously, if people are richer, they're gonna buy more refrigerators and engines and stuff like that. If people, you know, business isn't doing very well, they're not gonna buy that stuff. It's very correlated with the market.


[User] Please transcribe this text

[Assistant] Anti-AIDS, you know, if it cures AIDS, people are gonna buy it. If they've got AIDS, they're gonna buy it no matter what they're, you know, and otherwise they won't. So it's uncorrelated with the market.


[User] Please transcribe this text

[Assistant] So therefore the price of the AIDS company is going to be its expected payoff, discounted. The price of General Motors is going to be much less because it's going to be punished for having a correlation with the market. Therefore the return on General Electric is going to be much higher because the same expected payoff and a lower price.


[User] Please transcribe this text

[Assistant] Your return is going to be higher in General Electric. That's the shocking thing that Shakespeare couldn't have noticed. So yes, we're just doing what Shakespeare said from the beginning, but in a way he couldn't possibly have done without any mathematics.


[User] Please transcribe this text

[Assistant] Okay, so now the problem set, you'll have to see if you can do the problem set.


[User] Please transcribe this text

[Assistant] Tire Town School, Episode 16, Deep Mud.


[User] Please transcribe this text

[Assistant] Zippy spun his tires. He spun them very fast. Mud sprayed everywhere. But Zippy didn't move. Zippy sank deeper.


[User] Please transcribe this text

[Assistant] Said Dipper. Poppy gulped. Eddie looked at Jeb. Jeb looked at Eddie.


[User] Please transcribe this text

[Assistant] You're welcome.


[User] Please transcribe this text

[Assistant] Eddie zoomed along the path. I love driving off road.


[User] Please transcribe this text

[Assistant] One way to learn the parameters of the neural network so that it gives you a good encoding for your pictures of faces is to define and apply gradient descent on the triplet loss function. Let's see what that means. To apply the triplet loss, you need to compare pairs of images.


[User] Please transcribe this text

[Assistant] For example, given this picture, to learn the parameters of the neural network, you have to look at several pictures at the same time.


[User] Please transcribe this text

[Assistant] For example, given this pair of images, you want their encodings to be similar because these are the same person. Whereas given this pair of images, you want their encodings to be quite different because these are different persons.


[User] Please transcribe this text

[Assistant] In the terminology of the triplet loss, what you're going to do is always look at one anchor image.


[User] Please transcribe this text

[Assistant] And then you want the distance between the anchor and a positive image, really a positive example, meaning it's the same person, to be similar, whereas you want the anchor, when paired or compared with a negative example, for their distances to be much further apart.


[User] Please transcribe this text

[Assistant] So this is what gives rise to the term triplet loss, which is that you'll always be looking at three images at a time. You'll be looking at an anchor image, a positive image, as well as


[User] Please transcribe this text

[Assistant] And in particular you want this to be less than or equal to the, uhm, distance or the squared norm between the encoding of the anchor and the encoding of the negative. Where of course this is d of a p and this is d of a n. And you can think of d as a distance function, which is why we named it with the alphabet d.


[User] Please transcribe this text

[Assistant] Now, if we move the term from the right side of this equation to the left side, what you end up with is f of a


[User] Please transcribe this text

[Assistant] Minus f of p squared minus, I'm gonna take the right hand side now, minus f of n, uhm, squared. You want this to be less than or equal to zero.


[User] Please transcribe this text

[Assistant] But now we're gonna make a slight change to this expression, which is one trivial way to make sure this is satisfied, is to just learn everything equals zero. If f always outputs zero, then this is zero minus zero, which is zero, this is zero minus zero, which is zero, and so, well, by saying f of any image,


[User] Please transcribe this text

[Assistant] equals a vector of all zeros, you can, you know, almost trivially satisfy this equation.


[User] Please transcribe this text

[Assistant] So to make sure that the neural network doesn't just output zero for all the encodings, or to make sure that it doesn't set all the encodings equal to each other, right? Another way for the neural network to, uhm, give a trivial output is if the encoding for every image was identical to the encoding to every other image, in which case you again get zero, uhm, zero minus zero. So to prevent the neural network from doing that, what we're going to do is modify this objective,


[User] Please transcribe this text

[Assistant] To say that this doesn't need to be just less than or equal to zero.


[User] Please transcribe this text

[Assistant] It needs to be quite a bit smaller than zero.


[User] Please transcribe this text

[Assistant] So in particular, if we say this needs to be less than negative alpha, where alpha is, uhm, another hyperparameter, then this prevents the neural network from outputting the trivial solutions. And by convention, usually we write plus alpha instead of negative alpha there. And this is also called a margin, which is terminology that you'd be familiar with.


[User] Please transcribe this text

[Assistant] If you've also seen the literature on support vector machines, but don't worry about it if you haven't.


[User] Please transcribe this text

[Assistant] And we can also modify this equation on top by adding this margin parameter. So to give an example, let's say the margin is set to 0.2.


[User] Please transcribe this text

[Assistant] If in this example d of the anchor and the positive is equal to 0.5, then you won't be satisfied if d between the anchor and the negative was just a little bit bigger, say 0.51.


[User] Please transcribe this text

[Assistant] Right, even though 0.51 is bigger than 0.5, you're saying that's not good enough. We want D of A comma N to be much bigger than D of A comma P. And in particular, you want this to be at least 0.7 or higher.


[User] Please transcribe this text

[Assistant] Or alternatively, to achieve this margin or this gap of at least 0.2, you could either push this up or push this down so that uhm, there is this, at least this gap of this alpha, hyperparameter alpha at 0.2 between


[User] Please transcribe this text

[Assistant] And on the next slide, formalize it and define the triplet loss function.


[User] Please transcribe this text

[Assistant] The triplet loss function is defined on triples of images. So given three images, a


[User] Please transcribe this text

[Assistant] P and N, the anchor, positive and negative examples. So the positive examples is of the same person as the anchor, but the negative is of a different person than the anchor. We're going to define the loss


[User] Please transcribe this text

[Assistant] And then plus alpha, the margin parameter. And what you want is for this to be less than or equal to zero.


[User] Please transcribe this text

[Assistant] To define the loss function, let's take the max between this and zero. So the effect of taking the max here is that so long as this is less than zero,


[User] Please transcribe this text

[Assistant] Then the loss is zero because the max of something less than or equal to zero with zero is, uh, going to be zero. So, so long as you achieve the goal of making this thing of underlined and green, so long as you've achieved the objective of making that less than or equal to zero, then the loss on this example is equal to zero.


[User] Please transcribe this text

[Assistant] But if on the other hand, if this is greater than zero, then if you take the max, the max will end up selecting this thing of underlying green, and so you would have a positive loss.


[User] Please transcribe this text

[Assistant] So by trying to minimize this, this has the effect of trying to send this thing.


[User] Please transcribe this text

[Assistant] To be zero or less than or equal to zero, and then so long as it's zero or less than or equal to zero, the neural network doesn't care, uhm, how much further negative it is.


[User] Please transcribe this text

[Assistant] So this is how you define the loss on a single triplet and the overall cost function for your neural network can be sum over a training set of these individual losses on different triplets. So if you have a training set of


[User] Please transcribe this text

[Assistant] Say 10,000 pictures with a thousand different persons. What you'd have to do is


[User] Please transcribe this text

[Assistant] Take your 10,000 pictures and use it to generate, to select triplets like this, and then train your learning algorithm using gradient descent on this type of cost function, which is really defined on triplets of images drawn from your training set.


[User] Please transcribe this text

[Assistant] Notice that in order to define this, uh, data set of triplets, you do need some pairs of A and P, pairs of pictures of the same person. So for the purpose of training your system, you do need a data set where you have multiple pictures of the same person.


[User] Please transcribe this text

[Assistant] That's why in this example I said if you have 10,000 pictures of 1,000 different persons, so maybe you have 10 pictures on average of each of your 1,000 persons to make up your entire data set. If you had just one picture of each person, then you can't actually train this system. But of course, after training,


[User] Please transcribe this text

[Assistant] If you're applying this, but of course, after having trained the system, you can then apply it to your one-shot learning problem where


[User] Please transcribe this text

[Assistant] For your face recognition system, maybe you have only a single picture of someone you might be trying to recognize. But for your training set, you do need to make sure you have multiple images of the same person, at least for some people in your training set, so that you can have pairs of anchor and positive images.


[User] Please transcribe this text

[Assistant] Now, how do you actually choose these triplets to form your training set?


[User] Please transcribe this text

[Assistant] If you choose AP and N randomly from your training set, uh, subject to A and P being the same person and A and N being different persons, one of the problems is that if you choose them sort of at random, then this constraint is very easy to satisfy.


[User] Please transcribe this text

[Assistant] Because given two randomly chosen pictures of people, chances are A and N are much different than A and P.


[User] Please transcribe this text

[Assistant] And I hope you still recognize this notation. This DAP was what we had written.


[User] Please transcribe this text

[Assistant] On the last few slides, there's decent coding, so this is just equal to, uhm, this


[User] Please transcribe this text

[Assistant] But if a and n are two randomly chosen different persons, then there's a very high chance that this will be much bigger, more than the margin alpha, than that term on the left. And so the neural network won't learn much from it. So to construct your training set, what you want to do is to choose triplets a, p, and n that are hard to train on.


[User] Please transcribe this text

[Assistant] So in particular, what you want is for all triplets that this constraint be satisfied.


[User] Please transcribe this text

[Assistant] So a triplet that is hard would be if you choose values for ap and n, so that maybe dap is actually quite close to dan.


[User] Please transcribe this text

[Assistant] So in that case, the learning algorithm has to try extra hard to take this thing on the right and try to push it up or take this thing on the left and try to push it down so that there is at least a margin of alpha between the left side and the right side.


[User] Please transcribe this text

[Assistant] And the effect of choosing these triplets is that it increases the computational efficiency of your learning algorithm.


[User] Please transcribe this text

[Assistant] If you choose the triplets randomly, then too many triplets would be really easy and so gradient descent won't do anything because your neural network will just get them right pretty much all the time. And it's only by choosing hard triplets that


[User] Please transcribe this text

[Assistant] The gradient descent procedure, uhm, has to do some work to try to push these quantities further away from those quantities. And if you're interested,


[User] Please transcribe this text

[Assistant] The details are presented in this paper


[User] Please transcribe this text

[Assistant] By Florence Shroff, Dmitry Kalinichenko, and James Phobin, where they have a system called FaceNet, which is where a lot of the ideas I'm presenting in this video had come from.


[User] Please transcribe this text

[Assistant] By the way, this is also a fun fact about how, uhm, algorithms are often named in the deep learning world, which is if you work in a certain domain, uhm, let me call that blank, you often have a system called BlankNet or DeepLearning.


[User] Please transcribe this text

[Assistant] Blank. So we've been talking about face recognition. So this paper is called FaceNet and in the last video you just saw deep face.


[User] Please transcribe this text

[Assistant] But this idea of blank net or deep blank is a very popular way of naming algorithms in the deep learning world.


[User] Please transcribe this text

[Assistant] And uh, feel free to take a look at that paper if you want to learn some of these other details for speeding up your algorithm by choosing the most useful triplets to train on.


[User] Please transcribe this text

[Assistant] It is a nice paper. So just to wrap up, to trade on triplet loss, you need to take your trading set and map it to a lot of triples. So here is a triple with an anchor and a positive, both of the same person and a negative of a different person. Here's another one, uh,


[User] Please transcribe this text

[Assistant] Where the anchor and positive are of the same person, but the anchor and negative are of different persons and so on.


[User] Please transcribe this text

[Assistant] And what you do, having defined this training set of anchor positive and negative triples, is use gradient descent to try to minimize the cost function J we defined on an earlier slide. And that will have the effect of back propagating to all the parameters of the neural network in order to learn an encoding so that D


[User] Please transcribe this text

[Assistant] So that's it for the triplet loss and how you can use it to train a neural network to output a good encoding for face recognition.


[User] Please transcribe this text

[Assistant] Now it turns out that today's face recognition systems, especially the large scale commercial face recognition systems, are trained on very large datasets.


[User] Please transcribe this text

[Assistant] Data sets north of a million images is not uncommon. Some companies are using north of 10 million images and some companies have north of 100 million images with which to try to train these systems. So these are very large data sets. Even by modern standards, these data set assets are not easy to acquire.


[User] Please transcribe this text

[Assistant] Fortunately, some of these companies have trained these large networks and posted parameters online.


[User] Please transcribe this text

[Assistant] So rather than trying to train one of these networks from scratch, uh, this is one domain where because of the sheer data volume sizes, this is one domain where often it might be useful for you to download someone else's pre-trained model rather than do everything from scratch yourself.


[User] Please transcribe this text

[Assistant] But even if you do download someone else's pre-trained model, I think it's still useful to know how these algorithms were trained in case you need to apply these ideas from scratch yourself for some applications.


[User] Please transcribe this text

[Assistant] So that's it for the triplet loss. Uhm, in the next video I want to show you also some other variations on Siamese networks and how to train these systems. Let's go on to the next video.


[User] Please transcribe this text

[Assistant] I don't know why we are here. I'm in perfect health.


[User] Please transcribe this text

[Assistant] Now you look like one of our customers.


[User] Please transcribe this text

[Assistant] Everyone gets training so nobody gets hurt. We all work together. We all take care of each other. Everyone gets paid what they're supposed to get paid. Cooks, dishwashers, waiters, busboys. Nobody gets cheated. The boss knows what's good for workers is good for business.


[User] Please transcribe this text

[Assistant] Tell her you have two accounts. A checking account and a savings account.


[User] Please transcribe this text

[Assistant] A lawyer said he would open a bank account for him, but the guy lied, took the money and disappeared.


[User] Please transcribe this text

[Assistant] You can get in a lot of trouble with a credit card. When you buy things on credit, you have to pay it back right away. All at once. If you don't have the money, they charge you extra and extra and you have to pay more and more and more before you know it.


[User] Please transcribe this text

[Assistant] Miss Lopez, your dress will be perfect. And you will look beautiful. Mrs. Lee can fix anything. She is the best. That is why I send you to her.


[User] Please transcribe this text

[Assistant] It's tomorrow, I know. Everything will be mwah. The food will be delicious. You and your husband will dance all night. You will have beautiful photographs to send to your grandmother. Don't worry. You are in good hands.


[User] Please transcribe this text

[Assistant] Places, everyone. Here they come.


[User] Please transcribe this text

[Assistant] Sometimes, I think that New York is like one big wedding.


[User] Please transcribe this text

[Assistant] Lots of people working day and night to make it all happen. Mrs. Lee, Tanya, Manny, Ramon, Carlos, Tito, and Chi. And of course, Ahmed.


[User] Please transcribe this text

[Assistant] Without them, there would be no New York City.


[User] Please transcribe this text

[Assistant] He likes our shoes. Oh yea? You like our shoes? I bought them for her. Yes, it's the new me.


[User] Please transcribe this text

[Assistant] I can't. It's not good for me. They're delicious. Eat one.


[User] Please transcribe this text

[Assistant] These questions are long and difficult. Even when I read them in Arabic.


[User] Please transcribe this text

[Assistant] Let's give this to the parent coordinator. She can make copies and give it to the


[User] Please transcribe this text

[Assistant] I can't wait to go back home. Sometimes I think I'll never.


[User] Please transcribe this text

[Assistant] I was always the person in the family who took care of everyone. And now...


[User] Please transcribe this text

[Assistant] You know, I grew up on a farm.


[User] Please transcribe this text

[Assistant] And I used to climb a tree just like this when I was a child.


[User] Please transcribe this text

[Assistant] Oh, we don't have any photos of that.


[User] Please transcribe this text

[Assistant] I used to love to draw when I was growing up in Colombia.


[User] Please transcribe this text

[Assistant] Some people told me I was pretty good. When I came to this country I was so busy I stopped


[User] Please transcribe this text

[Assistant] Yes, I have something to show you.


[User] Please transcribe this text

[Assistant] Who's that? Just a wonderful nurse that I know.


[User] Please transcribe this text

[Assistant] Of course not. Just don't move.


[User] Please transcribe this text

[Assistant] Are you in charge of this hospital? Yes, I am.


[User] Please transcribe this text

[Assistant] Can you please come into my room for a minute?


[User] Please transcribe this text

[Assistant] Thanks to her, I am home with my family again.


[User] Please transcribe this text

[Assistant] It's Viviana Medina. I come from Columbia.


[User] Please transcribe this text

[Assistant] Four months ago, I had a terrible accident.


[User] Please transcribe this text

[Assistant] Yes, I'm an artist like you.


[User] Please transcribe this text

[Assistant] Why did you use pigeons as your main example throughout this whole video? Look, don't worry about it. It will be fine.


[User] Please transcribe this text

[Assistant] These are all valid points. When I was learning Spanish, I managed to expand my vocabulary incredibly quickly and this is thanks to a method that I created for myself.


[User] Please transcribe this text

[Assistant] And then I go ahead and I look at the words that I didn't know today on my phone and I add them into my diary, so.


[User] Please transcribe this text

[Assistant] So the words I saw today were pigeon, litter, bus stop, jewellers and skyscraper. And I don't know how to say those in Spanish. So I move on to the next page. And I go and search them on the internet.


[User] Please transcribe this text

[Assistant] You need money? Learn English. Get your diploma. Then, you make real money.


[User] Please transcribe this text

[Assistant] Crap. C-R-A-V. Crap.


[User] Please transcribe this text

[Assistant] Crap. C. R. A. B. Crap.


[User] Please transcribe this text

[Assistant] Crap. C R A B Crap.


[User] Please transcribe this text

[Assistant] Eel. Eel. Eel. Eel.


[User] Please transcribe this text

[Assistant] Eel. E. E. L. Eel.


[User] Please transcribe this text

[Assistant] F I S H Jellyfish


[User] Please transcribe this text

[Assistant] F I S H Jellyfish


[User] Please transcribe this text

[Assistant] O C T O B U S Octopus


[User] Please transcribe this text

[Assistant] O C T O B U S Octopus


[User] Please transcribe this text

[Assistant] Starfish S T A R


[User] Please transcribe this text

[Assistant] F I S H Starfish


[User] Please transcribe this text

[Assistant] F I S H Starfish


[User] Please transcribe this text

[Assistant] Shark. S-H-A-R-K. Shark.


[User] Please transcribe this text

[Assistant] W A L R U S Walrus


[User] Please transcribe this text

[Assistant] W A L R U S Walrus


[User] Please transcribe this text

[Assistant] Snail. S N A I L Snail.


[User] Please transcribe this text

[Assistant] Dolphin D O L P H I N Dolphin


[User] Please transcribe this text

[Assistant] S-I-E-L, seal.


[User] Please transcribe this text

[Assistant] Leopard L E O P A R D Leopard


[User] Please transcribe this text

[Assistant] Tiger. T I G E R. Tiger.


[User] Please transcribe this text

[Assistant] Tiger. T I G E R. Tiger.


[User] Please transcribe this text

[Assistant] R-A-B-B-I-T Rabbit


[User] Please transcribe this text

[Assistant] For information and advice call 3-1-1. For emergencies, to get help right away call 9-1-1.


[User] Please transcribe this text

[Assistant] Why are you here? Ask Zippy. This is a racetrack. It's for speedy cars.


[User] Please transcribe this text

[Assistant] Practice what? Ask Zippy. Racing.


[User] Please transcribe this text

[Assistant] Let's race, said Zippy. I'm going to win. Zippy shot after Eddie.


[User] Please transcribe this text

[Assistant] What? You, you didn't do anything wrong? I, I said I didn't technically.


[User] Please transcribe this text

[Assistant] Come on now Flanders, I don't complain about yer.


[User] Please transcribe this text

[Assistant] Good evening, sir. My name is Ross Geller. I'm one of the people who applied for the apartment, and I, I realize that the competition is fierce, but


[User] Please transcribe this text

[Assistant] That's weird. Are you sure you're punching right? Make a fist.


[User] Please transcribe this text

[Assistant] And as we talked about before, like I think it really, really inhibits people.


[User] Please transcribe this text

[Assistant] And, and, and it overwhelms them. They think, okay, so there is so much for me to learn. I will never be able to do this. And I'm like, no, as long as you can communicate your message.


[User] Please transcribe this text

[Assistant] You bel- English belongs to you, and you belong in English. And, and it doesn't matter if, if, you know, your parents spoke to you in that language or not. It doesn't matter. It doesn't matter if you only have a limited, um, number of words.


[User] Please transcribe this text

[Assistant] The moment you are able to communicate and express your feelings, your thoughts, your emotions, then this is your language too.


[User] Please transcribe this text

[Assistant] If that's what you just said. I actually have some exercises that can help you.


[User] Please transcribe this text

[Assistant] Well, you shouldn't blame yourself if that's what you just said. I actually have some exercises that can help you.


[User] Please transcribe this text

[Assistant] One of the other things that, that, that I would say was a,


[User] Please transcribe this text

[Assistant] I've already mentioned the importance of getting it in front of the customer. Uhm, and the last thing is, and this particularly I think is important to producers, one is, one is your vision.


[User] Please transcribe this text

[Assistant] That's right. If you call 311, you can get health insurance for your child.


[User] Please transcribe this text

[Assistant] Everybody has a right to healthcare. In New York, you don't need to have papers to see a doctor.


[User] Please transcribe this text

[Assistant] Even when your son feels good, he needs medicine. He needs to take it every day.


[User] Please transcribe this text

[Assistant] My asthma used to get worse at night. I found out that it was because of my pillow.


[User] Please transcribe this text

[Assistant] I had to get rid of my favorite pillow. And...


[User] Please transcribe this text

[Assistant] Wait. One more thing. Cockroaches.


[User] Please transcribe this text

[Assistant] Dr. Hunter. What are you telling? I'd urge him to think it through, carefully.


[User] Please transcribe this text

[Assistant] In the last video you learned about the GRU, the gated recurrent unit, and how that can allow you to learn very long range connections in a sequence. The other type of unit that allows you to do this very well is the LSTM, or the long short term memory unit, and this is even more powerful than the GRU. Let's take a look.


[User] Please transcribe this text

[Assistant] Here are the equations from the previous video for the GRU. And for the GRU, we had AT equals CT.


[User] Please transcribe this text

[Assistant] And uh, two gates, the update gate and the relevance gate, c tilde t, which is a candidate for replacing the memory cell. And then we use the gate, uh, the update gate gamma u to decide whether or not to update ct using c tilde t.


[User] Please transcribe this text

[Assistant] The LSTM is an even slightly more powerful and more general version of the GRU and is due to


[User] Please transcribe this text

[Assistant] Sepp Hochreiter and Juergen Schmidt-Huber. And this was a really seminal paper that had a huge impact on sequence modeling.


[User] Please transcribe this text

[Assistant] Although I think this paper is one of the more difficult ones to read, it goes quite a lot into the theory of vanishing gradients. And so I think more people have learned about the details of LSTM through maybe other places than from this particular paper, even though I think this paper has had a wonderful impact on the deep learning community.


[User] Please transcribe this text

[Assistant] But, uh, these are the equations that govern the LSTM. So the, we'll continue to the memory cell C and the candidate value for updating it, C tilde t, will be this. And so, um, notice that for the


[User] Please transcribe this text

[Assistant] At is equal to ct, so uhm, this is what we'll use. And so this is just like the equation on the left, except that we've now more explicitly used at there, or at minus 1 instead of ct minus 1.


[User] Please transcribe this text

[Assistant] And we're not using this gamma R, this relevance gate. Although you can have a variation of the LSTM where you put that back in, but maybe the more common version of the LSTM doesn't bother with that.


[User] Please transcribe this text

[Assistant] And then we will have an update gate, same as before. So w update, uhm, and I'm going to use a t minus one here. It's t plus u. And one,


[User] Please transcribe this text

[Assistant] New property of the LSTM is instead of having one update gate control both of these terms, we're going to have two separate terms. So instead of


[User] Please transcribe this text

[Assistant] Gamma u and 1 minus gamma u, we're gonna have gamma u here, and, and forget gate, which we're gonna call gamma f. So this gate gamma f is going to be sigmoid of pretty much what you'd expect, x t plus b f.


[User] Please transcribe this text

[Assistant] And then we're going to have a new outputs gate, which is sigma of w o. And then again, pretty much what you'd expect. Plus the o. And then the


[User] Please transcribe this text

[Assistant] Update value to the memory cell will be ct equals


[User] Please transcribe this text

[Assistant] Gamma u, then this asterisk denotes element-wise multiplication.


[User] Please transcribe this text

[Assistant] There's a vector-vector element-wise multiplication.


[User] Please transcribe this text

[Assistant] Plus, and instead of 1 minus gamma u, we're going to have a separate forget gate, gamma f, times c of


[User] Please transcribe this text

[Assistant] So this gives the memory cell the option of keeping the old value ct-1 and then just adding to it this new value c tilde of t. So use a separate update and forget gates. So this stands for update, forget, and output gates.


[User] Please transcribe this text

[Assistant] And then finally, instead of at equals ct, at is greater or equal to the output gate, element-wise multiplied with, uhm, ct.


[User] Please transcribe this text

[Assistant] So these are the equations that govern the LSTM.


[User] Please transcribe this text

[Assistant] And you can tell it has three gates instead of two, so uhm, it's a bit more complicated and it places the gates in slightly different places.


[User] Please transcribe this text

[Assistant] So here again are the equations governing the behavior of the LSTM.


[User] Please transcribe this text

[Assistant] Once again, it's traditional to explain these things using pictures, so let me draw one here. Uh, and if these pictures are too complicated, don't worry about it. I personally find the equations easier to understand than the picture, but I'll just show the picture here for the intuitions it conveys.


[User] Please transcribe this text

[Assistant] The particular picture here was very much inspired by a blog post due to Chris Olar, titled Understanding LSTM Networks. The diagram drawn here is quite similar to one that he drew in his blog post. But the key things to take away from this picture are maybe that you use 80 minus 1,


[User] Please transcribe this text

[Assistant] And then these values are combined in these complicated ways with element-wise multiplies and so on, uhm, to get ct from the previous ct minus 1.


[User] Please transcribe this text

[Assistant] Now, one element of this is interesting, is if you hook up a bunch of these in parallel, so that's one of them, and you connect them, you know, connect these, uh, temporally. So this is input x1.


[User] Please transcribe this text

[Assistant] Then x2, x3, so you can take these units and just hook them up as follows.


[User] Please transcribe this text

[Assistant] Where the output A at my previous time step is the input A at the next time step and similarly for C. And I've simplified the diagrams a little bit at the bottom.


[User] Please transcribe this text

[Assistant] And one cool thing about this you notice is that this is a line at the top that shows how, so long as you set the


[User] Please transcribe this text

[Assistant] Forget and the update gates appropriately, it is relatively easy for the LSTM to have some value C0 and have that be passed all the way to the right to have, you know, maybe C3 equals C0. And this is why the LSTM as well as the GRU is very good at memorizing certain


[User] Please transcribe this text

[Assistant] Values, uh, even for a long time, for certain real values.


[User] Please transcribe this text

[Assistant] Stored in the memory cell even for many, many time stamps.


[User] Please transcribe this text

[Assistant] Uhm, as you can imagine, there are also a few variations on this that people use. Perhaps the most common one is that instead of just having the gate values be dependent only on a t minus 1 x t, sometimes people also sneak in their, uhm, the value c


[User] Please transcribe this text

[Assistant] T minus one as well. This is called a peephole connection.


[User] Please transcribe this text

[Assistant] Not a great name maybe, but if you see peephole connection, what that means is that the gate values may depend not just on a t minus one, but and on x t, but also on the previous memory cell value. And a peephole connection can go into all three of these gates computations.


[User] Please transcribe this text

[Assistant] So that's one common variation you see of LSTMs. Uh, one technical detail is that these are, say, 100 dimensional vectors, if you have a 100 dimensional hidden memory cell unit, and so is this.


[User] Please transcribe this text

[Assistant] Let's say fifth element of CT minus one affects only the fifth element of the corresponding gates. So that relationship is one to one, where not every element of


[User] Please transcribe this text

[Assistant] The 100 dimensional CT-1 can affect all elements of the gaze, but instead, you know, the first element of CT-1 affects the first element of the gaze, the second element affects the second element and so on.


[User] Please transcribe this text

[Assistant] But if you ever read a paper and see someone talk about a people connection, that's what they mean. That CT minus one is used to affect the gait value as well. So that's it for the LSTM.


[User] Please transcribe this text

[Assistant] There isn't widespread consensus in this. And even though I presented GRUs first, in the history of deep learning, LSTMs actually came much earlier, and then GRUs were a relatively recent invention that were maybe derived as probably a simplification of the more complicated LSTM model.


[User] Please transcribe this text

[Assistant] Researchers have tried both of these models on many different problems, and on different problems, different algorithms will win out, so there isn't a...


[User] Please transcribe this text

[Assistant] Uh, universally superior algorithm, which is why I want to show you both of them. But uhm, I feel like when I am using these, the advantage of the GRU is that it's a simpler model, and so it's actually easier.


[User] Please transcribe this text

[Assistant] To build a much bigger network that only has two gates, so computationally it runs a bit faster, so it scales to building, you know, somewhat bigger models, but the LSTM is more powerful and more flexible since it has three gates instead of two.


[User] Please transcribe this text

[Assistant] If you want to pick one to use, I think LSTM has been the historically more proven choice. So if you had to pick one.


[User] Please transcribe this text

[Assistant] I think most people today would still use the LSTM as the default first thing to try, although I think in the last few years, uh, GRUs have been gaining a lot of momentum and I feel like more and more teams are also using GRUs because they're a bit simpler but often work, you know, just as well and it might be easier to scale them to even bigger problems.


[User] Please transcribe this text

[Assistant] So that's it for LSTMs. Uh, with either GIUs or LSTMs, you'll be able to build neural networks that can capture much longer range dependencies.


[User] Please transcribe this text

[Assistant] To provide ever greater control and accuracy of execution. It's the main method of sound production for the string section.


[User] Please transcribe this text

[Assistant] And each type of instrument has a bow of different size and weight. But all are built essentially the same.


[User] Please transcribe this text

[Assistant] A bow stick made out of a light, flexible, but strong wood called pernambuco, strung with a couple of hundred strands of horsehair.


[User] Please transcribe this text

[Assistant] As the Pernambuco tree is now an endangered species, sometimes substitute woods and carbon fibers are used.


[User] Please transcribe this text

[Assistant] The bow stick has a mild inward curve, which helps give the hair some tension.


[User] Please transcribe this text

[Assistant] The frog, or heel of the bow, is an adjustable block of wood that fixes the hairs into a flat position about a centimeter wide, and separates them from the bow stick by around an inch.


[User] Please transcribe this text

[Assistant] Tension is adjusted by means of a screw at the base, which pulls the frog back and tightens the strings between the frog and the protrusion at the opposite end of the bow, called the point.


[User] Please transcribe this text

[Assistant] The idea is to tighten the hairs to a certain degree of flexible stiffness, but still leaving a bit of bend in the bow stick.


[User] Please transcribe this text

[Assistant] After playing, the bow is always loosened once more, to preserve its flexibility and relieve tension on the hairs. It's a simple and elegant design, but just like the violin itself, the subtleties of design and craft can make all the difference. Some bows cost tens of thousands of dollars.


[User] Please transcribe this text

[Assistant] After the bow has been tightened, it's drawn several times across a cake of rosin, a hard lump of refined tree sap.


[User] Please transcribe this text

[Assistant] To give the horsehair some stickiness or drag on the strings.


[User] Please transcribe this text

[Assistant] Each instrument in the string section has its own grade of rosin. The violins and violas use a lighter, more powdery type.


[User] Please transcribe this text

[Assistant] While the cellos and double basses use darker, stickier rosin, as their bows have to grip the strings harder in order to get them to speak.


[User] Please transcribe this text

[Assistant] No matter which instrument though, the rosin tends to wear off after a while, and must be reapplied, often several times in the course of one performance.


[User] Please transcribe this text

[Assistant] It also leaves a bit of residue under the bridge, which must be wiped off daily so that it doesn't affect the tone of the instrument, not to mention making it look grimy.


[User] Please transcribe this text

[Assistant] Keep away from people who try to belittle your ambitions. Small people always do that.


[User] Please transcribe this text

[Assistant] The fear of death follows from the fear of life.


[User] Please transcribe this text

[Assistant] Conscience takes up more room than all the rest of a person's insides.


[User] Please transcribe this text

[Assistant] A lie can travel halfway around the world, while the truth is putting its shoes on.


[User] Please transcribe this text

[Assistant] Do something every day that you don't want to do. This is the golden rule for acquiring the habit of doing your duty without pain.


[User] Please transcribe this text

[Assistant] It is the little conveniences that make the real comfort of life.


[User] Please transcribe this text

[Assistant] Who's that? You're killing me. You don't know who that is? No.


[User] Please transcribe this text

[Assistant] The third largest social media inventor since Zuckerberg? Classmate? What planet are you from?


[User] Please transcribe this text

[Assistant] I'm very excited to bring you this super special lesson on Scottish English with Outlander. This is a popular historical fiction show based on a series of books written by Diana Gavildon.


[User] Please transcribe this text

[Assistant] The show follows the story of Claire, an English military nurse who in 1945 goes to Scotland on a second honeymoon with her husband Frank in an attempt to get to know each other again after their wartime separation.


[User] Please transcribe this text

[Assistant] However, while there, in a turn of events, she travels back in time to 1743 when Scotland and England didn't have a good relationship.


[User] Please transcribe this text

[Assistant] The Highlanders become suspicious of Claire, not only because she's from England,


[User] Please transcribe this text

[Assistant] While trying to figure out a way to go back to her life in the 20th century, Claire becomes a guest of the clan Mackenzie and befriends Jamie, the guy who she just healed from the gunshot, who she will later marry and fall in love with.


[User] Please transcribe this text

[Assistant] When Gayla says that the village is humming with talk about clay, she means that by having so many people discreetly commenting about her presence, it gives the impression that they are humming.


[User] Please transcribe this text

[Assistant] How do you think the following sentences would be pronounced in Scottish English? Yes, I can do it for you. I? I can do it for you? No, I won't go in your car. No, I won't go in your car.


[User] Please transcribe this text

[Assistant] Another of its characteristics is the strong rhotic R, that is, they roll and fully pronounce the R's. This is also very distinct in Irish English.


[User] Please transcribe this text

[Assistant] In fact, rolling Rs is the main thing people will do when trying to imitate Scottish or Irish accents. As we can see in this scene, which Monica mocks Ross because he adopted a British accent on his first day teaching at university.


[User] Please transcribe this text

[Assistant] It's incredible how friends seems to have a scene for every situation and use vocabulary in a way that is fun and natural, isn't it?


[User] Please transcribe this text

[Assistant] That's why we created a course for English learners based entirely on the sitcom, so that you can learn and practice expressions, pronunciation, and also learn about culture while having fun. Try it now for free with our three-part masterclass.


[User] Please transcribe this text

[Assistant] You may have seen in movies like 007 or Charlie's Angels that a spy is a person who secretly collects and reports information on activities or plans of an enemy or competitor.


[User] Please transcribe this text

[Assistant] As mentioned before, Scotland and England fought against each other many times over the centuries. So while Sassenach is mostly used by Scots in a derogatory way to refer to English people, in the series it became an affectionate word that only Jamie uses to address Claire, as we can see in these scenes.


[User] Please transcribe this text

[Assistant] That's why she is able to identify how hurt Jamie is.


[User] Please transcribe this text

[Assistant] Germs are the microorganisms like bacteria that cause disease.


[User] Please transcribe this text

[Assistant] In order to prevent that, wounds need to be disinfected, that is, cleaned with chemical substances such as iodine or thimerosal.


[User] Please transcribe this text

[Assistant] Back in the 18th century, none of this was part of popular knowledge. As we can see in this funny scene, in which years later she wants to treat another of Jamie's gunshot wounds with penicillin, which she brought from the future.


[User] Please transcribe this text

[Assistant] Run with water to control an epidemic on a ship.


[User] Please transcribe this text

[Assistant] Which words are pronounced as does-nay in this sentence? Do plus no, does plus not.


[User] Please transcribe this text

[Assistant] I'd love to, but I'm full. The spaghetti carbonara was really filling.


[User] Please transcribe this text

[Assistant] Jamie says he had to eat grass because he was living rough. This means he had no proper home.


[User] Please transcribe this text

[Assistant] In other words, he was homeless and had to sleep and live outdoors. He also uses some words that are particular to many Scottish dialects, such as lads, which means boys. In the previous scene, he also used the word wee, which means small or little.


[User] Please transcribe this text

[Assistant] Is this your little baby? I didn't know you were a father already. She is such a beautiful girl. Is this your wee bairn?


[User] Please transcribe this text

[Assistant] I didn't ken you were a dat already. She is such a bonny lassie.


[User] Please transcribe this text

[Assistant] This means that someone has offered money as a reward for capturing or killing him.


[User] Please transcribe this text

[Assistant] If you enjoyed this lesson, then be sure to check out this one next.


[User] Please transcribe this text

[Assistant] I'm sixteen, and don't worry, I'm old enough to know what sort of a place this is. Well, very pleased to meet you, Uncle Jamie's wife. When you see him, will you tell him I'm looking for him?


[User] Please transcribe this text

[Assistant] A benign fungus? Great title for your memoir. What's that? A benign fungus.


[User] Please transcribe this text

[Assistant] By now you've seen most of the key building blocks of RNNs.


[User] Please transcribe this text

[Assistant] So to motivate bi-directional RNNs, let's look at this network which you've seen a few times before in the context of named entity.


[User] Please transcribe this text

[Assistant] Why three should be zero or one? You need more information than just the first three words because the first three words doesn't tell you if they're talking about Teddy Bears or talking about the former U.S. President Teddy Roosevelt.


[User] Please transcribe this text

[Assistant] So this is a unidirectional or forward directional only RNN.


[User] Please transcribe this text

[Assistant] And this comment I just made is true, whether these cells are standard.


[User] Please transcribe this text

[Assistant] RNN blocks, or whether they're GRU units or whether they're LSTM blocks.


[User] Please transcribe this text

[Assistant] But, uh, all of these blocks are, uhm, in a forward omic direction.


[User] Please transcribe this text

[Assistant] So what a bidirectional RNN does, or a BRNN, is fix this issue.


[User] Please transcribe this text

[Assistant] So a bi-directional RNN works as follows. I'm going to use a simplified, um, for input.


[User] Please transcribe this text

[Assistant] We'll have a forward recurring component, so I'm going to call this A1.


[User] Please transcribe this text

[Assistant] Over that to denote this is the forward recurrent component, uhm, and so they'll be connected as follows. And so each of these forward recurrent units inputs the current x and then, uhm, feeds in


[User] Please transcribe this text

[Assistant] So, so far I haven't done anything, right? Basically redrawn the...


[User] Please transcribe this text

[Assistant] Uh, RNN from the previous slide, but with the arrows, you know, plays in slightly funny positions.


[User] Please transcribe this text

[Assistant] But I drew the arrows in just these slightly funny positions because what we're going to do is add a backward recurrent layer. So we'll have A1, left arrow to denote this is a backward connection, and then A2 backwards, A3 backwards, and A4 backwards. So the left arrow denotes that this is a backward connection. And so,


[User] Please transcribe this text

[Assistant] We're then going to connect the network up as follows.


[User] Please transcribe this text

[Assistant] A, backward connections will be connected to each other going backward in time.


[User] Please transcribe this text

[Assistant] So notice that this network defines a cyclic graph.


[User] Please transcribe this text

[Assistant] And so given an input sequence, uh, x1 through x4, the forward sequence will first compute a4 at 1, then use that to compute a4 at 2, then a4 at 3, then a4 at 4.


[User] Please transcribe this text

[Assistant] Whereas the backward sequence would start by computing a backward 4 and then go back and compute a backward 3. And notice you're computing network activations. This is not background, this is forward prompt. But the forward prompt goes as partially


[User] Please transcribe this text

[Assistant] But the forward prop has part of the computation going from left to right and part of the computation going from right to left in this diagram.


[User] Please transcribe this text

[Assistant] But having computed a backward 3, you can then use those activations to compute a backward 2, and then a backward 1. And then finally having computed all your hidden layer activations, you can then, uhm, make your predictions.


[User] Please transcribe this text

[Assistant] And so for example, to make the predictions, uh, your network would have something like


[User] Please transcribe this text

[Assistant] Y hat at time t is an activation function applied to W y with both the forward activation at time t


[User] Please transcribe this text

[Assistant] And the backward activation at time t being fed in to make that prediction at time t.


[User] Please transcribe this text

[Assistant] So, if you look at the prediction at time step 3, for example, then information from x1 can flow through here, forward 1 to forward 2, that also takes the information here, to forward 3,


[User] Please transcribe this text

[Assistant] So information from, uhm, x1, x2, x3 are all taken into account. Whereas information from x4 can flow through a backward 4 to a backward 3.


[User] Please transcribe this text

[Assistant] So this allows the prediction at time 3 to take its input, both information from the pause,


[User] Please transcribe this text

[Assistant] As well as information from the present, which goes into both the forward and the backward things at this step, as well as information from the future.


[User] Please transcribe this text

[Assistant] So in particular, given a phrase like, he said, Teddy, um, Roosevelt,


[User] Please transcribe this text

[Assistant] dot dot dot to predict whether Teddy was, uh, is a part of a person's name. You build a, take into account information from the past and from the future.


[User] Please transcribe this text

[Assistant] So this is the bi-directional recurrent neural network. And uhm, these blocks here,


[User] Please transcribe this text

[Assistant] And you have a complete sentence, you're trying to label things in a sentence, a bi-directional RNN with LSTM blocks, uh, both forward and backward would be a pretty reasonable first thing to try.


[User] Please transcribe this text

[Assistant] That's it for the bi-directional RNN, and this is a modification they can make.


[User] Please transcribe this text

[Assistant] The disadvantage of the bi-directional RNN is that you do need the entire sequence of data before you can make predictions anyway.


[User] Please transcribe this text

[Assistant] So for example, if you're building a speech recognition system,


[User] Please transcribe this text

[Assistant] Then the BRNN will let you take into account the entire speech utterance, but if you use this straightforward implementation, you need to wait for the person to stop talking to get the entire


[User] Please transcribe this text

[Assistant] But for a lot of natural language processing applications where you can't get the entire sentence all at the same time, the standard BRNN algorithm is actually very effective.


[User] Please transcribe this text

[Assistant] So that's it for the RNNs. In the next and final video for this week, let's talk about how to take all of these ideas, RNNs, LSTMs, or GRUs, and bi-directional versions, and construct deep versions of them.


[User] Please transcribe this text

[Assistant] After your dad fired him to tell you the truth. That's not a job sense.


[User] Please transcribe this text

[Assistant] Or maybe you have an internship that doesn't pay much. That's okay. Cause it's for a good cause.


[User] Please transcribe this text

[Assistant] Or in concrete, every day, all day long. To pass time, I...


[User] Please transcribe this text

[Assistant] To pass time, I would start composing these poems in my head.


[User] Please transcribe this text

[Assistant] It made me feel like I'd finally broken through to something real. About my life and what I should be trying to do with it. Music. Starting this year. Tomorrow.


[User] Please transcribe this text

[Assistant] No. Until I've got things more figured out, you're the only person I'm telling, okay?


[User] Please transcribe this text

[Assistant] It made me feel like I'd finally broken through to something real.


[User] Please transcribe this text

[Assistant] No. Until I've got things more figured out, you're the only person I'm telling, okay?


[User] Please transcribe this text

[Assistant] Last week you learned about RNNs, GRUs, and LSTMs. In this week, you see how many of these ideas can be applied to NLP, to Natural Language Processing, which is one of the fields of AI that is really being revolutionized by deep learning.


[User] Please transcribe this text

[Assistant] One of the key ideas you learn about is word embeddings, which is a way of representing words that lets your algorithms automatically understand analogies like that.


[User] Please transcribe this text

[Assistant] Man is the woman, as king is the queen, and many other examples. And through these ideas of word embeddings, you'll be able to build NLP applications even with modestly sized or even relatively small label training sets.


[User] Please transcribe this text

[Assistant] Finally, towards the end of the week, you see how to de-bias word embeddings as to reduce undesirable gender or ethnicity or other types of bias that learning algorithms can sometimes pick up.


[User] Please transcribe this text

[Assistant] So that, let's get started with a discussion on word representation.


[User] Please transcribe this text

[Assistant] So far, we've been representing words using a vocabulary of words, and our vocabulary from the previous week might be, say, 10,000 words.


[User] Please transcribe this text

[Assistant] We've been representing words using a one-hot vector. So for example, if man is word number 5391 in this dictionary, then you represent it with a vector with a 1 in position 5391. And I'm also going to use o subscript


[User] Please transcribe this text

[Assistant] And then if woman is word number nine eight five three, uhm, then you represent it with O subscript nine eight five three, which just has a one in position nine eight five three and zeros elsewhere. And then other words, king, queen,


[User] Please transcribe this text

[Assistant] Uhm, apple, orange would be similarly represented with one-hot vectors.


[User] Please transcribe this text

[Assistant] One of the weaknesses of this representation is that it treats each word as a thing unto itself.


[User] Please transcribe this text

[Assistant] And it doesn't allow an algorithm to easily generalize across words.


[User] Please transcribe this text

[Assistant] For example, let's say you have a language model that has learned that, uhm, when you see I want a glass of orange blank, well what do you think the next word will be? Very likely to be juice, but


[User] Please transcribe this text

[Assistant] Even if the learning algorithm has learned that I want a glass of orange juice is a likely sentence, if it sees I want a glass of apple blank, as far as it knows, the relationship between apple and orange


[User] Please transcribe this text

[Assistant] It's not any closer as the relationship between any of the other words, man, woman, king, queen, and orange. And so it's not easy for the learning algorithm to generalize from knowing that orange juice is a popular thing to recognizing that apple juice might also be a popular thing or a popular phrase.


[User] Please transcribe this text

[Assistant] And this is because the inner product between


[User] Please transcribe this text

[Assistant] Any two different one-hot vectors is zero. If you take any two vectors, say queen,


[User] Please transcribe this text

[Assistant] So it just doesn't know that somehow apple and orange are much more similar than king and orange or queen and orange.


[User] Please transcribe this text

[Assistant] So, wouldn't it be nice if instead of a one-hot representation, we can instead learn a featurized representation, where for each of these words, uh, man, woman, king, queen, apple, orange, or really for every word in our dictionary, we can learn a set of features and values for each of these words.


[User] Please transcribe this text

[Assistant] So for example, each of these words might have, uh, we want to know what is the gender associated with each of these things. So if gender goes from, uh, minus one for male to plus one for female, then the gender associated with man might be minus one for woman, might be plus one.


[User] Please transcribe this text

[Assistant] And then if you're actually learning these things, maybe for king you get minus .95, for queen plus .97, and for apple and orange, you know, sort of genderless. Another feature might be


[User] Please transcribe this text

[Assistant] Well, how royal are these things? And so the terms, man and woman,


[User] Please transcribe this text

[Assistant] On Optically Royal, so they might have features values close to zero, whereas


[User] Please transcribe this text

[Assistant] King and Queen are highly royal and Aqua and Orange are not particularly royal. Uhm, how about age? Well, man and woman.


[User] Please transcribe this text

[Assistant] Uh, doesn't connote much about age. Maybe men and women implies that they're adults, but you know, maybe neither.


[User] Please transcribe this text

[Assistant] Necessarily young, not old, so maybe values close to zero, whereas, uh, kings and queens are always, almost always adults, and apple and orange might be more neutral, with respect to age. And then another feature for, you know, is this a food? And, well, man is not a food, woman is not a food, uhm,


[User] Please transcribe this text

[Assistant] Neither are kings and queens, but apples and oranges are fruits.


[User] Please transcribe this text

[Assistant] And there can be many other features as well, ranging from uhm, what is the size of this, uh, what is the cost, uhm, is this something that is alive, is this an action, or is this a noun, or is this a verb, or is this something else, and so on.


[User] Please transcribe this text

[Assistant] So you can imagine coming up with many features and for the sake of illustration, let's say, uhm, 300 different features. And what that does is it allows you to take this list of numbers I've only written four.


[User] Please transcribe this text

[Assistant] This vector, this three-dimensional vector, a three-dimensional vector like this, I would denote E, nine, eight,


[User] Please transcribe this text

[Assistant] You use this representation to represent the words orange and apple. Then notice that the representations for orange and apple are now quite similar.


[User] Please transcribe this text

[Assistant] Some of the features will differ because maybe the color of an orange, the color of an apple, the taste, or the...


[User] Please transcribe this text

[Assistant] Or some other features would differ, but by and large, a lot of features of Apple and Orange are actually the same.


[User] Please transcribe this text

[Assistant] Or take on very similar values. And so this increases the odds of the learning algorithm that has figured out that orange juice is a thing to also quickly figure out that apple juice is a thing. So this allows it to generalize better across different words.


[User] Please transcribe this text

[Assistant] So over the next few videos, we'll find a way to learn word embeddings, which is basically learn high dimensional feature vectors like these.


[User] Please transcribe this text

[Assistant] That gives a better representation than one hot vectors for representing different words.


[User] Please transcribe this text

[Assistant] And the features we'll end up learning won't have a...


[User] Please transcribe this text

[Assistant] Uhm, easy to interpret, interpretation like that component 1 is gender, component 2 is royal, component 3 is Asian, and so on.


[User] Please transcribe this text

[Assistant] Exactly what they're representing will be a bit harder to figure out.


[User] Please transcribe this text

[Assistant] But nonetheless, the featurized representations we'll be able to learn will allow an algorithm to quickly figure out that apple and orange.


[User] Please transcribe this text

[Assistant] For each word, one of the popular things to do is also to take this 300 dimensional data and embed it.


[User] Please transcribe this text

[Assistant] Say in a two dimensional space so that you can visualize them.


[User] Please transcribe this text

[Assistant] And so one common algorithm for doing this is the T-SNE algorithm due to Lauren Vandermarten and Jeff Hinton. And if you look at one of these


[User] Please transcribe this text

[Assistant] Embeddings, one of these representations, you find that, uhm, words like


[User] Please transcribe this text

[Assistant] Man and woman tend to get grouped together, king and queen tend to get grouped together, and uh,


[User] Please transcribe this text

[Assistant] These are the people which tend to get grouped together. Those are, uhm, animals which tend to get grouped together. The fruits will tend to be close to each other. Numbers like 1, 2, 3, 4 will be close to each other. And then maybe the, uhm,


[User] Please transcribe this text

[Assistant] Animates objects as a whole will also tend to get grouped together, but you see plots like these sometimes on the internet to visualize some of these 300 or higher dimensional embeddings. And maybe this gives you a sense that word embedding algorithms like these can


[User] Please transcribe this text

[Assistant] Learn similar features for concepts that feel like they should be more related.


[User] Please transcribe this text

[Assistant] As visualized by that concept that seems to you and me like they should be more similar, end up getting mapped to more similar feature vectors.


[User] Please transcribe this text

[Assistant] And these representations we'll use, these sort of featurized representations in maybe a 300 dimensional space, these are called embeddings. And the reason we call them embeddings is you can think of a 300 dimensional space.


[User] Please transcribe this text

[Assistant] And again, I can't draw a 300 dimensional space, so let's make this a 3D one. And what you do is you take every word, like orange, and you, you know.


[User] Please transcribe this text

[Assistant] Have a three-dimensional feature vector, so the word orange.


[User] Please transcribe this text

[Assistant] gets embedded to a point in this three-hundred-dimensional space and the word apple gets embedded


[User] Please transcribe this text

[Assistant] To a different point in this 300 dimensional space. And uhm, of course to visualize it, algorithms like t-SNE map this to a much lower dimensional space so you can actually plot the 2D data and uh, and, and, and look at it. But that's where the term embedding comes from.


[User] Please transcribe this text

[Assistant] Word embeddings has been one of the most important ideas in NLP, in natural language processing.


[User] Please transcribe this text

[Assistant] In this video, you saw why you might want to learn or use word embeddings. In the next video, let's take a deeper look at how you'll be able to use these algorithms to build NLP algorithms.


[User] Please transcribe this text

[Assistant] In a big house in Brussels, Belgium.


[User] Please transcribe this text

[Assistant] A man took his daughter in his arms Little Audrey Hepburn looked up


[User] Please transcribe this text

[Assistant] And saw some beautiful lights above her. They were as white as snow.


[User] Please transcribe this text

[Assistant] She remembered those lights all her life. She always loved the color white. She saw white and remembered her father. Then she felt safe and warm.


[User] Please transcribe this text

[Assistant] Young Audrey enjoyed reading and loved animals and birds. But her greatest love was music. What's music? Audrey asked her mother one day. It's for dancing, her mother answered. Audrey wanted to be a dancer, but she was unhappy. I'm too fat, she thought.


[User] Please transcribe this text

[Assistant] They enjoyed sports and talked about their fathers. Audrey was quiet and sad. Her English was not good and she hated sports.


[User] Please transcribe this text

[Assistant] Audrey lived with her mother and two half-brothers in the Dutch town of Arnhem.


[User] Please transcribe this text

[Assistant] She became more and more interested in dancing.


[User] Please transcribe this text

[Assistant] She wanted to be a world famous ballet dancer. But one morning, Audrey's mother came into her bedroom. Wake up, she told her daughter. It's war.


[User] Please transcribe this text

[Assistant] In the early days of the war, Audrey did not leave school or stop dancing. Then


[User] Please transcribe this text

[Assistant] One day, the Germans sent everybody out of the town. Outside in the country, there was very little food.


[User] Please transcribe this text

[Assistant] Audrey and her family had to dig vegetables in the winter from the hard ground. When the war finished, Audrey was very thin and weak.


[User] Please transcribe this text

[Assistant] Chapter two. A little sun in her eyes.


[User] Please transcribe this text

[Assistant] Audrey and her mother moved to Amsterdam after the war. Her mother became a cook.


[User] Please transcribe this text

[Assistant] She was not rich, but she paid for Audrey's dancing lessons.


[User] Please transcribe this text

[Assistant] Audrey had the best dancing teacher in Holland. Then, when she was 17,


[User] Please transcribe this text

[Assistant] In 1948 Audrey and her mother moved to London. Her mother worked in a flower store. Friends gave them some money.


[User] Please transcribe this text

[Assistant] This was the most important ballet school in London. Audrey worked hard at her dancing.


[User] Please transcribe this text

[Assistant] Somebody remembered her from the ballet school and gave her a small part in a big London musical.


[User] Please transcribe this text

[Assistant] Three thousand girls tried to get the part, but the producers wanted Audrey.


[User] Please transcribe this text

[Assistant] She quickly found jobs in other musicals.


[User] Please transcribe this text

[Assistant] Everybody liked this thin girl with the pretty face and wide smile. I was not a great dancer, Audrey remembered later.


[User] Please transcribe this text

[Assistant] Her photo was in many magazines.


[User] Please transcribe this text

[Assistant] She had small parts in three cheap movies, and she was a cigarette girl in the famous movie The Lavender Hill Mob, 1951. One evening, she went to a party and met a rich Englishman.


[User] Please transcribe this text

[Assistant] James Hansen fell in love with Audrey.


[User] Please transcribe this text

[Assistant] She wanted to be in movies and the theater too. I don't want to get married now, she told him. Let's wait. In 1951,


[User] Please transcribe this text

[Assistant] Audrey had a small part in the movie Monte Carlo Baby. So she went to the south of France.


[User] Please transcribe this text

[Assistant] The famous French writer Colette was in the south of France too.


[User] Please transcribe this text

[Assistant] She wanted to find a girl for the Broadway musical of her book, Gigi.


[User] Please transcribe this text

[Assistant] When she saw Audrey, with her thin arms and legs, long dark hair, and big pretty eyes, she said


[User] Please transcribe this text

[Assistant] She is Gigi. Half woman, half boy. Wonderful. Audrey was excited, but nervous. I can't do that, she told the great writer.


[User] Please transcribe this text

[Assistant] I'm a dancer, not an actor. But Colette did not listen.


[User] Please transcribe this text

[Assistant] Chapter 3, Roman Holiday. Audrey was wonderful in Gigi, but she did not get too excited.


[User] Please transcribe this text

[Assistant] After Gigi, I'm going to marry James Hansen, she told the newspapers.


[User] Please transcribe this text

[Assistant] A Hollywood movie producer remembered Audrey from one of her early English movies.


[User] Please transcribe this text

[Assistant] He wanted her to be a princess in a big new movie, Roman Holiday. Audrey could not say no.


[User] Please transcribe this text

[Assistant] The most important actor in the movie was Gregory Peck. And Audrey was nervous. This was her first big movie.


[User] Please transcribe this text

[Assistant] And Gregory Peck was world famous. Maybe he won't like me, she thought. But she was wrong. He was nice to her and helped her.


[User] Please transcribe this text

[Assistant] It was the hottest summer in Rome for years, and the work was not easy.


[User] Please transcribe this text

[Assistant] Roman Holiday opened in London on August 20th, 1953. In the movie, Audrey is a princess.


[User] Please transcribe this text

[Assistant] But she does not wear expensive clothes or eat in expensive restaurants.


[User] Please transcribe this text

[Assistant] She wants to be the same as the other young girls in Rome.


[User] Please transcribe this text

[Assistant] Now she wanted to be in movies all the time.


[User] Please transcribe this text

[Assistant] She was not ready to be James Hansen's wife. James was sad, but he understood. They stayed friends.


[User] Please transcribe this text

[Assistant] Chapter 4 The Right Man


[User] Please transcribe this text

[Assistant] Hubert de Givenchy of Paris made her clothes.


[User] Please transcribe this text

[Assistant] A woman does not wear a dress, he told her. She lives in it. He became her friend for life.


[User] Please transcribe this text

[Assistant] There were two very famous actors in Sabrina William Holden and Humphrey Bogart. Holden liked Audrey very much. Bogart did not like Holden.


[User] Please transcribe this text

[Assistant] Don't be sad. Nobody likes Bogart, Holden told her. He's unfriendly to everybody. Holden fell in love with Audrey.


[User] Please transcribe this text

[Assistant] He wanted to leave his wife and marry her.


[User] Please transcribe this text

[Assistant] She was the love of my life, he said later. Audrey liked Holden too, but he was not the right man for her. He drank too much.


[User] Please transcribe this text

[Assistant] And he had a wife and three children. She did not want to hurt his family.


[User] Please transcribe this text

[Assistant] And children were very important to Audrey. When she married, she wanted to be a mother too. Later in the same year, Audrey went to a party in London and met the right man. He was 11 years older than her.


[User] Please transcribe this text

[Assistant] But to her, he was wonderful.


[User] Please transcribe this text

[Assistant] He was an actor, writer, producer, and director. He knew everybody in Hollywood. He was a good friend of Gregory Peck. He was intelligent and he could speak many languages. He had a warm and friendly face. His name was Mel Ferrer.


[User] Please transcribe this text

[Assistant] There was a photo of Audrey on the front of Time Magazine.


[User] Please transcribe this text

[Assistant] She shines with fire in her beautiful eyes.


[User] Please transcribe this text

[Assistant] And she acted in it with him for six months. People liked Ondine. And Audrey acted beautifully.


[User] Please transcribe this text

[Assistant] Three days later, she won a Tony for Best Actress in Ondine. These were exciting times for Audrey, but she was very tired. She had to have a vacation, so she flew to Switzerland. She loved Switzerland. She felt safe there.


[User] Please transcribe this text

[Assistant] Away from the people and noise of the big cities.


[User] Please transcribe this text

[Assistant] Chapter 6. A Good Wife.


[User] Please transcribe this text

[Assistant] On September 25th, 1954, she married Mel Ferrer in Switzerland. She became pregnant and stopped making movies. Sadly, she lost the baby. She was very unhappy, but Mel was strong. He loved her and was very nice to her.


[User] Please transcribe this text

[Assistant] After that, she wanted to be in movies with her husband.


[User] Please transcribe this text

[Assistant] She acted with him in War and Peace 1955.


[User] Please transcribe this text

[Assistant] Then she made Funny Face, 1956, with the famous dancer Fred Astaire. Mel was not in this movie, but he was near her, in Paris.


[User] Please transcribe this text

[Assistant] It was a happy musical with beautiful clothes by Givenchy.


[User] Please transcribe this text

[Assistant] This time, Mel could not be near her. He was in the United States.


[User] Please transcribe this text

[Assistant] Mel worked in Spain and Mexico and Audrey went with him. She cooked for him and helped him with his work.


[User] Please transcribe this text

[Assistant] A lot of people in Hollywood were angry with Mel and did not like him.


[User] Please transcribe this text

[Assistant] To them, he was her boss, more than her husband.


[User] Please transcribe this text

[Assistant] There was no love story. No music. There were no funny conversations. And no Givenchy clothes.


[User] Please transcribe this text

[Assistant] In the movie, Audrey was a nun, Sister Luke, and she had to wear nun's clothes.


[User] Please transcribe this text

[Assistant] The camera could only see her mouth and eyes, but she was more than a pretty face. She was a great actress too, and people loved her.


[User] Please transcribe this text

[Assistant] She was a bird girl in Green Mansions. It was her worst movie.


[User] Please transcribe this text

[Assistant] She was sad because she made the movie for love. Mel was the director.


[User] Please transcribe this text

[Assistant] This was a bad time for Audrey.


[User] Please transcribe this text

[Assistant] After Green Mansions, she became pregnant again. Mel wanted her to stay at home. But she went to Mexico.


[User] Please transcribe this text

[Assistant] In the middle of the movie The Unforgiven, she had an accident. She fell off a horse.


[User] Please transcribe this text

[Assistant] A big expensive Hollywood movie. He sent her the story and she read it. What do you think? Hitchcock asked. Will you do it? I'm sorry, Audrey told him. I don't like the story. It's not for me. Hitchcock was very angry with her.


[User] Please transcribe this text

[Assistant] He made a different cheap black and white movie with no world famous actors.


[User] Please transcribe this text

[Assistant] It will lose money and I'll never speak to Audrey Hepburn again.


[User] Please transcribe this text

[Assistant] He was wrong about his new movie. Its name was Psycho. But he was right about the second thing. He never spoke to Audrey again.


[User] Please transcribe this text

[Assistant] Hollywood wanted her to be in a new musical. West Side Story. But she stayed quietly in Switzerland.


[User] Please transcribe this text

[Assistant] She did not want to lose a third baby.


[User] Please transcribe this text

[Assistant] After Sean was born, Audrey stayed at home with him. She was happy in Switzerland.


[User] Please transcribe this text

[Assistant] In the day she went for long walks in the snow.


[User] Please transcribe this text

[Assistant] Then one evening, she read a story by Truman Capote.


[User] Please transcribe this text

[Assistant] She usually played nice girls.


[User] Please transcribe this text

[Assistant] Princesses, dancers, nuns.


[User] Please transcribe this text

[Assistant] But the girl in breakfast at Tiffany's is not nice.


[User] Please transcribe this text

[Assistant] She leaves her husband and goes to New York.


[User] Please transcribe this text

[Assistant] Audrey was also nervous because Truman Capote did not really want her in the movie. He wanted Marilyn Monroe. Maybe the part isn't right for me, she thought.


[User] Please transcribe this text

[Assistant] You're one of the five biggest actresses in the world. You can do anything!


[User] Please transcribe this text

[Assistant] It's time to do something different and dangerous.


[User] Please transcribe this text

[Assistant] Audrey made breakfast at Tiffany's in New York.


[User] Please transcribe this text

[Assistant] And it came to the movie theaters in 1961.


[User] Please transcribe this text

[Assistant] Audrey began to have problems after breakfast at Tiffany's. Mel was not happy.


[User] Please transcribe this text

[Assistant] She was a school teacher in The Loudest Whisper with Shirley MacLaine. But the movie was not successful. It was boring, people thought.


[User] Please transcribe this text

[Assistant] Then, in 1962, she made a very bad movie. Paris when it sizzles. William Holden was also in the movie. He fell in love with Audrey and Sabrina.


[User] Please transcribe this text

[Assistant] In the winter of 1962,


[User] Please transcribe this text

[Assistant] Chapter 10 My Fair Lady


[User] Please transcribe this text

[Assistant] Rex Harrison was Higgins. And Julie Andrews was Eliza Doolittle.


[User] Please transcribe this text

[Assistant] In the beginning, Rex Harrison was very unfriendly to Audrey.


[User] Please transcribe this text

[Assistant] Because Julie Andrews was not in the movie. He was also unhappy about money.


[User] Please transcribe this text

[Assistant] The newspapers also wanted to see Julie Andrews in the movie. Audrey felt sorry for Julie.


[User] Please transcribe this text

[Assistant] But what can I do? She thought. The producers want me.


[User] Please transcribe this text

[Assistant] She had to have singing lessons because she could not sing as well as Julie Andrews. One day the director said,


[User] Please transcribe this text

[Assistant] I'm sorry, Audrey. We'll have to use a different singer.


[User] Please transcribe this text

[Assistant] After this, she was difficult with everybody.


[User] Please transcribe this text

[Assistant] Everybody on the movie heard their fights. Sometimes, in front of the cameras, she started to cry.


[User] Please transcribe this text

[Assistant] My fair lady came to the movie theaters in October 1964.


[User] Please transcribe this text

[Assistant] But they liked her more in the second half. She was beautiful in her white hats and dresses. And when Professor Higgins hurt her, she acted wonderfully.


[User] Please transcribe this text

[Assistant] You could really feel it when she was angry.


[User] Please transcribe this text

[Assistant] My fair lady won eight Oscars, but Audrey did not win one.


[User] Please transcribe this text

[Assistant] Chapter 11. A Family Home.


[User] Please transcribe this text

[Assistant] Audrey left Hollywood as quickly as possible. She wanted to be near Mel.


[User] Please transcribe this text

[Assistant] So she moved to Spain with him for eight months. She wanted to be happy again and to stop fighting. She remembered when she was a child. Her father left her when she was six years old. Now her son, Sean, was the same age.


[User] Please transcribe this text

[Assistant] She then bought a house in a small town in the French part of Switzerland.


[User] Please transcribe this text

[Assistant] She had white English flowers in the yard, and Sean played with the other children.


[User] Please transcribe this text

[Assistant] For a short time, in 1965,


[User] Please transcribe this text

[Assistant] Audrey made a successful movie with Peter O'Toole. How to steal a million.


[User] Please transcribe this text

[Assistant] In 1966, she made Two for the Road.


[User] Please transcribe this text

[Assistant] This was a movie about the problems of married life. Audrey was very good in this movie