[User] Please transcribe this text

[Assistant] I'm just going to follow all the paths backwards.


[User] Please transcribe this text

[Assistant] So that I get both df dx and df dy.


[User] Please transcribe this text

[Assistant] Just by, you see the idea is to take the derivative of each step, each small step. That's really what we do in calculus. You know, if you think about the start of a calculus course.


[User] Please transcribe this text

[Assistant] What, what derivatives do we actually know? Do we actually use x, f at x plus delta x minus f? Do we, what derivatives do we grind out?


[User] Please transcribe this text

[Assistant] Every calculus book starts with x squared and finds the derivative of x to the n. Then you do sine x and cos x.


[User] Please transcribe this text

[Assistant] Then what others? Are there any more? Even e to the x, good, e to the x. And it's the inverse function log. In freshman calculus, you always write ln, just to be out of date.


[User] Please transcribe this text

[Assistant] Okay, and now that may be the list, is it? And then the chain rule. Are there others that you actually do a computation of?


[User] Please transcribe this text

[Assistant] Actually, e to the x is defined by the property that its derivative is e to the x.


[User] Please transcribe this text

[Assistant] Uh, and then you discover what log x has to be. And sine x, you, how do you do sine of x plus delta x? Well, compared, minus sine of x, how do you find the, you know, the hard way, once and for all way, you,


[User] Please transcribe this text

[Assistant] Draw a little unit circle and, and mess with some, some angles and you...


[User] Please transcribe this text

[Assistant] Discover that the derivative of the sine is the cosine.


[User] Please transcribe this text

[Assistant] That's if you've defined the sign from as a ratio of


[User] Please transcribe this text

[Assistant] A few ones that it has worked out in detail. But tangent of x, we would use the quotient rule. Secant of x, we would use the quotient rule, 1 over cosine.


[User] Please transcribe this text

[Assistant] And the products, we use the product rule. So really, calculus tends to seem fairly simple.


[User] Please transcribe this text

[Assistant] After, uh, when you look back to see what actually you did. And then integration. What is integral calculus about? More or less guessing the answer.


[User] Please transcribe this text

[Assistant] Right, you have to integrate f of x dx. And you have, so really what you have to do is sort of think, okay, what had this derivative? What function had that derivative?


[User] Please transcribe this text

[Assistant] And mess around and, and, uh, get it. So that's, really, it's, uh,


[User] Please transcribe this text

[Assistant] It's, it's a freshman course, I guess. Yeah, okay. So where am I?


[User] Please transcribe this text

[Assistant] Backward, right. That's the thing still to do. How does the backward system work? Okay, I'll try, I'll try my best. Okay, so here's the, here's the big goal. Back. So reverse mode AD.


[User] Please transcribe this text

[Assistant] A little notice to give you another example.


[User] Please transcribe this text

[Assistant] Where the order that you do the computations makes a big difference.


[User] Please transcribe this text

[Assistant] I mean, and that's not obvious that it will, right? There are many things in math that you could do in either order.


[User] Please transcribe this text

[Assistant] It seems like logically you've done the same thing. So another, another and simpler example.


[User] Please transcribe this text

[Assistant] Which shows how one way could be faster, way faster than another way, is when I'm multiplying two matrices, three matrices.


[User] Please transcribe this text

[Assistant] So I'm multiplying three matrices, A times B times C, and the question is,


[User] Please transcribe this text

[Assistant] Do I do BC first and then multiply by A? Or do I do AB first and then multiply that by C? And of course, I kept them in order.


[User] Please transcribe this text

[Assistant] In the order a, b, c, but the order of computations can be different.


[User] Please transcribe this text

[Assistant] You get the same, you get the right answer both ways but


[User] Please transcribe this text

[Assistant] Those can be completely, completely different. One can be a thousand times faster than the other.


[User] Please transcribe this text

[Assistant] So that's just to show, actually it kind of connects to this. And there's also another, so I'll do that too. And there's also, so this is example, example two, where this is meant to be example one. And example three would be, leads to something called the adjoint method in,


[User] Please transcribe this text

[Assistant] In, in differential equations, or in, in optimization, in, in computing, in, in, uh, computing, uh, optimum, in maximizing until, yeaah. And, uh, that has the same, really the underlying reason it gives a speed up is it makes the right choice in, in a product of three things. Yeaah.


[User] Please transcribe this text

[Assistant] So it'll be enough to do example one and example two. Okay, let me go with example one.


[User] Please transcribe this text

[Assistant] Our, our, this is now back propagation. Finally we got to it. Okay. So how do I, how do I, uh,


[User] Please transcribe this text

[Assistant] Well, I look at my notes. That's how I do it. Okay.


[User] Please transcribe this text

[Assistant] So the notes, this is section 7.2, does these computational graphs, and then here is reverse mode, uhm,


[User] Please transcribe this text

[Assistant] So it starts over here with the, so I'm going to use the chain rule. So df df is 1. And then, and then I going, going backwards.


[User] Please transcribe this text

[Assistant] To, and I have to use, of course, I have to take the derivatives.


[User] Please transcribe this text

[Assistant] I have to find, use the right rule. So I have to use the product rule.


[User] Please transcribe this text

[Assistant] And then soon I'll have to use these power rule and linear rule. So of course, no change there. The change is that by going backwards,


[User] Please transcribe this text

[Assistant] Oh, I don't know if I completed that sentence.


[User] Please transcribe this text

[Assistant] A hundred partial derivatives if the function depended on a hundred variables in about five times


[User] Please transcribe this text

[Assistant] The cost of one variable. Three, three to five times the cost of one. So, so that you would expect a hundred.


[User] Please transcribe this text

[Assistant] A hundred chain rules would cost a hundred times, but, but you see we're reusing the pieces in the chain and, uh, just having a larger, uh,


[User] Please transcribe this text

[Assistant] Our chain is wider, but it's not longer and it's not repeated anyway. So here I'm going to use


[User] Please transcribe this text

[Assistant] So wha- in my, yeaah, okay.


[User] Please transcribe this text

[Assistant] So, so df, dc is s, and df, ds is c. That was because, because, because f started out as c times s. It was a product. Okay, then we've got to evaluate those.


[User] Please transcribe this text

[Assistant] Now, I'll look again to see that I'm hopefully writing down some of the correct things.


[User] Please transcribe this text

[Assistant] Okay, so now what I've written down next is the FTC is 5.


[User] Please transcribe this text

[Assistant] Or no, five on the, on that example. What is it here? Df, dc?


[User] Please transcribe this text

[Assistant] x cubed, so df, oh sorry, df dc. Yeah, I want s. I'm looking for s here, yeah. I'm looking for s.


[User] Please transcribe this text

[Assistant] So I'm looking for S, and that's x plus 3y. Am I doing this well?


[User] Please transcribe this text

[Assistant] The derivatives with respect to x and y, the whole gradient. Okay, I think we're started right.


[User] Please transcribe this text

[Assistant] The first derivatives is to write c and s, and then let me leave these boxes open just to get the picture. Then I'm going to do, uh, then I'll need


[User] Please transcribe this text

[Assistant] And dSdy. I think that's right. Right?


[User] Please transcribe this text

[Assistant] Here I had a product of C and S, so I needed two, I had two derivatives. Here I have C and S each


[User] Please transcribe this text

[Assistant] Differentiate, so I get to have an x and a y derivative, x and a y derivative, and now it just, it's just a matter of putting in those numbers and following the chain backwards.


[User] Please transcribe this text

[Assistant] Maybe I'm not going to put those numbers in because if I didn't reach 140 you wouldn't believe in backpropagation and that would be like an unhappy outcome.


[User] Please transcribe this text

[Assistant] So I'll leave you to put them in maybe. Uh, or the notes have a separate example that you can see. But do you see the point that I'm going to find all the


[User] Please transcribe this text

[Assistant] In the end I'm going to find df dx and df dy.


[User] Please transcribe this text

[Assistant] From the, from the chain, all in, from one chain and not, not, not from a separate chain for x and a separate chain for y. That's, that's, to me that's the point of reverse mode. It's a little bit of magic.


[User] Please transcribe this text

[Assistant] Steps, the ingredient. And some of you know, have seen this before and maybe know a better exposition. I found this blog by Christopher Olah clear and these very simple things you'll see are clear in the notes. But maybe another blog makes, brings out other


[User] Please transcribe this text

[Assistant] Uh, points to make here. It's not obvious maybe that I could have a hundred variables and, and, and do the calculation in four or five times the, the, uh, cost for four or five times being instead of a hundred. Yeah. But it's possible. Okay.


[User] Please transcribe this text

[Assistant] So, can I close today with this one?


[User] Please transcribe this text

[Assistant] How could those be different? You're computing the same numbers, the same aij, bjk, ckl, and doing the sums, but it certainly is different. So let's just do that. Okay, I'll do it here.


[User] Please transcribe this text

[Assistant] After my time, and I guess it'll be after Professor Rao on Monday, on Friday and Monday, I'll come back to, uh, Professor Sra's, uh, short proof.


[User] Please transcribe this text

[Assistant] of the convergence of stochastic gradient descent.


[User] Please transcribe this text

[Assistant] What assumptions do you need? You need some assumptions on the


[User] Please transcribe this text

[Assistant] Gradient, some assumptions on the step size, and they, if, for a good proof, all the assumptions fit together and dong, out comes the conclusion. And the conclusion would be how fast it converges. A stochastic gradient descent. So there's some, there's some x,


[User] Please transcribe this text

[Assistant] Expected things, because it's stochastic.


[User] Please transcribe this text

[Assistant] We expect some assumptions about the mean and the variance to go into the, into the proof.


[User] Please transcribe this text

[Assistant] So you'll see that, but maybe it's too much for today. So for today I'll just, so I'll come back to that.


[User] Please transcribe this text

[Assistant] I might even put it on, uh, Stellar and, uh,


[User] Please transcribe this text

[Assistant] Just close with this. So suppose A is m by n.


[User] Please transcribe this text

[Assistant] Steps does it take to find A times B times C, the product of those three matrices? Well, if I go this way.


[User] Please transcribe this text

[Assistant] I have to multiply a, I have to do bc first, so bc.


[User] Please transcribe this text

[Assistant] Cross, how many operations to multiply that times that?


[User] Please transcribe this text

[Assistant] NPQ, nice formula, NPQ. Why is that?


[User] Please transcribe this text

[Assistant] Well, I could say that the answer is, is n by q.


[User] Please transcribe this text

[Assistant] And every number in there was an inner product.


[User] Please transcribe this text

[Assistant] Uh, the row and column of length p. So I have nq inner product.


[User] Please transcribe this text

[Assistant] And each one costs p, uh, multiply add. And then, so now, now I have bc, which will be, now, so it's, so now I have m by n, then, then I have m by n, which is the a.


[User] Please transcribe this text

[Assistant] Times b by c, which is now n by q. That's bc. This is abc. So this one costs


[User] Please transcribe this text

[Assistant] What's the cost here? m by n, n by q, by the same rule, it'll be m, n, q. Good.


[User] Please transcribe this text

[Assistant] That's the first way, A times BC. Now the second way is AB times C.


[User] Please transcribe this text

[Assistant] Let me write in again, m by n, n by p, p by q.


[User] Please transcribe this text

[Assistant] So now I'm doing this first, so a, b.


[User] Please transcribe this text

[Assistant] Costs, tell me again now what's the rule for the cost of a matrix multiplication. MNP, MNP.


[User] Please transcribe this text

[Assistant] I'm multiplying M by P, that's AB, times P by Q, that's C. So I have M, P, Q.


[User] Please transcribe this text

[Assistant] That sum, those two or, or these two.


[User] Please transcribe this text

[Assistant] And let's just, let's just recognize the most important example.


[User] Please transcribe this text

[Assistant] Suppose C is a column vector, C for a column vector. So Q is 1, there's only one column. So if Q is 1,


[User] Please transcribe this text

[Assistant] This way did NP, let's just specialize to that.


[User] Please transcribe this text

[Assistant] To c equal a column vector.


[User] Please transcribe this text

[Assistant] Which means that q is 1. I only have one column. So then A times BC is versus


[User] Please transcribe this text

[Assistant] AB times C. So let's just figure that out when Q is 1. So NPQ is just NP. And MNQ is just MN.


[User] Please transcribe this text

[Assistant] Where A B is M and P. Oh, that's a bad one. Disaster already. Multiplying, those are potentially two big matrices. Multiplying a column vector.


[User] Please transcribe this text

[Assistant] So here I've done a matrix multiplication. I never should have done that. This is a matrix vector. It gives me a vector. And then this is a matrix vector. So I get nice numbers here.


[User] Please transcribe this text

[Assistant] But I get a terrible number for a, b. And then I multiply that by c. So that's mpq.


[User] Please transcribe this text

[Assistant] M that's factoring out times


[User] Please transcribe this text

[Assistant] Let me, let me do just this thinking again. If q is 1, if I go this way, was that my final total when q was 1? And that's this? That's no. m factors out times n plus p. Look, let's just get that right. This is


[User] Please transcribe this text

[Assistant] Oh no, n factors out, sorry, n factors out times m plus p. And this way was, was all these things.


[User] Please transcribe this text

[Assistant] The whole point is, we've got this horrible multiplication of three big numbers. And this only had


[User] Please transcribe this text

[Assistant] Two big numbers. So this was, this is orders of magnitude.


[User] Please transcribe this text

[Assistant] Faster than that. And of course, you would have done the calculation that way. You would have multiplied the column vector by a matrix.


[User] Please transcribe this text

[Assistant] You crazily multiplied two big matrices together.


[User] Please transcribe this text

[Assistant] And then got a column vector. So it was the, there's the bad move. Okay, thanks. So we're past the time on this ABC. It's just to show that on a very familiar calculation, you have to do it in the right order. And back propagation is the right order for partial derivatives. Okay, thank you. And so bring laptops, uh, Friday.


[User] Please transcribe this text

[Assistant] And, uh, look forward to, to, uh, Professor Rao. Give him a good welcome.


[User] Please transcribe this text

[Assistant] Yes, they call me that because I am always dressed in green. But my real name is Eugene.


[User] Please transcribe this text

[Assistant] Come to give you my daughter's greeting. I've never met you or your daughter before.


[User] Please transcribe this text

[Assistant] But you are very welcome to stay in my palace for the night. The sun is setting down, and...


[User] Please transcribe this text

[Assistant] For you to go back through the forest now is not advisable.


[User] Please transcribe this text

[Assistant] Please carry this gift to your daughter. It contains my portrait, and when I come to see her, she will know me.


[User] Please transcribe this text

[Assistant] As you wish, but I cannot stay away from you any longer. Until the wedding day, I will meet you every day.


[User] Please transcribe this text

[Assistant] I will come early in the morning and leave when it is dark.


[User] Please transcribe this text

[Assistant] That way, your stepmother will never find out about us.


[User] Please transcribe this text

[Assistant] I am sorry, but that is the one thing I cannot grant.


[User] Please transcribe this text

[Assistant] My heart belongs to the most beautiful girl in the world, and she waits for me in her island palace.


[User] Please transcribe this text

[Assistant] I can't imagine being with the same person for 25 years. After my breakup with Peter, I don't know if I could get along with someone for that long. Do you ever have problems? Or argue?


[User] Please transcribe this text

[Assistant] I hope so. What about you, Luke? You have been seeing someone for a few months now. Any chance you might pop a question soon?


[User] Please transcribe this text

[Assistant] I thought it was love at first sight. He was exactly my type. He was tall, brunette, and athletic. I fell for him immediately.


[User] Please transcribe this text

[Assistant] I told him I would go out with him again if he promised to be faithful to me. We made up and things went well for a few weeks.


[User] Please transcribe this text

[Assistant] But then... Oh no. What happened?


[User] Please transcribe this text

[Assistant] That's terrible. I know. This time I broke up with Peter for good. I refuse to be in an on and off again relationship. I am still so upset with him and I am very mad at my friend. I worry I will never get over it. I really thought he was the one, but it seems it was not meant to be though.


[User] Please transcribe this text

[Assistant] Thanks, Beth. You were very nice, but I don't like going out on blind dates. I prefer to know my date.


[User] Please transcribe this text

[Assistant] I met one nice guy on the site, but he lives six hours away. And I'm not interested in a long distance relationship.


[User] Please transcribe this text

[Assistant] It's nice to meet you, Robert. My name is Maria. Welcome to the team. Thanks. We were announced that you would be joining our team. You're from Canada, right?


[User] Please transcribe this text

[Assistant] Well, I'm sure you're going to feel great here and make a lot of friends. We have a lovely team here. Come on, let me introduce you to our colleagues.


[User] Please transcribe this text

[Assistant] The grumpy old man. Once upon a time, in the city of Mowark, there lived an old man called Mr. Ray.


[User] Please transcribe this text

[Assistant] And blame everyone around him but himself for all the wrong that would happen in his life.


[User] Please transcribe this text

[Assistant] As he started reading the newspaper he heard someone call. Excuse me?


[User] Please transcribe this text

[Assistant] The man was really an angel, and to prove himself, he snapped his fingers and a pair of beautiful silver boots appeared out of thin air. Mr. Ray was shocked.


[User] Please transcribe this text

[Assistant] The thought of gold, silver and diamonds tempted him, and he accepted at once.


[User] Please transcribe this text

[Assistant] Saying that, the angel disappeared. Mr. Ray picked up the boots and went home.


[User] Please transcribe this text

[Assistant] The next day when he woke up, he heard the screaming of his neighbor, the old Mrs. Kyle.


[User] Please transcribe this text

[Assistant] Hearing her scream, Mr. Ray got very angry and remembered about the pair of silver boots.


[User] Please transcribe this text

[Assistant] This made her very angry, and she screamed out loud.


[User] Please transcribe this text

[Assistant] And within a second, Mr. Ray was back in his own house.


[User] Please transcribe this text

[Assistant] A teardrop rolled down his eyes. He immediately wiped it off.


[User] Please transcribe this text

[Assistant] As he reached the park, he sat on his usual spot and opened his newspaper, when he heard little Tim's voice again. Excuse me, sir.


[User] Please transcribe this text

[Assistant] At this, tears rolled out of his eyes as he realized he was wrong again.


[User] Please transcribe this text

[Assistant] Within a second, he was back at the park, sitting on the bench.


[User] Please transcribe this text

[Assistant] He looked up and saw little Tim walking away with his head hung low. He got up and ran after him.


[User] Please transcribe this text

[Assistant] Delighted, little Tim kissed him on his cheek and gave him all the cookies that he had.


[User] Please transcribe this text

[Assistant] That very day, Mr. Ray went to Mrs. Kyle's house and gave her all the cookies.


[User] Please transcribe this text

[Assistant] And thus, Mr. Ray went on from being a very angry man to a kind-hearted and generous person. He never shouted at anyone ever again. Tim and him became best of friends


[User] Please transcribe this text

[Assistant] And as for the pair of boots, well, he kept them right there in his living room as a reminder of the fact that it's very important to be empathetic and before judging anyone, we should always step into their shoes.


[User] Please transcribe this text

[Assistant] And so the three became best friends. Everyone has something they are very good at and something they are not so good at. So we must neither be egotistical about our talents nor depressed about our weaknesses.


[User] Please transcribe this text

[Assistant] Where does all my snot come from? It comes from the mayo you eat. Nah.


[User] Please transcribe this text

[Assistant] Snot is basically produced by the mucous membrane present in our nose.


[User] Please transcribe this text

[Assistant] Firstly, the snot contains proteins called mucins. Mucins give our snot its gel-like properties, helping it trap germs, dust, etc., and thus preventing them from entering deeper into our body. Also, sometimes bacteria clump together, forming a harmful biofilm.


[User] Please transcribe this text

[Assistant] The mucins found in our snot bind to these bacteria and increase their motility. As a result, bacteria don't clump together, rendering them less dangerous. Lastly, snot also contains antibodies, antimicrobial peptides, etc. Which help kill germ, thus keeping us healthy.


[User] Please transcribe this text

[Assistant] Why does sunlight make you sneeze? Cuz sun teases our nose.


[User] Please transcribe this text

[Assistant] No. Wait, I'll explain. Sneezing caused by sunlight or bright light is called photic sneezing.


[User] Please transcribe this text

[Assistant] When bright light suddenly stimulates the optic nerve. The optic nerve mistakenly transmits some signals to the nearby trigeminal nerve. Hence, the trigeminal nerve misinterprets this signal as an irritant in the nose and thus, triggers sneezing.


[User] Please transcribe this text

[Assistant] A sneeze is controlled by the trigeminal nerve present in our face. When irritants like dust and pollen enter our nose, the trigeminal nerve triggers sneezing to expel them out with huge force and pressure.


[User] Please transcribe this text

[Assistant] Normally, sneeze should not be stopped, but in exceptional situations. A relatively safe way to stop a sneeze is to press our finger above our upper lip.


[User] Please transcribe this text

[Assistant] Since the trigeminal nerve is also present above our upper lip. Such pressing action also sends a signal through that nerve to the brain. This interrupts the sneeze response and distracts the brain. Thus helping in stopping our sneeze.


[User] Please transcribe this text

[Assistant] Our nose consists of a mucus membrane which secretes a slimy substance called mucus. However, sometimes, this membrane produces excess mucus which comes out of our nostril.


[User] Please transcribe this text

[Assistant] Mucus also contains antibodies and immune cells which kill germs and other foreign particles.


[User] Please transcribe this text

[Assistant] Besides this, Eucas even warms and moisturizes the inhaled air. Thus preventing our breathing passage from getting damaged. Topic Human Nose. Why does spicy food make your nose run?


[User] Please transcribe this text

[Assistant] Because it is not that strong to make me run. Heh, nah.


[User] Please transcribe this text

[Assistant] Now, spicy food items like chili, mustard, horseradish and wasabi contain chemicals called capsaicin and allyl isothiocyanate.


[User] Please transcribe this text

[Assistant] Thanks to Linode Cloud Computing for supporting this episode of SciShow. Go to Linode.com slash SciShow to learn more and get a $100 60-day credit on a new Linode account.


[User] Please transcribe this text

[Assistant] Observe a candle. That little cotton wick. It is burning. And then, after producing heat and light for a number of hours, nothing is left. The candle is gone.


[User] Please transcribe this text

[Assistant] And that can be pretty confusing. But like, we all get that something is going on here. No one would expect to light a 10-inch long cotton string on fire and have it still be burning in an hour. So where did the candle wax go? Here's the short answer. You burned it.


[User] Please transcribe this text

[Assistant] Here's the long version. Wax is a fuel, like the gas in your car's gas tank. But it is an extremely convenient fuel. This is hard to believe if you're used to light bulbs, but at one point, candles were a tremendous innovation. Specifically, they were an innovation over oil lamps. The oil used in lamps was usually plant oil, often olive oil or seed oils.


[User] Please transcribe this text

[Assistant] The paintings in the cave are from around 17,000 years ago. And to make an oil lamp work, a wick would need to be laid in a shallow dish, and that dish was then filled with oil. Now, oil by itself...


[User] Please transcribe this text

[Assistant] It doesn't always burn easily. That's what the wick is for. The wick draws the oil up and increases its surface area enough that in the presence of a flame, a lot of it will vaporize into a gas.


[User] Please transcribe this text

[Assistant] Then, it can freely mix with oxygen, and all the energy added by the flame allows it to combust. Combustion is a chemical reaction in which a carbon-containing molecule and oxygen from the air are converted into carbon dioxide and water, and sometimes other products. In the case of oils, those carbon-containing molecules are fatty acids. In the case of modern candles, it's waxes.


[User] Please transcribe this text

[Assistant] Into its own lamp. By putting the wick into a solid fuel and then building both the wick and the candle carefully, most of the fuel in your basic taper candle remains solid until it gets close to the flame. The wax near the flame melts and then is drawn up into the wick where it heats up further


[User] Please transcribe this text

[Assistant] Wax molecules react with oxygen to form carbon dioxide and water. And that…


[User] Please transcribe this text

[Assistant] So it cools down and then condenses back into tiny solid wax droplets. So unburned wax is what you see and what you smell after you blow out a candle. This is also why you can hold a match to that stream of wax, and the flame will chase the trail of vaporized and condensed fuel back down to the wick.


[User] Please transcribe this text

[Assistant] It becomes air. It's weird, and it's cool. And now when people ask you, where does the candle wax go, you can tell them. Or you can just send them here. I know that's what I'll be doing. So candles become air, and


[User] Please transcribe this text

[Assistant] Water in the air becomes clouds and it's all kind of weird and amorphous.


[User] Please transcribe this text

[Assistant] To help keep your cloud infrastructure up to date and secure. A lot of people leave their computers vulnerable just because updating them is such a hassle. So Linode's guide shows you how to automate that process, and rest easy knowing your computer's security systems are up to date. To try it yourself, click the link in the description or head to Linode.com slash SciShow. That link gives you a $100, 60-day credit on a new Linode.


[User] Please transcribe this text

[Assistant] Picture the most intense competition you've ever seen. Maybe it was a rivalry football game or the final round in an Olympic archery match. Whatever you're picturing, the competitors probably don't look cheerful. They might look serious, angry, or even hostile. In other words, they've got their game face on.


[User] Please transcribe this text

[Assistant] The idea here is that a laser-focused expression can disarm opponents and make them feel uneasy and anxious. But according to psychology research, it can also do more than that.


[User] Please transcribe this text

[Assistant] Putting on your game face may help your performance, too. It's maybe not that surprising that many people avoid angry faces. But the cool thing is, those reactions are measurable, so we can get a sense of exactly how these expressions affect us and why. Like, in a 2018 study,


[User] Please transcribe this text

[Assistant] Researchers showed 37 people images of faces that were happy, neutral, or angry.


[User] Please transcribe this text

[Assistant] They closely measure the subject's electric dermal activity. Electric dermal activity can mean a couple of things. Either the electrical output of nerves, or how electric current travels as it passes over our skin. But basically, it's a way to see how much someone is sweating, and by extension,


[User] Please transcribe this text

[Assistant] That's because fear and anxiety flood your system with hormones that


[User] Please transcribe this text

[Assistant] It's part of the fight-or-flight response that helps protect you from danger, and part of that response is sweat. In the study, those who saw an angry face produced almost one and a half times more electrodermal activity than when they were exposed to a neutral one. So, the faces seemed to stress them out. And beyond that,


[User] Please transcribe this text

[Assistant] Researchers also found that the expression people were shown was related to the amount of personal space they needed. It varies a lot by person, but on average, studies suggest we tend to prefer a roughly 30 centimeter bubble around us. But in this study, people who saw an angry face said their comfort zone was about 30% bigger on average


[User] Please transcribe this text

[Assistant] Based on all this, you might think that a game face is definitely the way to go. Like, if you have a chance to stress out your opponent and keep them away from you, cool.


[User] Please transcribe this text

[Assistant] Things are more complicated than what we see in the lab. Like, stress is an important factor in sports performance, but it also affects people differently based on their personality and background.


[User] Please transcribe this text

[Assistant] A little stress might be motivating, but for others, the same amount will be totally overwhelming. Everyone's comfort zone is different and depends on a bunch of factors, including stress management and experience. Genetics even plays a role, since all kinds of genes are related to fear and anxiety. So putting on a game face might be a double-edged sword.


[User] Please transcribe this text

[Assistant] That doesn't mean you should retire your well-practiced scout.


[User] Please transcribe this text

[Assistant] Because studies also show that putting one on can help you perform better.


[User] Please transcribe this text

[Assistant] At least, on some tasks. This kind of connection between facial expression and physiology is called facial feedback hypothesis. The idea is that no matter what you're feeling,


[User] Please transcribe this text

[Assistant] The physical act of expressing an emotion will cause your body to emulate that. And it can affect how you feel, although how much depends on the situation. Now, evidence for this hypothesis is pretty mixed.


[User] Please transcribe this text

[Assistant] So not all psychologists are on board with it. But some researchers do think that it may be related to why a game face can help us out. For instance, take this study. In a 2019 paper,


[User] Please transcribe this text

[Assistant] A group of researchers had 62 participants do various challenges. In one, they were basically put through a perfectionist nightmare. They were told to put together a hundred-piece black-and-white puzzle.


[User] Please transcribe this text

[Assistant] The key was, before they started, some participants were shown examples of a game face from pro athletes and were asked to replicate it during the challenge. Other participants weren't given any instructions about facial expressions. In the end,


[User] Please transcribe this text

[Assistant] The second group completed about 7 pieces on average. But those who got their game face on completed an average of 11 pieces, about 60% more. As for why, the researchers proposed that this could be another application of the facial feedback hypothesis. Except instead of facial expressions triggering a certain emotion,


[User] Please transcribe this text

[Assistant] See, over time, your brain learns to group different things and sensations, which helps it sort new information faster and easier. And this includes facial expressions. So in this study, subjects' brains may have associated their game face with accomplishing a hard task.


[User] Please transcribe this text

[Assistant] The game face could have been signaling to their brains that they needed to pay more attention and work harder. Now when it comes to personal performance in physical challenges like sports,


[User] Please transcribe this text

[Assistant] But there is some support for them. Like, one study looked at more than 4,000 images of World Cup soccer players from over 300 teams. And it found that players who wore angry expressions gave up fewer goals. It's hard to say why without more controlled studies, but some researchers have suggested that anger can be helpful for physical tasks. Specifically, if the task is something you might naturally do while angry.


[User] Please transcribe this text

[Assistant] I don't know, kick something really, really hard. Though again, we'll need more detailed studies to know for sure.


[User] Please transcribe this text

[Assistant] The next time you're getting ready for the big game, or the next time you're preparing for a big recital or presentation, maybe give it a try. It might just give you the edge you need.


[User] Please transcribe this text

[Assistant] We make decisions that will profoundly influence the lives of the people we're going to become.


[User] Please transcribe this text

[Assistant] And then when we become those people, we're not always thrilled with the decisions we made. So young people pay good money to get tattoos removed that teenagers paid good money to get. Uh, middle-aged people rush to divorce people who young adults rushed to marry. Uh, older adults work.


[User] Please transcribe this text

[Assistant] Hard to lose, what middle-aged adults worked hard to gain, on and on and on. The question is, as a psychologist, that fascinates me is, why do we make decisions that our future selves so often regret?


[User] Please transcribe this text

[Assistant] But what is the name of this magical point in life where change suddenly goes from a gallop to a crawl? Is it teenage years? Is it middle age? Is it old age? The answer, it turns out, for most people is now. Wherever now happens to be.


[User] Please transcribe this text

[Assistant] What I want to convince you today is that all of us are walking around with an illusion. An illusion that history, our personal history, has just come to an end. That we have just recently become the people that we were always meant to be and will be for the rest of our lives.


[User] Please transcribe this text

[Assistant] So how does it do so? Well, we asked thousands of people. We asked half of them to predict for us how much their values would change in the next ten years.


[User] Please transcribe this text

[Assistant] And the other is to tell us how much their values had changed in the last 10 years. And this enabled us to do a really interesting kind of analysis. Because it allowed us to compare the predictions of people, say, 18 years old, to the reports of people who were 28. And to do that kind of analysis throughout the lifespan. Here's what we found. First of all, you're right.


[User] Please transcribe this text

[Assistant] To give you an idea of the magnitude of this effect, you can connect these two lines and what you see here is that 18 year olds anticipate changing only as much as 50 year olds actually


[User] Please transcribe this text

[Assistant] It's not just values, it's all sorts of other things. For example, personality.


[User] Please transcribe this text

[Assistant] Neuroticism, openness to experience, agreeableness, extroversion, and conscientiousness. Again, we ask people how much they expected to change over the next 10 years, and also how much they had changed over the last 10 years, and what we found. Well, you're gonna get used to seeing this diagram over and over, because once again, the rate of change does slow as we age,


[User] Please transcribe this text

[Assistant] And it isn't just ephemeral things like values and personality.


[User] Please transcribe this text

[Assistant] You can ask people about their likes and dislikes, their basic preferences. For example,


[User] Please transcribe this text

[Assistant] People can name these things. We ask half of them to tell us, do you think that that will change over the next 10 years? And half of them to tell us, do you did that change over the last 10 years? And what we find, well, you've seen it twice now and here it is again.


[User] Please transcribe this text

[Assistant] People predict that their friend, the friend they have now is the friend they'll have in ten years, the vacation they most enjoy now is the one they'll enjoy in ten years, and yet, people who are ten years older all say, eh, you know, that's really


[User] Please transcribe this text

[Assistant] Does any of this matter? Is this just a form of misprediction that doesn't have consequences? No, it matters quite a bit and I'll give you an example of why. It bedevils our decision making in important ways. Bring to mind right now for yourself your, your favorite musician today and your favorite musician 10 years ago. I put mine up on the screen to help you along.


[User] Please transcribe this text

[Assistant] We asked people to predict for us to tell us how much money they would pay right now to see their current favorite musician perform in concert 10 years from now. And on average, people said they would pay $129 for that ticket.


[User] Please transcribe this text

[Assistant] Why does this happen? We're not entirely sure, but it probably has to do with the ease of remembering versus the difficulty of imagining.


[User] Please transcribe this text

[Assistant] And then we mistakenly think that because it's hard to imagine, it's not likely to happen. Sorry, when people say, I can't imagine that, they're usually talking about their own lack of imagination, and not about the unlikelihood of the event that they're describing.


[User] Please transcribe this text

[Assistant] The bottom line is, time is a powerful force. It transforms our preferences, it reshapes our values, it alters our personalities. We seem to appreciate this fact, but only in retrospect. Only when we look backwards do we realize how much change happens in a decade.


[User] Please transcribe this text

[Assistant] Human beings are works in progress that mistakenly think they're finished.


[User] Please transcribe this text

[Assistant] The person you are right now is as transient, as fleeting, and as temporary as all the people you've ever been. The one constant in our life is change. Thank you.


[User] Please transcribe this text

[Assistant] School is okay. It's almost over, so I'm looking forward to being done with it.


[User] Please transcribe this text

[Assistant] Thanks dad. I'm still trying to sort out my plans for after graduation.


[User] Please transcribe this text

[Assistant] I'm not even sure if I can get in a good college or hold down a job.


[User] Please transcribe this text

[Assistant] You know I love gaming. If I could game all day, I would just do that.


[User] Please transcribe this text

[Assistant] I'll ask Mr. Watson at school tomorrow if he can figure out how to get me involved in this career path. That would be awesome.


[User] Please transcribe this text

[Assistant] Thanks for watching and be sure to subscribe to our YouTube channel for more videos like this.


[User] Please transcribe this text

[Assistant] You only carry the master.


[User] Please transcribe this text

[Assistant] Come on now, stop whining and go to sleep! Nighty night!


[User] Please transcribe this text

[Assistant] To begin with, it's a very exciting job because you get to travel to many amazing places.


[User] Please transcribe this text

[Assistant] For example, you could be in Mexico one day and in Australia the next.


[User] Please transcribe this text

[Assistant] You can visit the outermost corners of the world while getting paid for that. Isn't that amazing?


[User] Please transcribe this text

[Assistant] However, there are also some disadvantages to this job that I have to consider. Firstly, it is a stressful job.


[User] Please transcribe this text

[Assistant] Sometimes you have to deal with all kind of difficult situations.


[User] Please transcribe this text

[Assistant] In addition, it is a tiring job because flight attendants often work long hours.


[User] Please transcribe this text

[Assistant] Apart from this, you are required to work on holidays too.


[User] Please transcribe this text

[Assistant] To conclude, I believe that, although being a flight attendant can be challenging at times, it is worth it.


[User] Please transcribe this text

[Assistant] Yes, I need a bar of chocolate, a carton of milk, and a dozen eggs.


[User] Please transcribe this text

[Assistant] Great! Can you do me a favor? Can you go to the pharmacy too? And get me some aspirin? I have a headache.


[User] Please transcribe this text

[Assistant] You're right. I had to do many projects for work. And I was very stressed about them.


[User] Please transcribe this text

[Assistant] So I worked very carefully to do a perfect job. I always make sure there are no mistakes.


[User] Please transcribe this text

[Assistant] I know. I did my best to make a good impression on my boss.


[User] Please transcribe this text

[Assistant] My house was broken into last night while I was sleeping. Someone got inside through the window.


[User] Please transcribe this text

[Assistant] Yes, I'm fine. Luckily I wasn't hurt.


[User] Please transcribe this text

[Assistant] Luckily my car had been taken to the service department to get the paint fixed. So the thief couldn't steal it.


[User] Please transcribe this text

[Assistant] Yes, I did. They came and investigated the matter. I was asked a few questions by the police.


[User] Please transcribe this text

[Assistant] Well, surprisingly they do. Some fingerprints were found.


[User] Please transcribe this text

[Assistant] And the thief has been identified by the police computer.


[User] Please transcribe this text

[Assistant] Yes, he has been arrested and taken into police custody, and now is being questioned.


[User] Please transcribe this text

[Assistant] Thanks for watching, and be sure to subscribe to our YouTube channel for more videos like this.


[User] Please transcribe this text

[Assistant] Tell me child, what do you want? I will grant any wish you have.


[User] Please transcribe this text

[Assistant] But remember, you'll have to keep him busy all the time. He must never sit idle.


[User] Please transcribe this text

[Assistant] We are about to start, your majesty.


[User] Please transcribe this text

[Assistant] I know that he is from a good family and a very daring being.


[User] Please transcribe this text

[Assistant] Hey Marcus, how's the job search going? Not so great actually. It's really frustrating. Why is that?


[User] Please transcribe this text

[Assistant] I've submitted my resume and applications to over ten places and I've only gotten one interview.


[User] Please transcribe this text

[Assistant] It was okay. I was nervous, and I think I spoke really fast.


[User] Please transcribe this text

[Assistant] Are there any other places hiring? There are a few left on my list. I'm still going to apply to as many jobs as possible, but there are a lot of other people looking for the same jobs. There's a lot of competition these days for jobs in business.


[User] Please transcribe this text

[Assistant] I'll keep asking around to see if any of my other friends have openings in their companies. Thanks, I appreciate it.


[User] Please transcribe this text

[Assistant] Yeah, that's a big plus. Thanks, I appreciate the support. How about you guys? How's your work going?


[User] Please transcribe this text

[Assistant] I guess you can't really have your cake and eat it too. Sometimes you have to make a few compromises


[User] Please transcribe this text

[Assistant] How about you, Nate? My work has become topsy turvy these days. What exactly happened?


[User] Please transcribe this text

[Assistant] It's an accountant position, and I've got an interview tomorrow. If I get the job, I'll be thrilled. It's exactly what I've been looking for.


[User] Please transcribe this text

[Assistant] I have a degree in finance and I am fluent in English and Spanish. I worked as a part-timer in college for a startup agency. I translated financial documents for them.


[User] Please transcribe this text

[Assistant] Being a fresher, I think I'm very flexible and adaptive to learning new things. I'm a hardworking person and a fast learner. And I get along fine with people. I'm highly organized and respectful.


[User] Please transcribe this text

[Assistant] In college, I took quite a few courses each semester, while working at least 20 hours every week. I handled that situation pretty well.


[User] Please transcribe this text

[Assistant] Thank you. It has been a great pleasure coming here and talking to you.


[User] Please transcribe this text

[Assistant] Hello, I'm Marcus Brown. I was hired as part of the finance team.


[User] Please transcribe this text

[Assistant] Yes I did. I came by car. But it was a bit of a problem parking the car on the street.


[User] Please transcribe this text

[Assistant] Welcome Marcus! We're glad to have you. Thanks! Pleased to meet you. I'm excited to be here.


[User] Please transcribe this text

[Assistant] While I was searching for a job, I came upon this exciting new opportunity. My team is so supportive.


[User] Please transcribe this text

[Assistant] I feel like everyone gets along well and helps each other out whenever it's needed. It's a really friendly environment.


[User] Please transcribe this text

[Assistant] So, when can we expect a big treat? Whenever you say. How about lunch at Falcon's Nest restaurant on Sunday? That sounds good. I will see you on Sunday then.


[User] Please transcribe this text

[Assistant] Hello, my name is Alice White. I'm the sales manager. How can I help you?


[User] Please transcribe this text

[Assistant] The skin allergy may be caused by something else. Like some food, for example.


[User] Please transcribe this text

[Assistant] Our products are made according to the highest industry standards.


[User] Please transcribe this text

[Assistant] We have our own laboratory where we do various tests to ensure the best quality of our products and services. Your company is cheating its customers!


[User] Please transcribe this text

[Assistant] We are not cheating our customers. Our products are not harmful in any sense.


[User] Please transcribe this text

[Assistant] We'll also pay for a consultation with the dermatologist. Is that okay with you?


[User] Please transcribe this text

[Assistant] Are there any breakables? Is it fragile? No. Okay, let's see how much it weighs. It's about seven pounds. What is the address? I have it here. Please fill out this customs declaration form. Okay, when will my package arrive? It depends on your shipping choice.


[User] Please transcribe this text

[Assistant] If you send it express, it will get there in 5 to 10 business days.


[User] Please transcribe this text

[Assistant] Then you can choose the priority, which is $43.17. There is also first class, which is the cheapest way to mail the package.


[User] Please transcribe this text

[Assistant] It costs only $13.19, but it will get there in 2-8 weeks. Then it doesn't have a tracking number or insurance.


[User] Please transcribe this text

[Assistant] Okay, your total comes to $50.35. How would you like to pay? With my credit card. Is there anything else I can help you with today? No, that's it.


[User] Please transcribe this text

[Assistant] Thank you. Here's your receipt. Your tracking number is on the receipt. Have a nice day. You too.


[User] Please transcribe this text

[Assistant] Where can I get a used one that's in good condition? I actually have a friend who is selling hers. It's in perfect condition. There's nothing wrong with it.


[User] Please transcribe this text

[Assistant] And I think she bought it a year ago, so it's not that old. She just wants a different brand. Really? You're a lifesaver!


[User] Please transcribe this text

[Assistant] Uh, helped a little bit to teach this class last year.


[User] Please transcribe this text

[Assistant] We're happy to see that it's going great this year. So, uh, Professor Strang came to teach 1806, some of you may know, uhm, the introductory linear algebra course and Professor Strang came, uh, by and gave this


[User] Please transcribe this text

[Assistant] Uh, demonstration about the row rank equals the column rank, and I'm wondering if you did that in this class at any time, or would they have seen that?


[User] Please transcribe this text

[Assistant] It's in the notes. Well, in any event, uhm, just as Professor Strang walked out,


[User] Please transcribe this text

[Assistant] So here, I'll just grab this. This is Julia. I was actually going to start writing the code to do the quadrilateral, but I didn't have enough time. You could see me starting. But here's the point five for the triangle, which was easy. So what's the story with Julia in this class? Have they used it a little or a lot? In the labs you've used Julia, but that was just Matlab. So that was, uh,


[User] Please transcribe this text

[Assistant] But okay, so uhm, uh, so, so uhm, Professor Strang showed this proof where he would, he put down a three by three matrix, it had rank two, and he showed the, you know, took the columns that were, you know, the first two columns were independent and, and was easy, you know, it was easy to show the row rank equals the column rank.


[User] Please transcribe this text

[Assistant] After Professor Strang went out, I asked, would that work for the zero matrix?


[User] Please transcribe this text

[Assistant] So uhm, so here's the, the, the zero matrix and since I'm not really telling you the proof, I'll just say if I were to put


[User] Please transcribe this text

[Assistant] If I were to make a matrix of the literally independent columns of this matrix, what would I do?


[User] Please transcribe this text

[Assistant] So, zero sort of a, might seem tricky, it's not really that tricky, but.


[User] Please transcribe this text

[Assistant] This is, this is what I did the moment you walked out. So, uh, so yeaah. So I've got this three by three matrix and I need to make a, uh, the first step in, in whatever the proof was, I needed to take the columns of, of, the, the, the linearly independent columns of this matrix and place them in, in a matrix of their own. What would I do? It would be an empty matrix. What would be the size of this empty matrix?


[User] Please transcribe this text

[Assistant] Where every, every column is still in R3, you see. So the right answer, I hope this makes sense, is a 3 by 0 empty matrix.


[User] Please transcribe this text

[Assistant] And that's a concept that exists in Matlab and in Julia and in Python, I think, I'm sure, and in any computer language. So, if you had a full rank 3x3 matrix, the linearly independent columns would be 3x3. If you had a rank 2, it would be 3x2. If I had a rank 1 matrix, it would be 3x1. So if I had no columns, 3x0 makes sense.


[User] Please transcribe this text

[Assistant] And to finish the proof, again I'm not telling these students, it's in the notes apparently, the next matrix would be random 0, 3, and of course you multiply and you get a 3 by 3 matrix of zeros. So it's fun to see that that proof still works even for the zero matrix, without any real hitch. So that was just today.


[User] Please transcribe this text

[Assistant] Let me, uh, the other thing is let me, can I say a word or two, uhm, about


[User] Please transcribe this text

[Assistant] You all know machine learning is hot. That's probably why you're here in this class. I probably don't have to, to tell you. Uh, and yet.


[User] Please transcribe this text

[Assistant] I wouldn't be surprised if a number of you were sort of, wish the whole, this whole class was in Python or something, or maybe Matlab or something. I don't doubt that some of you might have wanted that to happen. Uh, and, uh, we get sort of bombarded with, with, you know, why not Python


[User] Please transcribe this text

[Assistant] Not so much Matlab anymore, but you know, why not Python is sort of the issue that comes up.


[User] Please transcribe this text

[Assistant] A lot. Uhm, and, uh, I could talk till the cows come home, but nobody would believe me, but Google came out last week and said that when it comes to machine learning, there really are two languages that are powerful enough to do machine learning.


[User] Please transcribe this text

[Assistant] Uhm, what they do is they basically sort of start the race with a whole bunch of programming languages. There's Python, there's R, Java, JavaScript. They, you know, they sort of look at all of these languages.


[User] Please transcribe this text

[Assistant] Uhm, and if you read the blog, you'll see, but we're going to filter them out on technical merits and, and uhm, right away a lot of them disappear, including Python and Java. Uhm, and, and they actually spend, if you go to the blog, you'll see they spend a great deal of time


[User] Please transcribe this text

[Assistant] On the Python story, because they know that people are going to want to hear that one.


[User] Please transcribe this text

[Assistant] Right, I mean, people want to be convinced, and so there's actually multiple screens full on the reason why Python's just not good enough for machine learning.


[User] Please transcribe this text

[Assistant] Uhm, so they, they leave four languages left, um, Julia, Swift, C++, and Rust, and then sort of the, the next sort of part of the, if you, if you go to the next part of, of the blog, they filter on usability, and then two more sort of bite the dust.


[User] Please transcribe this text

[Assistant] So C++ and Rust disappeared and uhm, then they go on to say that these are the only two languages they feel are appropriate for machine learning and they


[User] Please transcribe this text

[Assistant] I'll put this nice quote, you know, that, uhm, that, you know, it shares many common values and they actually go on about what machine learning really needs and, you know, I'd recommend you


[User] Please transcribe this text

[Assistant] Look at it, and then finally of course they're gonna push Swift, uhm, which they, which they should. So they had some more blah blah about more people are using Swift. Maybe it's true, I don't know. Uhm, but uhm, so that, you know, they're gonna...


[User] Please transcribe this text

[Assistant] They, they, they really said is they're more familiar with Swift than, than Julia, which you know, if I, if I was speaking, I'd say I'm more familiar with Julia than Swift, so maybe it's fair. Uhm, and then uhm,


[User] Please transcribe this text

[Assistant] I started to put the little cartoon on the psychology of programming languages just because it's, it's sort of a, something that I bump into with all the time. People always say all languages are equally good, it doesn't really matter, uhm, but the truth is, if you don't mention a language, if you mention a language that you're not using yet, you're gonna tune it out, uhm, at least until Google comes along. So that, that's where we are. Okay, enough about, I just put this together, I, you know, we're testing it out on you. Alright.


[User] Please transcribe this text

[Assistant] Now, uh, so now let me do some, two more mathematical things. Uhm, so the first thing I want to do is talk to you about forward mode automatic differentiation. So have you done any automatic differentiation in this course? Okay, so I think this is pretty fun. I hope you like it. I have a notebook in Julia.


[User] Please transcribe this text

[Assistant] And forward mode automatic differentiation. And this notebook came together because


[User] Please transcribe this text

[Assistant] Uh, I was trying to understand what the big deal was for a long time and, and I had a little trouble. I mean, it's, it's the usual story where on a line by line level, it's easy to understand, but like, what's the big deal part? That's, that's sometimes the harder thing to grasp. And, and, and the first notebook I'm going to show you, uh, is, is sort of,


[User] Please transcribe this text

[Assistant] The result of my trying to grasp what was the real picture here.


[User] Please transcribe this text

[Assistant] And the second thing I think I'll just do on the blackboard, it's not even really ready yet, but I'll sort of unleash it on you folks anyway, is to show you how to do a particular example of backward mode automatic differentiation, the back propagation that you see in a neural net. And I guess you have seen some neural nets here?


[User] Please transcribe this text

[Assistant] So, uh, I think by now everybody's seen neural nets, I think.


[User] Please transcribe this text

[Assistant] Let me, let me start things off, and really the two things that I'd love to convince you of is, let's, let me just find, here's my autodiff thing, uhm, the two things that I really want to convince you of, and maybe you already believe some of this, is, uh, one, that,


[User] Please transcribe this text

[Assistant] Well, maybe you don't believe this yet, but the language matters, uh, in a mathematical sense.


[User] Please transcribe this text

[Assistant] The right, uh, the right computer language can do more for you than just take some algorithm on a blackboard and implement it. It could do much more. And this is something that I hope to give a few examples of. And the other thing that I bet you all believe now because


[User] Please transcribe this text

[Assistant] Uhm, you've been in this class, is that linear algebra is the basis for everything. Every course should start with linear algebra. I mean, it's, to me it feels like an unfortunate accident of history that linear algebra came too late for too many reasons and so very often things that would be


[User] Please transcribe this text

[Assistant] I mean, to me it feels like doing physics without calculus. Like, I just don't get it. Like, I know high schools do that, but it just seems wrong. You know, to me, like, you know, all of engineering should all be linear algebra. I mean, I just believe that. Almost all, maybe not all.


[User] Please transcribe this text

[Assistant] But quite a lot, more than most people realize, I would say. Okay, so let me start with automatic differentiation. So, uh, so I'm going to start by this story by telling you that


[User] Please transcribe this text

[Assistant] I would go to conferences, I would go to numerical analysis conferences, I would hear people talk about automatic differentiation. I'm gonna be honest, I was reading my email, I was tuned out, like who cares about, like, I know calculus, you could teach a computer to do it, I don't, you know, it seems pretty easy to me, I mean, I'm sure there were technical details, but it didn't seem that interesting.


[User] Please transcribe this text

[Assistant] To teach a computer to differentiate. I sort of figured that it was the same calculus that I learned when I took a calculus class, you know, you memorize this table, you teach it to a computer, you know, you learn the chain rule and the product rule and the quotient rule and bump, the computer's doing just what I would do with paper and pencil. So big deal. I didn't pay attention.


[User] Please transcribe this text

[Assistant] And in any event, there was this little neuron in the back of my brain that said, hey, maybe I'm wrong, maybe it's doing finite differences, you know, the sort of thing where you take the dy by the dx in some numerical way, you do the finite differences.


[User] Please transcribe this text

[Assistant] In theoretical analysis, they're supposed to tell you, you know, if H is too big, you get truncation error. If you have H too small, you get round-off error. And the truth is, nobody ever tells you what's a good H.


[User] Please transcribe this text

[Assistant] But you go to an American Analysis class hoping somebody would tell you.


[User] Please transcribe this text

[Assistant] But, but, in any event, I thought maybe it was that kind of a numerical finite difference.


[User] Please transcribe this text

[Assistant] And, uh, and I think the big surprise for me was that automatic differentiation was neither the first nor the second thing. That there's actually a third thing, something different, that's neither the first or the second. And I found that fascinating. And maybe I'll even tell you how


[User] Please transcribe this text

[Assistant] How it hit me in the head that this was a story, because I really wasn't paying attention.


[User] Please transcribe this text

[Assistant] Uh, I love the singular value decomposition. I'm glad to see that people are drawing parabolas and quarter circles and figuring out what the minimum SVD value is. The singular value is just, you know, the, it's just, you know, God's gift to mankind. It's just a good factorization.


[User] Please transcribe this text

[Assistant] One of the things that I was playing with, with Julia, was to calculate the


[User] Please transcribe this text

[Assistant] Uh, the Jacobian matrix for the SVD. So, so, you know, all matrix factorizations are just changes of variables, right?


[User] Please transcribe this text

[Assistant] If you have a square matrix n by n, the SVD, I'm sure you know this, is, you know, the U matrix is really n times n minus 1 over 2 variables, so is the V and the sigma's got n variables. Put it all together, you got n squared. So it's just a change of variables, and every time you change variables,


[User] Please transcribe this text

[Assistant] You can form that big matrix, n squared by n squared of, of the, you know, dy dx. Compute its determinant.


[User] Please transcribe this text

[Assistant] Uhm, and get an answer. And, and I wanted to know what the, I wanted to, I actually knew the theoretical answer for that and I wanted to see a computer confirm that theoretical answer.


[User] Please transcribe this text

[Assistant] And uhm, I spoke to some people who wrote Autodiff in non-Julia languages, and I was surprised by the answer. They said,


[User] Please transcribe this text

[Assistant] Oh, yeah, we could teach the answer to our system. I said, what do you mean you could teach the answer? Why doesn't it just compute the answer? Why do you have to teach the answer?


[User] Please transcribe this text

[Assistant] I thought that was all wrong, uh, cause in Julia we didn't have to teach it, it actually, it would actually calculate it. Uh, and then I started to understand a little bit more about what Autodiff was doing and what Julia was doing, and so this is how this notebook came to be. So let me start, I'm saying too much, let me start with an example that, that


[User] Please transcribe this text

[Assistant] So, I'm going to compute the square root of x. A real simple example, you know, square root's pretty easy. I'm going to take one of the oldest algorithms known to mankind, uh, the Babylonian square root algorithm. It says, you know, start with a starting guess t, maybe it's a little bit too low for the square root of x. Get x over t, so that would be too large.


[User] Please transcribe this text

[Assistant] Go ahead and take the average and repeat, okay? This is equivalent to a Newton's method for taking the square root. Uhm, and, you know, it's been known for millennia to mankind, so it's not a terribly...


[User] Please transcribe this text

[Assistant] You know, it, it, it's not the latest research by any means for computing square roots. Uhm, but it, it, it works very effectively and uhm, here's a little Julia code that actually will uhm,


[User] Please transcribe this text

[Assistant] We'll, we'll implement it, it probably looks like code in any language. So, uh, I'm gonna start off at one, so literally I'm just gonna take one plus the starting value x and divide by two, uh, and then I'm gonna repeat. Okay, and, you know, you can,


[User] Please transcribe this text

[Assistant] We can check that the algorithm works. Here's alpha is pi. And so I'll take the Babylonian algorithm and compare it to Julia's built in and you see.


[User] Please transcribe this text

[Assistant] It gives, gives the right answer. Uhm, here it is with square root of 2. You know, always good to check your, your code works. Okay?


[User] Please transcribe this text

[Assistant] Even, I like to see things graphically, so I ran the algorithm.


[User] Please transcribe this text

[Assistant] Uh, for lots of values of x, and you could even, I love doing this.


[User] Please transcribe this text

[Assistant] Uhm, I kind of wished that in the previous talk, I was, this is what, if I had only worked fast enough, I was actually, I wanted to build a little GUI where I could move the points in front of your eyes. Maybe you have one in MATLAB, I bet you do. Uhm, but I wanted to build it, but I didn't get there fast enough.


[User] Please transcribe this text

[Assistant] Uh, this is the sort of thing, you know, I like to see the convergence, uhm, and so you can see the digits converging, the, the, the parabola on the bottom, the black is the square root, of course. So, uh, there it is, there's the Babylonian algorithm.


[User] Please transcribe this text

[Assistant] I would like to get the derivative of square root.


[User] Please transcribe this text

[Assistant] But the rules of the game are, I'm not going to do method one or method two. I'm not going to do, you'll never see me type one half x to the minus half. That's right, you all know that's the derivative. I will not type that. I will not, it's not going to come anywhere from Julia. Okay, and the second thing is I'm not going to do a finite difference.


[User] Please transcribe this text

[Assistant] I'm gonna get that square root, but not by, by, by sort of either of the two things that I'm sure you would think of.


[User] Please transcribe this text

[Assistant] But, uh, here's how I'm gonna do it, and, and I'm gonna do a little bit of Julia code. There'll be eight lines of Julia, but I'm not gonna completely say how it works yet. I'll...


[User] Please transcribe this text

[Assistant] I'll, I'll keep you in suspense for maybe about five minutes and then I'll tell you how it, how it works. Alright, so here's eight lines of Julia code that will get me the square root. So in these three lines, I'm going to create a Julia type, I'm going to call it a D for, for


[User] Please transcribe this text

[Assistant] For a dual number, which is a name that goes back at least a century, maybe more.


[User] Please transcribe this text

[Assistant] So I'm going to create a D type and all this is is a pair of floats. Okay, so uhm, it's a tuple with a pair of floats. It's going to be some sort of numerical function and derivative pair. Alright, so three of my eight lines is to create a D. This means in Julia language.


[User] Please transcribe this text

[Assistant] This means to use a subtype of a number, so we're going to treat it like a number, right? We want to be able to add, multiply, and divide these, these ordered pairs. Uh, but it's just a pair of numbers, right? Don't get, don't let the Julia scare you. It's just a function derivative numerical pair, okay? And what's these other five lines? Well, uhm, I want to teach it the sum rule and the quotient rule, so.


[User] Please transcribe this text

[Assistant] You all remember the sum rule? I guess that's the easy one. The quotient rule is still, I still have the, my teacher from high school, you know, ringing in the back of my ear, you know, the denominator times the derivative of the numerator minus the numerator. You all have that jingle in your brain too? I bet you do. Divided by the denominator squared. Can't even get it out of my head. So, so, so there's the quotient rule.


[User] Please transcribe this text

[Assistant] And, uh, so what are we doing in these five lines? Well, uh, first of all, uhm, I wanna overload plus and divide and a few other things.


[User] Please transcribe this text

[Assistant] Julia wants me to say, are you sure? So the way you say are you sure is that I'm going to import plus and divide.


[User] Please transcribe this text

[Assistant] The seventh line is, if I have a dual number, I want it to, to convert it. You know how like the reels are embedded in the complexes? You know, you just, we have to tell Julia to, to, to take a dual number and stick a zero in. And then, then dual numbers and regular numbers can complete nicely together.


[User] Please transcribe this text

[Assistant] Uhm, and this, this actually is the thing that actually says if I have a dual number and a number in operation, promote them so the work is dual numbers. So, uh, eight lines of code. So the first thing I'm going to tell you is


[User] Please transcribe this text

[Assistant] I'm gonna remind you, you never, no, I never typed one half x to the minus a half, do you agree? No one, I'm not importing any packages, it's not like it's coming in from the, I'm not sneaking it in from the side. There's no one half x to the minus a half. And, uh,


[User] Please transcribe this text

[Assistant] Uh, there's certainly not any numerical derivatives either, right? Just the, just the, just arguably a rule that almost feels symbolic, the quotient rule and the addition rule, but no numerical finite differences at all here.


[User] Please transcribe this text

[Assistant] I'm applying the Babylonian algorithm without rewriting code to a dual number now, before we applied it to numbers, but now I'm going to apply it to this dual number that I just invented.


[User] Please transcribe this text

[Assistant] I'm going to apply it at 49.1 because I know the answer.


[User] Please transcribe this text

[Assistant] Uhm, and then I'm gonna compare it with, I'm taking one half of the square root of x just for comparison purposes, right, and not, not in my own algorithm. And of course you see that


[User] Please transcribe this text

[Assistant] I'm getting magically the right answer without ever. So you should wonder, how did I do that? How did I get the derivative? We could take any number you like. Here's a hundred.


[User] Please transcribe this text

[Assistant] Uhm, if you prefer to see a number like pi, we can do that. I mean, you know, we can do whatever you like.


[User] Please transcribe this text

[Assistant] Uhm, it's going to work. So there you see the, this is the square root of pi, and this would be one half over the square root of pi numerically. So, and you see it matches these, these numbers.


[User] Please transcribe this text

[Assistant] Uh, to enough digits, in fact to all the digits actually. Yeaah. So this thing magically worked. You should all be wondering, how did that happen? Right?


[User] Please transcribe this text

[Assistant] I didn't rewrite any code. I actually wrote a code to just compute a square root. I never wrote a code to compute the root of a square root.


[User] Please transcribe this text

[Assistant] And by the way, this is a little bit of the Julia magic that we're pushing numerically.


[User] Please transcribe this text

[Assistant] In this world, people will write a code to do something, and then if you want to do something more, like get a derivative, somebody writes another code. With Julia, very often, you can actually keep to the original code, and if you just use it properly and intelligently, you can do magic things without writing new codes, and you'll see this again in a little bit.


[User] Please transcribe this text

[Assistant] But uhm, here's the derivative of, you know, this is the plot of 1 half over the square root of x in black, and again you can see the convergence over here. Alright, well I'm still not going to show you why it works just yet, I promise I will in probably a few more minutes more.


[User] Please transcribe this text

[Assistant] What I will do first is, is I'd like to show you something that most people don't ever look at. I never look at it. I want to show you, here's the same Babylonian code. I want to show you the assembler for the computation of the derivative. So I'm going to run Babylonian on a dual, dual number and we're going to look here.


[User] Please transcribe this text

[Assistant] And, uh, I don't know if anybody here reads Assembler, I'm betting there's zero or one of you actually reads this stuff. How many of you read Assembler?


[User] Please transcribe this text

[Assistant] Okay, we have, I got, it wasn't zero or one, we had a half. Right there is a half. He's kind of going like this. Here's zero, here's one, he's like this. Okay. So, so I think zero or one is like the right.


[User] Please transcribe this text

[Assistant] But, I'll bet you'll believe me if I tell you that when you have short assembler like this, and it's not very long, then you have efficient code. It's very tight, it'll run very fast.


[User] Please transcribe this text

[Assistant] So whatever this thing is doing, it's short. Uhm, and this you won't get from any other language. If you did try to do the same thing in Python, I promise you, there'd be screens and screens and screens full of stuff, if even if you could get it.


[User] Please transcribe this text

[Assistant] So, uh, so here's the Babylonian algorithm on the dual number, and here it is in the assembler, and it's short. So, so the other thing that I'm saying is not only does it work, but Julia also makes it efficient.


[User] Please transcribe this text

[Assistant] So, uh, before I finally tell you what's really going on and why it works, uh, I'm gonna grab a Python symbolic package, which will work nicely with Julia, and I'm going to run the same code


[User] Please transcribe this text

[Assistant] We're not, this is a symbolic computation. We're not doing a symbolic computation. This is mathematically equivalent to the function we would get if we were to like plot it or something. But of course we're not, symbolic computation is very inefficient. I mean, you get these big coefficients. I mean, look at this number. What is this? 5 million or?


[User] Please transcribe this text

[Assistant] Something. Anyway, you get these big numbers, uhm, these big, even bigger numbers here. Look at these huge numbers, right?


[User] Please transcribe this text

[Assistant] It takes a lot of storage, dragging these x's along, there's a big drag on memory, I mean, this is not the way to, this is why we do numerical computation. But, the Babylonian algorithm, in the absence of any round-off, is equivalent


[User] Please transcribe this text

[Assistant] To computing, uh, you know, above the line it's computing the square root here, and then below here, this is, these are the iterates towards the derivative. So it's not actually calculating.


[User] Please transcribe this text

[Assistant] Uhm, x, 1 half x to the minus 1 half, it's actually doing something iterative that is approximating 1 half x to the minus 1 half.


[User] Please transcribe this text

[Assistant] Alright, well let me tell you now, let me sort of reveal what's going on, just so that I can kind of show you how it's getting the answer.


[User] Please transcribe this text

[Assistant] Like I said, it was the SVD that sort of convinced me how this was happening, because the SVD is also an iterative algorithm like this Babylonian square root, but it's easier to show you the point with the Babylonian square root. So I'm going to do something that I would never want to do, which is explicitly write a derivative Babylonian algorithm.


[User] Please transcribe this text

[Assistant] What I'm doing is I'm going to take the derivative with respect to x of every line of my code. So, you know, if every even or odd line, I never know what's even or odd anymore, but uhm,


[User] Please transcribe this text

[Assistant] The original line of code had one plus over two, now I'm going to take the derivative, I'll get a half.


[User] Please transcribe this text

[Assistant] Uhm, here I had this line of code. If I take the derivative, I'll use the, the, the, the quotient rule, and this would be the derivative.


[User] Please transcribe this text

[Assistant] If I run this code, what I'm effectively doing is, I'm just using good old plus and times and divide, nothing fancy. There's not a square root to be seen. But what I'm doing is, as I run my algorithm, I'm also running, I'm actually


[User] Please transcribe this text

[Assistant] I'm not using anything other than plus, minus, times, and divide to make that happen. So if you rewrite any code at all, you could have any code, iterative, finite, it doesn't matter. If you just take the derivative with respect to your variable of every line of your code,


[User] Please transcribe this text

[Assistant] Then you can get a derivative out, and as I said, it's not a symbolic derivative like, you know, all of 18.01 or whatever, wherever we teach calculus these days, and it's not a numerical derivative like in the numerical courses, you know, the 18.3, the x, y, z's, whatever. It's a different beast. It's the, it's, it's using the, the quotient rule, the addition rule,


[User] Please transcribe this text

[Assistant] Uhm, at every step of the way to get the answer.


[User] Please transcribe this text

[Assistant] So, and, you know, here's this de-Babylonian algorithm. You could see it running and it'll get, it gives the right answer. Oop, I have to execute the code first to get the right answer. But you see, it gives, it gives the right answer.


[User] Please transcribe this text

[Assistant] Okay, oh, I was just in Istanbul and they challenged me to do sign. I forgot about that. It's still in my notebook. I did it in front of everybody. It worked. I got a cosign. Okay, but let me pass all of that.


[User] Please transcribe this text

[Assistant] So, so, so, let me go back and tell you then


[User] Please transcribe this text

[Assistant] How is this all working? Well, what's happening? Let's go back to the eight lines of code and now maybe you could see what's happening. Where's my eight lines of code from the very beginning?


[User] Please transcribe this text

[Assistant] And I gotta watch the time. I want to show you this one other thing too, so hopefully I'll have enough time to do that. But here, let's see, uhm, where are my eight lines of code? Where are they?


[User] Please transcribe this text

[Assistant] Here we go. Here are the lines of code. So what I'm doing is, instead of rewriting all your code by taking the derivative of every line the human way, I'm saying that why can't the software just do this in some automatic way? And this is where the automatic differentiation comes in.


[User] Please transcribe this text

[Assistant] And in the old, old days, when people in all the numerical code was in Fortran, there would be these source-to-source translators that would actually input


[User] Please transcribe this text

[Assistant] The Julia way, uh, the more modern way is to let the JIT compiler kind of do that for you. So, uh, here I needed plus and divide. Of course I would want to add minus and times, uh, but, uh, you just add a couple of things and then bump the


[User] Please transcribe this text

[Assistant] You don't, you don't have to rewrite the D Babylonian because the Babylonian with this type will just


[User] Please transcribe this text

[Assistant] We'll do the work for you, okay? And that's where the magic of a good piece of software will have it. So you don't have to write a translator. You don't have to...


[User] Please transcribe this text

[Assistant] That's forward-mode automatic differentiation. I've got 10 minutes to go backwards. So, but let me see if there's any, anybody have any questions about this? This, uh, it's really magic, right? But it's pretty wonderful magic. And, and, and, you know, I don't know what you've heard about machine learning.


[User] Please transcribe this text

[Assistant] But to be honest, uh, uh, machine learning these days, it's, it's, it's, it's, you know, forgetting about whether, you know, humans will be useless, which I don't believe by the way, but, but the, the, the, the big thing about machine learning is that it's really just a big optimization. That's all it is, right? One big minimum maximum problem.


[User] Please transcribe this text

[Assistant] Uhm, where you've all known from calculus that what you need to do is take derivatives, you know, set them to zero, right?


[User] Please transcribe this text

[Assistant] In the case of multivariate, it's a gradient, you set it to zero.


[User] Please transcribe this text

[Assistant] Really, all of this machine learning, all the big stories and everything in the end comes down to automatic differentiation as sort of like the workhorse of the whole thing. And so if we could have a language that gives you that workhorse in a good way, then machine learning really sort of benefits from that. So I hope you all see the big picture of machine learning.


[User] Please transcribe this text

[Assistant] It really does come down to taking derivatives. That, that's, that's the end of, that's how you optimize.


[User] Please transcribe this text

[Assistant] Any quick questions? Otherwise I'm going to switch topics and I'm going to move to the blackboard. Yeah?


[User] Please transcribe this text

[Assistant] There's a trick that basically lets you go to higher orders. Yeaah, you can basically make it a combo of two first order derivatives. So yeaah, it can be done. Did you have a question?


[User] Please transcribe this text

[Assistant] So, so, it's the biggest trick in the world. It's not this little thing, the idea of making a type to do what you, I mean, did you see Chronicle products in this class?


[User] Please transcribe this text

[Assistant] Okay, uhm, let me see, what would you have seen in this? Did you see tridiagonal matrices, your favorite? Okay, so here. So here's a built-in type. Let's see, n is, oh, n doesn't have to be 4. I'm going to create a string matrix, if I could spell it right.


[User] Please transcribe this text

[Assistant] Uhm, and it's going to be a sim tridiagonal, which is a Julia type.


[User] Please transcribe this text

[Assistant] I mean, this is built in, but you could have created it yourself just as easily. Uhm, and I don't like calling this, it's certainly not a dense matrix, and I don't like calling it a sparse matrix. I prefer to call it a structured matrix. Uhm, though the word sparse, it's a little tricky here, but the reason why I don't like to call this a sparse matrix is because


[User] Please transcribe this text

[Assistant] We're not storing indices in any, I mean there are a lot of fancy schemes for storing indices for sparse matrices. All we score, store is a diagonal vector, there's the twos on the diagonal.


[User] Please transcribe this text

[Assistant] There's this four vector with four twos, and there's a three vector, uh, for the off diagonal, and, you know, you don't have it twice, by the way. Most sparse matrix structures would, would, would, would have, uh, you know, the, the minus one vector twice, the super and the sub.


[User] Please transcribe this text

[Assistant] Uh, but really only the core information that's needed is stored.


[User] Please transcribe this text

[Assistant] And in a way, one uses types in Julia to basically, you only store what you need, not more, and then you define your operations to work. So for example, if I were to, uhm, take, you know, uh, string inverse times, oh, anything,


[User] Please transcribe this text

[Assistant] That while you can do it in languages like Python and Matlab, if you were able to look at the assembler, and Matlab would never let you, Python, you just would regret it. But uhm, you would see just how much overhead there is in doing this. So there would be no performance gain.


[User] Please transcribe this text

[Assistant] In a way, this is what you want to do. You want to use these things to match the mathematics. Uh, and so that's really the nice thing to be able to do.


[User] Please transcribe this text

[Assistant] Alright, I only have five minutes, I don't know if I'm going to pull this off, but I'll, let me see if I can give you the main idea in five minutes of, of reverse mode differentiations. But here, as long as you are familiar with neural networks, let me see if I can do this very quickly.


[User] Please transcribe this text

[Assistant] I'm going to start with scalars. Okay, I'm going to do a neural network of all scalars, but only for simplicity for starters, but I think you're going to see that this can generalize to vectors and matrices which are real neural networks. So what I'm going to do is I want to imagine that we have our inputs.


[User] Please transcribe this text

[Assistant] Alright, so we have a bunch of weights and biases here. Okay, and


[User] Please transcribe this text

[Assistant] We'll also have an x1, which will sort of start off our neural network. And we're going to compute, I'll write it in sort of Julia-like or Matlab-like notation, for i equals 1 through n, I will update x by taking some function


[User] Please transcribe this text

[Assistant] Input, maybe something like this. And what function h to use, I don't really care too much. In the old days, people used to talk about the sigmoid function. Uh, nowadays, it's the maximum of zero and t that gets used all the time. It's got this ridiculous name, relu, which


[User] Please transcribe this text

[Assistant] I really can't stand, but anyway, rectified linear unit. But in any event, I mean, it's just a function that's, you know, t of t is greater than or equal to zero, zero, if not.


[User] Please transcribe this text

[Assistant] So, uh, but whatever function you like, uhm, and here I'm just gonna update. Okay, and then ultimately you might also have some data y,


[User] Please transcribe this text

[Assistant] And you would like to, if everything's a scalar, like I said, this could be generalized pretty quickly.


[User] Please transcribe this text

[Assistant] So uhm, what we can do is we, we can minimize say one half y minus xn squared, you know, and you're going to want to find the data that would minimize that, okay. All this generalizes to matrices and vectors, which is what most neural nets do.


[User] Please transcribe this text

[Assistant] Since I'm not going to have a lot of time, maybe I can just sort of cut to the chase. If I were to differentiate the key line here, uh, got a little bit of Julia here, but if I were to differentiate the key line, what would I write?


[User] Please transcribe this text

[Assistant] This would be how, how, so I'm almost done, that's the goodness. Uhm, so if I make a little change, I like to think of this as like 0.001 changes. I don't like, I don't like infinitesimals, I like 0.001s, that's how I think of them. But you make a little change here, a little change here, a little change here, you get a change here. You, you, you'll get this, uh,


[User] Please transcribe this text

[Assistant] Uhm, you know, you'll get this linear, this linear function of the perturbations here gives you perturbations here.


[User] Please transcribe this text

[Assistant] Okay, well, I've only got one minute, so I'm going to write all this out with linear algebra because everything is better when written out with linear algebra. So I'm going to write down that, uh,


[User] Please transcribe this text

[Assistant] I'm gonna write down that I'm actually interested in the last element.


[User] Please transcribe this text

[Assistant] And I'm going to have a couple of matrices here. Let me just sort of get the structure right.


[User] Please transcribe this text

[Assistant] In fact, I'd like to use block matrices a little bit. So, uh, here I'm gonna have DW1, DB1. I'm gonna put the bias together. Sorry for the mess. But, uh, and DWN, DBN. Julia lets you...


[User] Please transcribe this text

[Assistant] Make block matrices and you can actually use them directly. There'd be a special type right there. Okay, and then what goes here? You could actually see what it would be. It would be, I hope I'm doing it right, but there'd be a delta 1 x1 and a delta n xn, and this would be a diagonal matrix. Okay, and then what do I have over here? Here,


[User] Please transcribe this text

[Assistant] I'd have the, the delta W's. And if you check, you'll see that this will be.


[User] Please transcribe this text

[Assistant] I'm not going to get the indices right and I don't have time, so I'm just going to write it like this. And now I'm just going to give you the end of the story because I've run out of time. You could write all this as dx is equal to a diagonal matrix times the derivative of the parameters plus a lower triangular matrix times dx again.


[User] Please transcribe this text

[Assistant] And if I only want the last element, let's say en is the vector that pulls out the last element.


[User] Please transcribe this text

[Assistant] Then this is all I'm going to need to get all my derivatives. And what's the moral of the story? I apologize for going one minute over, but the moral of the story is instead of backpropagating through your own hard work,


[User] Please transcribe this text

[Assistant] You probably know that when you solve a lower triangular matrix,


[User] Please transcribe this text

[Assistant] People have already written code that back solves the lower triangular matrix. The back, the big back piece has already been implemented for you. Why do, why reinvent the wheel in back propagation if the back, if linear algebra already has the back, you see? Uhm, and so if you just do this and you do it in a language that lets you get pull,


[User] Please transcribe this text

[Assistant] Anyway, linear algebra is a secret to everything. That's the big message.


[User] Please transcribe this text

[Assistant] Where does wind come from? Like me, it comes from Wonderland.


[User] Please transcribe this text

[Assistant] Oh, umsum. Wind can be defined as the movement of air molecules on a large scale. Sun is primarily responsible for the creation of wind. How?


[User] Please transcribe this text

[Assistant] Sun's rays heat up the earth's surface every day, but they do so unevenly. Air present above land heats up much faster as compared to air present above the surface of rivers and oceans.


[User] Please transcribe this text

[Assistant] This results in the creation of different air pressures. The heated air above land expands and rises up. The cooler air from the oceans rushes in and takes its place, resulting in a sea breeze.


[User] Please transcribe this text

[Assistant] Now, at night, inland temperatures drop, making oceans relatively warmer than land. Hence, air above oceans expands and rises up. The cooler air from land rushes in to take its place, resulting in a land breeze. In this way, wind is born.


[User] Please transcribe this text

[Assistant] What if wind suddenly stopped? Wind might stop, but Umsum the Great will never stop! Stop it, Umsum.


[User] Please transcribe this text

[Assistant] Firstly, if there was no wind in heavily industrialized places, pollutants would not disperse, making the air extremely toxic.


[User] Please transcribe this text

[Assistant] Secondly, equator would get more hotter while poles would get more cooler. Why?


[User] Please transcribe this text

[Assistant] Because in the absence of wints, heat at the equator will not be carried to the poles.


[User] Please transcribe this text

[Assistant] Fifthly, as wings are a major source of pollination, without wings, plant life would greatly suffer.


[User] Please transcribe this text

[Assistant] How does a fan give us cool air?


[User] Please transcribe this text

[Assistant] Both of these above ways create a stagnant layer of hot humid air around our skin, thus making it difficult to continue the processes of convection and evaporation.


[User] Please transcribe this text

[Assistant] Why does hot air balloon float? Cause it has invisible wings! No, it is because of density.


[User] Please transcribe this text

[Assistant] Density is the measure of mass present per unit volume. Lesser the density, lighter will be the object. Now, density varies with temperature. Oh dude. It is so complicated. Alright. I'll explain.


[User] Please transcribe this text

[Assistant] Maybe it applies a special perfume. Nah. Rain is just water and water doesn't have any smell. Yeah dude, so what's that smell?


[User] Please transcribe this text

[Assistant] The distinctive smell which frequently accompanies the first rain after a dry weather is scientifically called petrichor. It basically comes from plants and bacteria called actinomycetes, which live in the soil. Now during a long dry spell, the plants release oils into the soil to block other seeds from germinating.


[User] Please transcribe this text

[Assistant] Whereas, the actinomycetes produce a chemical called geosmin. Now when rain hits the ground,


[User] Please transcribe this text

[Assistant] Thanks to Brilliant for supporting this SciShow News video. You can keep building your STEM skills at Brilliant.org slash SciShow with 20% off an annual premium subscription.


[User] Please transcribe this text

[Assistant] Sharks are fascinating creatures, and none are more interesting than the biggest shark to ever swim.


[User] Please transcribe this text

[Assistant] Megalodon. This week in Nature Communications, scientists report a way to use fossilized shark teeth to figure out where different species, including megalodon, stood in the web of life. Not by size or shape, but by their zinc content.


[User] Please transcribe this text

[Assistant] And the biggest we know of is Otidus megalodon. After ruling the seas for a good 20 million years, megalodon abruptly went extinct roughly 3 million years ago. But there's a lot we don't know about it, like why it died out and how its massive teeth fit into the oceanic food web. These researchers were interested in megalodon's trophic level.


[User] Please transcribe this text

[Assistant] The ancestor to the modern great white shark lived alongside Megalodon, and the researchers believe the two sharks may have competed with one another.


[User] Please transcribe this text

[Assistant] Just because two species are predators doesn't necessarily mean they compete with each other. For example, some mid-level predators might have different food sources and themselves get eaten by top predators. To find out if Great White ancestors muscled in on Megalodon's apex turf, the team looked at the ratio of two types, or isotopes, of zinc.


[User] Please transcribe this text

[Assistant] That's because scientists think that plant-based diets are full of a substance that binds more to lighter zinc atoms.


[User] Please transcribe this text

[Assistant] Leaving the heavier ones floating around for an animal's body to use. So animals that eat a lot of soft animal tissues and not a lot of plants will have access to more Zinc 64 to start with. But that hadn't really been demonstrated for other animals, extinct or not.


[User] Please transcribe this text

[Assistant] So, the team first measured the zinc in 20 living shark and fish species. And the results were similar. Apex predators like great white sharks had significantly lower ratios of zinc-66 to zinc-64 in their teeth,


[User] Please transcribe this text

[Assistant] compared to sharks that feed at a lower trophic level, like basking sharks which eat plankton. So next, the team checked the zinc ratio of fossil teeth from 13 different species, including megalodon. Most had a similar zinc ratio to their modern shark relatives, suggesting a similar trophic level.


[User] Please transcribe this text

[Assistant] But there was a noticeable difference between the modern great white shark and its ancestors. Specifically, it seems like it climbed up the proverbial trophic ladder, becoming more of an apex predator. Since megalodon was already sitting at a high trophic level, this could suggest that the great white started budding into some of the megalodon's feeding grounds.