[User] Please transcribe this text

[Assistant] Bess, come. Bess, come. Good. There you go. Nice. Right past the distractions. This is what your leash is for, just in case she wants to go to them. That's what this handy little tool is for. You pull your dog right back in the direction of you. Bess, come. Come here. Good. Nice. Nice. Good girl.


[User] Please transcribe this text

[Assistant] Good girl. And now I'm gonna go four six feet. Same thing. Beth, come here. Come here. Good. Nice. Good girl. That's a cum. Good girl.


[User] Please transcribe this text

[Assistant] Same thing, right past all the distractions. I'm gonna go ahead and squeak one. Get our attention on it.


[User] Please transcribe this text

[Assistant] Oh yeaah. Oh yeaah. Oh yeaah. Bass, come here. Come here. Good girl. Nope.


[User] Please transcribe this text

[Assistant] Nope. And then again, she's distracted over here. Come here. Good girl. Leash brings her back to me. Good girl.


[User] Please transcribe this text

[Assistant] And one more time. Bass, come here. Good girl. That's a come. Good girl. Good girl.


[User] Please transcribe this text

[Assistant] And then simultaneously throw the toy and call at the same time.


[User] Please transcribe this text

[Assistant] Beth, come here. Come here. There you go. That's a good girl. Perfect. Look at you. Nice.


[User] Please transcribe this text

[Assistant] And when she gets to you, a lot of energy, a lot of positive interaction. Good girl. You want to make this what it's all about when training your dog to come, especially for a young dog. They're influenced by so many things. You want their number one influence to be you.


[User] Please transcribe this text

[Assistant] And one more thing I want to point out here. Not all dogs are meant to be off leash. Some dogs are great off leash and others have too much instinct that nor you or a trainer can ever conquer.


[User] Please transcribe this text

[Assistant] The best way to figure this out is talk to a trainer. Have the trainer assess your dog to see if your dog is a candidate for off-leash training.


[User] Please transcribe this text

[Assistant] To taper your dog off of treats, go to your workbook and learn the lottery system.


[User] Please transcribe this text

[Assistant] Hey guys, today we're gonna be implementing a depth system. So I've actually made a tutorial on this before, and we're gonna be implementing it here today. Just with a few tweaks to suit our game. And although this is obviously a continuation of our farming RPG series, it should just work as a standalone depth system tutorial, so it should be applicable across any games that you make.


[User] Please transcribe this text

[Assistant] Right now I'll just explain the kind of problem and then what we're hoping to do with our death system. So, in our game...


[User] Please transcribe this text

[Assistant] When we go above or below an object, nothing is really changing. But what we want is for the object to appear behind another when it's above it, and then come in front of it when we go lower. So in that way our game world appears to have a kind of depth.


[User] Please transcribe this text

[Assistant] And we are using depth in our project already. So for example, we have different layers in our rooms at different depths. So for example, the instances layer is at a lower depth than the tiles, which means the tiles are further away from us because their depth is higher. And that's why our instances appear to be on top of the grass tile.


[User] Please transcribe this text

[Assistant] And we can do something similar with our objects. So we could have all of our objects having different depths depending on their position in the room. That would mean they're no longer on the same instance layer. And you can do it that way, that's fine.


[User] Please transcribe this text

[Assistant] You have to be a little bit careful to make sure that you're gonna need a lot of sort of buffer room between your instances and the tiles, for example.


[User] Please transcribe this text

[Assistant] Depending on how big your room is, but we're not actually gonna do that. It's quite useful to have all the instances just in the same layer for organization, for applying effects, because there's a lot of functions that you can use that will affect an entire layer, such as deactivating instances.


[User] Please transcribe this text

[Assistant] So what we're gonna do instead is keep them in the same layer, but control the order that they're drawing themselves to the screen.


[User] Please transcribe this text

[Assistant] So imagine it like sticking a bunch of stickers to a blank canvas. The stickers you put on first are going to appear behind all of the later stickers.


[User] Please transcribe this text

[Assistant] So what we'll do is we'll start with the instances at the top and then one by one we'll go down the screen sticking on all of our instances. And the way that we're going to do that is we're going to use another DS grid, a data structure. And this grid is going to have a width of two and then the height of the grid is going to be the number of instances that are part of our dev system.


[User] Please transcribe this text

[Assistant] So the first column, which is column 0, this is going to contain all of the instances ID. Basically every step of the game we're going to fill this grid with all of the instances that are currently in the room.


[User] Please transcribe this text

[Assistant] And then in the second column, which is actually column one, because we start at zero, we put the corresponding y values of all of the instances.


[User] Please transcribe this text

[Assistant] And then all we have to do is sort in an ascending order, which means the lowest will be first, and then we ascend upwards.


[User] Please transcribe this text

[Assistant] And we sort the grid so that the instance that has the lowest y position, which means it's up the top of the room, because in game maker, zero is actually up the top. So it will be first in the grid and the instances with a higher y value will be last. And then we will just loop through the grid and draw the instances in that order to the screen. So fairly straightforward. Let's get started.


[User] Please transcribe this text

[Assistant] So the first thing we're going to do is we're going to create a new object. And this is the object that is going to be handling all of the drawing. So remember in our new depth system, the instances themselves, they're no longer going to be drawing themselves. It's all going to be handled by our master drawing object. So I'm going to call this one depth sorter.


[User] Please transcribe this text

[Assistant] Okay, so let's come into its create event and let's make the grid, so ds, and I'm going to call this depth grid. And we can go ahead and create it right away. ds, grid, create.


[User] Please transcribe this text

[Assistant] So the width is gonna be 2, and it's always gonna be 2. And the height for now we'll just put 1.


[User] Please transcribe this text

[Assistant] And now as with any data structure, we should make sure that we are deleting it when the object is destroyed or when the game ends. And actually I'm going to use the clean up event.


[User] Please transcribe this text

[Assistant] And this will be whenever it gets destroyed. So it could be when something is destroying it or it could be when the game ends. So whenever it's destroyed, the cleanup event is going to run. So we go ahead and we destroy the grid.


[User] Please transcribe this text

[Assistant] Let's come along to the drawer event and we're gonna set up the logic for what we're gonna be doing. So first off, there's a few things that we're gonna have to do every single frame of the game.


[User] Please transcribe this text

[Assistant] Now the number of instances in the room might be changing. We might have, for example, crops being destroyed and created every few seconds. So we're gonna have to basically check every frame, how many instances that we want part of the depth system, and then we're gonna have to resize the grid to that size.


[User] Please transcribe this text

[Assistant] We're gonna take advantage of object hierarchies to get the number of instances that we want as part of the depth system.


[User] Please transcribe this text

[Assistant] And basically, all of the objects that we want to be sorted and drawn by our depth system, we're gonna make them the child of a parent object. So let's go and make...


[User] Please transcribe this text

[Assistant] Actually, I'm going to drag the crops into beings and we're going to create an object called parent depth objects.


[User] Please transcribe this text

[Assistant] Okay, and this is gonna have, as it's children, all of the other objects. And actually, so note that I've called it par-depth-objects, so I've actually changed one of the naming conventions that we set up.


[User] Please transcribe this text

[Assistant] Because we called this objParentNPC, but to me that looks a little bit ugly, so I'm gonna just change it to parNPC. Okay, anyway, back to the parent object.


[User] Please transcribe this text

[Assistant] Let's now add all of its children that we need. So we need to add the player. And because we've actually made another parent


[User] Please transcribe this text

[Assistant] For the NPCs as a parent NPC, we just have to add that object.


[User] Please transcribe this text

[Assistant] And then we'll have sort of like, this is the grandparent, and then parent is the parent of the NPCs and then all of the children NPCs themselves. So you can set up these complicated hierarchies like this. So, parent, NPC, and then the final one is the crops. And for any other objects that you have in the room, you're gonna want to add them as a child of this.


[User] Please transcribe this text

[Assistant] Now, as something to point out, we did actually make the parent NPC previously a child of the collision object.


[User] Please transcribe this text

[Assistant] And we'll fix this later, but just, it's something to note about parent hierarchies, because we can't have more than one parent in Game Maker.


[User] Please transcribe this text

[Assistant] So anyway, let's come back to the dev soda. So now what we can do is we can use the function instance number parent.


[User] Please transcribe this text

[Assistant] Depth object. And that's gonna get the number of instances of parent depth object. And because they are children of this object, they are gonna be included in this.


[User] Please transcribe this text

[Assistant] So we're going to save whatever this returns into a variable and we'll call it instance number.


[User] Please transcribe this text

[Assistant] And what I'm also gonna do is, so remember how when we create a variable, it belongs to this object and it's not accessible inside any of the other objects? Well, we're gonna be using a few with statements in this code here, which means the objects that we go with, we will want them to be able to access the depth grid. So if we declare it as a local variable, so let's go ahead and do that.


[User] Please transcribe this text

[Assistant] Now, anything for this entire block of code is going to be able to access this variable. And then of course it gets deleted once the script is run.


[User] Please transcribe this text

[Assistant] Okay, so now we have the number of instances and we have access to the depth grid. Let's go ahead and just resize. So this takes in a few arguments here.


[User] Please transcribe this text

[Assistant] We have to give it the ID of the grid and then a new width and height for it to resize it to.


[User] Please transcribe this text

[Assistant] And so over here, we could give it D grid or you can use this variable because we're still inside just the depth sorter. I'm just going to use D grid.


[User] Please transcribe this text

[Assistant] Alright, so now we have the grid that is an appropriate height to store all of our values in, but of course it doesn't contain anything yet. So resize the grid, now we need to add


[User] Please transcribe this text

[Assistant] Instances to the grid. And basically what we're gonna do is we're gonna go with depth object.


[User] Please transcribe this text

[Assistant] So we're going to go with all of the instances and it's going to have all of them run this code. We can't really predict what the order is going to be. So each instance is going to have to do two things. It's going to have to add its ID and its Y value. It's going to look something like this.


[User] Please transcribe this text

[Assistant] So in the first column of the grid, we have it add its ID. And then in the second column, which is column one, remember because the first column is column zero, second is column one, we have it add.


[User] Please transcribe this text

[Assistant] It's y value and you'll notice here I've just put yy because we haven't talked about what row it should be in. So like I said, we can't really predict where an instance will be in the order that this is running, but all we can do is we can set up a variable here, yy.


[User] Please transcribe this text

[Assistant] So for the very first instance that gets to run this, yy will be zero. And then once it's done, it can just increment this value. So now the next one, we'll sort it in at the second row and then it will just keep going on and on like that. All right.


[User] Please transcribe this text

[Assistant] So then the next thing to do is sort the grid in ascending order so that the lowest y value instance will be at the top. So we can use a function called deus grid sort.


[User] Please transcribe this text

[Assistant] This function takes in three arguments, so we have to give it the id of the grid. So again, we can just use dgrid or dsdepthgrid, we've come out of the grid, but we'll just keep things consistent. And we have to give it a column to sort by. So it's going to look at the data and it's going to sort all of the values in an ascending or descending order.


[User] Please transcribe this text

[Assistant] So we want it to look at the second column, column number one, because that's the one that has all the y values. So we'll say one.


[User] Please transcribe this text

[Assistant] And then finally, do you, it's asking do you want me to sort it in ascending order or is it gonna be in descending order? So we want it in ascending, so we put true.


[User] Please transcribe this text

[Assistant] Okay, and then the last thing to do is loop through the grid and draw all the instances in the order that we have sorted it.


[User] Please transcribe this text

[Assistant] So we're gonna use something kind of similar to this, we're gonna use a repeat statement. Basically, we wanna go through the grid one row at a time and tell all of the instances to draw themselves. Which means we're gonna have to keep pulling out all of the ID and then basically tell it, go ahead and draw yourself.


[User] Please transcribe this text

[Assistant] And we're going to have to do this for however many instances there are. So we're going to use another repeat statement here. Repeat. And then the number of times that we have to repeat is of course the number of instances.


[User] Please transcribe this text

[Assistant] So inst num, and they're all gonna have to do this. And basically we're gonna, again, we're gonna kind of go through the grid one.


[User] Please transcribe this text

[Assistant] So we're going to need some kind of variable that is incrementing itself every time. So we will set up yy again to be zero.


[User] Please transcribe this text

[Assistant] And then this is going to be incrementing every single time we repeat.


[User] Please transcribe this text

[Assistant] And we're also going to need a variable to store the instance ID in. So we're going to set up inst.


[User] Please transcribe this text

[Assistant] So we'll just get that variable ready to use. All right. So the first thing we do is we pull out the current ID. So this is the row that we're starting at in the grid. So we're starting at row zero. This is the first row and the instances are stored in the first column. So we know that we're going to have to set inst equal to, so whatever is located at this cell of the grid.


[User] Please transcribe this text

[Assistant] And we could just use draw self, but that would mean it's just drawing the sprite that it is. And because we've set up some pretty complicated drawing systems, we might actually have quite a few things to run in the draw event. So we're going to instead just tell it to run its entire draw event. So we'll go.


[User] Please transcribe this text

[Assistant] Event, perform, ev, draw. So this is the draw event. And this number is just if it's applicable. So for example, if you tell an object to run an alarm, it will ask for what number of alarm that you want. But that is not applicable here and we just put zero. So finally,


[User] Please transcribe this text

[Assistant] The last thing we do is we just increment yy for the next instance in our repeat loop. So in that way, we are going through the grid one at a time.


[User] Please transcribe this text

[Assistant] So because this object is taking care of all the drawing for us,


[User] Please transcribe this text

[Assistant] We don't need the objects drawing themselves anymore, and the way we can stop them doing this...


[User] Please transcribe this text

[Assistant] Is if you come to OBJ player for example, and we untick visible,


[User] Please transcribe this text

[Assistant] So OBJ player will no longer be drawing itself. So basically what GameMaker does every frame when it's time to draw all of the objects to the screen, it kind of goes through all of the objects one by one by default and it will run the draw event if you have anything in the draw event and if it doesn't, it will just draw itself. And that means just drawing whatever sprite that you've set to the object.


[User] Please transcribe this text

[Assistant] So by unticking visible, we kind of, we remove that object from the rendering pipeline, and GameMaker is no longer gonna get that object to draw itself, which is what we want, because it is being handled by our depth sorter. So we need to do this for all of the objects, because they all get their own visible checks, it's not something that's handled in an object hierarchy.


[User] Please transcribe this text

[Assistant] Right now, we've added it to this instances layer in room 0, and this layer has a depth of 0. And because the object is persistent, what's basically gonna happen when it comes into the next room, is it's gonna stay at that depth of 0. And that means when we're actually going through this loop,


[User] Please transcribe this text

[Assistant] Yeah, and we're getting the instances to draw themselves. It will actually draw all of the instances at this layer. So when we do a with a statement like this, it will just use the depth of whatever object that is currently accessing it.


[User] Please transcribe this text

[Assistant] Which is not really what we want. We don't want it to be drawing at depth of zero. We want it to be drawing at whatever the instances is because remember how we set up these tiles that would always be above the instances and we want to keep that. So because there might be a different number of layers in all of the rooms.


[User] Please transcribe this text

[Assistant] So right now, the instances layer has a depth of 800, but in a different room, it could have a depth of 200.


[User] Please transcribe this text

[Assistant] So we need our depth object to be changing its depth every time a new room starts. So we're going to go into the room start event and we're basically going to change the layer that it's in. We're going to add the instance to the instances layer.


[User] Please transcribe this text

[Assistant] So, and that will add it to whatever the instances layer currently is in a new room.


[User] Please transcribe this text

[Assistant] Alright, so if we run that now, everything should be working.


[User] Please transcribe this text

[Assistant] So technically the tutorial is done at this point. We've implemented the depth system, but we still have that problem of the NPCs no longer having collisions. So we're going to fix that by again taking advantage of the parent system.


[User] Please transcribe this text

[Assistant] So we could do two things. We could make the parent depth object a child of obj collision, or we could do it the other way around.


[User] Please transcribe this text

[Assistant] But there are two problems with either. If we make the collision object a child of the depth system, that's going to mean that some of our invisible collision objects are going to be added to the depth system, which is not what we want. Those are supposed to be invisible. We don't want them as part of the depth system.


[User] Please transcribe this text

[Assistant] And then the other way around, if we make the depth object part of OBJ collision, there might be some objects that we don't want to be collidable. So, for example, the crops, we might not want them to have any collision with the player, we might just want to be able to walk over them freely.


[User] Please transcribe this text

[Assistant] And when I was making this video, I kind of, I had a look at both ways of doing it and you can do it either way, but it ended up being a bit easier to make the parent depth object, a child of the collision. So that's the way that we're going to do it. So let's go ahead and make the parent of the depth objects, the collision objects.


[User] Please transcribe this text

[Assistant] And this means that anything that is checking for a collision with OBJ collision, so any child of OBJ collision, if it meets any of those objects, it's going to have a collision event, right? So for example, we have that code in our player.


[User] Please transcribe this text

[Assistant] So we're going to have to kind of change some of this code to check not only if we're colliding with a collision object, but we're also going to check, we're going to make a variable called collidable. So we're going to check that an object is collidable before going ahead and doing this.


[User] Please transcribe this text

[Assistant] So we'll set up that variable first. So we'll go into the collision object and we're going to set up a variable called collidable.


[User] Please transcribe this text

[Assistant] And we'll set this to true by default. And then for any objects that we don't want to be collidable, we will change this to false. Because I anticipate most of the visible objects in the game will have a collision, and there will just be a few that don't.


[User] Please transcribe this text

[Assistant] Then we'll head over to the depth object, and the way inheritance works is if you don't have any events here,


[User] Please transcribe this text

[Assistant] It will just run all of its parent events, unless you go ahead and override them. So right now, because obj-collision has this in its create event,


[User] Please transcribe this text

[Assistant] Our parent depth object will also be running that exact same code. So it's going to be declaring the collidable variable as well. But if we go ahead and put create.


[User] Please transcribe this text

[Assistant] That's going to overwrite the default parent event and it will no longer be running that code. And what we would have to do is write event inherited.


[User] Please transcribe this text

[Assistant] That would mean it would run its parent's create event first here, and then we can go ahead and do other stuff in the create event.


[User] Please transcribe this text

[Assistant] So right now we don't need that, but a lot of our other objects, they have their own create events, which means they will all be overriding this. So we will have to add...


[User] Please transcribe this text

[Assistant] Event inherited right here to make sure that they're running their parents or their parents' parents' create event and we'll need to add this to every single object. So let's go ahead and do that.


[User] Please transcribe this text

[Assistant] Uh, actually that one is already inheriting from the parent mpc, so we just need to add that to this one.


[User] Please transcribe this text

[Assistant] Except we don't want the crops to be running collidable equals true because we don't want the crops to be collidable. So I'm going to say collidable equals false.


[User] Please transcribe this text

[Assistant] Alright, so now we have to change our collision events to be checking not only if there is a collision, but we also want to check if the object we're colliding with has the variable Collidable.


[User] Please transcribe this text

[Assistant] So we're going to have to change this slightly. So a place meeting just checks if there is a collision at a point and it returns true or false if there is a collision or if there isn't. But we can use a slightly different function called instance place. And this will actually return the ID of the instance that we're colliding with if it finds the collision there.


[User] Please transcribe this text

[Assistant] No collision or it will contain the ID of an instance. So what we can check is if this variable is not equal to no one, because that is what it's going to return if it does not find anything.


[User] Please transcribe this text

[Assistant] So if it's not equal to no one, that means there's something there. And so the collisions, whatever ID it's storing, so it's going to get the collidable variable from it. So it's going to check if this is equal to true, if that is easier to kind of read for you, or we can just leave it as that because that will either be true or false.


[User] Please transcribe this text

[Assistant] So if this is the case and this is true, then it will go ahead and do all this, but otherwise it won't.


[User] Please transcribe this text

[Assistant] So and we'll do the same thing here. So we change this to instance place and then we check.


[User] Please transcribe this text

[Assistant] If the collision, the vertical collision is equal to no one and this returns true.


[User] Please transcribe this text

[Assistant] Okay, so that is one way to go about that. Let's check if it's worked.


[User] Please transcribe this text

[Assistant] Okay, and we'll run into an NPC. Yep, that's working. And do they collide with us? Yes. Okay. And with that, we are done. That is it for today.


[User] Please transcribe this text

[Assistant] I'd like to thank everyone on Patreon who is supporting me to create these tutorials. And a special shout out to 3dmonkeybiz and Uthelion. Thanks for watching guys, I'll see you next time.


[User] Please transcribe this text

[Assistant] Warriors number two, Fire and Ice by Aaron Hunter. Performed by McLeod Andrews. Prologue.


[User] Please transcribe this text

[Assistant] Orange flames lapped at the cold air, throwing sparks up into the night sky.


[User] Please transcribe this text

[Assistant] The firelight flickered across a wasteland of ragged grass, making silhouettes of the two legs huddled there.


[User] Please transcribe this text

[Assistant] A pair of white lights appeared in the distance, heralding the approach of a monster.


[User] Please transcribe this text

[Assistant] It roared past on a thunder path that rose high into the sky, filling the air with sour fumes. At the edge of the wasteland a cat moved, its eyes glinting in the shadows.


[User] Please transcribe this text

[Assistant] Pointed ears twitched then flattened against the noise.


[User] Please transcribe this text

[Assistant] More cats followed one by one onto the filthy grass.


[User] Please transcribe this text

[Assistant] They carried their tails low and sniffed the bitter air with their lips curled. What if the two legs see us? hissed one of the cats.


[User] Please transcribe this text

[Assistant] A large Tom answered, his eyes like amber disks reflecting the firelight. They won't. Their night sight is weak.


[User] Please transcribe this text

[Assistant] As he padded forward, the flames lit up the black and white fur on his powerful shoulders. He held his long tail straight up, sending a message of courage to his clan.


[User] Please transcribe this text

[Assistant] But the other cats crouched low against the grass, trembling. This was a strange place.


[User] Please transcribe this text

[Assistant] The noise of the monsters battered their sensitive ear fur and the acrid stench stung their nostrils. Tallstar? A Grey Queen flicked her tail uneasily. Why have we come here? The black and white Tom turned to the She-Cat.


[User] Please transcribe this text

[Assistant] We've been driven from every place we've tried to settle, Ashfoot. Perhaps we can find some peace here, he meowed. Peace? Here? Ashfoot echoed in disbelief.


[User] Please transcribe this text

[Assistant] She pulled her kit toward her and sheltered it beneath her belly. With fire and monsters? My kits won't be safe. But we weren't safe at home, meowed another voice.


[User] Please transcribe this text

[Assistant] A black tom pushed his way forward, limping heavily on a twisted paw. He held Tallstar's amber gaze.


[User] Please transcribe this text

[Assistant] We couldn't protect them from ShadowClan, he spat. Not even in our own camp.


[User] Please transcribe this text

[Assistant] Anxious yowls rose from some of the cats as they remembered the terrible battle that had driven them from their home in the uplands at the edge of the forest. A young apprentice wailed


[User] Please transcribe this text

[Assistant] It stood unsteadily and stared toward the shadows. At once the cats fell silent, crouching lower. Even Tallstar lowered his tail.


[User] Please transcribe this text

[Assistant] The two legs shouted into the darkness and flung something toward them.


[User] Please transcribe this text

[Assistant] The cats waited for a few moments before Tallstar stood once more.


[User] Please transcribe this text

[Assistant] Ashford stood too, wincing at the new pain in her shoulder. Tallstar, I fear for our safety here. And what will we eat? I can't smell any prey.


[User] Please transcribe this text

[Assistant] Tallstar stretched his neck and rested his muzzle gently on the Queen's head.


[User] Please transcribe this text

[Assistant] But we'll be safer here than back in our old territory, or in the two leg fields and woods. Look at this place. Even ShadowClan wouldn't follow us here. There's no scent of dogs, and these two legs can hardly stand. He turned to the black tom with the twisted paw. Dead foot, he ordered.


[User] Please transcribe this text

[Assistant] Take one whisker and see if you can find anything to eat. If there are two legs, there must be rats.


[User] Please transcribe this text

[Assistant] Rats! spat Ashfoot, as Deadfoot and a smaller brown tabby bounded away. That's no better than crow food! Hush! hissed a tortoise shell beside her.


[User] Please transcribe this text

[Assistant] Rat meat is better than starving to death.


[User] Please transcribe this text

[Assistant] Ashfoot scowled and dipped her head to lick her kit behind its matted ears.


[User] Please transcribe this text

[Assistant] We must find a new place to settle Ashfoot. The tortoise shell went on more gently. Morning flower needs to rest and eat. Her kits will be born soon. She needs to be strong.


[User] Please transcribe this text

[Assistant] The lean shapes of deadfoot and one whisker emerged from the shadows.


[User] Please transcribe this text

[Assistant] You were right, Tallstar, called Deadfoot. There are rat scents everywhere, and I think I've found somewhere we can shelter. Show us, Tallstar ordered, gathering the rest of his clan with the flick of his tail.


[User] Please transcribe this text

[Assistant] Cautiously, the cats padded across the wasteland after deadfoot. He led them toward the raised thunder path.


[User] Please transcribe this text

[Assistant] The firelight making their shadows loom against its huge stone legs. A monster roared overhead and the ground shook. But even the tiniest kit sensed the need for silence and trembled without crying out. Here, meowed Deadfoot, stopping beside a round hole two cats high. A black tunnel sloped down into the ground. A constant stream of water trickled into it. The water's fresh.


[User] Please transcribe this text

[Assistant] We'll have wet paws day and night, Ashfoot complained. I've been inside, the Black Tom told her. There's some space away from the stream.


[User] Please transcribe this text

[Assistant] At least we'll be safe from two legs and monsters. Tallstar stepped forward and lifted his chin.


[User] Please transcribe this text

[Assistant] WindClan has traveled for long enough, he declared.


[User] Please transcribe this text

[Assistant] It's nearly a moon since ShadowClan drove us from our home. The weather is turning colder, and Leafbear will be here soon. We have no choice but to stay. Ashfoot narrowed her eyes, but said nothing.


[User] Please transcribe this text

[Assistant] Silently she joined her clan as, one by one, they filed into the shadowy tunnel.


[User] Please transcribe this text

[Assistant] Still kinda hot, that's fine. I'm gonna leave that sauce there.


[User] Please transcribe this text

[Assistant] But today, they are in a hurry to get back and play with Holly Parrott.


[User] Please transcribe this text

[Assistant] George is a little bit shy.


[User] Please transcribe this text

[Assistant] In. Written by Cheryl Ryan. Illustrated by Nora Voodis.


[User] Please transcribe this text

[Assistant] Welcome back to the Farming RPG. And today we are going to be continuing with the dialogue system and we're going to be implementing player choices and allowing our dialogue to kind of branch depending on what the player chooses to say.


[User] Please transcribe this text

[Assistant] So, for example, I've just brought up one of my NPC objects, RBJ Alina here.


[User] Please transcribe this text

[Assistant] And basically, what I wanna do is, so remember here we were saying...


[User] Please transcribe this text

[Assistant] In response to her asking us who we were, I'm the player, I'm kind of a big deal. So instead of just this, I want to give the player a few options. So for now, I'm just gonna actually tab this over multiple lines so that I can see it a little bit better.


[User] Please transcribe this text

[Assistant] And the way I'm gonna want to set this out is that I can, instead of just putting one string right here, I wanna have


[User] Please transcribe this text

[Assistant] An array of choices, right? So I'll make that into an array, even if there's only one choice, it'll still be an array, if that's something that you wanna do.


[User] Please transcribe this text

[Assistant] And I could say something like, hey, I'm Alina, too.


[User] Please transcribe this text

[Assistant] And what I also might do is just make this line a little bit longer because as we'll see a little bit later, we might run into some trouble with the text wrapping with dialogue choices because they'll be appearing in a kind of different way than what we've set up using our string wrap function, if you remember that. So I'll just add something to this.


[User] Please transcribe this text

[Assistant] There we go, and if you want you can also tab these over multiple lines as well.


[User] Please transcribe this text

[Assistant] So I think this is quite a nice way to structure this for our own sakes when we're writing out the dialogue, but


[User] Please transcribe this text

[Assistant] Now, as the textbox object, we have to imagine that we're receiving a new kind of input. So, if you remember, I'm just gonna hit Ctrl T.


[User] Please transcribe this text

[Assistant] To find my text box object. There it is. So just come over to it, hit enter. In the draw GUI event, that was where we were drawing the text.


[User] Please transcribe this text

[Assistant] What was happening was, up here, we were drawing all of the textbox elements, that's all fine, we're gonna do that regardless of whether we're displaying the player choice or not.


[User] Please transcribe this text

[Assistant] We set the font, we draw the name, that's gonna be happening every time.


[User] Please transcribe this text

[Assistant] But this stuff, the drawing of the text, this is what is going to be different depending on whether we have a player choice dialogue. Right, so whether we're selecting it.


[User] Please transcribe this text

[Assistant] Or whether it's just an NPC saying something and it's gonna be spelling out. So what I'm going to do is we need some way of checking whether it is a player choice or not.


[User] Please transcribe this text

[Assistant] And that's actually fairly easy because if we head back to Alina, we know it's a player's choice just if this line is an array or not. So if it's an array,


[User] Please transcribe this text

[Assistant] Then we know it's a player choice, if it's not, then we know it's just a normal line.


[User] Please transcribe this text

[Assistant] So we could write if it's not an array.


[User] Please transcribe this text

[Assistant] Text page. So if the current text page that we're on, if that is not an array, then we're going to be doing this as normal. But otherwise...


[User] Please transcribe this text

[Assistant] Now, what I want to do is draw out all of our different choices. So, I'm going to do this with a repeat loop.


[User] Please transcribe this text

[Assistant] So basically, whereas right here we only had...


[User] Please transcribe this text

[Assistant] One line to draw out. This time we have potentially two or three or four lines that we want to draw out. So we'll do this with a repeat. So repeat. And how many times do we want to repeat? Well we want to repeat however long


[User] Please transcribe this text

[Assistant] In this array right here, so we just get the array length. Let's head back down. So array length, 1D. And now what are we going to put in here?


[User] Please transcribe this text

[Assistant] So we don't want to just put text, again, we want to put text page.


[User] Please transcribe this text

[Assistant] And actually just to make it really clear that just so that we remember, this is an array, not a single line. I'm going to make a variable right here called text.


[User] Please transcribe this text

[Assistant] Array. And we'll make that text page. And now we'll put that in here. Okay. So now we're repeating the correct amount of times. And I will declare


[User] Please transcribe this text

[Assistant] i equals zero, we'll increment that each time, and now what we want to do is I'll just copy this.


[User] Please transcribe this text

[Assistant] Paste it in. Although I could just move this outside, I don't need that in here.


[User] Please transcribe this text

[Assistant] I think that C was being declared outside of our if statement, but let's just check. Yep, it's, it's declared right here. So there we go. So we can use that. So now it will be drawing out, what will it be drawing out? So,


[User] Please transcribe this text

[Assistant] Substream was from here, but that's not what we want. We actually want to be drawing out each entry of the text array. So we can access that using i. So let's go. Text array.


[User] Please transcribe this text

[Assistant] But at the moment, as you can see, it is drawing them out at this location. So it will be drawing them on top of each other.


[User] Please transcribe this text

[Assistant] Which is not what we want. So we want text y to be getting larger each time we loop. So I'm going to introduce another variable, yAdd.


[User] Please transcribe this text

[Assistant] We'll start at zero because we want this to be at zero for the first entry, but then each time we want to add something to this y position. And what do we want to add? So we can add


[User] Please transcribe this text

[Assistant] However high this text is going to be, so however many lines that text choice is going to take up, so as we can see though, some lines are going to be larger than others.


[User] Please transcribe this text

[Assistant] We will probably go back and refine this because it's not the most performant way to do this. So we will be adding the string height we can get.


[User] Please transcribe this text

[Assistant] And we're going to do ext because with this function we can actually set it to take into account a maximum text width and then wrap it. And the problem that we were having before, the reason that we set up that script


[User] Please transcribe this text

[Assistant] Here to wrap it ourselves is because we were spelling the text out, but since we're not spelling it out, we can just use this. And we'll actually, we'll use it for this line as well because we want the text to wrap. So let's just add ext.


[User] Please transcribe this text

[Assistant] And we did actually create the variables for the separation, which is the number of pixels that it's going to drop down every time we need a new line and the width, the maximum width that the line can be. So we had declared them in the create, if you want to have a look, but their names were text height.


[User] Please transcribe this text

[Assistant] Alright, that's the separation between the lines we want, and then text max width.


[User] Please transcribe this text

[Assistant] So that will wrap it for us. And now here, when we get the height, we can do the same thing. So we want to get the text array eye, text height, text max width. And before we test this, we are going to just have to hop into here and not do this if it's a choice dialogue.


[User] Please transcribe this text

[Assistant] Text that's going to cause an error. So we don't want to be doing any of this if it's a choice dialogue. So we can just go if exactly as we did before, it's not an array.


[User] Please transcribe this text

[Assistant] Otherwise, actually what we could do is just kind of set ourselves up for, well, let's put a couple of these in. So we could take this out.


[User] Please transcribe this text

[Assistant] We can also take this out and let's just call this instead text array length. And we'll put that in here.


[User] Please transcribe this text

[Assistant] I'll leave that for now, like I said, just because it's a little bit trickier to organise. But that should all be good. Alright, let's run the game.


[User] Please transcribe this text

[Assistant] Alright, so let's go talk to Alina. There we go, so the first dialogue's working. And the second dialogue. So it is printing them out correctly. It's getting the right wrapping.


[User] Please transcribe this text

[Assistant] You might want to test this a little bit more extensively, so you might want to have a few varying lines of dialogue and see if they all spell out correctly.


[User] Please transcribe this text

[Assistant] So the text is drawing correctly, but we can't choose any dialogue. And if we hit E, it just goes to the next line.


[User] Please transcribe this text

[Assistant] So let's get it displaying our choice. And for this, I'm going to create a few variables. So I'm going to make a choice equal zero.


[User] Please transcribe this text

[Assistant] What we wanna do is just to give some visual feedback to the player. If...


[User] Please transcribe this text

[Assistant] choice, that variable that we just set, which is 0 at the moment, if it is currently equal to whatever i is,


[User] Please transcribe this text

[Assistant] So if choice is equal to zero, which it is at the start, then what we want to do is change the color. So it kind of looks like we're selecting it. So let's do that first. Let's go.


[User] Please transcribe this text

[Assistant] Choice is equal to i, then let's change.


[User] Please transcribe this text

[Assistant] C to equal, so something different. So let's say instead of text cull, let's go twice cull.


[User] Please transcribe this text

[Assistant] So we haven't said this variable, let's just quickly do that. So we'll take that back to the create down here and you can make that whatever color you want. I might just do white for now.


[User] Please transcribe this text

[Assistant] Because uhm, at the moment, if it changes it to this, it's going to be changed for the resulting loop. So we have to change it back. If it's not, then we'll just set it as text column.


[User] Please transcribe this text

[Assistant] And maybe additionally, so we're just changing the colour at the moment, but how about we also add a little indicator, because when you have two choices it can be a little tricky for the player to understand.


[User] Please transcribe this text

[Assistant] Which one that they're selecting. So if you add some kind of visual indicator like a little arrow or something to the text, it can be a little bit clearer. So what I'm going to do.


[User] Please transcribe this text

[Assistant] You could draw a sprite, but I'm just gonna draw a little, uh, one of these. I'm gonna draw a text.


[User] Please transcribe this text

[Assistant] Color. And the place that I want to draw this at is actually gonna be the end of the string, so like right over.


[User] Please transcribe this text

[Assistant] And then, but not outside the box. So, if you remember, in our create event, we had a few different x locations, and the one I'm going to use...


[User] Please transcribe this text

[Assistant] Is I'm going to go the box X and then I'm going to add the maximum text width.


[User] Please transcribe this text

[Assistant] So that it's still within the text box, but it's after any text that could be displayed. Because for me personally, otherwise it goes outside the box, but you might want to play with this value. So I'm gonna go box x plus text max width.


[User] Please transcribe this text

[Assistant] And we want it to be drawing at the same text y as the string, so we can do the same for that. Y add.


[User] Please transcribe this text

[Assistant] So the string, like I said, I'm gonna draw one of these little arrows. And the color, we want it to be matching whatever this is. And 1 alpha. Okay. Great. So, let's just test that.


[User] Please transcribe this text

[Assistant] It should just, it should be displaying that first option as white. At the moment we can't change it, but we should get a visual indicator. There we go. Okay.


[User] Please transcribe this text

[Assistant] So now we want to allow the player to press up and down to change the choice.


[User] Please transcribe this text

[Assistant] And let's do that in step, because that's where we were kind of changing the interact button. So again, I want to be checking if it's a choice dialogue, then take in the up and down input of the player. And because I've been


[User] Please transcribe this text

[Assistant] Repeating myself a bit here. We've written, I think...


[User] Please transcribe this text

[Assistant] Choice dialog here, we want to set to false if it's not an array.


[User] Please transcribe this text

[Assistant] Okay, so we can replace that with the Toys dialog.


[User] Please transcribe this text

[Assistant] And in a similar way to how we gathered inputs way back for the


[User] Please transcribe this text

[Assistant] Movement of the player, what I'm gonna do is have choice. Basically...


[User] Please transcribe this text

[Assistant] Plus equals, so we're going to add to it, and then what we're going to add is keyboard, check, pressed. So vk down.


[User] Please transcribe this text

[Assistant] Minus keyboard check pressed vk up. So, now, let's think about that.


[User] Please transcribe this text

[Assistant] Either the down button, which means the choice will kind of be going down, which means we want to be going down the array. So if that's true, it would be one.


[User] Please transcribe this text

[Assistant] Let's just say our input is 1, minus if we're pressing up.


[User] Please transcribe this text

[Assistant] That would make it minus one, and we're gonna throw an error because that's an invalid entry for the text array. So.


[User] Please transcribe this text

[Assistant] When it equals minus one, or choice is larger than the kind of number of entries that are in here, then we want to wrap it back around.


[User] Please transcribe this text

[Assistant] Array length, so let's do that one first. So if choice is larger than, so in this case, there are two entries, which means text array length is going to be equal to two.


[User] Please transcribe this text

[Assistant] But, as you know, the entries themselves, they will start at 0, so we have to subtract.


[User] Please transcribe this text

[Assistant] Minus one, so we don't want choice getting bigger than one, is what this means. Which is what we want, because if it gets zero, one, and then gets to two, then we want it to wrap around to zero. So then we'll set choice.


[User] Please transcribe this text

[Assistant] Let's go talk to Alina. There we go, and we press up. Or down actually. There we go, and down again. Good, so it wrapped back around. Now let's press up. Perfect. Alright. So now...


[User] Please transcribe this text

[Assistant] At the moment, if I hit enter, it's just going to go to the next page.


[User] Please transcribe this text

[Assistant] But what we might want to do is, depending on what choice is at the time that we hit interact key, we might want to send the player to a different line.


[User] Please transcribe this text

[Assistant] So what I'm gonna do here is set up a new variable in the NPCs, and actually I might go to the NPC object first.


[User] Please transcribe this text

[Assistant] And again, we're going to put in an array of entries. And what I want this to do is basically you can set what line that we want to go to next. If I put zero,


[User] Please transcribe this text

[Assistant] I'm going to have that be my case for if I just want to go to the next line. So this is going to be the default value. Zero is just going to go to the next. You can choose something different.


[User] Please transcribe this text

[Assistant] I recommend it being 0 or something less than 0, because personally I don't think I'm ever gonna send the player back to the very first line. But you might, so you might not want to actually use 0. And you don't want it to be a number greater than that, because that could be a line that you actually want to send the dialogue to. So you wanna use probably negative numbers.


[User] Please transcribe this text

[Assistant] I'm going to use, like I said, I'm going to use zero to mean next line and minus one is going to be quit the dialogue. Those are going to be my two special values. So we've initialized it there. Let's head over to Alina. So I'm going to set next line.


[User] Please transcribe this text

[Assistant] So that one's going to be zero, but this choice dialogue, just like how the text has an array within here, I'm going to have an array of different lines that we could send the dialogue to. So I'll have that be one line, but I'll also have this line be the case if you say, hey, I'm a linear as well.


[User] Please transcribe this text

[Assistant] So, in the first case, we want to send the player to the next line, or we could say send them to 0, 1,


[User] Please transcribe this text

[Assistant] Two, so this is line two. We could say, send them to line two.


[User] Please transcribe this text

[Assistant] And in this case, we want to send them to line three.


[User] Please transcribe this text

[Assistant] So now for this line, I don't want to go and play this line of dialogue too. I want to just quit the dialogue immediately. Right? And the same for this. Okay, cool. So that is the structure of the dialogue. Now we have to set that up in the text box. So we're going to have to change this right here.


[User] Please transcribe this text

[Assistant] So first off, I only want to be doing this if it's not a choice dialogue. So this was that thing checking if the counter was larger than the string length. So I can just say if not, choice dialogue and this. Alright, good.


[User] Please transcribe this text

[Assistant] NextLine variable, and it just occurred to me that we haven't actually sent it over to the textbox, so remember how in the player.


[User] Please transcribe this text

[Assistant] We were sending over all the NPC's variables when we collide with it and interact. Right. Here.


[User] Please transcribe this text

[Assistant] Right, so within this, we are sending over text and speakers. We haven't sent over next line. And we're gonna need to change this script to also get the next line. So it'll come up in the argument now.


[User] Please transcribe this text

[Assistant] And we can set, so remember this is the text boxes variable. Next line will be equal to.


[User] Please transcribe this text

[Assistant] Argument two. So whatever the NBC is sending over. All right. So we can head back to the text box.


[User] Please transcribe this text

[Assistant] Alright, so now we do have a variable called nextLine within the textbox. We can access it, and we want to access whatever page that we're currently on, right?


[User] Please transcribe this text

[Assistant] Now, remember that this could be an array or it might not be, depending on whether it's a choice dialogue or not. So, if it's a choice dialogue, then I want to be getting whatever number that we put in here


[User] Please transcribe this text

[Assistant] That is the same as the choice variable, right? So choice, whatever choice is currently equal to, if it's equal to 0, 1, 2, we want to be getting that entry from this array because they kind of match each other.


[User] Please transcribe this text

[Assistant] Right, so we're going to be setting line equal to line choice. So now we've got the right thing contained in line.


[User] Please transcribe this text

[Assistant] Now what we wanna do is instead of just incrementing page, like I said, we want to change page equal to line.


[User] Please transcribe this text

[Assistant] That's if we haven't put zero or minus one, because remember those were kind of code for just go to the next one or stop the dialogue entirely. So we want a couple cases here. So if line is equal to zero,


[User] Please transcribe this text

[Assistant] Then in that case we do just want to increment page, but if line is equal to minus one, so that should be else if, if it's equal to minus one,


[User] Please transcribe this text

[Assistant] Then what we actually want to do is this. We want to destroy it, right? That was our exit case.


[User] Please transcribe this text

[Assistant] And what I'm also going to do is just exit the year code block at this point because I don't need to perform any of the subsequent code. And then finally.


[User] Please transcribe this text

[Assistant] If it's not equal to zero and it's not equal to minus one, then it must be a valid input to change the page to a certain line. Then that's what we'll do. We'll set page equal to whatever we've put here.


[User] Please transcribe this text

[Assistant] And it will go ahead and perform event one, which will set up the dial again. Okay.


[User] Please transcribe this text

[Assistant] Whoops, actually I forgot to add the rest of my entries for speakers. So I'll just quickly add ID. So remember when we were creating the text box.


[User] Please transcribe this text

[Assistant] Every entry for each line of your dialogue.


[User] Please transcribe this text

[Assistant] Okay, so now we can select an entry and let's try this one and hope that it sends us to the right line. It did. And if we talk to her again, then we pick the first one, then it sent us to that line and it's exiting correctly as well. Okay, so we've added choices to the dialogue system.


[User] Please transcribe this text

[Assistant] So that's all that we'll do for today, and in the next tutorial I also want to add the ability to pass along a script to the textbox in a similar way that we were passing along text, speakers, and nextline. I want to be able to send over a script and arguments to pass into the script.


[User] Please transcribe this text

[Assistant] Like I had initially said, we could change the location of the camera. So basically, allowing us to change anything about the game that we want. We can perform any script that we make up.


[User] Please transcribe this text

[Assistant] Yes, and you can paint us another lovely picture. And what will you paint next time, George?


[User] Please transcribe this text

[Assistant] I know. I don't care. Cause they know if they tell anyone, I'll-


[User] Please transcribe this text

[Assistant] I have been trying to get my new room in order. This has not been easy.


[User] Please transcribe this text

[Assistant] Because I own over a thousand books. To make matters worse, the room is rather small. So I have temporarily put my books on the floor. At the moment,


[User] Please transcribe this text

[Assistant] My sister helped me to carry one of my old bookcases up the stairs.


[User] Please transcribe this text

[Assistant] I got a big surprise when she saw all those books on the floor.


[User] Please transcribe this text

[Assistant] This is the prettiest carpet I have ever seen, she said. She gazed at it for some time and then added, You don't need bookcases at all.


[User] Please transcribe this text

[Assistant] You can sit here in your spare time and read the carpet.


[User] Please transcribe this text

[Assistant] George, I'm gonna throw the ball, and then you have to catch it. George, please stop hiccuping.


[User] Please transcribe this text

[Assistant] George, I know how to cure hiccups. You have to do what I say.


[User] Please transcribe this text

[Assistant] George, you've done it all wrong. I know a better way to cure hiccups.


[User] Please transcribe this text

[Assistant] First, jump up and down three times on one leg.


[User] Please transcribe this text

[Assistant] Remember, this is just pretend scaring.


[User] Please transcribe this text

[Assistant] You know, it's pretty telling that you snatches can't even keep your stupid mirror shtick together. It ain't cute.


[User] Please transcribe this text

[Assistant] Professor and chair of the UCLA Department of Art, it's my pleasure to introduce tonight's speaker, James Benning.


[User] Please transcribe this text

[Assistant] This lecture was made possible through the generous support of the William D Feldman Family Endowed Lecture Fund. And as always, we're grateful to the Hammer Museum for continuing to host the UCLA Department of Art lectures.


[User] Please transcribe this text

[Assistant] James Benning is a widely acclaimed experimental filmmaker. He's received...


[User] Please transcribe this text

[Assistant] That same year, Benning embarked on the strange ambition to replicate the cabins of a famous 19th century author and an infamous 20th century author, Henry David Thoreau's shelter from Walden Pond.


[User] Please transcribe this text

[Assistant] And Ted Kaczynski's aka the Unabomber's plywood shack from Montana.


[User] Please transcribe this text

[Assistant] Both of which Benning completed the following year on his property in the Sierra foothills.


[User] Please transcribe this text

[Assistant] Today, three very different film meditations have come from the two cabins.


[User] Please transcribe this text

[Assistant] Benning's installations have been staged at the Walker Art Center, Minneapolis, the Whitney Museum of American Art, New York,


[User] Please transcribe this text

[Assistant] Uh, Argo Center for Art and Media, Brussels. Earlier this year, he opened a solo museum exhibition titled Decoding Fear at Kunsthaus in Graz, Austria. And his films have shown throughout Asia, the Americas, and Europe.


[User] Please transcribe this text

[Assistant] He teaches film video at CalArts. Please join me in welcoming James Benning.


[User] Please transcribe this text

[Assistant] Once a year, a race is held for old cars.


[User] Please transcribe this text

[Assistant] A lot of cars entered for this race last year.


[User] Please transcribe this text

[Assistant] And there was a great deal of excitement just before it began. One of the most handsome cars was a Rolls Royce Silver Ghost. The most unusual car was a Benz, which had only three wheels. Built in 1885, it was the oldest car taking part.


[User] Please transcribe this text

[Assistant] And some drivers spent more time under their cars than in them.


[User] Please transcribe this text

[Assistant] A few cars, however, completed the race.


[User] Please transcribe this text

[Assistant] The winning car reached a speed of 40 miles an hour.


[User] Please transcribe this text

[Assistant] Much faster than any of its rivals. It sped downhill at the end of the race.


[User] Please transcribe this text

[Assistant] And its driver had a lot of trouble trying to stop it.


[User] Please transcribe this text

[Assistant] It was very different from modern car races, but no less exciting.


[User] Please transcribe this text

[Assistant] Oh, good morning. Uhm, I'm flying to Toronto Airport next week and I need to get to a town called, uhm, Milton. Could you tell me how I can get there?


[User] Please transcribe this text

[Assistant] Oh, good morning. Uhm, I'm flying to Toronto Airport next week and I need to get to a town called, uhm, Milton. Could you tell me how I can get there?


[User] Please transcribe this text

[Assistant] Right. Well, I don't really want to drive myself, so I'd like more information about public transport.


[User] Please transcribe this text

[Assistant] I think for that kind of distance a cab would be way beyond my budget. But the bus sounds okay. Can you tell me how much that would cost?


[User] Please transcribe this text

[Assistant] Ah, that's quite cheap, great. Uh, whereabouts does it stop in Milton?


[User] Please transcribe this text

[Assistant] Ah, of course. Hang on. We're due to get there at 11.30am.


[User] Please transcribe this text

[Assistant] Oh, I see. Well, what about the shuttle you mentioned?


[User] Please transcribe this text

[Assistant] Oh, that doesn't sound too bad. Especially if it'll take me straight to the hotel.


[User] Please transcribe this text

[Assistant] Is it possible to make a booking right now, through you?


[User] Please transcribe this text

[Assistant] The 16th of October. Oh, oh no. Sorry, that's my departure date. I arrive on the 17th, so book it for them please.


[User] Please transcribe this text

[Assistant] Yeah, that sounds fine. As long as we land on time.


[User] Please transcribe this text

[Assistant] Yeah, that'll be fine. Provided I can book the return trip once I'm there.


[User] Please transcribe this text

[Assistant] Yes, it's called the Vacation Motel, and I think it's near the town centre.


[User] Please transcribe this text

[Assistant] Anyway, the address is 24 Kitchener Street. That's K-I-T.


[User] Please transcribe this text

[Assistant] Welcome to all of you. Can everybody see and hear me? Good.


[User] Please transcribe this text

[Assistant] I'm Sally, your guide for this tour of the Bicentennial Park.


[User] Please transcribe this text

[Assistant] I hope that you're all wearing your most comfortable shoes and that you can keep up the pace.


[User] Please transcribe this text

[Assistant] So, let's get underway on our tour around this wonderful park.


[User] Please transcribe this text

[Assistant] I'll start today with some general background information.


[User] Please transcribe this text

[Assistant] There used to be a lot of factories in this area until the 1960s.


[User] Please transcribe this text

[Assistant] Creating the park required the demolition of lots of derelict buildings on the site.


[User] Please transcribe this text

[Assistant] So most of the exciting park space all around you was originally warehouses and storehouses. The idea of building a public park here


[User] Please transcribe this text

[Assistant] was first discussed when a property developer proposed a high-rise housing development, but the local community wasn't happy. If the land was to be cleaned up, they wanted to use the site for recreation.


[User] Please transcribe this text

[Assistant] Residents wanted open space for outdoor activities.


[User] Please transcribe this text

[Assistant] Rather than housing or even an indoor sports complex.


[User] Please transcribe this text

[Assistant] Now, to the Bicentennial Park itself. It has two areas, a nature reserve and a formal park with man-made features and gardens.


[User] Please transcribe this text

[Assistant] The tall blue and white building in front of us is called the Tower and is the centre point for the formal gardens. It stands 12 metres high.


[User] Please transcribe this text

[Assistant] So follow me up the stairs to where we can take advantage of the fantastic views.


[User] Please transcribe this text

[Assistant] Here we are at the top of the tower and we're going to look at the view from each direction.


[User] Please transcribe this text

[Assistant] Out to the east, the large buildings about a kilometre away are on the Olympic site.


[User] Please transcribe this text

[Assistant] There's an indoor arena for gymnastics, a stadium for track and field, and a swimming pool for races and synchronised swimming, and also diving.


[User] Please transcribe this text

[Assistant] If you look carefully down there you can see the train lines. The Olympic site has its own station to encourage the use of public transport.


[User] Please transcribe this text

[Assistant] There is also a car park, but it only holds a limited number of cars.


[User] Please transcribe this text

[Assistant] The formal park has some specially created water features. If you look out here to the south, you can see a circular ornamental pond.


[User] Please transcribe this text

[Assistant] And around to the west, you can relax and sit on a bench to smell the flowers and the rose garden. And finally up to the north, if you look in front of you now, there's a lake with a small island in the center. You can hire rowing boats at the boat shed, which you, uhm,


[User] Please transcribe this text

[Assistant] Can't see from here, but if you look through the trees, you can see the cafe, which has lovely views across the water. Okay, let's climb down now.


[User] Please transcribe this text

[Assistant] We will go now and have a look at the nature reserve section of the park, which has opened up natural wetland to the public.


[User] Please transcribe this text

[Assistant] You'd think that people would come here to look at the unusual plant life of the area, but in fact it's more often used for cycling and is very popular with the local clubs.


[User] Please transcribe this text

[Assistant] This is the far end of the park, and over there you can see the frog pond, a natural feature here long before the park was designed.


[User] Please transcribe this text

[Assistant] Just next to it we have our outdoor classroom, a favourite spot for school parties. The area is now most often used by primary schools for biology lessons.


[User] Please transcribe this text

[Assistant] And finally, let's pass by the Waterbird Refuge. This area is in a sheltered part of the estuary.


[User] Please transcribe this text

[Assistant] That's why the park's viewing shelter is a favourite spot for birdwatchers who can use it to spy through binoculars.


[User] Please transcribe this text

[Assistant] You can watch a variety of water birds, but most visitors expect to see black swans when they come to the shelter. You might spot one yourself right now. Well, here we are back at our starting point.


[User] Please transcribe this text

[Assistant] James, thanks a lot. Great presentation. I really appreciate it. I showed your film Tabletop, uh, in my Pixelvision festival. It's 24 years still showing toy video camera films. I can get you a copy, no problem.


[User] Please transcribe this text

[Assistant] Uhm, tell us what attracted you to using this toy video camera and did you ever make a second pixel vision film?


[User] Please transcribe this text

[Assistant] There are many different ways of getting round the city, and we'd like to suggest some you may not have thought of. How about a city trip by boat? There are four main stopping points. From west to east, stop A, Greenbanks, stop B, City Bridge, stop C, Roman Landing, and stop D, Newtown.


[User] Please transcribe this text

[Assistant] You can find the main booking office at stop A. The first boat leaves at 8am and the last one at 6.30pm.


[User] Please transcribe this text

[Assistant] There are also many attractions you can visit along the river. At stop A, if you have time


[User] Please transcribe this text

[Assistant] You can visit the fine 16th century palace here, built for the king, with its beautiful formal gardens. It's very near the booking office.


[User] Please transcribe this text

[Assistant] Now you can enjoy every corner of this superb residence. Stop B. Why don't you visit Tower Restaurant with its wide range of refreshments?


[User] Please transcribe this text

[Assistant] This is a place where you can sit and enjoy the wonderful views over the old commercial and banking center of the city. Stop C


[User] Please transcribe this text

[Assistant] Is the area where in the first century AD invading soldiers crossed the river. This was much shallower than it is now. That's why this area is called Roman Landing. There's an interactive museum to visit here with a large shop which has a good range of local history books. At the furthest point of the trip, Stop D, the most exciting place to visit,


[User] Please transcribe this text

[Assistant] ...is the new entertainment complex with seven-screen cinema, bowling alley, and video games arcade.


[User] Please transcribe this text

[Assistant] Besides the boat tours, there are city buses. Two companies offer special services.


[User] Please transcribe this text

[Assistant] The Top Bus Company runs all its tours with a live commentary in English. Tours leave from 8.30am every 20 minutes. There are departures from Central Station, Castle Hill and Long Walk. This is a hop-on hop-off service and tickets are valid for 24 hours.


[User] Please transcribe this text

[Assistant] The number one sightseeing tour is available with a commentary in eight languages. Buses depart from Central Station


[User] Please transcribe this text

[Assistant] Every five to six minutes, from about 9am.


[User] Please transcribe this text

[Assistant] With the last bus at around 7pm.


[User] Please transcribe this text

[Assistant] There are also number one services with an English speaking guide.


[User] Please transcribe this text

[Assistant] We're pleased to welcome Dr Martin Meriwether of the Antarctic Centre in Christchurch, New Zealand.


[User] Please transcribe this text

[Assistant] Who has come along to talk to us today about the role of the Centre and the Antarctic Treaty.


[User] Please transcribe this text

[Assistant] Now my first question is about the choice of location for the centre. Why Christchurch? Was it because of the climate?


[User] Please transcribe this text

[Assistant] Right, so it's because of where we are, coupled with our historical role.


[User] Please transcribe this text

[Assistant] So tell us, what is the main purpose of the centre?


[User] Please transcribe this text

[Assistant] And which countries are actually located at the centre?


[User] Please transcribe this text

[Assistant] Really? And what does the visitor centre offer?


[User] Please transcribe this text

[Assistant] What is it actually like at the South Pole? I know you've been there on a number of occasions.


[User] Please transcribe this text

[Assistant] Amazing. To think that it was once attached to Africa.


[User] Please transcribe this text

[Assistant] Now, let's just have a look at the Antarctic Treaty. How far back does the idea of an international treaty go?


[User] Please transcribe this text

[Assistant] What do you see as the main achievements of the treaty?


[User] Please transcribe this text

[Assistant] That's important, since the territory belongs to everyone.


[User] Please transcribe this text

[Assistant] Well, I'm afraid we're going to have to stop there because I'm afraid we've run out of time.


[User] Please transcribe this text

[Assistant] Thanks for coming along today and telling us all about the Centre and its work.


[User] Please transcribe this text

[Assistant] International events are about moments. We don't know if that is the series defining moment, but for BDD and Kingzone to clutch that fight, we were walking up here expecting to be talking about a 2-0 for RNG. The game changed on one big team fight.


[User] Please transcribe this text

[Assistant] It's to which degree we can teach systems to act, to perceive and act in this world from data.


[User] Please transcribe this text

[Assistant] So let's take a step back and think of what is the full range of tasks that an artificial intelligence system needs to accomplish. Here's the stack.


[User] Please transcribe this text

[Assistant] From top to bottom. Top the input, bottom output.


[User] Please transcribe this text

[Assistant] The environment at the top, the world that the agent is operating in.


[User] Please transcribe this text

[Assistant] Sensed by sensors. Taking in the world outside and converting it to raw data interpretable by machines. Sensor data. And from that raw sensor data, you extract features.


[User] Please transcribe this text

[Assistant] And as we discussed, you form higher and higher order representations, a hierarchy of representations based on which the machine learning techniques can then be applied.


[User] Please transcribe this text

[Assistant] Once the machine learning techniques, the understanding, as I mentioned, converts the data into features into higher order representations and into simple, actionable, useful information, we aggregate that information into knowledge. We take the pieces of knowledge extracted from the data,


[User] Please transcribe this text

[Assistant] Through the machine learning techniques and to build a taxonomy.


[User] Please transcribe this text

[Assistant] A library of knowledge. And with that knowledge, we reason. An agent is tasked to reason.


[User] Please transcribe this text

[Assistant] To connect pieces of data seen in the recent past or the distant past to make sense of the world it's operating in.


[User] Please transcribe this text

[Assistant] And finally to make a plan of how to act in that world based on its objectives. Based on what it wants to accomplish. As I mentioned, a simple but commonly accepted definition of intelligence.


[User] Please transcribe this text

[Assistant] It's a system that's able to accomplish complex goals.


[User] Please transcribe this text

[Assistant] So a system that's operating in an environment in this world must have a goal, must have an objective function, a reward function, and based on that it forms a plan and takes action.


[User] Please transcribe this text

[Assistant] Effectors with which it applies the actions to change something about the world. That's the full stack of an artificial intelligence system that acts in the world. And the question is


[User] Please transcribe this text

[Assistant] What kind of task can an artificial intelligence system learn as we understand AI today?


[User] Please transcribe this text

[Assistant] We will talk about the advancement of deeper enforcement learning approaches in some of the fascinating ways it's able to take much of the stack and treat it as an end-to-end learning problem.


[User] Please transcribe this text

[Assistant] But we look at games. We look at simple, formalized worlds. While it's still impressive, beautiful, and unprecedented accomplishments, it's nevertheless formal tasks.


[User] Please transcribe this text

[Assistant] Can we then move beyond games and into expert tasks of medical diagnosis, of design,


[User] Please transcribe this text

[Assistant] And into natural language and finally the human level tasks of emotion, imagination, consciousness.


[User] Please transcribe this text

[Assistant] Let's once again review the stack in practicality in the tools we have. The input for robots operating in the world from cars to humanoid to drones.


[User] Please transcribe this text

[Assistant] There's LiDAR camera, radar, GPS, stereo cameras, audio, microphone, networking for communication, and the various ways to measure kinematics with IMU.


[User] Please transcribe this text

[Assistant] It's then processed, features are formed, representations are formed and multiple higher and higher order representations. That's what deep learning gets us before neural networks, before the advent of


[User] Please transcribe this text

[Assistant] Before the recent successes of neural networks to go deeper and therefore be able to form high order representations of the data.


[User] Please transcribe this text

[Assistant] That was done by experts, by human experts. Today, networks are able to do that. That's the representation piece. And on top of the representation piece, the final layers of these networks are able to accomplish the supervised learning tasks, the generative tasks, and the unsupervised clustering tasks.


[User] Please transcribe this text

[Assistant] Through machine learning, that's what we talked about a little in lecture one and we'll continue tomorrow and Wednesday. That's supervised learning.


[User] Please transcribe this text

[Assistant] And you can think about the output of those networks as simple, clean, useful, valuable information. That's the knowledge.


[User] Please transcribe this text

[Assistant] And that knowledge can be in the form of single numbers, it can be regression, continuous variables, it can be a sequence of numbers, it can be images, audio, sentences, text.


[User] Please transcribe this text

[Assistant] Speech. Once that knowledge is extracted and aggregated, how do we connect it in multi-resolutional ways, form hierarchies of ideas, connect ideas?


[User] Please transcribe this text

[Assistant] The trivial silly example is connecting images, activity recognition and audio for example.


[User] Please transcribe this text

[Assistant] If it looks like a duck, quacks like a duck, and swims like a duck, we do not


[User] Please transcribe this text

[Assistant] Currently have approaches that effectively integrate this information.


[User] Please transcribe this text

[Assistant] To produce a higher confidence estimate that is in fact a duck. And the planning piece.


[User] Please transcribe this text

[Assistant] Fusing the sensory information and making action, control and longer term plans based on that information as we'll discuss today.


[User] Please transcribe this text

[Assistant] Are more and more amenable to the learning approach, to the deep learning approach, but to date have been the most successful and non-learning optimization based approaches, like with the several of the guest speakers we have, including the creator of this robot, Atlas, in Boston Dynamics.


[User] Please transcribe this text

[Assistant] So the question, how much of the stack can be learned end to end from the input to the output? We know we can learn the representation.


[User] Please transcribe this text

[Assistant] And the knowledge from the representation and to knowledge even with the kernel methods of SVM and certainly


[User] Please transcribe this text

[Assistant] With, with neural networks mapping from representation to information.


[User] Please transcribe this text

[Assistant] Mapping from raw sensory data to knowledge. That's where the success, the automated representation learning of deep learning has been a success. Going straight from raw data to knowledge. The open question for us.


[User] Please transcribe this text

[Assistant] Today and beyond is if we can expand the red box there of what can be learned end to end. From sensory data to reasoning, so aggregating forming higher representations of the extracted knowledge.


[User] Please transcribe this text

[Assistant] And forming plans and acting in this world from the raw sensory data. We will show the incredible fact that we're able to do learn exactly what's shown here end to end with deeper enforcement learning on trivial tasks.


[User] Please transcribe this text

[Assistant] In a generalizable way, the question is whether that can then move on to real world tasks of autonomous vehicles, of humanoid robotics, and so on.


[User] Please transcribe this text

[Assistant] That's the open question. So today let's talk about reinforcement learning. There's three types of machine learning.


[User] Please transcribe this text

[Assistant] Unsupervised are the categories at the extremes.


[User] Please transcribe this text

[Assistant] Relative to the amount of human input that's required. For supervised learning, every piece of data that's used for teaching these systems is first labeled by human beings. And on supervised learning on the right is no data is labeled by human beings.


[User] Please transcribe this text

[Assistant] In between is some sparse input from humans. Semi-supervised learning is when only part of the data is provided by humans, ground truth. And the rest must be inferred, generalized by the system. And that's where reinforcement learning falls. Reinforcement learning, as shown there with the cats,


[User] Please transcribe this text

[Assistant] As I said, every successful presentation must include cats.


[User] Please transcribe this text

[Assistant] They're supposed to be Pavlov's cats.


[User] Please transcribe this text

[Assistant] And, uh, ringing a bell, and every time they ring a bell, they're given food and they learn this process.


[User] Please transcribe this text

[Assistant] The goal of reinforcement learning is to learn from sparse reward data, from learn from sparse supervised data and take advantage of the fact that in simulation or in the real world there is a temporal consistency to the world. There is a


[User] Please transcribe this text

[Assistant] Temporal dynamics that follows from state to state to state through time.


[User] Please transcribe this text

[Assistant] And so you can propagate information, even if the information that you received about the supervision, the ground truth is sparse. You can follow that information back through time to infer something about the reality of what happened before then, even if your reward signals were weak.


[User] Please transcribe this text

[Assistant] So it's using the fact that the physical world evolves through time.


[User] Please transcribe this text

[Assistant] Some sort of predictable way to take sparse information and generalize it over the entirety of the experience that's being learned.


[User] Please transcribe this text

[Assistant] So we apply this to two problems. Today we'll talk about deep traffic.


[User] Please transcribe this text

[Assistant] As a method, as a way to introduce deeper enforcement learning. The deep traffic is a competition that we ran last year and expanded significantly this year.


[User] Please transcribe this text

[Assistant] And I'll talk about some of the details and how the folks in this room can, on your smartphone today, or if you have a laptop, train an agent while I'm talking. Training a neural network in the browser. Some of the things we've added are we've added the capability


[User] Please transcribe this text

[Assistant] We've now turned it into a multi-agent deep reinforcement learning problem. Where you can control up to 10 cars within your own network. Perhaps less significant.


[User] Please transcribe this text

[Assistant] But pretty cool is the ability to customize the way the agent looks.


[User] Please transcribe this text

[Assistant] So you can upload, and people have, to an absurd degree have already begun doing so, uploading different images instead of the car that's shown there.


[User] Please transcribe this text

[Assistant] As long as it maintains the dimensions, shown here as a SpaceX rocket.


[User] Please transcribe this text

[Assistant] The competition is hosted on the website, selfdrivingcars.mit.edu slash deep traffic. We'll return to this later.


[User] Please transcribe this text

[Assistant] The code is on GitHub with some more information, a starter code.


[User] Please transcribe this text

[Assistant] And the paper describing some of the fundamental insights.


[User] Please transcribe this text

[Assistant] That will help you win at this competition is an archive.


[User] Please transcribe this text

[Assistant] So, from supervised learning in Lecture 1 to today.


[User] Please transcribe this text

[Assistant] Supervised learning we can think of as memorization.


[User] Please transcribe this text

[Assistant] of ground truth data in order to form representations that generalizes from that ground truth.


[User] Please transcribe this text

[Assistant] Reinforcement learning is, we can think of as a way to brute force propagate that information, the sparse information.


[User] Please transcribe this text

[Assistant] Quality reward to state that does not directly have a reward.


[User] Please transcribe this text

[Assistant] To make sense of this world when the rewards are sparse but are connected through time. You can think of that as reasoning.


[User] Please transcribe this text

[Assistant] So, the connection through time is modeled in most reinforcement learning approaches.


[User] Please transcribe this text

[Assistant] Very simply, that there's an agent taking an action in a state and receiving a reward. An agent operating in an environment executes an action, receives an observed state, a new state, and receives a reward. This process continues over and over.


[User] Please transcribe this text

[Assistant] In some examples, we can think of any of the video games, some of which we'll talk about today, like Atari Breakout.


[User] Please transcribe this text

[Assistant] As the environment, the agent, is the paddle.


[User] Please transcribe this text

[Assistant] Each action that the agent takes has an influence on the evolution of the environment. And the success is measured by some reward mechanism. In this case, points are given by the game. And every game


[User] Please transcribe this text

[Assistant] It has a different point scheme that must be converted, normalized into a way that's interpretable by the system. And the goal is to maximize those points, maximize their award.


[User] Please transcribe this text

[Assistant] The state is the angle, the angular speed, the position, the horizontal velocity.


[User] Please transcribe this text

[Assistant] The actions are the horizontal force applied to the cart. And the reward is one at each time step if the pole is still upright.


[User] Please transcribe this text

[Assistant] All the first person shooters, the video games, and all StarCraft. The, uh, strategy games.


[User] Please transcribe this text

[Assistant] In case of first person shooter in Doom, what is the goal? The environment is the game. The goal is to eliminate all opponents. The state is the raw game pixels coming in. The actions is moving up, down, left, right, and so on. And the reward is positive when eliminating an opponent and negative when the agent is eliminated.


[User] Please transcribe this text

[Assistant] The state is the raw pixels of the real world that the robot observes. The actions are the possible actions of the robot, the different degrees of freedom and moving through those degrees, moving the different actuators to realize the position of the arm. And the reward is positive when placing a device successfully and negative otherwise.


[User] Please transcribe this text

[Assistant] Everything can be modeled in this way. Markov decision process.


[User] Please transcribe this text

[Assistant] There's a state as zero, action a zero, and reward received.


[User] Please transcribe this text

[Assistant] A new state is achieved. Again, action reward state. Action reward state. Until a terminal state is reached.


[User] Please transcribe this text

[Assistant] And the major components of reinforcement learning is a policy, some kind of plan of what to do in every single state, what kind of action to perform, a value function,


[User] Please transcribe this text

[Assistant] Some kind of sense of what is a good state to be in, of what is a good action to take in a state.


[User] Please transcribe this text

[Assistant] And sometimes a model that the agent represents the environment with.


[User] Please transcribe this text

[Assistant] Some kind of sense of the environment it's operating in, the dynamics of that environment that's useful for making decisions about actions. Let's take a trivial example.


[User] Please transcribe this text

[Assistant] A grid world of three by four. Twelve squares. Will you start at the bottom left?


[User] Please transcribe this text

[Assistant] End our task with walking about this world to maximize reward.


[User] Please transcribe this text

[Assistant] Is a punishment or is a negative reward of 0.04? So what is the optimal policy in this world?


[User] Please transcribe this text

[Assistant] Now when everything is deterministic, perhaps this is the policy.


[User] Please transcribe this text

[Assistant] When you start at the bottom left, well, because every step hurts, every step has a negative reward, then you want to take the shortest path to the maximum square with the maximum reward. When the state space is non-deterministic,


[User] Please transcribe this text

[Assistant] As presented before, with a probability of .8, when you choose to go up, you go up, but with probability .1,


[User] Please transcribe this text

[Assistant] You go left and point one, you go right. Unfair. Again, much like life. That would be the optimal policy.


[User] Please transcribe this text

[Assistant] What is the key observation here? That every single state in this space must have a plan.


[User] Please transcribe this text

[Assistant] Because you can't, because then the non-deterministic aspect of the control


[User] Please transcribe this text

[Assistant] You can't control where you're going to end up, so you must have a plan for every place. That's the policy. Having an action, an optimal action to take in every single state.


[User] Please transcribe this text

[Assistant] Now suppose we change the reward structure, and for every step we take, there's a negative, uh, our reward is a negative two. So it really hurts. There's a high punishment for every single step we take. So no matter what,


[User] Please transcribe this text

[Assistant] We always take the shortest path. The optimal policy is to take the shortest path to the, to the only spot on the board that doesn't result in punishment.


[User] Please transcribe this text

[Assistant] If we decrease the reward of each step to negative point one, the policy changes.


[User] Please transcribe this text

[Assistant] Were there some extra degree of wandering encouraged?


[User] Please transcribe this text

[Assistant] Every step is, increases the reward. Then there's a significant incentive to, to stay on the board without ever reaching the destination.


[User] Please transcribe this text

[Assistant] So the value function, the way we think about the value of a state or the value of anything in the environment is


[User] Please transcribe this text

[Assistant] The reward we're likely to receive in the future. And the way we see the reward we're likely to receive is we discount the future reward. Because we can't always count on it. Here are gamma.


[User] Please transcribe this text

[Assistant] Further and further out into the future, more and more discounts decreases the reward, the importance of the reward received.


[User] Please transcribe this text

[Assistant] And the good strategy is taking the sum of these rewards and maximizing it. Maximizing the discounted future reward. That's what reinforcement learning hopes to achieve. And with Q learning, we use


[User] Please transcribe this text

[Assistant] Any policy to estimate the value of taking an action in a state.


[User] Please transcribe this text

[Assistant] We move about the world and use the Bellman equation here on the bottom.


[User] Please transcribe this text

[Assistant] To continuously update our estimate of how good a certain action is in a certain state.


[User] Please transcribe this text

[Assistant] So we don't need, this, this allows us to operate in a much larger state space and a much larger action space. We move about this world through simulation or in the real world, taking actions and updating our estimate of how good certain actions are over time.


[User] Please transcribe this text

[Assistant] The new state at the left is the, is the updated value. The old state is the starting value for the equation and we update that old state estimation with the sum


[User] Please transcribe this text

[Assistant] And the maximum reward that's possible to be received in the following states. Discounted.


[User] Please transcribe this text

[Assistant] That update is decreased with the learning rate. The higher the learning rate, the more value we, the faster we learn, the more value we assign to new information.


[User] Please transcribe this text

[Assistant] That's simple. That's it. That's Q learning. This simple update rule allows us to, to explore the world and as we explore


[User] Please transcribe this text

[Assistant] Get more and more information about what's good to do in this world. And there's always a balance in the various problem spaces we'll discuss. There's always a balance between exploration and exploitation.


[User] Please transcribe this text

[Assistant] As you form a better and better estimate of the Q function of what actions are good to take, you start to get a sense of what is the best action to take.


[User] Please transcribe this text

[Assistant] But it's not a perfect sense. It's still an approximation. And so there's value of exploration. But the better and better your estimate becomes, the less and less exploration has a benefit.


[User] Please transcribe this text

[Assistant] So usually we want to explore a lot in the beginning and less and less so towards the end. And when we finally release the system out into the world and wish it to operate its best, then we


[User] Please transcribe this text

[Assistant] Have it operate as a greedy system, always taking the optimal action according to the Q value function.


[User] Please transcribe this text

[Assistant] And everything I'm talking about now is parameterized and are parameters that are very important for winning the deep traffic competition.


[User] Please transcribe this text

[Assistant] Which is using this very algorithm with a neural network at its core.


[User] Please transcribe this text

[Assistant] So for a simple table representation of a Q function, where the y-axis is state, four states, s1, 2, 3, 4, and the x-axis is actions, a 1, 2, 3, 4,


[User] Please transcribe this text

[Assistant] We can think of this table as randomly initiated or initiated, initialized in any kind of way.


[User] Please transcribe this text

[Assistant] That's not representative of actual reality. And as we move about the world and we take actions, we update this table with the Bellman equation shown up top.


[User] Please transcribe this text

[Assistant] And here, slides now are online. You can see a simple pseudocode algorithm of how to update it, of how to run this Bellman equation. And over time,


[User] Please transcribe this text

[Assistant] The approximation becomes the optimal Q table.


[User] Please transcribe this text

[Assistant] The problem is when that Q table becomes exponential in size.


[User] Please transcribe this text

[Assistant] When we take in raw sensor information as we do with cameras with deep crash.


[User] Please transcribe this text

[Assistant] Or when we take Go, the game of Go, when it's taking the units, uhm, the, the board, the raw state of the board.


[User] Please transcribe this text

[Assistant] As the input, the potential state space, the number of possible combinatorial variations of what states are possible.


[User] Please transcribe this text

[Assistant] Is extremely large. Larger than, we can certainly hold the memory and larger than we can, uh, ever be able to accurately approximate through the Bellman equation over time through simulation.


[User] Please transcribe this text

[Assistant] So this is where deep reinforcement learning comes in. Neural networks are really good approximators. They're really good at exactly this task of learning this kind of Q table.


[User] Please transcribe this text

[Assistant] So as we started with supervised learning, where neural networks help us memorize patterns using supervised ground truth data, we moved to reinforcement learning that hopes to propagate outcomes to knowledge.


[User] Please transcribe this text

[Assistant] Deep learning allows us to do so on much larger state spaces, on much larger action spaces, which means it's generalizable. It's much more capable to deal with the raw


[User] Please transcribe this text

[Assistant] Stuff of sensory data, which means it's much more capable to deal with a broad variation of real world applications.


[User] Please transcribe this text

[Assistant] And it does so because it's able to learn the representations as we discussed on Monday.


[User] Please transcribe this text

[Assistant] Converting the raw sensory information into, into simple useful information based on which the action in this particular state can be taken in the same exact way. So instead of the queue table, instead of this queue function, we plug in a neural network where the input is the state space, no matter how complex.


[User] Please transcribe this text

[Assistant] And the output is a value for each of the actions that you could take.


[User] Please transcribe this text

[Assistant] Input is the state, output is the value of the function. It's simple.


[User] Please transcribe this text

[Assistant] At the core of the success of DeepMind, a lot of the cool stuff you see about video games, DQN or variants of DQN are at play. This is what a first with a nature paper from DeepMind


[User] Please transcribe this text

[Assistant] The success came of playing the different games, including Atari games.