[User] Please transcribe this text

[Assistant] Then we'll brew some coffee with it. And then after that, we'll talk about whether it fits your specific needs as a solution for travel for coffee on the go. So let's open it up.


[User] Please transcribe this text

[Assistant] A scoop, but we know how I feel about scoops, so we'll leave that in there. And there's also the vase for the kettle.


[User] Please transcribe this text

[Assistant] I think metal mesh filter, which we're going to brew some coffee with. You could obviously put whatever kind of V60 papers you wanted in this little pouch on the side. So inside the kettle body are


[User] Please transcribe this text

[Assistant] This is your thermos, we're going to brew into the thermos directly. It's dual walled stainless steel. And then you've got...


[User] Please transcribe this text

[Assistant] Interplace and then it's a slightly larger cone. It's still pretty small actually, which is interesting, but it sits the top.


[User] Please transcribe this text

[Assistant] That seems like a little bit of a design flaw. I was not being particularly rough with it, I don't think, but it just, by just opening it out, I think I just squeezed a bit too much at the bottom and that is broken. So I guess for the rest of this test I will brew with paper. I had brewed with this before and I'll share some thoughts on it at the end, but that's a bit


[User] Please transcribe this text

[Assistant] Bit frustrating. So let's brew some coffee. To do that...


[User] Please transcribe this text

[Assistant] Let's start with the kettle. Now the base thankfully is dual voltage, which means you can travel with it anywhere and it should work completely fine. You do need to adapt the plug on the end of it. So we do need to get a UK adapter to make this work and then I can plug it in, put some water in this. We can turn it on.


[User] Please transcribe this text

[Assistant] Obviously it's cool touch by the jacketing on the outside of this, though this little bit exposed here is hot, don't touch that. I've got to say, that's a very pleasurable pouring experience as they go. Very easy to work with, very easy to pour.


[User] Please transcribe this text

[Assistant] Getting a bit of stalling with this paper filter as well, which is interesting, so just had to lift that up.


[User] Please transcribe this text

[Assistant] There. Now the brew did not go perfectly according to plan. I had some stalling in that brew. Uhm, it seemed like the metal filter and the paper really stuck together and that seemed to cause a stall and just detaching the paper filter slightly from the wall.


[User] Please transcribe this text

[Assistant] That's a nice brew. It certainly hasn't suffered horribly for a bit of a stall, just a bit slow. This as a travel cup is interesting as a decision to have such a large hole here. It makes it pleasurable to drink from, but maybe not an ideal travel container. This would spill very easily and it would spill completely if you knocked it over.


[User] Please transcribe this text

[Assistant] A collapsible filter cone and a little container for ground coffee. You could put those things together as a separate kit of things much cheaper than that.


[User] Please transcribe this text

[Assistant] When you finish with the kettle, you'll obviously have to dump it out, but you need to cool it back down again. It's really hot, it's well insulated, so it stays really hot for a while, so there's a bit of a process there to cool it back down with cold water, throw that out, dry it down, and you want everything to be dry before you put it back together and travel with it, so there's a bit of a process involved there.


[User] Please transcribe this text

[Assistant] Putting in a little container that's a decent size, a decent capacity, but not actually super functional.


[User] Please transcribe this text

[Assistant] This is an interesting choice to me. I feel like that is sufficient volume to have got a hand grind into there.


[User] Please transcribe this text

[Assistant] A grinder. I would want a solution that fits with this whole thing. I don't want to have this big pouch of stuff and then.


[User] Please transcribe this text

[Assistant] I'm gonna go through more coffee than that pretty quickly if I'm travelling for any real period of time. So I'm probably gonna be taking a bag of coffee with me rather than using this. The fact that this failed


[User] Please transcribe this text

[Assistant] On camera is a bit embarrassing for me, but at the same time also...


[User] Please transcribe this text

[Assistant] Kind of frustrating. I did as they show in their video. I was trying to pinch it open to make it stay open. I pinched on the seam as they do.


[User] Please transcribe this text

[Assistant] And it just split. And I know how these things can happen. It just looks like they went too thin on the seam and it's just not strong enough to cope with just a little bit of pressure from my fingers. I liked this. I liked the coffee it made. It's just frustrating that it broke.


[User] Please transcribe this text

[Assistant] So let's answer the final question, which is, who is this for and does it work well for them? I guess it's kind of pitched at people who spend a lot of their lives in hotels or in Airbnbs or that kind of thing, which in the past had been someone like me. Kettles are nice to have. The idea of having a kettle wherever you go is, is good.


[User] Please transcribe this text

[Assistant] But obviously you're going places that have power and those honestly often have kettles. I know not every hotel does, but many do. I know not every Airbnb does, but.


[User] Please transcribe this text

[Assistant] I hope. I don't. Anyway, like for many of us, the volume is as important as the weight. And so having something so heavy and so large,


[User] Please transcribe this text

[Assistant] Doesn't feel super flexible. For me, yeah, I can see a use for this. I could see it fitting into aspects of my life when I travel, but it would be going into checked luggage. It would be for a worst case scenario where I got to the hotel and there's just nothing to brew good coffee with, but you still need a grinder. And that's what makes this not a complete kit for me.


[User] Please transcribe this text

[Assistant] What's held you back? Is it the price? Is it something else? I would love to hear your thoughts down in the comments below. But for now, I'll say thank you so much for watching and hope you have a great day.


[User] Please transcribe this text

[Assistant] He'd obviously told the servants that he had accidentally set fire to his room by knocking over a lighted candle.


[User] Please transcribe this text

[Assistant] I saw Grace Poole sitting inside, calmly mending the curtains.


[User] Please transcribe this text

[Assistant] She certainly did not look desperate or mad enough to have tried to murder her master, but I decided to investigate. Good morning, Grace, I said, entering the room. Tell me what happened last night. The servants are talking about it. Good morning, Miss, she replied, looking up innocently. Well, master was reading in bed and fell asleep, so he must have knocked the candle over. It set fire to the sheets, but luckily he managed to put the flames out with some water.


[User] Please transcribe this text

[Assistant] How strange, I said quietly. Didn't anybody hear what was happening? At this, she seemed to examine me carefully.


[User] Please transcribe this text

[Assistant] Mrs. Fairfax is a heavy sleeper like most old people and didn't hear anything. But you're young, Miss. Perhaps you heard a noise. I did. I whispered. I'm sure I heard a strange laugh.


[User] Please transcribe this text

[Assistant] Sorry! Finally! It's been way too long. We did the trailer like, probably a year ago.


[User] Please transcribe this text

[Assistant] I was worried, I was like, oh my god, did she get captured again? Like, I was wondering what happened to her.


[User] Please transcribe this text

[Assistant] Because they're not going to obey the orders, right? The orders were to flee.


[User] Please transcribe this text

[Assistant] They can't keep up. They're getting thrown off. They can't keep up. Look at that guy!


[User] Please transcribe this text

[Assistant] Every time we upload a new video. If you enjoyed this video, please let us know by clicking the thumbs up button and leaving your thoughts in the comments down below. You can send in recommendations to our email irhsubtitles.gmail.com or you can post them to the subreddit.


[User] Please transcribe this text

[Assistant] In this video, I want to talk about the benefits of taking the log of the dependent variable, opposed to just running a regression in levels.


[User] Please transcribe this text

[Assistant] So the first of these benefits is if I take the log of my dependent variable and I leave my independent variables in non-logged form,


[User] Please transcribe this text

[Assistant] But I have a model which looks something like this. So I have log y is equal to alpha plus beta 1 times x1i plus beta 2 times x2i all the way through to xp, or beta p rather, times xpi.


[User] Please transcribe this text

[Assistant] Take the inverse of the log, which is the exponential function. I can then re-express this model in terms of exponents. So I can write this model as y i is equal to e to the power alpha times e to the power beta 1 times x 1 i.


[User] Please transcribe this text

[Assistant] And you can see that essentially by going from, well, by looking at our model in this form, we are allowing for a very nonlinear and therefore quite general relationship between our variables.


[User] Please transcribe this text

[Assistant] And often a linear model just doesn't cut it. So having this multiplicative form as we do here often is a much better way of modeling the dependent variable than just using a linear model. So that's one particular benefit.


[User] Please transcribe this text

[Assistant] log y is equal to alpha plus beta one times log


[User] Please transcribe this text

[Assistant] Then these coefficients here on the individual log terms actually have immediately an economic interpretation. They are the elasticities of that particular variable or rather of the dependent variable with respect to that particular variable.


[User] Please transcribe this text

[Assistant] So that's beneficial because elasticities in general are better to deal with than just these sort of level terms because that allows us to compare disparate markets and disparate situations.


[User] Please transcribe this text

[Assistant] Notice also that if I was to, like I did for the previous model, invert this, I would have a model which looks something like yi is equal to x1i to the power beta 1.


[User] Please transcribe this text

[Assistant] So again, we have got some sort of complicated multiplicative model here, implied by estimating a model like this.


[User] Please transcribe this text

[Assistant] And often this is actually a much more realistic way of modeling situations than just using a model which is linear in all your parameters. Okay, the third benefit of taking the log of the dependent variable


[User] Please transcribe this text

[Assistant] Often it's the case that we have a dependent variable, which is something like sales, perhaps, or perhaps it's wages.


[User] Please transcribe this text

[Assistant] And it has a minimum value which is minus infinity. So our dependent variable now, if we're running a model which has got log yi as a dependent variable, is no longer a limited dependent variable model. So theoretically that is a better situation than dealing with a limited dependent variable.


[User] Please transcribe this text

[Assistant] The fourth situation is to do with heteroskinasticity.


[User] Please transcribe this text

[Assistant] Which look something like this. So I'm getting a higher variance in yi as xi is increasing. So that's my xi, that's my yi.


[User] Please transcribe this text

[Assistant] And that will be reflected in my sort of distance of my points away from my fitted line.


[User] Please transcribe this text

[Assistant] Well, essentially what the log transform does is it suppresses variation because it, the log function itself has a decreasing marginal returns to xi. So,


[User] Please transcribe this text

[Assistant] By actually taking a log of my dependent variable, and often taking a log of my independent variable, then you can actually transform to situations which


[User] Please transcribe this text

[Assistant] Practically, you've actually got rid of this heteroskedasticity, because it's just been suppressed so much by the log function. So, perhaps your fitted model looks something like that now, and perhaps there's still a bit of heteroskedasticity, but practically it's not, no longer an issue.


[User] Please transcribe this text

[Assistant] So taking the log of dependent variables and independent variables can be a good idea because it can reduce heteroskedasticity.


[User] Please transcribe this text

[Assistant] The last reason for taking the log of a dependent variable is because of the fact that by taking the log of the dependent variable, we actually make our data more normal, or technically make the error more normal.


[User] Please transcribe this text

[Assistant] And the way in which this works is similar to the example with heteroscedasticity. So assume that we have an error which looks something like this. So we have a positively skewed error distribution.


[User] Please transcribe this text

[Assistant] By taking the log, we are suppressing each of these variations. So, essentially we're making our distribution more symmetric. So, perhaps we're only taking the log of the dependent variable, it might look something more like


[User] Please transcribe this text

[Assistant] That, it will still have a bit of positive skew, but we have made our data much more normal than it was previously. And as we know from our other discussions, it is much better to deal with normal errors than it is to deal with non-normal errors for inference purposes.


[User] Please transcribe this text

[Assistant] Well, stream, I would consider our job here done. GG.


[User] Please transcribe this text

[Assistant] This is a very serious question. For a long time ago. I'm gonna answer very seriously. You ready? Really? Who is better at Fortnite? Me or CourageJD? I don't know who CourageJD is. I don't know who CourageJD is either, dude.


[User] Please transcribe this text

[Assistant] That's like a... like a... He does those nipples. He does that a lot.


[User] Please transcribe this text

[Assistant] A single tear rolls down Tyler's face. Embrace. Like two lovers having not seen each other in many moons. Do you know how I know that Tyler enjoyed it? I could feel his tiny boner.


[User] Please transcribe this text

[Assistant] It was the end of his stream, he just disconnected.


[User] Please transcribe this text

[Assistant] Man, we've done a lot of stupid stuff in this game. Remember I actually found a lot of-


[User] Please transcribe this text

[Assistant] That was a long time ago too. You got em!


[User] Please transcribe this text

[Assistant] The C4- Oh! What did I just get knocked off by? Who just said it pulls me off a mountain?


[User] Please transcribe this text

[Assistant] Got so much stuff from so long ago. ClipSci I have not seen in forever. That was pretty good. The second half definitely, uh, fantastic.


[User] Please transcribe this text

[Assistant] Sometimes kids aren't counted absent until they have failed to log on for five days in a row. And some are never required to attend class. But the state still requires the schools to report attendance. So most just report 100 percent, even though that's not what's really going on.


[User] Please transcribe this text

[Assistant] Miss Temple did not reply. She looked straight in front of her, and her face was as cold and hard as marble. Mr. Brocklehurst, on the other hand, now looked round at the girls, and almost jumped in surprise. Who? What is that girl with red hair? With curls, madam? With curls everywhere? That is Julia Severn, said Miss Temple quietly.


[User] Please transcribe this text

[Assistant] Miss Temple, that girl's hair must be cut off.


[User] Please transcribe this text

[Assistant] I have said again and again that hair must be arranged modestly and plainly. I see other girls here with too much hair.


[User] Please transcribe this text

[Assistant] Yes. I shall send someone tomorrow to cut all the girl's hair.


[User] Please transcribe this text

[Assistant] Mr. Brocklehurst, began Miss Temple. No, Miss Temple, I insist. To please God, these girls must have short, straight hair and plain, simple clothes.


[User] Please transcribe this text

[Assistant] Who had unfortunately not heard his comments on dress and hair. They all wore the most expensive clothes and had beautiful long curly hair.


[User] Please transcribe this text

[Assistant] I heard Miss Temple greet them as the wife and daughters of Mr. Brocklehurst.


[User] Please transcribe this text

[Assistant] I'm glad to be here and as Michael said, this will be an interdisciplinary talk.


[User] Please transcribe this text

[Assistant] Maybe sort of like a, uhm, a fusion cuisine of education and psychology and computer science.


[User] Please transcribe this text

[Assistant] Uh, I wanna talk about, uh, big data and discovering tacit implicit knowledge and using that to improve learning. And my overall argument is here on this slide. Uh, I wanna first try to convince you that we are not aware of most of what we know and learn. And, uh, as a consequence,


[User] Please transcribe this text

[Assistant] The typical approach for designing instruction


[User] Please transcribe this text

[Assistant] Uh, whether that's writing a textbook or authoring lectures is fundamentally flawed by a basic lack of information. That self-reflection that we use to design


[User] Please transcribe this text

[Assistant] Approaches we've taken to draw upon data to tap into what we, what, what we as experts are, uh, what students trying to acquire expertise know and learn, uhm, and as well as examples of using computational modeling to dig, dig into those issues.


[User] Please transcribe this text

[Assistant] Importantly, we want to then demonstrate that those insights, uh, really work by redesigning courses based on them and comparing them in experiments to, to show you that you get improvement. So that's the closed loop.


[User] Please transcribe this text

[Assistant] Uhm, I'm gonna start with a little background, uh, uh, then, then give some examples of these data-driven models, uh, and give you a brief sense of this computational theory that we're developing of student learning in these areas. So, uh, going back, uh, to some of the early, uh, foundational work, uh, at, uh,


[User] Please transcribe this text

[Assistant] This is sort of a CMU take on early AI and cognitive science history, but it's one that's relevant to...


[User] Please transcribe this text

[Assistant] So my core way of thinking. Uhm, so Newell and Simon did this work on, uh, human problem solving way back in the early, uh,


[User] Please transcribe this text

[Assistant] And, you know, that work led to things that came later, uh, I mean, they built the first chess-playing computer, but, you know, of course we see today how powerful those technologies have evolved to be.


[User] Please transcribe this text

[Assistant] But they also had insights into the nature of expertise, including the notion that experts acquire something like 10,000 chunks of knowledge to become an expert.


[User] Please transcribe this text

[Assistant] Probably already an indicator that it's maybe hard to reflect on what one knows if the volume is that high.


[User] Please transcribe this text

[Assistant] But another thing that, that was very compelling about some of the work they did, uh, for example in chess was this notion of, uh, perceptual chunks and, and is best illustrated by this task of giving a master, uh, chess player, uh, board position in the middle of the game, say there's 28 pieces on the board, uhm,


[User] Please transcribe this text

[Assistant] Five second exposure, sweep it away, ask the player to reproduce that board and they can do that very accurately and quickly.


[User] Please transcribe this text

[Assistant] Novices can't do that, and interestingly, experts with those same 28 pieces randomly placed on the board can't do it either.


[User] Please transcribe this text

[Assistant] So they've learned particular patterns of pieces, just like you learn patterns of letters that form words. And you can quickly remember what those letters are when they form words. So too do chess players learn these patterns.


[User] Please transcribe this text

[Assistant] Uhm, and it's not like they're trying to do that. They're trying to play chess, right? But it turns out that their brains acquire these patterns and it might actually even be functional, but they don't realize that they're doing it.


[User] Please transcribe this text

[Assistant] Herb Simon and others did some, uh, later work in developing not only models of-


[User] Please transcribe this text

[Assistant] Doing complex tasks, but adaptive systems that could acquire, uh, uh, knowledge of doing those tasks. So this was early symbolic machine learning.


[User] Please transcribe this text

[Assistant] Uh, adaptive production systems was a key approach. Uhm, see one of the really interesting papers in this is sort of little known, uh, proceedings of the National Academy of Sciences paper that Simon was involved with.


[User] Please transcribe this text

[Assistant] And, uh, this among other papers really made a big emphasis on the hard part of becoming an expert is learning the essentially the retrieval cues or if part of our knowledge. Uhm, we often can


[User] Please transcribe this text

[Assistant] Articulate what we do in an expert area, those are the actions in the then part, but when and why we do them is, is the hardest part, uh, of learning and also probably the most tacit of, of what we learn.


[User] Please transcribe this text

[Assistant] They did some demonstrations actually of using this to guide better instruction in math and physics.


[User] Please transcribe this text

[Assistant] There was also work developing cognitive architectures like ACT-R and SOAR and some of that led to intelligent tutoring systems like early work on a programming tutor teaching the list programming tutor had an impressive demonstration of teaching programming in about one third the time it takes to learn programming in a typical situation.


[User] Please transcribe this text

[Assistant] So, uh, one of the key ideas in ACT-R, uh, helps to highlight this tacit knowledge thing. Um, and it's not simply, uh, uh,


[User] Please transcribe this text

[Assistant] I guess in, in philosophical terms, these distinctions between declarative and procedural might tempt you to think that's, uh, declarative is what you can say in procedural is not. But, uh, ACTAR cuts it a little bit differently. Um, there are declarative elements like the structure of the perceived world.


[User] Please transcribe this text

[Assistant] That uh, are uh, not necessarily easy to verbalize. Uhm, things like,


[User] Please transcribe this text

[Assistant] The first language grammar, you know, your natural language, that grammar, you don't learn that explicitly. You'd have trouble explaining, for instance, why you use the in some places in a noun phrase and not in the other. You pick that up like the chess master through experience and through exposure.


[User] Please transcribe this text

[Assistant] These perceptual chunks are really powerful in expanding expert's memory, organizing their intuitive reasoning, allowing them in essence to make leaps of inference, uh, uhm, uh, by seeing the world in bigger chunks.


[User] Please transcribe this text

[Assistant] On the procedural side, these cues help, for instance, surgeons make choices in surgery, uh, uh,


[User] Please transcribe this text

[Assistant] Academics plan, uh, proofs, uhm, social design decisions, get at the conditions and logic behind decisions and planning.


[User] Please transcribe this text

[Assistant] If we ask how much of our knowledge is each of these, there are, uh, uhm, some empirical investigations that suggest


[User] Please transcribe this text

[Assistant] These things we can't articulate, this tacit knowledge is about 70% of what we know. Uh, uh, the remaining 30% that we can directly articulate is actually quite small. And I'll say a little bit more about that.


[User] Please transcribe this text

[Assistant] But if you want to ask me later, you can probe a little bit about how those numbers came about.


[User] Please transcribe this text

[Assistant] Um, prefacing the, this, uh, simulated student theory I'll get to later, wanted to put on the table a few of the claims that we've, uh, that are,


[User] Please transcribe this text

[Assistant] Part of that theory. So following this tradition, the notion that much of our learning comes from inducing from experience, from examples and by doing. And that, that kind of learning is mostly outside of our conscious awareness.


[User] Please transcribe this text

[Assistant] That kind of learning involves challenges for our brains corresponding to the things I was emphasizing earlier, learning cues for decisions or the if part of the production, and learning the deep representational structure of a domain from perceptual input. Uh, and the computational modeling framework we have


[User] Please transcribe this text

[Assistant] have mechanisms to perform these two challenges.


[User] Please transcribe this text

[Assistant] And part of this approach is, is to suggest that, you know, what we actually spend a lot of our conscious time in learning and instruction, verbal forms of instruction, is somewhat secondary in the process. And, and so,


[User] Please transcribe this text

[Assistant] Uh, something challenging there in the sense that it's in our conscious awareness, so it feels more primary. But in fact, if you...


[User] Please transcribe this text

[Assistant] Look deeply into how learning works. It, uh, it guides us but doesn't really get us there. So.


[User] Please transcribe this text

[Assistant] Uh, visually, here's the dilemma. Experts, uh, if you believe this, uh, uh, notion can describe only 30% of what they know. Uhm, the rest, we don't know what we know, so instruction based on intuitive design lacks information and thus is likely to be


[User] Please transcribe this text

[Assistant] A surgery procedure catheter insertion. Uhm, and this is recording for different surgeons how many


[User] Please transcribe this text

[Assistant] Of the steps in the final procedure that they came up with through these multiple interviews and probing and some rational extension of what the procedures need to be. How many did they original, what proportion did they originally mention?


[User] Please transcribe this text

[Assistant] And this 30% comes from essentially that the decision steps are a minority of what they're mentioning.


[User] Please transcribe this text

[Assistant] So how, this thing I'm illustrating here is a form of cognitive task analysis. Uh, there are a number of ways to do it. Think-alouds that Newell and Simon did are one of those ways.


[User] Please transcribe this text

[Assistant] Clark and colleagues have done this structured interview form.


[User] Please transcribe this text

[Assistant] One thing that's really powerful about what they've done from a education perspective is they've taken the insights and redesigned instruction and then compared that instruction to existing instruction. So in this surgery domain, the control condition in these studies was


[User] Please transcribe this text

[Assistant] Teaching that these surgeons in med schools have developed before. So these are really smart people, right?


[User] Please transcribe this text

[Assistant] Subtleties of, uh, the surgery procedure, uh, like even the way in which the, uh, person's holding the needle here that weren't being, uh, articulated in the course.


[User] Please transcribe this text

[Assistant] In the original course and when they modified the course based on, uh, these kinds of insights, they got much better outcomes and they, they interestingly not only


[User] Please transcribe this text

[Assistant] Uh, better at the surgical procedures. As demonstrated for instance, they had many fewer extra insertions of the needle. Uhm, so very practical, important and powerful outcome. Uh, there have been, uh, not a ton of these kinds of studies. You can imagine they're kind of expensive to do both the, uh,


[User] Please transcribe this text

[Assistant] This intensive, uh, interviewing process as well as the redesign and the experiment, but about six or so in a small medicine analysis really showed


[User] Please transcribe this text

[Assistant] Build upon this extremely informative approach but find methods that get beyond some of the issues of subjectivity and time cost that these methods incur and take a more quantitative approach that uses data.


[User] Please transcribe this text

[Assistant] Does so by asking two qu- key questions. Uh, one form of data is to look at the difference in difficulty of related tasks.


[User] Please transcribe this text

[Assistant] And by seeing those difficulties, we can infer through the difference what cognitive processes might account for those empirical differences in difficulty. And the second is looking at task performance over time to see


[User] Please transcribe this text

[Assistant] Give an example of that in, in a more, uh, academic domain. In early work that we did, uh, developing, uh, an algebra course and an intelligent tutor for algebra, I was interested in, uh,


[User] Please transcribe this text

[Assistant] What I was hearing, you know, from working with teachers, uh, and I guess to some extent it, it felt like my own experience, uh, that story problems in algebra were difficult. So here's a, this kind of,


[User] Please transcribe this text

[Assistant] Yeah, I was actually designing this thinking I could get some insight into how, uh, you know, story problems are solved, translated into equations. This might be a sort of intermediate form and I could unpack that. Uh, you guys are mostly consistent with what math educators believe.


[User] Please transcribe this text

[Assistant] Uhm, and unfortunately most of us were wrong. Uhm, what the data showed is that you were right.


[User] Please transcribe this text

[Assistant] Maybe not for the right reason, but uhm, uh, yeaah. The big difference is actually between the equation and the word problem. So it's not about the situation. It's actually a small effect of the situation on decimal arithmetic operations. We never see students subtracting $66 from 90 cents in the top problem.


[User] Please transcribe this text

[Assistant] They do in this problem sometimes subtract 66 from the 90. Um, so that's the- that's essentially accounts for the difference between 70 and 61 percent. Uh, but the big difference is between the problems expressed in English and the one expressed in the language of algebra. Yeah.


[User] Please transcribe this text

[Assistant] Yeah, uhm, we've replicated this number of times and, uh, you know, one of the things we've manipulated is whether the operators are made explicit or whether they're implicit in a more natural way of saying that they'd be implicit. But that makes almost no difference.


[User] Please transcribe this text

[Assistant] You'll be not surprised to, to hear that the bars shoot up, uh, like to 98%, uh, but that one only goes up to 94.


[User] Please transcribe this text

[Assistant] So it actually replicates with, with, you know, mean SATs of like 690 or something like that, yeaah.


[User] Please transcribe this text

[Assistant] Uhm, but, uh, the other thing we did is we manipulated the difficulty of the problem. So one feature of difficulty that flip-flops this relationship is if the story or equation refer to the unknown twice.


[User] Please transcribe this text

[Assistant] So x minus .2x equals 38 or Ted bought a coat at a 20% discount and he paid $38. How much was the original price? There, the story problem is harder than the equation and it's because there they need the equation to solve it. Here they're actually able to solve these problems without using the equation.


[User] Please transcribe this text

[Assistant] And why we don't experience it as being hard.


[User] Please transcribe this text

[Assistant] Uh, and what our analysis suggested is basically it's kind of a linguistic phenomena. There's a language of algebra that your brain and its ability to learn language without language, that's how you learned your first language, is still soaking up the grammatical patterns and so forth. It needs to because you don't naturally know how to read this. You don't know how to chunk.


[User] Please transcribe this text

[Assistant] The phrase structure that you should do multiplication before addition. You can be told that, but that doesn't mean you're gonna, in the moment, do it.


[User] Please transcribe this text

[Assistant] Nine months you were asked to say what this means versus say what that means. It'd be easier to do in English than it would be in Greek. Yeah.


[User] Please transcribe this text

[Assistant] We actually replicated this in Russia because a colleague of mine was saying they start using the algebraic language in second grade. Problems like x plus two equals five. And as a matter of fact,


[User] Please transcribe this text

[Assistant] Yeah, it turns out, we thought that might be harder, but it turns out some students don't know what the asterisks mean. So there's a lexical issue here, and 6x is again a grammatical issue, right? That's a, a particular grammatical form that juxtaposition means multiplication. And just cause I tell you at once,


[User] Please transcribe this text

[Assistant] Just like I could tell you in Russia how to conjugate a verb doesn't mean you can do it, right?


[User] Please transcribe this text

[Assistant] Well, it's interesting you asked that. We did do surveys of, uh, of teachers and in one of those surveys we had elementary, middle school and algebra teachers and the algebra teachers were the worst at this prediction task. They were the ones who


[User] Please transcribe this text

[Assistant] Like many of us, including me, want to say yeah, the equations are easy.


[User] Please transcribe this text

[Assistant] Because they're easy for them. Uh, so yeaah, we started to use this phrase, expert blind spot. The more expert you become in something, uh, the less reliable you are at predicting how hard it might be for somebody to use. Especially if that expertise is rather tacitly acquired and you don't have much conscious awareness of it. Was I getting at the nature of your question?


[User] Please transcribe this text

[Assistant] Yeah, well, uhm, I think it depends on what you mean. Certainly people's judgments of teaching can go to, like, personality and, yeah.


[User] Please transcribe this text

[Assistant] But I think ultimately what, to the extent that, that teacher is being perceived as effective because they're able to select tasks that, uh, make


[User] Please transcribe this text

[Assistant] The subtle distinctions that let me see, let me understand something that wasn't obvious to me, you know, that other professors would have


[User] Please transcribe this text

[Assistant] Different, even though the then part's the same, you know, you need to learn it two different ways. And I give you both of those examples. Uh, I'm probably gonna be a better instructor than the one who just gives you the one example and expects you to be able to induce and generalize to the other context. Yeah.


[User] Please transcribe this text

[Assistant] So, uh, yeaah, I spent more time on that than maybe I planned, but I think it's worthwhile. Uhm, here's a screenshot from the middle of the Algebra Cognitive Tutor that we developed. There's a lot of activities like this. Some are more like practicing your serve or your backhand. They're more isolated. This one's a bigger task, like playing the whole game of tennis.


[User] Please transcribe this text

[Assistant] Intendedly authentic problem situation like comparing two different cell phone plans and they're using multiple representations, not just equations, they're not shown but that's part of it, uh, tables and graphs and, uh, developing a model of the problem situation and using that model to answer questions and as they're filling in this table,


[User] Please transcribe this text

[Assistant] Students can, when they're challenged with hard questions and aren't able to figure out how, how to go about finding out when the two plans, uh, uh, intersect, they can, uh, go up here and click the hint button and, uh, and get help. So, two forms of instruction here, feedback on errors,


[User] Please transcribe this text

[Assistant] Or next step hints to, for what to do next. And if the student is able to figure out from this higher level hint, they can ask for a next one. And eventually, it grounds out in giving them an example. Not of the whole solution, but of the, of the, just the next step. And then it's back in the student's hands to learn by doing.


[User] Please transcribe this text

[Assistant] And instruction comes only when needed. Not when I think you need to hear it, but when you demonstrate that you need instruction because you've made an error. And it, it, it starts with verbal instruction.


[User] Please transcribe this text

[Assistant] Which actually the data suggests maybe not all that valuable. It ends with an example of what you should do and that actually seems like that's probably the most power. If you don't know the best thing maybe to just instead of being told to be given an example and then you can induce from that given a later chance to learn it.


[User] Please transcribe this text

[Assistant] With respect to giving a later chance to learn it, if you're making errors or tracking what, uh, skills, uhm, you're having difficulty with and then assigning new activities, uhm, based on this skill profile so that better students can skip past stuff fast, uhm, students that need more help get more help in the areas they need it.


[User] Please transcribe this text

[Assistant] Uh, strategies, uh, representatives, if then rules here for dealing with a problem like that one, uh,


[User] Please transcribe this text

[Assistant] Either these are correct, and here's a common error. If you have a model like this, then the crucial thing is to get these right. Uhm, uh, this is kind of a made up example, but I'll give you examples later of how getting this right can be subtle.


[User] Please transcribe this text

[Assistant] Uh, all the instruction that the tutor, uh, can deliver is based on this model. It will give context-sensitive instruction depending on the structure of the model, uhm, as I illustrated earlier. And it's also going to, uh, update the,


[User] Please transcribe this text

[Assistant] Model of what the student knows, there's a Bayesian estimation scheme here underlying it to, uh,


[User] Please transcribe this text

[Assistant] Select upcoming activities and determine the pacing so that those skill bars are based on for each student there's an estimate of the probability that they know each skill and that can be then used to


[User] Please transcribe this text

[Assistant] There's been lots of large-scale, uh, field trials of this. It's also been very widely disseminated.


[User] Please transcribe this text

[Assistant] About 600,000 students per year are using these courses about two days a week. Uhm, and those numbers, those are not just starting numbers, those are finishing numbers, you know, those are students getting through these courses. About, it's about almost 3,000 schools are using it across the country.


[User] Please transcribe this text

[Assistant] Geo trials have been going on since the earliest quasi-experiment we did is reflected in the graph here. In particular, this represents one of the results we often saw is that if we design assessments to get at some of our key goals like a


[User] Please transcribe this text

[Assistant] Many studies in between, but one of the most recent is actually the largest yet. This was about a hundred and forty or fifty schools where classes in those schools were randomly assigned either to use the cognitive tutor algebra course, by the way, it's a full course. We developed the text teacher professional development and the software.


[User] Please transcribe this text

[Assistant] We're, they're either randomly assigned to that or to use their regular algebra course and uhm,


[User] Please transcribe this text

[Assistant] Uh, the, the effects here are particularly strong in the second year of implementation and, uh, uhm, the,


[User] Please transcribe this text

[Assistant] A somewhat oversimplified result of a very complicated analysis is that students in the cognitive tutor course over the school year learned about twice as much.


[User] Please transcribe this text

[Assistant] They're treated as though they got it, even if they didn't. Uh, we've seen in some of these implementations, some of the schools intend to have students use this curriculum, but don't actually use it. And these policy level studies, uh, require that you count, uh, those students as having used it.


[User] Please transcribe this text

[Assistant] Uh, certainly interesting issues there about methodology. But the real point I wanted to make about these is while these are some pretty good results, uhm, there's certainly, uh,


[User] Please transcribe this text

[Assistant] Insufficient data or insufficient science to guide those decisions. So we created this Learn Lab Center, uh, about nine years ago with generous funds from the National Science Foundation around the idea that we could use these widely disseminated technologies


[User] Please transcribe this text

[Assistant] Develop that CMU through the online, uh, learning initiative that, uh, uh, Candice Till was directing who is now here at Stanford.


[User] Please transcribe this text

[Assistant] Also, uh, work in language learning, uh, domains, uh, uh, and in a number of data sets, uh, using educational games.


[User] Please transcribe this text

[Assistant] So, the key idea here is kind of like, you know, Google-izing education, if you will, by having these technologies out in the field. We can embed studies in them.


[User] Please transcribe this text

[Assistant] We can collect the raw data and we can build our better theory, uh, through the technologies as kind of a deeper instrumentation of what's, what's happening in the learning process.


[User] Please transcribe this text

[Assistant] So the social part of this infrastructure was facilitating researchers, some of these laboratory psychologists we wanted to get out into the real world, help them run studies in schools by organizing that process for them.


[User] Please transcribe this text

[Assistant] And part of that organization is technical in that we'd help psychologists to implement studies in these technologies. So we, uh,


[User] Please transcribe this text

[Assistant] Have, through this process and actually increasingly just through people adding to our data repository, we now have, I think it's the world's largest open repository of educational technology data. It's about 500 data sets, uhm, from a wide variety of domains.


[User] Please transcribe this text

[Assistant] Uhm, but we've also ran many of these, uhm, in vivo experiments or, or they use the language of i- industry, these A-B tests where


[User] Please transcribe this text

[Assistant] Instead of that study that I showed you, you know, we're manipulating all kinds of things, right? That course is different not just because of the technology, the text is different, the teacher professional development is different. In these studies, they're, they're tighter principle testing studies where we're varying one thing. They might last for maybe a couple weeks.


[User] Please transcribe this text

[Assistant] Or even a day session with a pre-test and a post-test between them. And, uh, so,


[User] Please transcribe this text

[Assistant] Done over 300 of those. Uh, this graph, uh, represents a subset of the content, uh, areas that we're focused on in those studies. So, some language learning things, uhm, math and science things. Uhm, and on the other axis here are, are the things that were manipulated, like,


[User] Please transcribe this text

[Assistant] And we, uh, use this experience to put together a framework.


[User] Please transcribe this text

[Assistant] Essentially making some dichotomies or taxonomies of different kinds of knowledge.


[User] Please transcribe this text

[Assistant] mappings of variable situations into dependent variable responses.


[User] Please transcribe this text

[Assistant] And then articulated verbal knowledge of principles. Increasing in complexity here, both complexity in the sense of you could look at description length, they get longer to describe, but also collect complexity in the sense that if we look at the time to execute the corresponding operations, the time gets longer. Uh, and then we also have this dimension of


[User] Please transcribe this text

[Assistant] Or to facilitate induction, or to facilitate understanding, and again these get increasingly complex too, uhm, as sort of characterized both theoretically and temporally. Uh, these books by the way are some sources for


[User] Please transcribe this text

[Assistant] Uh, many studies that, you know, various paradigms of research in educational and cognitive psychology explored a lot of these dimensions and been published in a lot of these resources. But we are trying to put some org- organization to that.


[User] Please transcribe this text

[Assistant] Uh, what's in this content of this table is some examples of particular studies. So here for instance, this shows three different domains we manipulated adding worked examples to our, our tutors. Um, and the outcome was positive for the treatment.


[User] Please transcribe this text

[Assistant] Complexity of your knowledge goals, uh, should be matched with the complexity of the kind of instructional intervention that you focus on. And that complex forms of instruction, like having a, a classroom dialogue, may not be very effective for learning Chinese vocabulary, for instance. So, uh,


[User] Please transcribe this text

[Assistant] And that one, that one's rather intuitive, but it gets more subtle in some of these cases. We're prompting students to self-explain why they chose a particular English article.


[User] Please transcribe this text

[Assistant] Actually enhances learning, but it takes more time relative to not doing it without getting any more positive gain.


[User] Please transcribe this text

[Assistant] Uhm, and the fundamental theoretical idea here is it's because that most of that activity is not enhancing memory. Uhm, it's targeted at enhancing understanding, but memory is the primary learning process for acquiring vocabulary. Uh, so uhm,


[User] Please transcribe this text

[Assistant] I could give a whole talk about that upper corner and we have a recent book chapter that talks about that. But I, uh,


[User] Please transcribe this text

[Assistant] Not only has a lot of our work been more in the middle here, uhm, but I think this is really sort of fundamental and it's somehow under-emphasized.


[User] Please transcribe this text

[Assistant] Cognitive psychology loves to do studies about memory, and because we're consciously aware of the verbal stuff at the top, that's what we mostly think about when we think about instruction, you know, how we're communicating to each other in lectures.


[User] Please transcribe this text

[Assistant] And the middle part often gets left out because it's happening tacitly. Uh, so that's where we've been doing most of our exploration.


[User] Please transcribe this text

[Assistant] I think just to give a kind of sense of a status of where we are, we learned as much in this center about what we don't know as we learned about what we do know and


[User] Please transcribe this text

[Assistant] I think we know a lot. I would characterize it in a metaphor with transportation. It's like we have Model T cars now. It's much better than walking or having a horse-drawn carriage, right?


[User] Please transcribe this text

[Assistant] What we could have in education is jet airplanes.


[User] Please transcribe this text

[Assistant] What it will take to get there is sort of analogous. It's gonna take a lot of work, it's gonna take a lot of advances in the science and engineering, you know, not of jet engines or


[User] Please transcribe this text

[Assistant] Uh, how, uh, uh, planes work but in, in deeply understanding how learning works and how we can use technology to make it easier and faster and better. Uhm.


[User] Please transcribe this text

[Assistant] So part of that development of understanding goes to thinking more deeply about what are the best forms of instruction. And we often get caught up, uh, in debates about instruction as if there were just two ways to teach. Uhm, you know, you've probably heard about the math wars or the reading wars. Like the reading wars are


[User] Please transcribe this text

[Assistant] Uh, here are some examples, uh, and the bold italics is an indication of where the literature seems to lean on a lot of these issues. Although, uh, and, and some of them, there's, there's, uh, work in, in both directions, and some of these, uh, you'd find people disagreeing, uh, uh, on the status of them.


[User] Please transcribe this text

[Assistant] And some of them, which are apparently different in the, in that there are different paradigms of work and different people involved, if you examine them closely, these two things are essentially


[User] Please transcribe this text

[Assistant] The same dimension of manipulation, but one paradigm's coming out with a different answer than the other. And it turns out, I think, this is one of the things that inspired that knowledge instruction dependency. That most of these studies are about facts.


[User] Please transcribe this text

[Assistant] In memorizing facts and their practice is much more important than study. Whereas most of these studies were about skills and their studying examples is much, uh,


[User] Please transcribe this text

[Assistant] Doing that, uh, much more than we usually do, say, in a math and science class. You know, we get homework assignments where you get problems to solve. Might be ten problems to solve. What this literature says is that you learn more.


[User] Please transcribe this text

[Assistant] Half of the problems were solved for you, and you study.


[User] Please transcribe this text

[Assistant] Those half, half of those problems and solve the other half. Uhm, that gives you more information to induce the patterns that you need to learn.


[User] Please transcribe this text

[Assistant] And of course we always give examples, but the problem is instead of giving five examples, the lecture usually only has one or two. And then some of the subtleties that you need to learn aren't represented in those examples. So you have no information on which to base them.


[User] Please transcribe this text

[Assistant] So, uh, extending this idea out a little bit, I started to think about how many different instructional options there really are and the fact that these binary distinctions, you know, already hide some of the, uh, subtlety in the sense that, uh, you can take a distinction of like,


[User] Please transcribe this text

[Assistant] Uh, master focused practice versus distributed practice and, and come up with a number of intermediate solutions like gradually widen the, the time between practice intervals.


[User] Please transcribe this text

[Assistant] Uh, and you can combine these dimensions together. So, uh, here we can take the dimension of study versus, uh, test and combine it with the other. And now we're not at two, we're at nine.


[User] Please transcribe this text

[Assistant] That's only two of the dimensions, right? Uhm, there's, should the examples be concrete or abstract or some mixture? Should the feedback be immediate or delayed? Should I block topics or interleave them? Should the instructor be doing the explaining or the student doing the explaining?


[User] Please transcribe this text

[Assistant] And you can again combine these with, with, for the most part, there's some exceptions, but for the most part all these can be combined with others, uh, and there are many more such dimensions which we've articulated in this paper.


[User] Please transcribe this text

[Assistant] Uhm, and so how many choices are there?


[User] Please transcribe this text

[Assistant] 205 trillion is the, is the number we came up with. So that's the design space.


[User] Please transcribe this text

[Assistant] If we want to, uh, get to the jet airplane of, uh, effective instruction, there's a lot of work to be done.


[User] Please transcribe this text

[Assistant] It's not going to be purely empirical. We need some theory to help prune this space and guide us there. And part of what we did in this paper was suggest some recommendations for, uh, for getting there both by leveraging large scale experimentation online, but also theoretically by, uh,


[User] Please transcribe this text

[Assistant] I want to say, give you some examples now, sort of, uh, of, of this data driven process I mentioned, uh, and, and how we've been going about that.


[User] Please transcribe this text

[Assistant] We have done a number of studies using data to, to do various things and, and, uh, one of those is to do, uh,


[User] Please transcribe this text

[Assistant] A s- predictive assessment by using online interactions to predict end of course, uh, uh, tests. For example, state tests, uh, this is data from the Massachusetts, uh, uhm, eighth grade.


[User] Please transcribe this text

[Assistant] They're needed and we get metrics of uh, essentially work ethic embedded in these models that are


[User] Please transcribe this text

[Assistant] The log data from these system interactions to detect, uh, various, uh, emotions or disengagement gaming the system. And this is a, uh,


[User] Please transcribe this text

[Assistant] Uh, screen from an intervention, Scooter the Tutor, who gets increasingly, uh, looks a little frustrated with you if you're looking like you're not engaged in the system, is one indirect form of intervention.


[User] Please transcribe this text

[Assistant] Uh, there's also this help seeking tutor that was built getting at, uh, helping students acquire learning skills and again driven by data.


[User] Please transcribe this text

[Assistant] Here we challenge folks to build better models of how students learn. And it was a real effort to try to engage the machine learning research community, community in, uh, uh, understanding, uh, educational data.


[User] Please transcribe this text

[Assistant] It was structured, similarly, if you're familiar with the Netflix prize, uhm, instead of having, uhm, movie watchers making ratings of movies, we had students here.


[User] Please transcribe this text

[Assistant] Uh, solving, these are observations of their performance on steps, uhm, in these tutors, uhm, over time. And the, the data here is not their movie rating, but, uh,


[User] Please transcribe this text

[Assistant] One, if they got it right on the first try without any help, and zero otherwise. Either they got it wrong or they asked for help.


[User] Please transcribe this text

[Assistant] And, uh, in the competition, there was a held out, uh, section of this data, uhm, uh, the end of every unit in essence and the,


[User] Please transcribe this text

[Assistant] Uh, 130 participants, a lot of predictions, uh, and the accuracy greatly improved. Uhm, there was a 15% reduction in error over the span of the competition, uh, in essence. Actually, there were models that were even worse than this, but realistic attempts, uh, started about there. Uh, it's, uh,


[User] Please transcribe this text

[Assistant] It's actually a kind of interesting question, uh, uh, what we learned here. I think, uh,


[User] Please transcribe this text

[Assistant] The, these better predictive models kind of have a sort of black box character to them. And, and so that was one of the sort of frustrating things about this. But one thing we definitely learned is that there is, uh, there is some signal in this data. And there's some things to be discovered. Uh,


[User] Please transcribe this text

[Assistant] One of the things that makes this possible in a rich way and I think is really important to reflect upon as we're seeing more and more, uh, online.


[User] Please transcribe this text

[Assistant] Uh, activities in which, uh, you know, we're getting data on students problem solving is the difference between what's mostly out there in, in MOOCs and Khan Academy these days, which is you get a question.


[User] Please transcribe this text

[Assistant] It may be reasonably complex. You might sit down and do some work on paper and 10 or 15 minutes later you enter a solution online.


[User] Please transcribe this text

[Assistant] Maybe it's two minutes later, but uhm, here there's actually two different parts that are entered. In these intelligent tutoring systems and, and other kinds of more, uhm, graphical user interface, more scaffolded environments, uhm, here's an, uh, an intelligent tutor, Kurt Van Lane, built for physics, uhm,


[User] Please transcribe this text

[Assistant] They really try to unpack the problem solving process, so it's not just the answer, but the work students do towards the answer. So the green stuff here is, is elements of the force body diagram that students are drawing and getting, in this case positive feedback, because the green indicating that they got it right.


[User] Please transcribe this text

[Assistant] They're entering definitions of variables up here, and here they'd be writing down multiple equations that describe the situation and accumulate towards a final answer. Uh, here they've gotten that equation wrong. So,


[User] Please transcribe this text

[Assistant] Essentially the same kind of problem, but instead of two data points, we have twenty data points. And so that means much better sense of what students are doing and learning, but also pedagogically more powerful.


[User] Please transcribe this text

[Assistant] Let me illustrate how that works. Here's one of, uh, uh,


[User] Please transcribe this text

[Assistant] The domains we, we did some analysis in geometry, uh, we were particularly interested in students' ability to not just bring formulas to bear to solve problems, but to do that in more novel situations where they have to combine formulas to, to say find the, uh, what's left over when you take that circle, uh, and cut it out of the square there.


[User] Please transcribe this text

[Assistant] When we get data like this, every one of these steps is recorded in the data set and we can look at the error rate across students, the average error rate over time, uh, to produce learning curves. And so a learning curve


[User] Please transcribe this text

[Assistant] Looks like this where on the x-axis we have opportunities to practice a particular, uh, skill and the error rate. And the trick here is to, uh, the insight is that we can use this kind of data to identify what the skills are. So if we think that geometry is a kind of skill you acquire,


[User] Please transcribe this text

[Assistant] And every time you practice a geometry problem, you get better at it. Uhm, we can categorize that data that way and, and we can create a graph and say,


[User] Please transcribe this text

[Assistant] Does that look like learning is happening? Is the error rate going down? And this is one of the visualizations that's in Datashop. This is a case where the knowledge component model is suggesting that there's just one component that you're acquiring. Yeah.


[User] Please transcribe this text

[Assistant] Yeah, well, if we take this same data and, and we label it more finely, so we make a distinction, uhm, between what we think the skills are. So now we have 12 categories.


[User] Please transcribe this text

[Assistant] We're gonna renumber that to say what's the first opportunity you had for circle area. And what used to be two in that graph might be the first opportunity to, to use circle diameter, right? And then we can regroup and re-average, and when we do that,


[User] Please transcribe this text

[Assistant] So this is empirical evidence that there are components of knowledge that drive learning.


[User] Please transcribe this text

[Assistant] Up there, we don't see any evidence of transfer if we think the components, this one big geometry muscle that's getting stronger. Uhm, here, when we look at these finer grain components, we do. And what's particularly powerful about this technique is that it continues to apply


[User] Please transcribe this text

[Assistant] It is the curve going down. So when we do that, we see some curves that are sort of reasonable. We also see some things like, like this one which suggests that


[User] Please transcribe this text

[Assistant] Hey, this is starting off pretty easy, staying pretty easy, and we're wasting students' time. That's a quick win to improve the system by reducing that. More interestingly though, we sometimes see curves that look like that original one, where


[User] Please transcribe this text

[Assistant] The error rate's not going down, it's staying high. And that could mean that they're not learning, but it could mean like that first curve in which we know they're learning, right? It just wasn't showing up because we had the, the model of knowledge wrong.


[User] Please transcribe this text

[Assistant] That, that, those labels are off. So we can dig in. We can say, well what, that composed by addition, what was it labeling? It was labeling these steps and these problems.


[User] Please transcribe this text

[Assistant] And if we look at the error rate, it's high up there, it's pretty low for the next two, and it's intermediate here, uhm, and, and,


[User] Please transcribe this text

[Assistant] Here you have to switch on your cognitive, psychology, math, ed, head and think hard about what's really going on to try to do this, to understand substantively what, what's happening. Uhm, after doing that, we...


[User] Please transcribe this text

[Assistant] Uh, concluded, uhm, that tentatively at least that, uh, that first step there requires that I really engage in planning to figure out how I'm gonna combine the two areas. Once I've got the plan, all I need to do with new input is, is repeat the arithmetic. I just have to do the subtraction.


[User] Please transcribe this text

[Assistant] In this problem, it's different because it's actually scaffolding, computing the needed intermediate sub-goals.


[User] Please transcribe this text

[Assistant] Uh, so the planning is in part being done by the scaffolding in the system, and that makes it slightly easier. So now we take that data, uh, I'll skip past this next graph, uhm, and we, uh,


[User] Please transcribe this text

[Assistant] Here's the, the, the twenty different steps across the different problems that were labeled with this and the range of error rates. We re-label it so that we get three different skills. Instead of the one represented in blue, they've seen the average here doesn't do a great job. We've got three different skills. It's not perfect, but it's much better fit to the data. Uhm, more,


[User] Please transcribe this text

[Assistant] Precisely, we can, uh, use, uh, a logistic regression model. This is a generalization of item response theory to build.


[User] Please transcribe this text

[Assistant] Predictive models that, you know, we can get quantitative tests of how well, uh, changes in the model, uh, fit the data. The changes are reflected in changing this Q matrix that maps each step, uh, to the skills involved.


[User] Please transcribe this text

[Assistant] The middle components representing difficulty, the last components representing learning and transfer over time, and that's what's producing these, uh,


[User] Please transcribe this text

[Assistant] The, the predictions shown in, in the green and blue lines here. Uh, so in Datashop, uh, you can re-enter these models and as you do so, you get updated prediction fit statistics like AIC and BIC and root mean squared error on various kinds of cross-validation. And,


[User] Please transcribe this text

[Assistant] What we often find is that the improvement, uh, the reduction in error isn't necessarily that big.


[User] Please transcribe this text

[Assistant] But as a matter of fact, we didn't change these models very much. Ten components went to twelve by changing one out of these, keeping the other nine the same to three here. And so most of these models are similar. So,


[User] Please transcribe this text

[Assistant] Unlike the KDD cup, we saw a large difference, but we didn't know what it meant. Here we have a small difference, but it's very highly meaningful. We understand, we can interpret it, and we can use it to make action. Yeaah.


[User] Please transcribe this text

[Assistant] Who know some cognitive psychology. They may know the domain or they may work with subject matter experts to, to do, to use this data to redesign courses.


[User] Please transcribe this text

[Assistant] And so we either need to get cognitive psychologists working with instructors or get more instructors doing discipline based education research and learning some cognitive psychology.


[User] Please transcribe this text

[Assistant] So here's how we take an insight like this and redesign a tutor. We're gonna add new skills to this, uh, to this skillometer that'll, uh, change the way the pacing goes and, and better isolate the, the, the practice that's needed.


[User] Please transcribe this text

[Assistant] We'll, uh, sequence differently. And, and what was most powerful actually in this study is we designed new tasks that got at this hidden planning skill we hadn't well enough identified.


[User] Please transcribe this text

[Assistant] And this isn't sophisticated intelligent tutoring in this case. This is the psychology.


[User] Please transcribe this text

[Assistant] Uh, guiding us towards a multiple choice question actually, but one that gets at the essential need here, which is to learn how to plan better. So you're just picking which of these is the best plan to solve that problem.


[User] Please transcribe this text

[Assistant] And so we ran one of these in vivo studies, uh, embedded in a school, uh, pre-test, post-test, looked at time to mastery and, uh,


[User] Please transcribe this text

[Assistant] Uhm, at the same time we actually increased the time they spent on tests.


[User] Please transcribe this text

[Assistant] Uhm, so, yeaah, I realized that, uh, I'm running low on time here, but we've been pretty interactive along the way. I did want to give, do one more thing, which is to give you a sense for


[User] Please transcribe this text

[Assistant] How we can, so this involved a heavy amount of cognitive psychology expertise to do this interpretation. Can we build a theory that can produce some of these features automatically? And the strategies there been to, uh, create, uhm,


[User] Please transcribe this text

[Assistant] Actually, I'm gonna skip over, this next section is about a search mechanism that searches through these spaces of factors that people can


[User] Please transcribe this text

[Assistant] I wanted to talk about is can these cognitive models be automatically generated and it goes back to this earlier slide here where we have this sim student theory that's been implemented in a computational model.


[User] Please transcribe this text

[Assistant] Uhm, it actually has four learning mechanisms.


[User] Please transcribe this text

[Assistant] Three of them are about learning production rules, uh, the how is the then part, where to look and when to do are part of the if part, and a fourth mechanism for learning representations. This is the grammar thing I referred to before.


[User] Please transcribe this text

[Assistant] There's a app, a key application here is the potential that we could build cognitive models and intelligent tutors essentially by teaching the computer how to do the task and then we'd have the model and the tutor would come from that.


[User] Please transcribe this text

[Assistant] So this representation learning part is the, the, the newest and most novel part of the, uh, uh, maybe the most powerful part of it. And it goes back to this, uh, chunking idea I mentioned earlier. And we've used probabilistic context free grammar induction to, to, to learn the structure of domains


[User] Please transcribe this text

[Assistant] By patterns of co-occurrence uh, uh, frequency.


[User] Please transcribe this text

[Assistant] Uhm, to build these chunks. So it's an unsupervised learning mechanism that just like the chess master who's seeing lots of boards, is gonna learn chunks because certain patterns frequently co-occur. And you can represent those patterns uhm, in, in a grammar uhm,


[User] Please transcribe this text

[Assistant] We have actually a two-dimensional version of a grammar. We haven't tried it with chess, but uhm, there's some potential that would work there as well. So, sim student, uh,


[User] Please transcribe this text

[Assistant] It works by being tutored. Uhm, you build some kind of graphical user interface for your domain. You can enter problems into that. You say this is the start state. Now try to solve it. And at first, student says, I'm stuck. I don't know what to do. Show me what to do. So in the interface, you demonstrate what to do.


[User] Please transcribe this text

[Assistant] It, it does its whatever version of parsing it's learned so far. It learns a production rule. Uh, the next step, it says I don't know what to do, show me. Um, it'll infer another production rule. We give it a little clue with, with labeling that. And again, the, when these labels repeat, that's when we collect together, uh, comp positive examples.


[User] Please transcribe this text

[Assistant] So the interesting thing is when you go to the next problem, what can SIM student do?


[User] Please transcribe this text

[Assistant] So uhm, it's inducing from examples here. So it may or may not get it right. So notice what it does here. Uh, this is a slightly different problem. It's a less sophisticated grammar. It doesn't know.


[User] Please transcribe this text

[Assistant] About the minus going with the three, so it said divide by three and, and the author says no, that's wrong. That becomes a negative example. The production rule, uh, will modify its if part to learn more. So we now show it what it should have done and it'll adapt and go from there. So we've used this, uhm, in, uh,


[User] Please transcribe this text

[Assistant] Uh, four domains here and we've compared the model that it came up with with the model that was generated by humans. Uh, most of the time it produces more distinctions than the human modelers did.


[User] Please transcribe this text

[Assistant] And most of the time, the consequence is that the prediction is more accurate.


[User] Please transcribe this text

[Assistant] Again, you can interpret this because you can look at the rules, right? Uhm, and so it was a pretty interesting insight that came out of this that I hadn't really fully recognized before. Uh, what should you think is the hardest of these four problems?


[User] Please transcribe this text

[Assistant] There's no number there, right? It's grammatically different in a subtle way. If you actually write the grammar, you have to write a different grammatical structure. Sim student learns a different rule for that one than for the other three.


[User] Please transcribe this text

[Assistant] Actually, there's a little bit of subtlety here, if you call dividing by 4 reasonable, if not...


[User] Please transcribe this text

[Assistant] Ideal thing to do, uhm, that's actually what the tutor is doing. So uhm, uh, but yeaah, it's actually quite a big difference and it's a consistent one. And that's one that the, that this machine system now discovered and we can verify that it better predicts the data in the way I was illustrating before. Yeaah.


[User] Please transcribe this text

[Assistant] Uhm, and it might be because this tutor pretty deliberately, uh, uh, presents problems on both sides.


[User] Please transcribe this text

[Assistant] But yeaah, that is one of those subtleties that might matter, but that's the cool thing about the data. You know, you have a hypothesis that it matters, you may or may not be right. Uhm, yeaah, and it turns out, I, uh,


[User] Please transcribe this text

[Assistant] It's actually an interesting question. If we train students where the problems were x is always on the left, it would probably, uh, do poorly when it first encountered those on the right. So, you know, part of it goes to the variety of the experience. Uhm, so, uh,


[User] Please transcribe this text

[Assistant] So yeaah, that's the bottom line message. You don't know what you know. Intuitive design is not reliable. We need data-driven learner models to do better. Uhm, and that better is both in terms of improving the science but also improving the practice of learning.


[User] Please transcribe this text

[Assistant] But, uh, uhm, yeaah, there's a lot of potential here.


[User] Please transcribe this text

[Assistant] And, you know, I'd invite you to, uh, look into some of the opportunities we have. I actually had, can, if you're interested in finding more about our summer schools or, uh, our, uh, the Datashop repository, I actually have some handouts about that. But, thank you very much for your time and the conversation.


[User] Please transcribe this text

[Assistant] In the default loss function, all samples, such as these code snippets, are treated the same, irrespective of their content. But there are scenarios where it could make sense to weight the samples differently. If, for example, one sample contains a lot of tokens that are of interest to us, or if a sample has a favorable diversity of tokens.


[User] Please transcribe this text

[Assistant] We can also implement other heuristics with pattern matching or other rules.


[User] Please transcribe this text

[Assistant] For each sample, we get a loss value during training and we can combine that loss with a weight. Then we can create a weighted sum or average over all samples to get the final loss for the batch.


[User] Please transcribe this text

[Assistant] Let's have a look at a specific example. We want to set up a language model that helps us auto-complete common data science code. For that task, we could, we would like to weight samples stronger where tokens related to the data science stack, such as PD or NP, occur more frequently.


[User] Please transcribe this text

[Assistant] Here you see a loss function that does exactly that for causal language modeling. It takes the model's input and predicted logits as well as the key tokens as input.


[User] Please transcribe this text

[Assistant] Then the loss per sample is calculated, followed by the weights.


[User] Please transcribe this text

[Assistant] Finally, the loss and the weights are combined and returned.


[User] Please transcribe this text

[Assistant] This is a pretty big function, so let's take a closer look at the loss and the weight blocks.


[User] Please transcribe this text

[Assistant] During the calculation of the standard loss, the logits and labels are flattened over the batch. With the view, we unflatten the tensor to get the matrix with a row for each sample in the batch and a column for each position in the sequence of the sample.


[User] Please transcribe this text

[Assistant] We don't need the loss per position, so we average the loss over all positions for each sample.


[User] Please transcribe this text

[Assistant] For the weights, we use Boolean logic to get a tensor with ones where a keyword occurred and zeros where not.


[User] Please transcribe this text

[Assistant] This tensor has an additional dimension as the loss tensor, which is so because we get the information for each keyword in a separate matrix.


[User] Please transcribe this text

[Assistant] We only want to know how many times keywords occurred per sample, so we can sum over all keywords and all positions per sample.


[User] Please transcribe this text

[Assistant] Now we're almost there. We only need to combine the loss and with the weight per sample. We do this with element-wise multiplication and then average over all samples in the batch. In the end, we have exactly one loss value for the whole batch. And this is the whole necessary logic to create a custom weighted loss.


[User] Please transcribe this text

[Assistant] In Accelerate, we just pass the input IDs to the model to get the logits, and then we can call the custom loss function.


[User] Please transcribe this text

[Assistant] After that we continue with the normal training loop by for example calling backward.


[User] Please transcribe this text

[Assistant] For the trainer, we can override the compute loss function of the standard trainer. We just need to make sure that we return the loss and the model outputs in the same format. With that, you can integrate your own awesome loss function with both the trainer and accelerator.


[User] Please transcribe this text

[Assistant] There is neither egg in eggplant nor ham in hamburger. Neither apple nor pine in pineapple. English muffins weren't invented in England. Or french fries in France.


[User] Please transcribe this text

[Assistant] Sweet meats are candies while sweet breads, which aren't sweet, are meat. We take English for granted.


[User] Please transcribe this text

[Assistant] But if we explore its paradoxes, we find that quicksand can work slowly. Boxing rings are square.


[User] Please transcribe this text

[Assistant] And a guinea pig is neither from guinea, nor is it a pig. If the plural of tooth is teeth, why isn't the plural of booth, beef?


[User] Please transcribe this text

[Assistant] Doesn't it seem crazy that you can make amends, but not one amend?


[User] Please transcribe this text

[Assistant] That you comb through amals of history, but not a single amal.


[User] Please transcribe this text

[Assistant] If you have a bunch of odds and ends and get rid of all but one of them, what do you call it?


[User] Please transcribe this text

[Assistant] In what language do people recite at a play and play at a recital?


[User] Please transcribe this text

[Assistant] Ship by truck and send cargo by ship.


[User] Please transcribe this text

[Assistant] Park on driveways and drive on parkways.


[User] Please transcribe this text

[Assistant] How can a slim chance and a fat chance be the same?


[User] Please transcribe this text

[Assistant] While a wise man and wise guy are opposites.


[User] Please transcribe this text

[Assistant] How can the weather be hot as hell one day and cold as hell another?


[User] Please transcribe this text

[Assistant] English was invented by people, not computers.


[User] Please transcribe this text

[Assistant] That is why, when the stars are out, they are visible. But when the lights are out, they are invisible.


[User] Please transcribe this text

[Assistant] And why, when I wind up my watch I start it.


[User] Please transcribe this text

[Assistant] But when I wind up this essay, I end it.


[User] Please transcribe this text

[Assistant] Since the 1950s, most of the stars of pop music have come from Britain and America. However, in the last 10 years,


[User] Please transcribe this text

[Assistant] When many different kinds of music have established themselves on the pop scene, more and more stars have come from other countries.


[User] Please transcribe this text

[Assistant] Pop music changes all the time, and new stars appear and become famous.


[User] Please transcribe this text

[Assistant] Many of today's stars started out in the 1960s and have changed their music to suit the time.


[User] Please transcribe this text

[Assistant] Although most stars take many years to become famous, their fame does not usually last long.


[User] Please transcribe this text

[Assistant] For a musician to stay popular and still produce good, original music over a long period of time is a sign of a true star. Most stars start their careers in a simple way.


[User] Please transcribe this text

[Assistant] Playing in unknown night clubs or dance halls where people want to dance to the music, not listen to it.


[User] Please transcribe this text

[Assistant] They can be sure that their lives will change once they are successful.


[User] Please transcribe this text

[Assistant] Ordinary teenagers living at home with their parents may suddenly find themselves rich enough to buy their own houses.


[User] Please transcribe this text

[Assistant] An established superstar may be able to buy several.


[User] Please transcribe this text

[Assistant] For a lot of them, this means they have no home life and their personal relationships suffer.


[User] Please transcribe this text

[Assistant] In spite of great public success, life at the top can be very lonely.


[User] Please transcribe this text

[Assistant] Today we're going to review this, the new grinder from Wilfa, the Wilfa Uniform. Now it's important to say right at the start that Wilfa sent me this unit for free.


[User] Please transcribe this text

[Assistant] It's important, not just because you should declare these things in product reviews, but actually we'll touch on this a little bit later in the review. Now I won't lie, when this grinder was announced, it got me pretty excited. It's in collaboration with Tim Wendelboe, uh, the famous.


[User] Please transcribe this text

[Assistant] Coffee Roaster, Coffee Person, uhm, I don't know. Tim Wendelbo. It feels a little bit like a grinder made for me.


[User] Please transcribe this text

[Assistant] For how I brew coffee in the mornings. Up until this point, at home I'd been using the cheaper Wilfer grinder, a grinder I had recommended because as grinders go, it was surprisingly good considering how cheap it was. This really is a whole different ballgame.


[User] Please transcribe this text

[Assistant] First world war grinder was pretty small, conical burr set, some pretty easy grind adjustments, a few good features, but, but this is a much more ambitious project.


[User] Please transcribe this text

[Assistant] And they've moved from conical burrs to flat burrs and the price has jumped up from about a hundred pounds to about 350 pounds. So expectations have gone up a lot along with the price. You know, this is, is in an interesting price point where actually there's not a whole load of competition.


[User] Please transcribe this text

[Assistant] And in many ways it stands alone in that category. It's pretty hard to buy a flat burr set, filter focused coffee grinder for home.


[User] Please transcribe this text

[Assistant] Without getting into sort of commercial equipment and the kind of ridiculous prices that commercial equipment costs.


[User] Please transcribe this text

[Assistant] This had me pretty pumped. And when I say it's made for someone like me, it's made for someone who gets up in the morning.


[User] Please transcribe this text

[Assistant] Gets a scale out, weighs their beans, grinds those beans, and then away and brews them. Right? I don't have a hopper full of beans, I'm definitely using a scale as part of my brewing routine. However I'm brewing, I'm definitely weighing my coffee.


[User] Please transcribe this text

[Assistant] It's kind of built for me. I want a more uniform grind profile.


[User] Please transcribe this text

[Assistant] And this is supposed to offer that. And it unquestionably does. It is a significant improvement in grind quality from the first wheel for grinder. You know, that those flat burs typically.


[User] Please transcribe this text

[Assistant] That is true. And the grinds coming out of this are much, much, much better. In terms of extraction, which is probably one of the few ways we have to comfortably talk about how good a job a grinder is doing.


[User] Please transcribe this text

[Assistant] Bruise 21 and a half, 22% extractions, if that means anything to you.


[User] Please transcribe this text

[Assistant] These taste really good, right? So it's certainly not having issues at slightly higher extractions. Certainly much better than the other Wilfer grinder would taste at those higher extractions. So, so the burrs are better.


[User] Please transcribe this text

[Assistant] They're not EK43 burrs, they're not top, top, top tier burrs, but for the home, they're a significant jump up, certainly from Wilther's other grinder. What I haven't been able to do yet is compare this to some similarly priced coffee grinders, perhaps the Sete from Barazza.


[User] Please transcribe this text

[Assistant] So that's, that's on the list of things that I do want to have a look at. So let's dive in a little bit more into the simple features of this grinder. So first off, the lid for the hopper. The lid for the hopper is a set of scales, right?


[User] Please transcribe this text

[Assistant] Nice, neat little set of scales. Uh, pretty simple functionality. You know, they do grams and the other.


[User] Please transcribe this text

[Assistant] Thing. No timer function on here because this does have Bluetooth and it does connect to an app.


[User] Please transcribe this text

[Assistant] I've used this grinder for about a month at home and mostly to make my first coffee of the day. And at that point, I know what I want to do. I'm brewing it with, uh, actually the Sage Precision Brewer, which I'll talk about another time. So I'm brewing it with an automatic brewer. So really my routine in the mornings is pretty simple. I don't want to have to worry about.


[User] Please transcribe this text

[Assistant] Apps, connectivity, any of that stuff, but it is part and parcel of this. And if you want bring recommendations from Tim, those are bundled into the app. So you have the scales on top.


[User] Please transcribe this text

[Assistant] This is designed really to be a single dose grinder, i.e. it's just gonna grind what you pour in the top. This isn't really designed to be a storage hopper for your bins. Ideally you're keeping them in something dark, dry, cool, airtight.


[User] Please transcribe this text

[Assistant] For brewing, let's say 45 grams to 750 grams of water. So like three quarters of a liter of brewed coffee. I'm at about 28 at the moment with my brewing setup. We're not going to get too deep into what kind of pre-infusion times we're going to flow it to have, but just to give you an idea of where this would sit. We'll talk about espresso a bit more towards the end of the review. And then the controls of the grinder.


[User] Please transcribe this text

[Assistant] Which is a simple on button. And when I heard about this, one of the killer features for me is that you turn this grinder on and it will switch itself off when it's finished grinding. I presume it's detecting some level of resistance in the motor.


[User] Please transcribe this text

[Assistant] Now they say this does help mitigate static electricity and there's a little black plastic piece that you need to have on sits on top of this while it's grinding. And that very simply is, is the grinder.


[User] Please transcribe this text

[Assistant] You know, it's really designed to be a filter coffee focused grinder. It can do espresso, and we'll pull some shots of espresso towards the end of the review.


[User] Please transcribe this text

[Assistant] But it's really, really focused and built around better filter coffee at home. Now I've wanted a filter coffee focus grinder at home for a long time and now I have one. And this is definitely going to be a part of my dealing brewing setup.


[User] Please transcribe this text

[Assistant] In the foreseeable future. Like I've had this for about a month. I'm really enjoying using it.


[User] Please transcribe this text

[Assistant] Though I have to say there are inevitably a few caveats. There are a few things for me that I didn't completely love that I do have to talk about. So let's start with the scale in the top.


[User] Please transcribe this text

[Assistant] Now I like a lot about having an integrated scale in the grinder. I think it's a really neat, clever place to put it. I think it looks good.


[User] Please transcribe this text

[Assistant] It does, however, lag a little bit. It's a little bit slow when you are weighing stuff out. For example, coffee beans. So most mornings I've tended to overshoot a little bit, one or two grams.


[User] Please transcribe this text

[Assistant] It's not the end of the world, and I can and should adapt to this, but it does bother me just a little bit that the scales are compared to others.


[User] Please transcribe this text

[Assistant] Is a killer feature for me. It's one of those things where, why isn't this normal? Why doesn't everything have this? However.


[User] Please transcribe this text

[Assistant] On occasions, it's detection of finished grinding hasn't been fantastic and it's run just a little bit longer than I've wanted. Just to the point where my brain has kind of clicked in and gone, do I need to switch this off myself?


[User] Please transcribe this text

[Assistant] Its detection can be a little slow, especially as those last few beams kind of bounce around in there. It does detect that it is grinding though still. Uh, so in theory, it's, it's getting everything through, but it can be a little bit slow.


[User] Please transcribe this text

[Assistant] So one of the things we should talk about is accessing the burrs in this grinder. To do that, you just take it coarse.


[User] Please transcribe this text

[Assistant] One of my great sort of pet hates in this world, in the world of grinders, is having threading.


[User] Please transcribe this text

[Assistant] Right, anyone who has tried to clean out a matzah grinder and been tightening the burrs back up again and hit a little bit of ground coffee in the threads will know that pain. I really, really, really don't like threading.


[User] Please transcribe this text

[Assistant] In and around ground coffee. That's like a personal thing. And there are other ways around this. You don't have to have threading in the burr chamber. It's easy to access. It's easy to clean. It's a small, neat burr chamber. I like all of those things. I just wish that they'd gone a different route on the threading.


[User] Please transcribe this text

[Assistant] Moving down to the grounds collection bin, uh, I like the stainless steel. I like that design. I like that material quite a lot. It doesn't do.


[User] Please transcribe this text

[Assistant] That much more. For me, for static mitigation, I still prefer to do the little water drop trick. Add just the tiniest amount of water to the coffee beans before you grind them and then you get a much better result. The lid that sits on top of it


[User] Please transcribe this text

[Assistant] To put back on every time, and I do worry about losing it at some point. I wonder if there could be a different materials choice or a different fit.


[User] Please transcribe this text

[Assistant] Yeah, just to make that a slightly more enjoyable experience.


[User] Please transcribe this text

[Assistant] Quite intentionally, this grinder is pretty slow.


[User] Please transcribe this text

[Assistant] Uh, Tim talked about this, that they wanted to have a slower motor to prevent less heat transfer to the coffee beans.


[User] Please transcribe this text

[Assistant] temperature and coffee beans, but really the best evidence is that incredibly cold coffee does grind better. But beyond that, I'm not sure that there's a ton of evidence that going that bit slower really does produce


[User] Please transcribe this text

[Assistant] Significantly better or noticeably better coffee. Like I get the idea behind it. I get, I get the hypothesis, but I just don't think we actually have any evidence and I would like this to have been just a little bit quicker in the mornings.


[User] Please transcribe this text

[Assistant] The good. I really like this grinder. I like the grounds it produces. I think they're very good for the money. I think to get better than this, you're spending significantly more money, even though this will be quite an expensive grinder. I like the design. I like the footprint and I like the build.


[User] Please transcribe this text

[Assistant] There are, however, a couple of things that annoy me slightly and...


[User] Please transcribe this text

[Assistant] I find it hard to really decide how I feel about those because I didn't pay for this grinder. I received this unit to review.


[User] Please transcribe this text

[Assistant] I would have happily paid for one. I think I'd have offered to pay for one, but they sent one to me. I'm going to keep it. I'm going to use it every day. But without having spent the money, I don't really know how much those things.


[User] Please transcribe this text

[Assistant] Would it bother me less, in fact? There are a couple of things that, that are imperfect, but I don't think that would stop me recommending this as a good grinder. And so before I finally, finally, finally wrap up, I did say I would pull some shots. Now, what I don't have here in the studio right now is an espresso machine.


[User] Please transcribe this text

[Assistant] So for the very, very last piece of this review, I'm going to go to where I can get an espresso machine, uh, down to some friends at VA machinery. And I'm going to pull some shots on a black eagle with this thing to see how we go. And I'll conclude the last bits of the review over there.


[User] Please transcribe this text

[Assistant] I think it's a great filter grinder for the home, and it does have its issues. It's not a great espresso grinder, but it's not supposed to be.


[User] Please transcribe this text

[Assistant] And there are other issues that are brought up, but if Wilfa fix them, then release the second version.


[User] Please transcribe this text

[Assistant] Well, they're not such big deals that I consider it worthwhile upgrading to a newer version, so...


[User] Please transcribe this text

[Assistant] That's kind of an interesting metric for me to consider when I think about is that really an issue or not.


[User] Please transcribe this text

[Assistant] I will be making it a part of my daily brewing routine. That's the Wilfer Uniform.


[User] Please transcribe this text

[Assistant] There's a blue bath mat, a blue fake fur cover on the toilet seat.


[User] Please transcribe this text

[Assistant] All this bathroom lacks from the time before is a doll whose skirt conceals the extra roll of toilet paper. Except that the mirror over the sink has been taken out and replaced by an oblong of tin and the door has no lock. And there are no razors, of course. There were incidents in bathrooms at first. There were cuttings, drownings. Before they got all the bugs ironed out. Cora sits on a chair outside in the hall to see that no one else goes in.


[User] Please transcribe this text

[Assistant] In a bathroom, in a bathtub, you are vulnerable, said Aunt Lydia. She didn't say to what. The bath is a requirement, but it is also a luxury.


[User] Please transcribe this text

[Assistant] Merely to lift off the heavy white wings in the veil. Merely to feel my own hair again with my hands is a luxury. My hair is long now. Untrimmed. Hair must be long but covered. And Lydia said, St. Paul said it's either that or a close shave. She laughed. That held back neighing of hers. As if she'd told a joke. Cora has run the bath. It steams like a bowl of soup.


[User] Please transcribe this text

[Assistant] I take off the rest of the clothes, the overdress, the white shift and petticoat, the red stockings, the loose cotton pantaloons.


[User] Please transcribe this text

[Assistant] Without caring that my legs, my arms, my thighs, and back were on display, could be seen. Shameful. Immodest.


[User] Please transcribe this text

[Assistant] I step into the water, lie down, let it hold me. The water is soft as hands. I close my eyes, and she's there with me. Suddenly, without warning.


[User] Please transcribe this text

[Assistant] Which was when Luke and I did the week shopping because both of us had jobs.


[User] Please transcribe this text

[Assistant] She was sitting in the little baby seats they had then in supermarket carts, with holes for the legs. She was happy enough, and I turned my back, the cat food section I think it was.


[User] Please transcribe this text

[Assistant] She was crying and saying it was her baby. The Lord had given it to her. He'd sent her a sign. I felt sorry for her. The store manager apologized and they held her until the police came. She's just crazy, Luke said.


[User] Please transcribe this text

[Assistant] I thought it was an isolated incident. At the time.


[User] Please transcribe this text

[Assistant] Behind my closed eyes I can see myself as I am now, sitting beside an open drawer or a trunk in the cellar where the baby clothes are folded away. A lock of hair, cut when she was two, in an envelope, white blonde. It got darker later.


[User] Please transcribe this text

[Assistant] If you have a lot of things, said Aunt Lydia, you get too attached to this material world, and you forget about spiritual values. You must cultivate poverty of spirit. Blessed are the meek.


[User] Please transcribe this text

[Assistant] I lie, lapped by the water beside an open drawer that does not exist. And think about a girl who did not die when she was five. Who still does exist, I hope, though not for me. Do I exist for her?


[User] Please transcribe this text

[Assistant] Am I a picture somewhere in the dark at the back of her mind?


[User] Please transcribe this text

[Assistant] I ain't got all day, says Cora's voice outside the door. It's true, she hasn't.