[User] Please transcribe this text

[Assistant] Uhm, but all the telemetry is always, you know, at least 90 minutes old. So it's like all you ever know about where the spacecraft is is where the software says it is. It's the only kind of object that you have to point at.


[User] Please transcribe this text

[Assistant] So, these were kind of complicated problems to, uhm.


[User] Please transcribe this text

[Assistant] To resolve and the, the, the, the, the resolution actually at first was we'll run both systems in parallel for two months and then we'll switch over. Well, it's been years at this point and nobody's switched over and whether they actually switch over is, you know, is, is, is, is uncertain. But, uhm, yeaah, they actually called it going the fool, Monty. That's not my pun. Uhm, the, but the, uh,


[User] Please transcribe this text

[Assistant] But it does, this sort of produces, this moment of anxiety around the navigational software and the idea of the uncertainty of knowing where the object is or even of knowing how it is that one might know where the object is is a really interesting one, right? Because that's the anxiety that's provoked.


[User] Please transcribe this text

[Assistant] It's like, if this software is correct and is telling us something new, then what does that mean about what we knew before? But we knew that what we knew before was okay because we hadn't crashed it into anything yet. So, so we must be at least approximately correct. So it's like it produces this anxiety about how the organization might even ever know what's going on.


[User] Please transcribe this text

[Assistant] And the attempts they made to reconcile the two were confounded by the fact that the software


[User] Please transcribe this text

[Assistant] Difficult if not impossible to read the algorithm back off the software. We were talking about algorithms earlier, earlier lunch and, and the kinds of


[User] Please transcribe this text

[Assistant] What the algorithm is in here that's giving it different kind of answers, especially where it's dependent


[User] Please transcribe this text

[Assistant] Uhm, you can't take the slice, the core sample through all those different kind of layers and pull it out and say that's the algorithm and oh look it's incompatible here with what that, what that piece of code is doing.


[User] Please transcribe this text

[Assistant] Uhm, so the material configurations...


[User] Please transcribe this text

[Assistant] Of, uh, of, of software objects and hardware objects and their sort of, their complicated sort of entanglements, uhm, become, uh, you know, begin to obscure what it is the organization even knows about how to navigate spacecraft and where a space, where a spacecraft is.


[User] Please transcribe this text

[Assistant] Uhm, it obscures the sort of the, the, the, uhm, the expertise that the organization has despite the fact that there could be no more expertise about space navigation than there is, than there is at NASA, right? These guys are clearly expert, but the ways in which they're expert are not even necessarily available to them because of the ways they're entwined with the software system.


[User] Please transcribe this text

[Assistant] Is a, is another important material consideration, which is the temp, which are the temporalities of this stuff.


[User] Please transcribe this text

[Assistant] One of the things, one of the problems for what, one of the aspects of the Cassini project that manifests itself in this stuff is that you can't stop. Uhm.


[User] Please transcribe this text

[Assistant] Uhm, and in fact the temporality here is not a punctuated temporality as it is for various other kinds of things. It's a continual ongoing thing and the ongoingness of it.


[User] Please transcribe this text

[Assistant] Uhm, actually we have two, but the other one's still here. And the other is, uhm, it took us a long time to learn how to do that.


[User] Please transcribe this text

[Assistant] So that was number one, that, that, going the full monte. The second of the sort of three things I want to talk about, uhm, that sort of give us some kind of perspective on these questions of socio-materiality, uhm, arises around ways of accounting for technological degradation. So.


[User] Please transcribe this text

[Assistant] Spacecraft has to tumble in space and move around to point at different kinds of instruments at the, at the, whatever the object of astronomical and scientific attention is at any given moment. It has two ways of doing that.


[User] Please transcribe this text

[Assistant] Uhm, but the spacecraft can also sort of rotate on its own axis.


[User] Please transcribe this text

[Assistant] And it does this by having a set of what they call reaction wheels.


[User] Please transcribe this text

[Assistant] With a series of wheels, all working just fine.


[User] Please transcribe this text

[Assistant] Uhm, and after, but after a while they began to notice that the wheels weren't behaving exactly to spec. One of the wheels was breaking down. Uhm.


[User] Please transcribe this text

[Assistant] Now the actual process of interpreting what the wheels do is really complicated because you also actually used, it was dragging.


[User] Please transcribe this text

[Assistant] But drag is actually one of the ways that they do gravitational measurements as they get close to things and actually at some points you deliberately fly the spacecraft through the outer atmosphere of some bodies in order to asses- and use the drag to assess


[User] Please transcribe this text

[Assistant] They sort of went, oh, we need to be paying attention to the, to the performance of the reaction wheels as a way of understanding what the spacecraft's doing and what things are doable, what things we can actually support. So they built a piece of software.


[User] Please transcribe this text

[Assistant] Codrbot, the reaction wheel bias optimization tool, that one I remember.


[User] Please transcribe this text

[Assistant] Uhm, and our bot is a mathematical model of the behavior of the wheels that allows them to, uhm, evaluate potential movements of the spacecraft, pointing arrangements of the spacecraft, observations the spacecraft's going to make.


[User] Please transcribe this text

[Assistant] In terms of the reaction, the impact upon the reaction wheels, whether it's going to take the reaction wheels into dangerous frequency bands that are sort of problematic and cause the reaction wheels to degrade faster, or whether this is, uhm, a safe maneuver, what they often call arbot friendly.


[User] Please transcribe this text

[Assistant] It's kind of a slow piece of software. It just, I mean, there's a lot to figure out here, and it takes a long time. It takes a kind of, more than it takes a long time, it takes an unpredictable amount of time. I can tell you,


[User] Please transcribe this text

[Assistant] So uhm, as somebody who lives in Los Angeles, any studies that are ever done of like traffic and the things that cause stress on your commute are not about how long your commute is, it's about how predictable your commute is. The problem with Arbot and the thing that causes organizational stress is not that it takes a long time to run, but that nobody quite knows how long it's going to take to run.


[User] Please transcribe this text

[Assistant] And nobody quite knows what it is that it's going to say. We know the algorithm, we know the math, but we don't know it. Nobody's got a sort of intuitive understanding of it. At least not on the science side. The engineers have a better understanding of it. But from the science side, all they know is...


[User] Please transcribe this text

[Assistant] That once upon a time they could say, this is the instrument pointing that I need to do, and go do it. And the engineers would say, okay, here's your data.


[User] Please transcribe this text

[Assistant] And now they say, here's the instrument pointing that I need to do. And the engineers say, no, we're not going to do that. That's not friendly to the wheels.


[User] Please transcribe this text

[Assistant] But, that they might say that at an unpredictable moment. Uhm.


[User] Please transcribe this text

[Assistant] At the same time as it's the absoluteness of its decisions or its statements is sort of a problem.


[User] Please transcribe this text

[Assistant] Uhm, and so the organization had to sort of switch things around in order to try to get to a point where they could do science in a way that would not bring it into conflict with, with, with, with engineering. The scientists wanted a set of guidelines.


[User] Please transcribe this text

[Assistant] saying what kinds of things are arbot friendly and what kinds of things are not but they arbot doesn't work that way you can't say you can point it this way and you can point it that way and it's going to be okay because it's all about how you get from over here to over there and it depends on where the spacecraft is and again the gravitational forces that impinge upon it at different kind of moments


[User] Please transcribe this text

[Assistant] Uhm, and so, nobody can quite get to what the, uhm, what Orbot Friendly is.


[User] Please transcribe this text

[Assistant] The software, the algorithm has inserted itself into the organizational planning process. And in fact, the organization had to completely change its scientific planning process to place our bot assessment early on. Because now,


[User] Please transcribe this text

[Assistant] There was a, there was a moment of bizarre breakdown where the engineers said to the scientists, well you need to, so that we can figure out which of these things we're going to be able to do, you need to give us the science.


[User] Please transcribe this text

[Assistant] And the scientists are like, are you guys kidding? Everything on here is a priority. That's why it's on the list. We spent six months fighting with each other, figuring out what things we were going to put on the list. The result of that is, is like, it's just, everything's a high priority. Uhm.


[User] Please transcribe this text

[Assistant] So, so the, the, the organization actually had to respond by also making all sorts of changes in order to incorporate Rbot into what they're doing. Now this is an interesting thing, they're not really in some ways incorporating the reaction fields into what they're doing, they're incorporating Rbot into what they're doing. Rbot is a piece of software that was after all built.


[User] Please transcribe this text

[Assistant] So it's kind of complicated how we incorporate this thing. So that then takes me to the second point, which is this one about sort of where's the material here.


[User] Please transcribe this text

[Assistant] If we talk about social material, we say the social organization has responded in some ways. It's responded to material constraints. But you could argue that the material constraints to which it's responding are not the degradation of the wheels, but rather the operation of a piece of software on a particular computer.


[User] Please transcribe this text

[Assistant] That is, if this is something that actually the organization could respond to differently because I can put more memory into my computer and have it, you know, come up with an answer faster. That's a kind of interestingly alternative material constraint.


[User] Please transcribe this text

[Assistant] It will be no surprise to you, a lot, a lot of software going on in this system.


[User] Please transcribe this text

[Assistant] And some software is absolutely mission critical, you know, the kind of software which, you know, if this software was to go wrong, a spacecraft filled with radioactive fuel would explode in orbit and rain the fuel down on


[User] Please transcribe this text

[Assistant] A term people use, category D, which refers to uncategorized software, right, this residual category, all the other stuff.


[User] Please transcribe this text

[Assistant] The category D software is like the kind of little things that you build for yourselves. Your, your Python programs and your Perl scripts and your random little bits of wizardry that hack together the different pieces of your life and make it possible for you to actually get anything done. Uhm, the Tour Atlas is, is a piece of category D software.


[User] Please transcribe this text

[Assistant] So the Tour Atlas is this, is this, uhm, set of scripts that generates HTML pages that describe things about how, uhm, where the spacecraft has been in the past and how certain kinds of, uhm, arrangements, navigational and pointing arrangements were achieved.


[User] Please transcribe this text

[Assistant] It takes a slice through lots of databases, uhm, and pooling information from all sorts of different kinds of places in order to just help people to, uh, to, uhm, get their work done more easily by reusing knowledge from the history of the, from the history of the mission.


[User] Please transcribe this text

[Assistant] The tour atlas was a set of scripts put together by a guy. We'll call him Jim.


[User] Please transcribe this text

[Assistant] And the thing is that Jim was building these software pretty much for his own benefit.


[User] Please transcribe this text

[Assistant] But pretty soon everybody got the idea that it was there. People would go, you know, you would know that if you could save yourself some time by going and getting, working, sitting down at Jim's desk with Jim and having Jim take you through whether the particular kind of thing you've done, you're wanting to do is something we've ever done before. How is the organization able to do this? Well, here's, here's, here's its, here's its history. Uhm.


[User] Please transcribe this text

[Assistant] You don't want to go through all that process and then get to the end and discover you can't do it.


[User] Please transcribe this text

[Assistant] Uhm, because NASA has a matrix model, matrix organization, and uhm, and, and, uh, and so people often move around inside the organization, different sort of, different missions.


[User] Please transcribe this text

[Assistant] He was the guy who knew how to make this work. And this wasn't actually about, uhm, using Jim's software. This was about working with Jim. It was always about working with Jim. It was Jim who could, uhm,


[User] Please transcribe this text

[Assistant] He knew how to navigate the archive in terms that the Tour Atlas provided. Uhm, and the organization, I mean, it hasn't lost knowledge in the sense that all the databases that say where they've been and how they got there and what they did when they got there are still there.


[User] Please transcribe this text

[Assistant] Uhm, and Arbot still, or, sorry, the Tour Atlas still works as a way to extract, no, as a way to extract stuff.


[User] Please transcribe this text

[Assistant] And yet, the way in which it's going to be entwined into a sort of informal planning process has sort of broken down because it ended up sort of relying so much on, on, on Jim. Uhm, so the software evolves.


[User] Please transcribe this text

[Assistant] Is that uhm, we can maintain the software, we can maintain the code, but what they actually find more problematic is maintaining the set of relations around the code that actually make it.


[User] Please transcribe this text

[Assistant] As I say, the issue with making the Tour Atlas work is not making the code work. Using the Tour Atlas was not running a piece of software. Using the Tour Atlas is sitting with Jim.


[User] Please transcribe this text

[Assistant] Uhm, and, and so the Tour Atlas was always and only an adjunct to a series of things that Jim was doing, even as it was a sort of database browser, right? So your database browser only works if you know how to, how to look through it.


[User] Please transcribe this text

[Assistant] Uhm, that are deeply entwined into sort of organizational life and scientific practice.


[User] Please transcribe this text

[Assistant] Three modes of figuring, uhm, three ways of seeing the spacecraft, seeing the mission, producing a representation of it. In the case of Monty, like literally in figures, numbers, digits.


[User] Please transcribe this text

[Assistant] Uhm, each of which speak to questions of socio-materiality by giving us points of purchase on the relationship between social forms and material forms.


[User] Please transcribe this text

[Assistant] Uhm, so, so what would this tell us about the ways that people have tried to analyze socio-materiality as, or employ socio-materiality as a way of, uhm, as a way of analyzing organizational needs and the relationship to, of societies to, to, of social structures to, to technology. Well, well, the first one is, is sort of,


[User] Please transcribe this text

[Assistant] You know, the first kind of socio-material, uh, uh, uh, conception is one of like the social and the material is separate. Yeaah. And then, that the analysis that we want to understand is one that speaks to the relationality between them, right? Speaks to how these two go together. There's the social world.


[User] Please transcribe this text

[Assistant] So arguably, what people interact with is not the spacecraft, what they interact with is all sorts of stuff in the ground systems, which are apparently telling a story about where a spacecraft might be, but you know, it might have been captured by the Martians and they could be beaming us all back, all sorts of weird telemetry, right? Uhm, they, uh,


[User] Please transcribe this text

[Assistant] As a way of understanding the relationship between a world that is purely social here and a world that is purely material there that somehow have some kind of uhm, uhm, link. That just, that, that just doesn't seem to work very well in this case. Even in fact if we don't draw the dividing line at


[User] Please transcribe this text

[Assistant] Uhm, even if we, even if we look for other kinds of separations, uhm, there's a much more complicated tangle at work here.


[User] Please transcribe this text

[Assistant] Social and material are two aspects of the kind of sort of like contemporary organizational systems, contemporary social systems that sort of mirror each other in the ways in which they


[User] Please transcribe this text

[Assistant] Push on, uhm, the, push on the world, uh, they sort of constitute, they both constitute the world.


[User] Please transcribe this text

[Assistant] That's a, you know, that's a, I like that a little better than the separation argument. Uhm, but we don't find that working terribly well analytically for us in this data either.


[User] Please transcribe this text

[Assistant] Because in fact, in our situation, what we seem to have, I mean the symmetry argument again, I remember at the beginning I was saying these things all sort of figure around where do you locate the agency. The symmetry argument says you locate it, uhm, equally, continually in both of the, in both of these domains. But we sort of find that kind of rocking back and forth here. Sometimes,


[User] Please transcribe this text

[Assistant] Interventions from the Saturn system where broken down or degrading, um, wheels or bad telemetry suddenly forces a reorganization of both technical and social material practice on the, practice on the ground and so forth. So, so a symmetry doesn't need to work because it's more of a sort of tacking back and forth that we're finding, that we're finding at work.


[User] Please transcribe this text

[Assistant] Is sort of the third of the ways in which again many people have tackled the social material, which is primarily that sort of, you know, it's still harkening back to the social constructionist. The idea that places...


[User] Please transcribe this text

[Assistant] Uhm, that one just to sort of, you know, is we don't see this kind of unidirectionality. As I said, we've got this sort of tacking, tacking back and forth. So,


[User] Please transcribe this text

[Assistant] By placing an emphasis, I sort of began with, and the paper on which this talk is based sort of really sort of argues instead for a kind of understanding this in terms of reconfiguration.


[User] Please transcribe this text

[Assistant] Alright. Figuring the figural practices by which particular kinds of models of reality are produced.


[User] Please transcribe this text

[Assistant] Doing so, con-figuring, doing that figuring in a way that is deeply connected to what they describe because, you know, uh, uh, you know, you can't be decoupled from that.


[User] Please transcribe this text

[Assistant] Rather than, uhm, uh, rather than a sort of, uhm, this


[User] Please transcribe this text

[Assistant] The idea here is to pay attention to the representational practices that technologies provide us with. To think about the technologies that we're used to thinking about as things that do stuff.


[User] Please transcribe this text

[Assistant] Has, uhm, material properties, granularity, temporality, uhm, um, malleability and so forth. That's a larger project, uhm, that I would have talked about if I hadn't already talked for so long. So I will not talk more about that. Uhm, but, uhm, I don't know, maybe I'll...


[User] Please transcribe this text

[Assistant] Uhm, especially over time. And so the other part is actually I think that uhm, the dynamics of this and the temporal, and the temporal dynamics of it. So the fact that this is sort of always ongoing becomes really, really important.


[User] Please transcribe this text

[Assistant] I think that changes the way that, uhm, a lot of the, a lot of the, uhm, argumentation or associative materialities, particularly in organizational studies, uhm, happens. And I think it's, uhm, much, much broader than simply that, simply the, the Cassini stuff.


[User] Please transcribe this text

[Assistant] But because we do frequently have this, uhm, this idea of, uh, the, the, the running, this idea of the, for instance, the separation between, uhm, between individually and separately constituted domains of social and material that somehow, you know, oh yeaah, I have to build a, a conceptual bridge there, but they, they, that, you know,


[User] Please transcribe this text

[Assistant] Puts them together but keeps them apart at the same time is, is, is I think what we're trying to, what we're trying to do. So Cassini turns out to be a useful place to do that because of some of the, uhm, separations that it has. Uh, but I think that, I think the points speak much more broadly to the way that, that the notions of the social and the material are invoked as that sort of analytic frame.


[User] Please transcribe this text

[Assistant] Digital traces, uhm, this question of am I working with the data or am I working with what it is that data represents, right? Am I reaching through the data or is the data, is the data my, my, my object? Uhm.


[User] Please transcribe this text

[Assistant] And I do think again, sort of going to, to what I was saying to Fred as well, that the, it's, it's actually often the temporal dynamics that pull this stuff back together. Because, because, you know, what makes


[User] Please transcribe this text

[Assistant] Uhm, and, and that we often, I think, forget about that sort of active role. I mean, this goes against things we were chatting about earlier. It's like, this, this relationship between


[User] Please transcribe this text

[Assistant] Which is the complicated thing. So I think trying to find what the right sort of figure of ground shift is that allows us to keep both of those in view at the same time, uhm, is, is really important and I think that is something that would tie most of those sort of data practices to the kinds of things that these guys are doing too.


[User] Please transcribe this text

[Assistant] Uhm, there is lots of turnover and lots of churn and the kind of turnover and the kind of churn there is turns out to be fascinating. So,


[User] Please transcribe this text

[Assistant] On the engineers, people certainly move around. The matrix organization kind of like forces that. People move into this project. Navigators, for instance, move into this project because they want to do the kind of like cool navigation stuff. Uhm, people sort of, people certainly cycle around a good deal. Uhm, the,


[User] Please transcribe this text

[Assistant] One of the things that is really fascinating, and actually Marissa is trying to write about this just now, on the engineering side, this project has become the province of a number of very senior, uhm, uh, older women. The men,


[User] Please transcribe this text

[Assistant] The way to make progress in the organization is to pick up some skills here and then move somewhere else. So the organization kind of forces them into that sort of turn mode. But it has played out differently in science and engineering, and it's played out differently for people at different levels of seniority, and it's played out really differently across gender divides in ways that I find, I find fascinating.


[User] Please transcribe this text

[Assistant] Today I'm gonna be talking about how we can leverage foundation models for robotics, uh, and kind of get the best out of both of them. Uh, to start to sort of like motivate, uh, I'm gonna do a brief demo.


[User] Please transcribe this text

[Assistant] So one really nice thing about language models is essentially, uh, you know, they act as this like knowledge base of even like very esoteric things like the skittles are tasting the rainbow and you can see the robot actually doing it is over here and this is the robot's view here.


[User] Please transcribe this text

[Assistant] And I just said I'll take Reese's and it remembers from before that I just said which bowl is which, whose it belonged to, and


[User] Please transcribe this text

[Assistant] Yeah, again, you get this like very kind of abstract knowledge that taste of the rainbow means skittles. You can also do things like, uh,


[User] Please transcribe this text

[Assistant] I don't know, really impressive to us that I can just say put them all in different corners, and it understands that that means that not the same corners, it understands what the corners are, it knows what all the candies are, and it's able to like loop through this. Like, this was something that, uh, I don't know, about a year ago sort of like blew our mind that this is possible. Uhm, so I think,


[User] Please transcribe this text

[Assistant] It might have missed, I think it, because they're in the same bowl, it like, it put the Reese's on top, the Skittles, and then did that. So this is like one area that we're sort of trying to like, fix, is that sometimes this grounding between the code or the language model, the fact that it can't see things means that it makes sort of simple mistakes like this. And I'll talk through sort of like,


[User] Please transcribe this text

[Assistant] Why this is a big problem in robotics and sort of like where we're gonna go, uh, to fix it. So, let's see, what else can I do? Uhm, the...


[User] Please transcribe this text

[Assistant] So something like put the M&M's in the center of the bowls, and this is actually like, so essentially this would just like, knows that the center is, you know, the middle, and then let's say, okay.


[User] Please transcribe this text

[Assistant] So something like making a line is like also a somewhat abstract concept. It really like requires you to understand the way that like, you know, geometry works and, uh, and you can kind of, the code is maybe hard to see, but essentially it should output something. I would miss the, uh, re-sees because I think it's too far off the screen.


[User] Please transcribe this text

[Assistant] Oh, I think it actually got the M&M's in between the Skittles, so, and then the Reese's is like so far off that it's falling off the table. But essentially like, it actually understood like what's there in a wrote code to be able to do some like abstract geometric reasoning like this. And it's not like, you know, we've like trained this or shown it anything, this is just like falling directly out of the language model. So uhm, and let's say, uh,


[User] Please transcribe this text

[Assistant] So something like, also like, move the M&Ms down, like, it has to understand that, like, the direction of down is, is like in the negative Y direction, uhm.


[User] Please transcribe this text

[Assistant] And wait till that updates, and it's able to successfully move that, and then...


[User] Please transcribe this text

[Assistant] And something like move them a little bit up requires you to not only like understand sort of like general geometry, but also that like a little bit is not all the way up, it's partial, part of the way. And let's see if, move.


[User] Please transcribe this text

[Assistant] Essentially, uhm, I'm gonna talk through sort of like how we've gotten to this point and sort of some directions forward that we've been looking at to expand beyond this to fix some of the issues like when it picked up the wrong thing or, uh, you know, didn't do the, didn't do exactly what we expected.


[User] Please transcribe this text

[Assistant] To start, I mean this is all about foundation models and the main idea here is that you take the vast trove of information on the internet, throw them through a transformer, uhm, and we get performance out of it.


[User] Please transcribe this text

[Assistant] I think a lot of times this takes a lot of work for us to collect the amount of data that was just given to us for free from here. So we should really like strive to figure out how to use them. And these capabilities are sort of like shooting up like starting from language models that were able to answer trivia or explain jokes.


[User] Please transcribe this text

[Assistant] We can also now generate images, we can answer questions about images and do these VQA tasks, uhm, and we can even like hear audio and directly like map it through a foundation model to get the text out of it. And the real thing is that all of these performances like, they're great as they are and they're like continuing to improve and improve and to become more and more general.


[User] Please transcribe this text

[Assistant] Uh, easy reset so that it can collect data, like it can close this, uh, these two bins so that it moves one thing from another. It's got, uh, very like short horizon tasks of pick and place. It's got a kind of structured tasks and the robots aren't really moving around if we're doing any sort of like mobile manipulation. It's usually like at a very limited scale. So we sort of want to, you know,


[User] Please transcribe this text

[Assistant] I guess general knowledge that's in there that is important for a robot to act in the world.


[User] Please transcribe this text

[Assistant] Uh, there's also a lot of semantic knowledge of how things map to each other, what order to do things in, uh, these are things that traditionally robotics has really struggled with. But there's actually like a lot more challenges than there really are like areas of overlap. I think in some ways like other fields,


[User] Please transcribe this text

[Assistant] Have been very quickly revolutionized by having foundation models be able to solve them. Robotics on the other hand, has all sorts of challenges like, just general grounding. You need to know what your robot is capable of, what the environment has. A language model can't see anything, this like, is sort of like a huge gap for, for them.


[User] Please transcribe this text

[Assistant] Uh, interactive decision making, everything they've been trained on is this like unsupervised, just internet data with no interaction, no making the wrong decision, making the right decision. Uhm, this is like a core part of robotics that they basically don't have in them.


[User] Please transcribe this text

[Assistant] Uhm, so it's really hard to get this like expert data that we could, that we got for free from the internet essentially.


[User] Please transcribe this text

[Assistant] Uhm, vision and just general sensing modalities are, I guess, slowly getting better in vision language models but have traditionally also not been an area of strength.


[User] Please transcribe this text

[Assistant] So to first start, I'm going to talk about how we can ground these models in the world and in their environment. Then I'll talk about what just powered that demo, this code as policies.


[User] Please transcribe this text

[Assistant] Then I'll talk about kind of like steps forward for building foundation models or using foundation models to give new, uh, knowledge to, uh,


[User] Please transcribe this text

[Assistant] Robotics and then finally I'll talk about maybe training our own foundation model and what that sort of looks like.


[User] Please transcribe this text

[Assistant] As I said before, a robot can't see, or a language model can't see, so we sort of imagine this, this task that like, you're a language model and you open your eyes, uh, or you can't open your eyes and you're trying to reason over the entire environment.


[User] Please transcribe this text

[Assistant] Once you, like, open it and you realize that I've asked you, I've said I've spilled my drink, can you help? You realize that what's in the scene...


[User] Please transcribe this text

[Assistant] Or things like a spilled drink, a sponge, which might be able to help. And also I've got a lot of experience in the world of doing different things. So as we sort of like go through we see, you know, everything that's like where I might want to throw that away, what the gripper looks like in the embodiment.


[User] Please transcribe this text

[Assistant] And then also like in the, in the past I've been able to do things like pick up these objects. So this is all like crucial to the embodiment. It's both what you can do and it's what's in the environment. And we basically want to like bring this into a language model. Give it in some ways the hands and eyes necessary to make it do these tasks.


[User] Please transcribe this text

[Assistant] But if we just take a language model out of the box and say, you know, ask it this question, it answers pretty reasonably. I mean, it says, like, you could try using a vacuum cleaner or calling a cleaner. Uhm, it even, like, takes credit for the spill, which is, like, a very reasonable thing from a language model side but not necessarily actionable for robotics.


[User] Please transcribe this text

[Assistant] And yeaah, so, you know, we wanted to be able to speak this language of robots. We wanted to be able to like, actually be.


[User] Please transcribe this text

[Assistant] And so the first, uh, algorithm that I'll kind of put forth is, uh, called Seikan and the idea is essentially


[User] Please transcribe this text

[Assistant] Instead of letting the language model, where normally it would just generate sort of like the max probability next token, uhm, we are instead going to fix the way that it can respond. So these are like the only ways that, that it can possibly respond, it's called like a scoring mode, and basically see how likely is it to respond in each of these ways. So if I said like put the apple on the table,


[User] Please transcribe this text

[Assistant] And we say, like, these are the things you can do. You can, like, find an apple, find a sponge, pick up an apple, uhm, and these are, like, actual probabilities that would come out of a language model. Uh, and you can see, like, pick up the apple is really high. So, like, that makes a lot of sense, but if you then, like,


[User] Please transcribe this text

[Assistant] Open your eyes, uh, you see that, well, there's not an apple in front of me, I'm just sort of like in the middle of the environment.


[User] Please transcribe this text

[Assistant] So how do we like bridge this gap? And one way we can do this is robotic policies usually come with trained value functions. And these value functions are the probability of getting the reward of succeeding at a task given the state, given the image of the environment. So here we might see that the value functions say, well,


[User] Please transcribe this text

[Assistant] Picking up an apple is like pretty low because there's not an apple there.


[User] Please transcribe this text

[Assistant] And together, if you multiply these, we get like the most useful and possible thing. So, in this case, it becomes find an apple, which, you know, makes a lot of sense. First you need to identify it. So,


[User] Please transcribe this text

[Assistant] Once you do that, you can then update this prompt that's going into the language model here. And so now it knows, okay, I found an apple. The value function gets updated saying there's an apple in front of it, and so probably it picks, uh, pick up the apple.


[User] Please transcribe this text

[Assistant] And what's really nice is we have this, like, interpretable way of looking at its decision making. We can actually see that, uhm, you know, the language model is reasoning over this, the affordances are reasoning over this, and we can see how it comes to its decision. And we can actually, like, which makes it, like, super easy to debug and understand. But here if I said, like, I split a code, can you bring a replacement? You actually notice that, like,


[User] Please transcribe this text

[Assistant] And it slowly can make these decisions throughout. So we have this like really nice sort of like interpretable framework and we can, uh,


[User] Please transcribe this text

[Assistant] So essentially like robotics actually has something to bring to the table. This like experience in the environment is crucial for the language model to be able to successfully plan and command the robot to what to do. And a quick video of this like in action.


[User] Please transcribe this text

[Assistant] We, uh, type some command, uh, I think this is like, I spilled a Coke and bring me a replacement and something to clean up, though maybe the internet's cutting out too much.


[User] Please transcribe this text

[Assistant] And as I was saying, like, we'll see, uh, on the right side that it pops up with the actual, like, decision-making process so we can, you know, as it goes, understand what it's trying to do, why it's trying to do it. It's all, you know, in language so you can actually understand it rather than some, like,


[User] Please transcribe this text

[Assistant] Yeah, we essentially asked it to do two things. We asked it to both, like, find, like, throw away the Coke can and also bring me something to clean up and it's able to, like, put these both together to form this, like, long-horizon task and do a, do a task that takes, like, a couple minutes to roll out and is, like, truly long-horizon and sequential.


[User] Please transcribe this text

[Assistant] And so one issue that arises with this is I just like had it so that I enumerated these skills and I like scored over these skills. But if you go into like a world like this.


[User] Please transcribe this text

[Assistant] Uh, it's pretty computationally attractable for me to like write down everything I can do. There's like pairwise things, like I can stack something on something else.


[User] Please transcribe this text

[Assistant] And so Seikan doesn't really like scale to this like complexity, this like intractability of the real world. So how do we kind of go from this idea of how we ground, uh, in value functions to a more complicated world like this?


[User] Please transcribe this text

[Assistant] Essentially, we can, instead of picking specific scoring things, just like very specific primitives, we can instead just bias it at the token level. So a language model is always predicting probabilities over the next token.


[User] Please transcribe this text

[Assistant] We can also do this with a policy. We can have that policy conditioned on some arbitrary amount of natural language. So just like, put the, and then the, uh,


[User] Please transcribe this text

[Assistant] The next thing, the next object has to have a high probability, uh, in this scene, so we see that like, the Pepsi is pretty high, the cake is also there, I think these are like little cakes, uhm, as well as like certain letters are, are visible. And essentially we have this like token level, uh, grounding of what's possible and what's useful.


[User] Please transcribe this text

[Assistant] So this means that we can also include things like safety and preferences of the user and be able to decode in like a much more varied like combinatorially complex world.


[User] Please transcribe this text

[Assistant] And we can do things like, uh, pack a picnic box and ignore things like the knife that maybe the robot's not capable of doing because the safety function can sort of like biases away from it. And so it's just like, it's a very general way to sort of like ground the language model in the world and the capabilities of the robot without losing the expressibility of the language model.


[User] Please transcribe this text

[Assistant] Uh, we can also use it for navigation, so choosing like where to navigate and even the language model can trigger when it should be grounded and when not. So this is in a way like a tool use out of the language model. We see that, uh, given that we wanted, uh, some object,


[User] Please transcribe this text

[Assistant] You say, I'll go find, and then when it gets to the object, it doesn't know what's in the world, and so it can actually, like, trigger that I need to ground this next statement. And it can look, based on the world, in this case using CLIP, what have I seen in the past, and realize that, like, a Pepsi can was there, and an apple was there, and then make its decision based on that.


[User] Please transcribe this text

[Assistant] Uh, one thing that still goes wrong though is, let's say, uhm,


[User] Please transcribe this text

[Assistant] And we saw it in the, uh, in the initial demo, it failed at a task. But it didn't get any feedback that, hey, I failed at that task. It just failed that and then kept going on to the next one.


[User] Please transcribe this text

[Assistant] And in the wor- in the world we might, like, get this, like, you're trying to unlock a door and you realize you put the key in and it didn't work so I need to switch to a different one. Okay, I turned that and now it opened. A language model isn't necessarily getting this feedback in these earlier, uh, in these earlier setups.


[User] Please transcribe this text

[Assistant] So what we need is to have some like inner monologue thinking of, okay, I tried this, did that succeed, what do I see in the world? And there's a lot of sources of feedback that we can provide. We can use object recognition to tell the language model what's available.


[User] Please transcribe this text

[Assistant] Uh, we can use success detection to say what, uh, whether it succeeded at something and basically build this all


[User] Please transcribe this text

[Assistant] Into a real prompt. So we get some query, like bring me a drink from the table. We use say can to determine the next task.


[User] Please transcribe this text

[Assistant] But without knowing what's there it's very hard to sort of like blindly reason what possibly could be there. And instead of the object recognition you can just write this description into the prompt and then it's very clear, uh, you know, what should be done next.


[User] Please transcribe this text

[Assistant] The robot can also ask questions and basically in this intermodal it realizes like an underspecified problem, I should ask the user something. And then when it tries to do something like pick up the coke and it fails, it can even recognize that it failed and just try this again.


[User] Please transcribe this text

[Assistant] So what it looks like in the real world is that we have all these sources of feedback and we can just build them all into a prompt and let the language model essentially handle this monologue of what's going on in its head of, okay, I tried this, that failed, what should I do next, should I, uh,


[User] Please transcribe this text

[Assistant] I think in this case we have it, uh, try something and it fails because we, like, explicitly stopped it from working. So picking up a Coke can and then... But now the Coke can's not there anymore. So...


[User] Please transcribe this text

[Assistant] The bring me a soda maybe doesn't work so well. So maybe I should go back to the user and ask, hey, uh, you know, this didn't work.


[User] Please transcribe this text

[Assistant] I don't see a Coke can anymore, what would you like instead? So I see like an orange soda, and it says there's this, this, and we say no, that doesn't work for us. So it goes back to the planning stage and says, okay, now I need to find this, uh, lime soda that we asked for instead.


[User] Please transcribe this text

[Assistant] So this allows the language model to be much more grounded in the world by using these additional tools of, uh, a success detector of human feedback, uhm, of a scene descriptor. And in this case, it successfully delivers it, uh, to the person.


[User] Please transcribe this text

[Assistant] Another nice thing that we see is that it can even, it actually like terminates at the end. It says like, okay, I've accomplished a task. And the last thing that it does is, and this is like our U position, but it'll even like raise the thing saying like, okay, I've like completed the task.


[User] Please transcribe this text

[Assistant] But there are some limits of what language can do, uh, there's,


[User] Please transcribe this text

[Assistant] And so language isn't necessarily the best thing for this. Also this one where we have to wait until we see something to make another decision.


[User] Please transcribe this text

[Assistant] This, uh, feels a lot more like, uh, how kind of code works. And actually code is this like linguistic representation of actions. It's, it's, uh,


[User] Please transcribe this text

[Assistant] And most importantly, there is a, like, ton of data of what this is. Like, it's sort of the equivalent, uh, I mean,


[User] Please transcribe this text

[Assistant] Even I think like the best like GPT models now start code trained on code and then are fine tuned on like instruction following and language. So code is like this really nice structured and we know it runs otherwise it wouldn't be there so there's like a good signal that it's high quality. So it's actually like a really good fit and it's what we run on robots.


[User] Please transcribe this text

[Assistant] So, we should just use these language models to write code and essentially, uh, run that directly on the robot. What's kinda nice is we can both use APIs, so,


[User] Please transcribe this text

[Assistant] Maybe we have like, perception APIs that can like, detect where things are, uhm, we can use for loops out of this. And we can even use the language model to write new APIs. So if it generates something like, is empty or stack objects as a function,


[User] Please transcribe this text

[Assistant] It can realize actually these weren't already defined and then just generate the next, uh, the next thing. So this hierarchical code generation actually, uh, does extremely well to sort of like.


[User] Please transcribe this text

[Assistant] So, they've also looked at like every textbook that you've seen, so it can do, it can write a full controller for something as simple as a cart pull, uh, it can, uh,


[User] Please transcribe this text

[Assistant] Move up, move right, move back is something versus turn left, move forward. So this is the way that the robot moves and then the code is made, is able to immediately adapt to that to be able to do new things. So it's like really showing this power of uh, I guess what prompting can do and especially when you're prompting in a language that's directly accessible by the robot. And we put this on like, I guess,


[User] Please transcribe this text

[Assistant] Two, three robots here with different end effectors, and like, the best thing is like, you could...


[User] Please transcribe this text

[Assistant] I guess like, I really like these ones where you're like drawing shapes because I think this would be like an extremely complicated control problem but in code it's extremely trivial to just write how to do it.


[User] Please transcribe this text

[Assistant] And you can just put it on and, uh, like within a day have this running. Uh, and it can run like fairly robustly. You can also do very abstract reasoning like make this square bigger, move something, uh, to the side, or write these like while loops and if loops that, that wait for things to happen before it makes a decision. Uhm, so it's a very like, I guess, powerful concept.


[User] Please transcribe this text

[Assistant] But there's also, you know, some limitations of how well we can write things through code. There's a reason why sometimes we want these end-to-end policies, something like


[User] Please transcribe this text

[Assistant] So, uh, this is a paper called Robotics Transformer or RT1 and the idea was basically like a simple, uhm, transformer architecture with a few things to make it efficient enough to run quickly on a robot. We tokenized the whole input, the instruction, the image,


[User] Please transcribe this text

[Assistant] And we put it through a couple like efficient layers and then we go through a transformer and then we just direct the output, tokenized output.


[User] Please transcribe this text

[Assistant] Scenes like this where we're randomizing which objects are there and where the objects are. But it can also actually do pretty well, uh, with unseen objects, uhm, with unseen tasks, like adding in a lot more objects and like really cluttering up the space here and like progressively making it more cluttered.


[User] Please transcribe this text

[Assistant] It's just, uh, regular behavior cloning without a transformer. We tried GATO, uh, which has like some subtle differences and found that in general RT1 actually works pretty well and most excitingly starts to generalize across things. But I think the, the things that like, they were most excited about are its ability to ingest, uh, heterogeneous data.


[User] Please transcribe this text

[Assistant] We collected, again, a ton of episodes on this robot in this, uh, scene. And then...


[User] Please transcribe this text

[Assistant] We put in simulated data, uhm, which is very easy to collect and we put in data from a different robot, from a kooka that was collected for a completely different project and we just kinda like made the action space the same and then threw it in. And what you see is first the sim data


[User] Please transcribe this text

[Assistant] It went from like 10% success or 20% success to 85%, so like 65% more. Uhm, it's able to even like do new skills with those objects that it hasn't seen before in either one, so it sort of like can cross-domain transfer. And then adding in these new objects with a different robot also works pretty well, and we're able to suddenly pick those up.


[User] Please transcribe this text

[Assistant] So we're actually like starting to see this like scaling, uhm.


[User] Please transcribe this text

[Assistant] We do see that it like overfits pretty quickly on the environment that we have. I think we had like 97% there and like 80% in the next scene which is great for robotics but maybe not the level of performance that we hope to see. Uhm, but it can


[User] Please transcribe this text

[Assistant] We fixed essentially the data s- are we slowly lowered the data size and we lowered it by either uniformly taking out data across all tasks or


[User] Please transcribe this text

[Assistant] Cutting out like the lowest data performing tasks. So here we lose I think like 5% of data but across like the tasks that are underrepresented but like the more diverse tasks and we see the performance drops really quickly.


[User] Please transcribe this text

[Assistant] If we just pull out data kind of uniformly across the board and keep our diversity up, we actually see that like our performance doesn't go down, or at least not nearly as quickly. So really like diversity is the key to this. But diversity is sort of hard to get, especially for robotics, you need to have them in a lot of different scenes.


[User] Please transcribe this text

[Assistant] Uhm, you need to, yeah, like, and also like scaling up in the one scene. So we have this like classroom where we just collect most of our data.


[User] Please transcribe this text

[Assistant] But the performance, so one way we can vary this data collection is that we can imagine that we were in different scenes. So maybe we've collected some data here.


[User] Please transcribe this text

[Assistant] Let's use diffusion models to imagine something different. This is kinda, essentially what we've done here is we recognize which areas we've seen and then we vary everything else that isn't important to this task. So it's picking up the Coke can, but it's imagining that it's in a kitchen. It's imagining that it's in a living room, in an office.


[User] Please transcribe this text

[Assistant] And all this becomes useful data for the robot and essentially we've just used these massive diffusion models to be able to inject new imagined experience for the robot and give it this ability to


[User] Please transcribe this text

[Assistant] And then we throw that in as though that was real robot experience. And we see, after this page, after this one I think I have like actual numbers, but another sort of like trippy video of,


[User] Please transcribe this text

[Assistant] Policy that is, oh yeaah, and like we can act as though the whole time it picked up like a cloth or things that are deformable, uhm, and we see that we actually like...


[User] Please transcribe this text

[Assistant] Are able to do this at a higher rate. I think there's a lot of numbers here, but the main thing is, when we use Rosie on all the bolded numbers are when we added in this Imagine experience. It's sort of like across the board, it helps and it doesn't really hurt anything. So it's sort of getting like this free experience just from the foundation model.


[User] Please transcribe this text

[Assistant] So I think what we really want to do is at like the lowest levels use the knowledge contained in foundation models for robotics. So the next thing we try to do is actually like train our own. And we called it POM-E which starts with POM which is this


[User] Please transcribe this text

[Assistant] We also train the VIT to output essentially like text tokens or some like co-embedding space between the two.


[User] Please transcribe this text

[Assistant] It can be images, it can be anything else, but you can see essentially the language model is just reading in these tokens as though they're all text tokens.


[User] Please transcribe this text

[Assistant] And these other smaller models are outputting things that then go in as vision tokens or, uh, object tokens or things like that. And we train the whole thing end to end so that it learns to essentially like recognize the text tokens and the image tokens as though they're all the same thing.


[User] Please transcribe this text

[Assistant] And not only were we able to do sort of VQA tasks, like, and kind of complicated ones, like giving this image, uh, what's in there and answering emojis, uhm, or


[User] Please transcribe this text

[Assistant] One of the most exciting things though is that we really see positive transfer here. So, if we train on only, like, on the same model, but only with this data, and only this data, and only this data,


[User] Please transcribe this text

[Assistant] There's reasons to throw everything sort of like into one pile and it's, and even though these are like pretty different visually, especially the, uh,


[User] Please transcribe this text

[Assistant] Like, between this, like, language table and this, and the SACAN data, they're extremely, like, diverse visuals, the action space is different, and it's still able to actually, like, cross-domain transfer.


[User] Please transcribe this text

[Assistant] So we also see that as we get to larger and larger models, the performance doesn't go down as much when we add in this vision. So essentially these are just, uh, language only tasks.


[User] Please transcribe this text

[Assistant] And with an 8 billion parameter palm model, this is what it started with for performance, and when we added in this like training, it dropped like massively, like it basically is a terrible language model at this point.


[User] Please transcribe this text

[Assistant] So we see that like, if you get to a really large model, it's able to ingest all this information without actually forgetting or like having any catastrophic forgetting of its previous knowledge. So this is like, extremely promising that maybe we don't lose too much by, by doing this.


[User] Please transcribe this text

[Assistant] Some examples that I think we were pretty excited about or just thought were kind of fun of visual, uh, reasoning are here. So it's able to do things like read off here, uh,


[User] Please transcribe this text

[Assistant] The price of these items and then do math based on that to tell you how much it would cost to, uhm, to get two custom pizzas. Uhm, we're able to, I think like this one is particularly complex reasoning because it's a mix of like words.


[User] Please transcribe this text

[Assistant] And rules and so it's saying you can't enter unless it's a bike but it has to understand that like it's saying accept but then there's also a picture of a bike and emergency vehicles and it's able to reason through this to recognize that like on a bike it's okay to go through.


[User] Please transcribe this text

[Assistant] We can also use multiple images so we can compare, I think, and these are like, definitely not in the training set, this is like, one of us went to, I think, like, Marine later and took some pictures and said, what's the difference between these two images, uhm, what matches, and it's able to like,


[User] Please transcribe this text

[Assistant] Put this complex reasoning to work even over these like images that it's never seen before. Uh, which is like very exciting, especially for robotics tasks.


[User] Please transcribe this text

[Assistant] Use it to, on the same model, do these tasks on the right.


[User] Please transcribe this text

[Assistant] And, and neither of these is it putting, it's not putting out actions, but it's commanding a low level policy that's putting out actions. But it's able to handle adversarial disturbances, like here we like block it from picking something up.


[User] Please transcribe this text

[Assistant] We feel like this is a real step towards doing this, uh, kind of multi-embodiment brain that the, this foundation model for robotics. So,


[User] Please transcribe this text

[Assistant] This is kind of my last slide, but uh, what's next is we want to go beyond language. We want to continue pushing on this, like, visual language, uh, domain.


[User] Please transcribe this text

[Assistant] But we also recognize that there's a lot of issues with applying foundation models to robotics. They're really like not, uhm,


[User] Please transcribe this text

[Assistant] Perfectly mated for each other, uhm, which I think as like a robotics researcher is kind of exciting that there's a lot of areas to still figure out.


[User] Please transcribe this text

[Assistant] Uh, but in the areas that we can leverage them, we should sort of leverage them as best we can and do so in like, in a way that makes sense for robotics. So things like grounding them in the scene, uh,


[User] Please transcribe this text

[Assistant] While language models continue to increase in performance and vision language models, the interaction side of robotics is still, uh, definitely like the bottleneck of the overall system. Uhm, but we do think that like we can kind of move


[User] Please transcribe this text

[Assistant] There, there's a lot of really positive signs that like in embodied foundation models possible, this positive transfer across things, this uhm,


[User] Please transcribe this text

[Assistant] This ability to ingest heterogeneous data between different robots from SIM, uhm, is extremely promising and the fact that we don't get catastrophic forgetting as we train these models with more robotics data means that we can kind of leverage everything that was there but still add in our own data.


[User] Please transcribe this text

[Assistant] So I think there's a lot of people who worked on this, uhm, most of which are pictured here, but not all, uhm,


[User] Please transcribe this text

[Assistant] But yeaah, I think that's, that's it. So, I think I, I can obviously take questions and then we also have the robot running in the back end if there's like anything we want to try on that, but.


[User] Please transcribe this text

[Assistant] Our policies are trained to, uh, like terminate or have failure if they like hit something like the table too hard or things that mostly like could cause any sort of damage. So those are sort of like both baked into the robotics stack and then also trained into the policy that that's like


[User] Please transcribe this text

[Assistant] That that would be a negative outcome. Uhm, so there weren't anything that was like...


[User] Please transcribe this text

[Assistant] But I think as like language models continue to get better, like it probably if we went back and like re-benchmark this with like, this was, we probably ran this a year ago.


[User] Please transcribe this text

[Assistant] Our teleoperation is pretty basic and really, I mean I've tried to like pick things up with it and it's like hard to do. Uhm, so I think things like adding in haptic feedback would be incredibly impactful. I think


[User] Please transcribe this text

[Assistant] I mean, we see things like, uh, the, like, recent, like, Aloha paper, uh, where it's like, uh, just a much better controller, and it seems like dexterous manipulation is much easier. So I think this is like a huge area that there's...


[User] Please transcribe this text

[Assistant] Yeah, I mean I think there's a lot of tasks that like not having depth in there is...


[User] Please transcribe this text

[Assistant] And not having torque sensing is probably like a little bit limiting. I think the fact that I guess the way we think about it is we're trying to push as far as we can in the simplest domain to see how, what level of performance we can get. If we were like a product trying to get to the best one, we might be, we might add these in. But the fact that we can get to 97% on all these tasks, this is like,


[User] Please transcribe this text

[Assistant] You can get pretty far with images and since they're available,


[User] Please transcribe this text

[Assistant] The kind of across the board, we want to take like the simplest approach, but I do think probably closing that last 3%.


[User] Please transcribe this text

[Assistant] Yeah, I mean I think there's a lot of areas around like distilling policies that seem to be relatively promising. Uhm, I also think like the fact that these are hosted in the cloud and there's somewhat minimal latency like the


[User] Please transcribe this text

[Assistant] And then output the next action that the person took. So uhm, if they have like some trajectory, we're chunking this up into single time steps where we just predict the very next action to take, given those like last six images.


[User] Please transcribe this text

[Assistant] Yeah, so to get the data, we essentially, like, someone takes a, uh, Oculus controller and moves as though they're going to pick it up. They can also do like a handheld controller, which moves the robot hand to then like grasp the object. We then take all that demonstration data and turn it into single time steps where we take an image and we output an action.


[User] Please transcribe this text

[Assistant] Yeah, so, uh, the level controller that Pommy uses is actually actually RT1, so it's like relatively reliable as long as it's in like the scenes that it's been in. But we do...


[User] Please transcribe this text

[Assistant] Going forward though, we're definitely pushing in this direction of what levels can we train into these models and still get pretty high performance. And I think it's somewhat of an open question of


[User] Please transcribe this text

[Assistant] Where, where do you get better performance? Maybe the language model or the vision language model does much better generalization. So potentially it's, it's, you need to find like the sweet spot and trade-offs between performance and generalization across the two. So I think like the, the small local model is probably higher performance and the other one has higher generalization, uhm, is like the answer.


[User] Please transcribe this text

[Assistant] Yeah, so, I guess, initially, about a year ago when we did this, it was, like, 50-50 across whether it was, uh, the high level making a mistake or the low level making a mistake. Now, I think if you do, like, palm-y or something like this, it's much more on the low level making mistakes. Like, it's probably like,


[User] Please transcribe this text

[Assistant] 80-20 now, uhm, and uh, yeaah, the performance of foundation models improving has certainly put sort of robotics in the...


[User] Please transcribe this text

[Assistant] So, I mean, it does have like this list of objects that are, uh, coded in for it to look for.


[User] Please transcribe this text

[Assistant] Just stay here for a moment. I'm going up to the top floor. Don't call anyone. I'll be back soon.


[User] Please transcribe this text

[Assistant] I waited, cold and tired in his room for what seemed a very long time.


[User] Please transcribe this text

[Assistant] Then I saw the light of his candle approaching through the darkness, and he appeared, looking pale and depressed.


[User] Please transcribe this text

[Assistant] Did you see anything when you opened your bedroom door? He asked, glancing sharply at me. No sir, only a candle on the floor. But you heard a strange laugh, did you say?


[User] Please transcribe this text

[Assistant] Yes, I've heard it before. Grace Poole laughs like that. That's it. It must have been Grace Poole. You guessed it.


[User] Please transcribe this text

[Assistant] I shall consider what to do about it. But meanwhile, I'm glad you're the only person who knows anything about all this. Say nothing to anybody else, and now go back to your own room. Good night then, sir, I said, moving towards the door. What? Are you leaving me already? He said, seeming surprised, although he had just told me to go, and so coldly. You said I should go, sir.


[User] Please transcribe this text

[Assistant] Young men! I understand your intentions.


[User] Please transcribe this text

[Assistant] I feel like this squirtle. That's how you feel. Can you make that expression?


[User] Please transcribe this text

[Assistant] And now the first ones emerged 25 years ago as places to experiment with new educational approaches, and since then they've exploded. There are now over 6,700 charter schools educating almost 3 million students, and some have celebrity backers, like Puff Daddy, Andre Agassi, and even Pitbull, who helped launch Miami's Slam Academy.


[User] Please transcribe this text

[Assistant] And look, when Pitbull has a charter school, it seems like it might be worth taking a look at them. And first, let me acknowledge, this is a controversial area. Charter proponents will point to positive news stories like this one about the Kip Charter Schools Network.


[User] Please transcribe this text

[Assistant] Now honestly, any philosophy that can get those kind of results might be worth considering. In the same way that if we found out they boosted our immunity, we'd seriously consider eating koalas.


[User] Please transcribe this text

[Assistant] But critics, critics argue charters overstate their successes, siphon off talented students, and divert precious resources within a school district. Now for this piece, and I know this is going to make some people on both sides very angry, we're gonna set aside


[User] Please transcribe this text

[Assistant] Whether or not charter schools are a good idea in principle, because whether they are or not, in 42 states and DC, we're doing them.


[User] Please transcribe this text

[Assistant] So instead, we're gonna look at how they operate in practice. One group found on average charters had a slight edge over traditional public schools in reading and did about the same in math, but acknowledged charter quality is uneven across the states and across schools. And that is putting it mildly.


[User] Please transcribe this text

[Assistant] Because around the country, there have been charter schools so flawed, they don't make it through the school year.


[User] Please transcribe this text

[Assistant] And charters in some states can have an alarming failure rate. Two years ago, a Florida paper found that since 2008, 119 charter schools had closed there, 14 of which had never even finished their first school year. So 14 schools in Florida were outlasted by NBC's Mysteries of Laura.


[User] Please transcribe this text

[Assistant] So how did those schools get approved? Well, Florida's charter process begins with a lengthy application, and Ivy Academy's was 400 pages long.


[User] Please transcribe this text

[Assistant] They get paid on a per-student basis. On average, that's about $7,000 for every enrollment. And that adds up.


[User] Please transcribe this text

[Assistant] And I'm not even sure Pennsylvania deserves to be called the worst because Ohio's charter law was for decades so lax, even charter advocates have called it the Wild West. The state has around 360 charters and their governor, John Kasich, speaks often about how much he loves choice and competition in schools.


[User] Please transcribe this text

[Assistant] But Ohio's charters have had huge problems with lack of oversight. A review of one year's state audits found charters misspent public money nearly four times more often than any other form of taxpayer-funded agency. And some cases are incredible.


[User] Please transcribe this text

[Assistant] Like that of Lisa Hamm, a school superintendent who was accused of spending money for her school on spas, jewelry, luggage, plays, veterinary care, and trips to Europe and to see Oprah.


[User] Please transcribe this text

[Assistant] There are ways to profit off of charter schools perfectly legally in Ohio, and there have been for years. Look at this episode of Frontline from 2000.


[User] Please transcribe this text

[Assistant] That man's company, White Hat Management, worked under contracts where each charter would pay 95% or more of its government funding to White Hats, which, as a private company, isn't obligated to provide the same level of transparency as, say, a school district.


[User] Please transcribe this text

[Assistant] So taxpayers could have little idea how that money was being spent. And who can say if that's a good system or not? All I know is Whitehat ran 32 of the lowest performing schools in the state. And if you do essentially the same terrible thing more than 30 times in a row,


[User] Please transcribe this text

[Assistant] Because they are approved and overseen by what are called authorizers. And while some states sharply limit who can be an authorizer, Ohio allowed many different groups, including non-profits, to do it. Meaning, well let's say I wanted to open the John Oliver Academy for Nervous Boys.


[User] Please transcribe this text

[Assistant] Harris denies a conflict of interest because she claims she wasn't directly involved in decision making. And maybe, maybe the schools just chose Kids Count because it had a proven track record of great oversight. So let's, let's just check in on one of the other schools they oversaw.


[User] Please transcribe this text

[Assistant] And look, some kids might need online education, but it has got to be monitored better because one major study found compared to kids in traditional public schools, students in online charters lost the equivalent of 72 days of learning in reading and 180 days in math during the course of a 180 day school year.


[User] Please transcribe this text

[Assistant] We'll tell you that even they are concerned about online schools, and they'll argue some states have much better oversight than the ones that we've seen, and that is true. Though for the record, some may even be worse. One charter researcher told Ohio, be very glad that you have Nevada, so you are not the worst.


[User] Please transcribe this text

[Assistant] Exactly. The problem with letting the free market decide when it comes to kids is that kids change faster than the market. And by the time it's obvious a school is failing, futures may have been ruined. So if we are going to treat charter schools like pizza shops,


[User] Please transcribe this text

[Assistant] I had hoped to hide my face behind my slate while Mr. Brocklehurst was talking, so that he would not recognize me. But suddenly the slate fell from my hand and broke in two on the hard floor. I knew only too well what would happen next. A careless girl, said Mr. Brocklehurst quietly, almost to himself. The new girl, I see.


[User] Please transcribe this text

[Assistant] I must not forget to say something to the whole school about her. And then to me aloud. Come here, child. I was too frightened to move, but two big girls pushed me towards him. Miss Temple whispered kindly in my ear. Don't be afraid, Jane. I saw it was an accident. Her kindness touched me, but I knew that soon she would hear the lies about me, and then she would hate me.


[User] Please transcribe this text

[Assistant] Put the child on that chair, said Mr. Brocklehurst.


[User] Please transcribe this text

[Assistant] Someone lifted me up onto a highchair so that I was close to his nose. Frightened and shaking, I felt everyone's eyes on me.


[User] Please transcribe this text

[Assistant] I did so. I'm happy, Jane, she continued. You mustn't cry. By dying young, I'll avoid suffering. I'm going to heaven. Does heaven really exist? I asked. Yes, I'm sure of it. I'm sure our souls go there when we die. She answered firmly. Will I see you again, Helen, when I die? Yes.


[User] Please transcribe this text

[Assistant] You will go to heaven too, Jane. I could not quite believe that heaven existed, and I held tightly to Helen. I did not want to let her go. We kissed goodnight and fell asleep. In the morning, Miss Temple found me asleep with Helen Burns dead in my arms. She was buried in the local churchyard.


[User] Please transcribe this text

[Assistant] Chapter 11 The Thornfield House Party


[User] Please transcribe this text

[Assistant] Two disappointing weeks passed before we heard from Mr. Rochester again.


[User] Please transcribe this text

[Assistant] During this time, I tried hard to forget my feeling for him. I reminded myself that he paid me to teach Adele nothing more and that no other relationship could exist between us. When his letter finally came, Mrs Fairfax announced with great excitement that he was planning a house party at Thornfield. He was going to return in three days' time


[User] Please transcribe this text

[Assistant] And it invited a large number of ladies and gentlemen to stay for several days. We all worked extremely hard in the next few days, cleaning all the rooms and preparing the food.


[User] Please transcribe this text

[Assistant] The only person in the house who did not appear excited was Grace Poole, who stayed in her room upstairs, coming down once a day for food and drink. None of the servants seemed at all curious about her, but I once heard two of the maids talking, and I listened when I caught her name.


[User] Please transcribe this text

[Assistant] Be sure to like the video and leave a comment on what you think of it.


[User] Please transcribe this text

[Assistant] Also be sure to subscribe to my channel and ring the bell to keep up with further audiobook readings.


[User] Please transcribe this text

[Assistant] It would be a big wall, roughly 20 feet high by 20 feet long.


[User] Please transcribe this text

[Assistant] I remember looking at that gaping hole in excruciating disbelief. I was utterly certain that there would never be a wall there ever again.


[User] Please transcribe this text

[Assistant] My brother and I would go to my father's shop after school to work on that wall.


[User] Please transcribe this text

[Assistant] We did everything ourselves. We dug the footing, mixed the mortar, and carried the buckets. I still remember the formula.


[User] Please transcribe this text

[Assistant] Two parts cement, one part sand, one part lime. Harry was in charge of the hose.


[User] Please transcribe this text

[Assistant] We'd mix the pile with shovels out on the sidewalk, and then fill two gallon buckets and lay our separate bricks.


[User] Please transcribe this text

[Assistant] We did it without any rebar or wood forms. Just one of those levels with the water bubble in the middle.


[User] Please transcribe this text

[Assistant] If you know anything about construction, you know this is a loony way to do this.


[User] Please transcribe this text

[Assistant] If we can keep it real, this is chain gang kind of labor. Today we would just call child protective services. This is a job so tedious and unnecessarily long that what ended up taking two kids most of a year would have only taken a team of grown men a couple of days at most. My brother and I worked weekends, holidays, vacations. We worked through the summer that year. It didn't matter.


[User] Please transcribe this text

[Assistant] My father never took a day off, so neither could we.


[User] Please transcribe this text

[Assistant] There were so many times I remember looking at that hole, totally discouraged. I couldn't see how this was ever going to end.


[User] Please transcribe this text

[Assistant] The dimensions became unfathomably large in my mind.


[User] Please transcribe this text

[Assistant] I was certain that I would grow old and die still mixing concrete and carrying those buckets. I just knew it. But Daddio wouldn't let us stop.


[User] Please transcribe this text

[Assistant] Every day we had to be there, mixing concrete.


[User] Please transcribe this text

[Assistant] Carrying buckets, laying bricks. It didn't matter if it was raining, if it was hot as, ehm, if I was mad, if I was sad, if I was sick.


[User] Please transcribe this text

[Assistant] If I had a test the next day, there were no excuses.


[User] Please transcribe this text

[Assistant] My brother and I tried to complain and protest, but it made no difference to Daddy-O. We were trapped.


[User] Please transcribe this text

[Assistant] One day, Harry and I were in a particularly stank mood. We were dragging our feet and grumbling, impossible this, and ridiculous that. Why'd we have to build a wall for anyway? This is impossible. It's never gonna get done.


[User] Please transcribe this text

[Assistant] Daddy-o overheard us, threw down his tools, and marched over to where we were yappin'.


[User] Please transcribe this text

[Assistant] Stop thinking about the damn wall, he said. There is no wall. There are only bricks. Your job is to lay this brick perfectly.


[User] Please transcribe this text

[Assistant] Then move on to the next brick. Then lay that brick perfectly. Then the next one.


[User] Please transcribe this text

[Assistant] Don't be worrying about no wall. Your only concern is one brick. He walked back into the shop.


[User] Please transcribe this text

[Assistant] Harry and I looked at each other, shook our heads, this dude's a kook, and went back to mixin'.


[User] Please transcribe this text

[Assistant] Some of the most impactful lessons I've ever received, I've had to learn in spite of myself. I resisted them. I denied them. But ultimately the weight of their truth became unavoidable.


[User] Please transcribe this text

[Assistant] My father's brick wall was one of those lessons.


[User] Please transcribe this text

[Assistant] The days dragged on and as much as I hated to admit it, I started to see what he was talking about. When I focused on the wall, the job felt impossible, never ending. But when I focused on one brick, everything got easy. I knew I could lay one damn brick well.


[User] Please transcribe this text

[Assistant] As the weeks passed, the bricks mounted and the hole got just a little bit smaller. I started to see that the difference between a task that feels impossible and a task that feels doable is merely a matter of perspective. Are you paying attention to the wall? Or are you paying attention to the brick? Whether it was ace in the test to get accepted into college,


[User] Please transcribe this text

[Assistant] Hitting it big as one of the first global hip hop artists, or constructing one of the most successful careers in Hollywood history, in all cases,


[User] Please transcribe this text

[Assistant] What appear to be impossibly large goals could be broken down into individually manageable tasks.


[User] Please transcribe this text

[Assistant] Insurmountable walls comprised of a series of conceivably layable bricks.


[User] Please transcribe this text

[Assistant] For my entire career, I have been absolutely relentless. I've been committed to a work ethic of uncompromising intensity, and the secret to my success is as boring as it is unsurprising. You show up, and you lay another brick. Pissed off? Lay another brick. Bad opening weekend? Lay another brick.


[User] Please transcribe this text

[Assistant] I have dealt with failure, loss, humiliation, divorce, and death.


[User] Please transcribe this text

[Assistant] I've had my life threatened, my money taken away, my privacy invaded, my family disintegrated, and every single day, still, I got up, mixed concrete, and laid another brick.


[User] Please transcribe this text

[Assistant] The only question is, are you going to get up and lay it?


[User] Please transcribe this text

[Assistant] I've heard people say that a child's personality is influenced by the meaning of their name. Well, my father gave me my name. He gave me his name. And he gave me my greatest advantage in life. My ability to weather adversity. He gave me will.


[User] Please transcribe this text

[Assistant] It was a cold, overcast day, nearly a year after my brother and I had begun.


[User] Please transcribe this text

[Assistant] By that time, the wall had become such a fixture in my life that thoughts of finishing it felt like delusions. Like, if we ever did finish...


[User] Please transcribe this text

[Assistant] But on that frigid September morning, we mixed the final pie dough, mixed the final bucket, and laid the final brick.


[User] Please transcribe this text

[Assistant] Daddio had been standing there watching the last few bricks being set into place. Cigarette in hand.


[User] Please transcribe this text

[Assistant] He stood quietly admiring our work.


[User] Please transcribe this text

[Assistant] Harry and I set and leveled the final brick. Then silence. Harry kinda shrugged.


[User] Please transcribe this text

[Assistant] What now? Do we jump? Do we cheer? Do we celebrate?


[User] Please transcribe this text

[Assistant] We gingerly stepped back and stood on each side of Daddio. The three of us surveyed our family's new wall.


[User] Please transcribe this text

[Assistant] Daddio plucked his cigarette to the ground, twisted his boot to put it out,


[User] Please transcribe this text

[Assistant] Exhaled the final drag of smoke and, never taking his eyes off the wall, he said,


[User] Please transcribe this text

[Assistant] Now don't y'all ever tell me there's something you can't do. Then he walked into the shop and got back to work.


[User] Please transcribe this text

[Assistant] Okay, we also got a bunch of other visitors. We got a Lamborghini Huracan 610. We got a Lamborghini Aventador. Uhm.


[User] Please transcribe this text

[Assistant] Alright, it looked like Yvonne took that one bro. Crazy.


[User] Please transcribe this text

[Assistant] I don't remember that. I'm cool. As in with the school uniform.


[User] Please transcribe this text

[Assistant] I think that the, the more that I haven't been to the gym, just being at home, I think I've, I think I've realised that.


[User] Please transcribe this text

[Assistant] Yeah. Yeah. So when I'm not training, it's like, Oh, hang on. Like, like my abs have gone down. It's it's and it's like, yeah. I mean, it sounds so dumb, right? You still get abs, but it's like, yeah, it's just like that dumb stuff.


[User] Please transcribe this text

[Assistant] Uh, new subscriptions, like I, what's, what's the pushback against ABC? Like what's why there's no subscription there. Yeah. So that's what I mean. So why, why, why would you sign up someone? I haven't looked at ABC.


[User] Please transcribe this text

[Assistant] Okay. Like in terms of like resources, I think it'd be very similar.


[User] Please transcribe this text

[Assistant] Yeah. I mean, that's what Brave, the browser is there for. I mean, so there's, it's, it's like a fork of Google Chrome. It's a fork. It's like they've taken the code base and then just edit it and modified it to however they want.


[User] Please transcribe this text

[Assistant] Yeah. And so anyway, so they've, I think they've got brave rewards, which is essentially like micro transactions. So every time you visit a publisher's website and they're set up with brave, you're giving, it's based on like cryptocurrencies and stuff, but you're giving a micro transaction to them.


[User] Please transcribe this text

[Assistant] And so they're earning, they're earning money through traffic and through views directly to them.


[User] Please transcribe this text

[Assistant] Maybe, maybe not, but definitely the worst build to a Survivor series of all time. I'm Oli Davis, I'm joined by Luke Owen, D.A.D. This is the Raw Review Podcast, which today is brought to you by Our Fantastic Skin.


[User] Please transcribe this text

[Assistant] We'll dive into that box after the Ultra Chats. Right, this episode of Raw, much like the last six weeks of Raw, there's nothing to talk about.


[User] Please transcribe this text

[Assistant] No, actually, no, no, no. Post-Draft Raw, there's nothing to talk about. Because pre-Draft Raw, they pooped the bed. And they did everything. Big E cashing in. Roman Reigns turned up. It looked like we were getting a Bloodline vs New Day feud. And then, the draft happened.


[User] Please transcribe this text

[Assistant] Yeah, you've got a lot of time there. Five hours, you could argue, in the week of main roster WWE time.


[User] Please transcribe this text

[Assistant] Uh, but what they instead decided to do is announce...


[User] Please transcribe this text

[Assistant] The teams on social media two Sundays ago, randomly, over the space of like 20 minutes,


[User] Please transcribe this text

[Assistant] Which was the first hint, or like, reference to anything about an elimination match. And then, ever since then, on both


[User] Please transcribe this text

[Assistant] Blah blah blah, stuff happened on the show, the main event, which I expect to do no viewership whatsoever.


[User] Please transcribe this text

[Assistant] But this week, and I love Rey Mysterio, I really love Bobby Lashley, but putting them in the main event here? I was just like...


[User] Please transcribe this text

[Assistant] Who's this? Who's tuning into this? And why is this the go-home? People who loved 2005.


[User] Please transcribe this text

[Assistant] This is the go-home segment for the pay-per-view on Sunday and what happened, Rey is like, Piers, why did you book my boy in a match against Bobby? You know his history with sons. He's gonna murder that guy. And


[User] Please transcribe this text

[Assistant] Piers is like, well if you don't like that, you also have a match with Bobby tonight. And they had a match, and Ray was a bit off his game actually, a few botches from Ray. But overall I enjoyed the match, Bobby is freakishly impressive. When he


[User] Please transcribe this text

[Assistant] When he got splashed on the ground, laying down, and stood up with Rey Mysterio. That's cool. I thought it was a very fun match and I thought Rey's comeback at the end.


[User] Please transcribe this text

[Assistant] I'll announce it later. Hey, you random guy in the ring!


[User] Please transcribe this text

[Assistant] I like the cut of your jib, sir. You're in Team Raw!


[User] Please transcribe this text

[Assistant] Create the best spectacle so he wants the best people in the matches but why if that was the case?


[User] Please transcribe this text

[Assistant] Which surely is the only explanation for this.


[User] Please transcribe this text

[Assistant] You would want the biggest stars, and definitely the biggest star is Rey Mysterio.


[User] Please transcribe this text

[Assistant] So, yeaah, what, what possi- Okay, so, the whole thing about Piers Sane, only want WWE champions on the team.


[User] Please transcribe this text

[Assistant] That mate, and then like, but he does the same thing to Rey, who is a WWE Champion. Austin Theory isn't a WWE Champion.


[User] Please transcribe this text

[Assistant] That seems like it contradicts itself. However, I can make an allowance for that because Pierce is


[User] Please transcribe this text

[Assistant] Okay, I get that. But then why, but why does he have beef?


[User] Please transcribe this text

[Assistant] And why would you choose Austin Theory to go on the team in a kayfabe reason?


[User] Please transcribe this text

[Assistant] Because von Wagner is now Pierce's muscle from SmackDown.


[User] Please transcribe this text

[Assistant] I like Adam Pearce's work pre-WWE. I remember really enjoying the NWA best of seven series with Colt Cabana. That was my first real proper exposure to the NWA concept in the modern day. And, you know, he's had like


[User] Please transcribe this text

[Assistant] He just does stuff on screen now. What he is given, he performs adequately, right?


[User] Please transcribe this text

[Assistant] I've never seen him stand out in this role and that's not a criticism of him as a performer, it's the role he's been given. Now, he is...


[User] Please transcribe this text

[Assistant] Actively annoying me as a character. Not in a good kayfabe heat kind of way, but in a, I really have no idea what your purpose is.


[User] Please transcribe this text

[Assistant] You told Adam Pearce, hey, go crazy authority figure.


[User] Please transcribe this text

[Assistant] It is literally my- this is an actual quote. This is the actual quote that he walks out to talk about the Mysterio situation.


[User] Please transcribe this text

[Assistant] It's literally my job to make sure we have the best possible team for Survivor series. It's just, it's so flat. And this whole like, the authority,


[User] Please transcribe this text

[Assistant] Thing, you know, Stephanie and Triple H played up that kind of we are the corporate people.


[User] Please transcribe this text

[Assistant] We talk flatly. We are business, business, business. And that worked, quite frankly. This, this is just an imitation of that. It's a stock photo.


[User] Please transcribe this text

[Assistant] Elsewhere, it's just champion vs champion by the way.


[User] Please transcribe this text

[Assistant] Was that the plan all along? Working yourself into a shoot, brother.


[User] Please transcribe this text

[Assistant] Brother work. Uh, any other bits you want to say or shall we see what the SWAFT nation has to say?


[User] Please transcribe this text

[Assistant] I don't know what to say, because it's so confusing. I guess it's just bad.


[User] Please transcribe this text

[Assistant] Greg Murphy says, now that pay-per-views aren't even a main source of revenue, network subscriptions, ratings, you'd wonder why WWE don't halve the amount of pay-per-views. Do six full builds rather than 12 half builds. The six pay-per-views you'd keep are pretty obvious too. Like any attraction, less is more. Ah, difference is there. And I agree with you, Greg.


[User] Please transcribe this text

[Assistant] You know. Oh no, the pay-per-views do make money. It was the house shows that lose money.


[User] Please transcribe this text

[Assistant] They wouldn't, the thing is, they wouldn't do it right. I agree with you. I agree with you. Have one of those, just make every other month a special episode of Raw. It's a mini pay-per-view, right? I mean, it's three hours long, it's long enough. But we've been through the six pay-per-views a year.


[User] Please transcribe this text

[Assistant] Relatively recently with the draft split. So you get six raw pay-per-views and they're alternate with Smackdown pay-per-views. They can't tell stories properly. They're so conditioned in the four week cycle of climaxes, they don't even do that right.


[User] Please transcribe this text

[Assistant] I don't think that'll fix it, unfortunately. Jack Nichols, how can WWE follow full gear? Welcome everyone to Mundane Night Raw. He swapped the word Monday.


[User] Please transcribe this text

[Assistant] That's pretty good. That's good. You're right, Raw is better when it's bad. When it's bearable, it's boring bollocks at best. Boo WWE, boo. Remember Story? Uh, lead Survivor Series matches?


[User] Please transcribe this text

[Assistant] So you're telling me, they didn't have Team Blue invade and have a locker room brawl with Team Red to build Survivor Series? Were people even wearing Team Raw shirts? Is it even actually Survivor Series then?


[User] Please transcribe this text

[Assistant] You want to put yourself over everybody else, really.


[User] Please transcribe this text

[Assistant] I love that because it doesn't make any sense and it gave me this screenshot of Triple H.


[User] Please transcribe this text

[Assistant] You know, at this point, not doing Survivor series from 22, so next year, would be a Halouva budget cut. I'm not a fan of fans cheering randomly, but a tiny part wanted some guy to get bored enough to get a live cowboy S-word chant going, unless it happened and I missed it.


[User] Please transcribe this text

[Assistant] I bet like Monday after Raw, you know, Monday after WrestleMania, those sorts of big shows. Royal Rumble crowd maybe if it's particularly bad.


[User] Please transcribe this text

[Assistant] Survivor Series is a Chad Gable push. It's futile to ever think it's going to properly


[User] Please transcribe this text

[Assistant] What's gonna happen here? You've got a thing under the table. You're getting it out. It's a camp-


[User] Please transcribe this text

[Assistant] For geology to be the debut sponsor camp.


[User] Please transcribe this text

[Assistant] It feels like, uh, like a Vox pop suddenly happened. We're gonna cut to our man on the street.


[User] Please transcribe this text

[Assistant] Okay, so this episode, everybody get your Ultra Chats in to WrestleTalk.com forward slash support or read everyone out over $5 by the end of the episode.


[User] Please transcribe this text

[Assistant] So, this show opened with, okay, I'm just gonna, I'm just gonna lay this out, what happened, and then we can dive into it.


[User] Please transcribe this text

[Assistant] Big E came out to cut a promo on Roman Reigns, then he also called out Kevin Owens. Kevin Owens came out, cut a decent promo, we'll go into that more in a sec. Then the Usos jumped Big E, Kevin Owens walked off, Matt Riddle made the save on Big E, then Sonya Deville booked a tag match between them that she said, we don't do this very often.


[User] Please transcribe this text

[Assistant] Seth Rollins suddenly makes his entrance immediately after Sonya makes that announcement. Because he's coming out for commentary. Because he's coming out for commentary. But this was a, it was an impromptu match, so he was just waiting there.


[User] Please transcribe this text

[Assistant] Now it's a six-man tag with the Team Raw captain Seth Rollins tagging with Smackdown team The Usos.


[User] Please transcribe this text

[Assistant] It was kind of good because you had Orton and Riddle going against the Usos, which is the match on the weekend, and you get to build Big E vs Seth as well, which is definitely a direction post.


[User] Please transcribe this text

[Assistant] Uh, Survivor series. Kevin Owens is lurking in the background. I thought Kevin Owens' promo was really good.


[User] Please transcribe this text

[Assistant] I'm not a heel. I'm a good guy. But you're all booing me. I can't do anything. If you want to boo me, boo me. If you want to cheer me, cheer me. I'm gonna do my own thing. I thought that was a really nice approach to being a heel. Ultimately though, this was an absolutely nonsensical segment and it made me feel


[User] Please transcribe this text

[Assistant] Yeah, and later on, Randy is shouting at Riddle backstage. That was well done. He said, I don't care about anybody else. I care about our team and our titles. And Riddle went,


[User] Please transcribe this text

[Assistant] To turn more babyface? Well, before he then turns mega-heel. Yeah, yeah, yeah. So like, he'll run down on his own accord. Maybe riddles out one week.


[User] Please transcribe this text

[Assistant] The whole show actually had a lot of structural issues for me, where people would make their entrance, and they'd be standing there in the ring, I know, I saw them come out, they'll go to break, they'll run a promo segment, they'll have a backstage interview, they'll do a recap package. Ten minutes later, you get to the ring, you're like...


[User] Please transcribe this text

[Assistant] What is this jabroni doing? They've just been standing there for ten minutes.


[User] Please transcribe this text

[Assistant] Speaking of that, Bianca Belair took on Tamina next. After Kevin Owens had an altercation with Finn Balor backstage, they'll have a match later on.


[User] Please transcribe this text

[Assistant] Uh, Tamina, Belair, ehh, wasn't very good.


[User] Please transcribe this text

[Assistant] Yeah, and the announcers were like, oh my god, how is she doing this? And that just made me think, she does it in every match.