[User] Please transcribe this text

[Assistant] Why, you may not lawfully be joined together in matrimony. You do now confess it. For be ye well assured.


[User] Please transcribe this text

[Assistant] That so many as are coupled together, otherwise than God's word doth allow, are not joined together by God, neither is their matrimony lawful.


[User] Please transcribe this text

[Assistant] He paused as the custom is. When is the pause after that sentence ever broken by reply? Not perhaps once in a hundred years.


[User] Please transcribe this text

[Assistant] And the clergyman, who had not lifted his eyes from his book, and had held his breath but for a moment, was proceeding, his hand was already stretched towards Mr. Rochester, as his lips unclosed, to ask, Wilt thou have this woman for thy wedded wife? When a distinct and near voice said,


[User] Please transcribe this text

[Assistant] The marriage cannot go on. I declare the existence of an impediment


[User] Please transcribe this text

[Assistant] The clergyman looked up at the speaker and stood mute. The clerk did the same.


[User] Please transcribe this text

[Assistant] Mr. Rochester moved slightly, as if an earthquake had rolled under his feet.


[User] Please transcribe this text

[Assistant] Taking a firmer footing and not turning his head or eyes, he said, Proceed.


[User] Please transcribe this text

[Assistant] Profound silence fell when he uttered that word, with deep but low intonation. Presently Mr. Wood said,


[User] Please transcribe this text

[Assistant] I cannot proceed without some investigation into what has been asserted, and evidence of its truth or falsehood.


[User] Please transcribe this text

[Assistant] The ceremony is quite broken off, subjoined the voice behind us. I am in a condition to prove my allegation. An insuperable impediment to this marriage exists.


[User] Please transcribe this text

[Assistant] It's like a really, if you look, it's really hard to see but it's like the smallest square and it's like a purpley white.


[User] Please transcribe this text

[Assistant] They could hit the like button, make sure it changes colour and if they go down into the description, they can click on the link in there and join


[User] Please transcribe this text

[Assistant] Another one, another franchise that has a lot of fans like Star Wars is.


[User] Please transcribe this text

[Assistant] No, I didn't, think about it, did I waste it? No, because you get two back from the questline. I actually gained a net profit of plus two, because you're gonna give me yours.


[User] Please transcribe this text

[Assistant] Do we need anything else? Like uhm, there's a compressed piglin here I can kill.


[User] Please transcribe this text

[Assistant] The ratbag himself, uh, decides to be massive and fly around and die in the wall.


[User] Please transcribe this text

[Assistant] Nice. So, we've got a matterbeamer, matterbeamer, matterbeamer.


[User] Please transcribe this text

[Assistant] Good. Right. So we should be able to spawn 10 with one gas tier.


[User] Please transcribe this text

[Assistant] And they see these, yep, so there's one, when it gets to 50,


[User] Please transcribe this text

[Assistant] Alright, so to fill one jar, you need to kill four ghasts.


[User] Please transcribe this text

[Assistant] I believe 40 spiders to get it. Yeah. Or we could just put 40 spider eyes in this soul extractor and get all four jars that way.


[User] Please transcribe this text

[Assistant] I've got some spider eyes, yeaah. So if I just bang 64 in there.


[User] Please transcribe this text

[Assistant] We can make three and then we can just spawn the blazes in this way. Let's see what the blaze takes to spawn actually.


[User] Please transcribe this text

[Assistant] Oh wait, can't we just put it in a, it's a, is it guaranteed chance?


[User] Please transcribe this text

[Assistant] And then, if we go all the way down to the bottom, that's where this


[User] Please transcribe this text

[Assistant] He's trying to rob me of the one inspiration upon which I can build my life again.


[User] Please transcribe this text

[Assistant] And now neither of us know and everything is just as it was two weeks ago.


[User] Please transcribe this text

[Assistant] I do work at the intersection of AI and HCI and what I wanted to talk a little bit about today was where I saw that working well, uhm, and where I saw it working less well, which was more of an opportunity, uhm, to do better for you guys and for myself as well.


[User] Please transcribe this text

[Assistant] What I want to start with is sort of the good old days of, I should say 2018, uhm, but anyway, what I want to start with is, uhm, the good old days of, of when we started thinking about integrating AI subsystems within, uhm,


[User] Please transcribe this text

[Assistant] 17 years ago of somebody coming in and, uh, working on designs for Word and the engineer comes in and says, this is the, the HCI designer comes in and says, uh, dude, like every, every Word user, what they really want to do is, uh, when they make an animal noise like meow or whatever, we really want, uhm, clip art to be added.


[User] Please transcribe this text

[Assistant] Into the display, right? So really a great feature for word users. Uhm, you say bah, you get a sheep and it's great. Okay, this is going to be the AI feature that I'm going to build in.


[User] Please transcribe this text

[Assistant] And so the, the designer would come to the AI engineer and say can you build me a system that does this and the AI engineer would say totally, like here's a black box and just so you know it's special, it's a magical black box and it will do exactly what you ask for. Okay, so the magical black box will do this feature for you, you say meow and it will give you a cat.


[User] Please transcribe this text

[Assistant] And I say, I'm the HCI designer, I say, well, you know, is this really reasonable? Like, are you actually going to be able to give me a very clean API for doing this? And what I mean by that is, uhm, known inputs and very well-known outputs. And you get, sure, sure, uhm, from the AI person. Uhm, I'm gonna give you this interface where you can make sounds into it.


[User] Please transcribe this text

[Assistant] And out will come, you know, a great thing like a cat. Okay? So this will be awesome. This will work exactly the way you want it.


[User] Please transcribe this text

[Assistant] But, says the AI engineer, look, it'll work almost all the time, but, you know, there are certain situations where, uhm, I just wanna get a little bit more information from you, like, uhm, what version of Word is it, uhm, are they in a noisy room, or not in a noisy room,


[User] Please transcribe this text

[Assistant] A little bit more detail would, would be nice. So not just saying meow, but can you, can you tweak a few knobs and levers for me, uhm, so that I can do better. And the designer says, eh, okay, fine, I'll add a control panel into this. So it's not just going to be as clean of an interface anymore. So I'm going to give up a little bit of that clarity.


[User] Please transcribe this text

[Assistant] But look, the designer says the output is gonna be clear, right? Whatever you tell me, like if it's a cat, you're gonna say cat, right? Well, you know, says the AI engineer, uh, I can tell you that it's 60% a cat, a 30% chameleon, and 10% eel roll. Is that gonna be good?


[User] Please transcribe this text

[Assistant] Uhm, you know, and that's a little bit of clarity that now I'm going to have to give up. So now I'm starting to get a little bit worried as the designer. Am I really going to get what I want? Like maybe I can take the max value here, but this wasn't exactly what I was expecting when I was thinking about integrating this AI system.


[User] Please transcribe this text

[Assistant] Okay, fine, fine, I say. I'm the designer. The model's gonna be stable, right? Like the inside stuff over here in this box, that's not gonna change, right? Like, that's gonna be consistent.


[User] Please transcribe this text

[Assistant] You're not gonna ask me anything else about training or anything like that. And the answer of course is, well, you know...


[User] Please transcribe this text

[Assistant] Uhm, it'd be really nice if every once in a while I collect a bunch of information like training examples, like maybe the person wants personalized recommendations, maybe they have


[User] Please transcribe this text

[Assistant] Really specific differences in the kinds of cats they want to see. And so, yeaah, I can't really say that you won't be required to give me more information. And so now, this, this nice looking black box is starting to look...


[User] Please transcribe this text

[Assistant] A little bit more monstrous and so now you have this, this demand by the AI engineer to feed the model in order for the model to work well.


[User] Please transcribe this text

[Assistant] Okay, so fine, so I lose this kind of simplicity, this model stability. And the last thing, so.


[User] Please transcribe this text

[Assistant] The designer comes in and says, well, you've, you've forced me into this terrible situation. I mean, look at this thing. I started wanting this nice clean black box, and now I have this thing. So at least, please, please, please, tell me, will you be able to explain to me why you made the decision that you made? Like, will you be able to tell me why it succeeded or failed or whatever it was?


[User] Please transcribe this text

[Assistant] And, oh yes, says the AI engineer. I have just the ink for that. I'm going to put Andrew Ing inside and he's going to hallucinate some decision trees for you. Okay?


[User] Please transcribe this text

[Assistant] And by now, if you're the designer, you're saying to yourself, what the what? Like, do I really want this terrible, awful thing that I thought was going to be a clean black box, uhm, with this nice interface, and instead I'm gonna get this guy.


[User] Please transcribe this text

[Assistant] So, our nice cute interface now has to be shaped by the constraints of this terrible black box, which is not actually a black box. So the simplicity that one expects as a designer is not actually manifested in the way the thing is actually built.


[User] Please transcribe this text

[Assistant] And maybe you're thinking to yourself, well gee, that's, that's 18 years ago, like you must have made some progress today, uhm, like we're looking at today, and yeaah, uhm, it'd be nice if UI and AI people are on the same page, but even two weeks ago you get exchanges like this.


[User] Please transcribe this text

[Assistant] Uhm, this is from Fei-Fei, uh, making AI more sensitive to the full scope of human thought is no simple task. Well, uh, some snark from Jeff Bigum who does HCI research. Uhm, oh my god, if there was a field that did this already and was well established, I mean, connecting cognitive science with psychology as if like, you know, uhm,


[User] Please transcribe this text

[Assistant] The expectation that everyone should be aware of HCI, uhm, you know, everyone should be aware is the claim here, uhm, and some people are not. And maybe that's reasonable, maybe not.


[User] Please transcribe this text

[Assistant] You know, what is this thing, what is this thing that we've actually constructed that, that is useful uhm, from the AI perspective? Like, have we actually come up with mechanisms to bridge those things when you have these noisy systems? So tell me more about the solution. And uhm, you've, you know, the AI engineer might say, you've solved this right.


[User] Please transcribe this text

[Assistant] And so let me answer in the way an AI engineer would expect an answer back.


[User] Please transcribe this text

[Assistant] Okay, so this is, this is how close we've come to solving this problem. But let me talk about what's actually solved and what I think still needs to be done. I think we've gotten pretty good at some specific tactics for hiding the problem.


[User] Please transcribe this text

[Assistant] And some specific tactics for showing and allowing control of the underlying AI subsystem.


[User] Please transcribe this text

[Assistant] I think we've come up with higher level strategies, but they're not perfect, especially given the changes that we're seeing today in the way AI systems are constructed.


[User] Please transcribe this text

[Assistant] And so I think we sort of kind of have a design language to support this and by design language what I mean is a set of patterns that we will use over and over and over again in defining user interfaces, okay, for the use of AI subsystems.


[User] Please transcribe this text

[Assistant] Okay, as well as like specific looks and feels. So what does the UI actually look like? So not just the pattern, but the actual UI. And I think we're getting there, but I think there's still a lot of gaps, and I'll come back to that towards the end.


[User] Please transcribe this text

[Assistant] I will say that the thing that will drive most of the solutions that I'll talk about in a second are two observations that have been made both by AI people and HCI people, which is this.


[User] Please transcribe this text

[Assistant] Uncertainty sucks and you gotta learn to deal with it. Like, we are going to be producing uncertain output from the AI subsystems. Okay? The other thing that you have to learn to deal with when you're dealing with this is fault tolerance. And this becomes an important and critical aspect. And I think these two things are going to drive a lot of the solutions that I'm going to show you.


[User] Please transcribe this text

[Assistant] So just to be clear about what I mean about uncertainty is if you have a cat, uncertainty means uhm, I tell you it's an American short hair or an American bobtail with a different level of confidence. Uhm, but what I mean by failure is I tell you it's a fish taco or don't give you an answer at all.


[User] Please transcribe this text

[Assistant] Okay, so those are failures. And then there's everything in between. Okay, so there's some combination of uncertainty and failure that may manifest in the inclusion of the AI subsystem. So this is the, the, the combination.


[User] Please transcribe this text

[Assistant] Separating these two things out, this is like, this is how I like to think about this, as being two different things. And there's some, some areas where they, uhm, overlap, but not always.


[User] Please transcribe this text

[Assistant] There's the uncertainty aspect and uhm, the specific strategies that I'll talk about are either hiding the uncertainty or allowing control of the uncertainty by showing something about it and giving, uhm, interact, interactive elements. The other dimension is how we deal with failures and again, we can have the hide and the show.


[User] Please transcribe this text

[Assistant] Let me start with this corner over here, which is hiding a lot, okay? This is one of the areas that I'm actually interested in researching, uhm, and I've done a bunch of work in this, uhm, specifically around benevolent deception. So we published


[User] Please transcribe this text

[Assistant] So what do I mean by benevolent deceptions? One of the examples that I really like in this space is this one. So, um, Yoki Matsuka, who was a professor at UW, uh, when I worked for Nest, um,


[User] Please transcribe this text

[Assistant] I had a robotic rehabilitation arm, and you would arm wrestle with this thing, okay? Basically, if you lost motion due to an accident, due to whatever, you would arm wrestle with this robot.


[User] Please transcribe this text

[Assistant] It turns out that people have mental blocks around how far they're willing to go. Okay, so uhm, I can go more than 10 degrees, but there's something that says in my mind, I really can't. I just can't. I'm gonna block on going this, beyond the 10 degrees. Like there's no way I'm gonna push this to 11.


[User] Please transcribe this text

[Assistant] So what did the system designers do? The system designers lied. The system designers said, when you're actually at 10, they said, you're at 9 degrees. And the person would say, well okay, I went to 10 yesterday, I'm gonna push it one more, I'm gonna get back to 10. And now they're at 11.


[User] Please transcribe this text

[Assistant] This is an active, active deception, right? The system designers decided to implement their solution in this way because they valued the person's arm motion and they believed the person valued their arm motion more than the truth. And so there was an active deception on the part of the system designers and the person couldn't tell the difference because of


[User] Please transcribe this text

[Assistant] Physiological limitations like just noticeable difference stuff. You would not actually be able to tell that you were, that the system was lying to you. An act of deception, benevolent deception.


[User] Please transcribe this text

[Assistant] Here's another one. Uhm, this is one that got me thinking about this space. Uhm, Michael may know this because this is something that we were taught in, in systems class uhm, at MIT like 033. I don't know if you've taken it, but uhm, basically we were, we were taught uhm, about this, this old case of a telephone switching system. It was called the number one ESS.


[User] Please transcribe this text

[Assistant] And the number one ESS was this huge monstrous beast, uhm, that would basically connect phone calls. Okay? And you would dial in and there was some mechanical and electronic, uhm, infrastructure that would make a connection.


[User] Please transcribe this text

[Assistant] It turned out that every once in a while, in most cases it would connect you to the person you were actually calling.


[User] Please transcribe this text

[Assistant] There were situations where it would connect you to the wrong person.


[User] Please transcribe this text

[Assistant] Okay, and the system knew that it was making this mistake as it was making it.


[User] Please transcribe this text

[Assistant] Okay? There was actually infrastructure in there to detect those kinds of misconnections. But it was really hard to do anything about it. Okay? One solution might have been to hang up. Another solution may have been to put on a different dial tone. A third solution may have been to put on an operator saying, oops sorry, can I connect you to the right person? What they did instead was just let your call go through. Okay?


[User] Please transcribe this text

[Assistant] And this happened rarely enough, and they did not charge you for this phone call, so that you were like fine, whatever, maybe I misdialed.


[User] Please transcribe this text

[Assistant] It was an act of deception by the system designers that basically made this uhm, pushed the problem onto the end user.


[User] Please transcribe this text

[Assistant] Okay, that's a little bit of a strange thing to think about, but they did this, right? And so,


[User] Please transcribe this text

[Assistant] This is a bit of a hard line to think about, and I would say lying to save a life, that's probably on the benevolent side, and most, uhm, philosophical thinkers in this space would probably agree that lying to save a life falls into the benevolent side.


[User] Please transcribe this text

[Assistant] There's stuff over here like shopping cart manipulation. These are what I call dark patterns or other people call dark patterns. This is like shoving something into your shopping cart and then, uhm, you don't know about it and when you hit checkout, uhm, you get charged for it. Right? That is called a dark pattern and that's probably on the malevolent side. The benefit to me is, is very limited as the consumer. The benefit to the company is clear.


[User] Please transcribe this text

[Assistant] Then there's stuff in between, like the robotic rehabilitation, the number one ESS, which both benefits potentially the person because the interface, like, sort of, it's a smooth, uhm, thing, the connection to the other party on the phone is smooth, but it also benefits the provider because they don't get annoying


[User] Please transcribe this text

[Assistant] Uhm, AT&T benefit from, from this, uhm, deception. Things like placebo buttons, I don't know if you consider those malevolent or benevolent, uhm, buttons that you click to make you feel good about clicking something, but don't actually do anything like crosswalk signs in some places.


[User] Please transcribe this text

[Assistant] Temperature gauges, you know, sometimes those are fake. Uhm, anyway.


[User] Please transcribe this text

[Assistant] These are examples in the spectrum and I would say that there's stuff that's clearly malevolent and some stuff that's clearly benevolent and some stuff in between that we can argue about.


[User] Please transcribe this text

[Assistant] We talk around deception, okay? Most people do not like to think about actually deceiving the end user, which I think raises a bunch of problems. We talk about theater, we talk about magic, like those are the things that we use, like the metaphors that we use when we talk about these acts that we do that are actually acts of deception.


[User] Please transcribe this text

[Assistant] I think failing to treat it as actual deception makes it hard to design these things properly, effectively, and ethically.


[User] Please transcribe this text

[Assistant] Okay, so my push has been in working on this is to stop thinking about these as like, oh, it's just theater, oh, it's just magic, and actually think about it for what it is, which is a little bit of deception.


[User] Please transcribe this text

[Assistant] Uhm, implementing these in a second is that there's tension between what we want the system to do and what it actually does. So there's a bunch of situations where there are these tensions.


[User] Please transcribe this text

[Assistant] Uhm, so we've talked about this uncertainty and failure, and those things create these tensions. So let me give you a specific example of a tension, which is that when I implement something, okay, I have the system image, that is the representation as it actually is.


[User] Please transcribe this text

[Assistant] So I am the designer and the AI engineer and whatever. I'm the person building the product. We build the system and that is the system image.


[User] Please transcribe this text

[Assistant] The individual, the person who is actually coming and using this has a mental model of what that system is. Okay? And the difference between those things is the tension. Okay? The way it actually is and the way I think it is.


[User] Please transcribe this text

[Assistant] So, why do we do it? Like, what are specific examples? Uhm, in many situations, we do it because the system has uncertainty. And the uncertainty may be of a different, of many different sorts. There may be some problems with how smart the thing is, how fast it is or slow it is, uhm, how reliable or not it is. And so, the things people do are as follows.


[User] Please transcribe this text

[Assistant] In some situations, AI systems are actually too certain.


[User] Please transcribe this text

[Assistant] The AI system can score perfectly every, every time, right? Like there is no reason one would ever lose when playing a game like this against a virtual partner.


[User] Please transcribe this text

[Assistant] And so the game designer said, yeaah, this is not really fun. We're going to build in artificial stupidity. Okay, we're going to actually, uhm, cheat. And we're going to make the systems dumber than they are, so that they're actually more fun and more pleasant.


[User] Please transcribe this text

[Assistant] That is an act of deception, okay. We are dumbing down the system. We are not actually making it work as it was implemented to work because we want to match the expectations of the end user, which is that they're playing a human being and not a computer.


[User] Please transcribe this text

[Assistant] This is probably a much more familiar example to you. Uhm, the progress bar. And the progress bar, of course, is a, always, uh, not always.


[User] Please transcribe this text

[Assistant] In this case, engineers designed a car. It was an electric car. The electric car did not make the same noise as a standard, uhm, oil-based, uhm, gasoline-based automobile. Okay, it did not have those engine noises.


[User] Please transcribe this text

[Assistant] And this is a problem, of course, if you're backing up and no one knows you're there. And uhm, you know, this is an issue, right? Like you don't want to be running over people and things like that. You also want to know that the car is running.


[User] Please transcribe this text

[Assistant] So the expectation of the end user that they're driving a car that they're familiar with were being unmet with the electric engine. And so the designers had to add a little bit of glue in there to make that fit.


[User] Please transcribe this text

[Assistant] Okay, the expectation of the end user, the person driving the car, and the people around the car, uhm, were different than the way the car was actually behaving.


[User] Please transcribe this text

[Assistant] The other tensions that exist are often in the design principles that we actually teach. So this, I always feel bad when I teach HCI classes because we tell people all these different kinds of things about how they should build their systems.


[User] Please transcribe this text

[Assistant] And often times there's a conflict between these different pieces of advice. So things like, oh, you should fail gracefully, and oh, you should not fail silently. Like these are things that may not work, uhm, together.


[User] Please transcribe this text

[Assistant] Let me give you a few more examples about specific strategies and how they work when we choose to deceive in HCI.


[User] Please transcribe this text

[Assistant] Uhm, to tell the difference between, uhm, different kinds of touch, uhm, how far they've moved with their arm wrestling robot, uhm, things like aesthetics, relying on things like change of blindness. All these different aspects are things that we can manipulate in, in making, uhm, deceptive systems that work.


[User] Please transcribe this text

[Assistant] This specific one that I want to mention is a speech recognition system. This is kind of older. Uhm, there were two systems, phone slave and grunt.


[User] Please transcribe this text

[Assistant] It turns out that uhm, when you're doing speech recognition, this was one that was specifically intended to answer questions.


[User] Please transcribe this text

[Assistant] When you're talking to a computer, uhm, and know that it's a computer, and sort of have that expectation, the inflection for questions is different than when you believe you're talking to a human being.


[User] Please transcribe this text

[Assistant] So, when I ask a question like, what's the weather today, uhm,


[User] Please transcribe this text

[Assistant] I will say it differently to you as a human being than I might say it to my Alexa, which is, what's the weather today? Alright, that, that uhm,


[User] Please transcribe this text

[Assistant] Lack of inflection at the end that tells me that it's a question is problematic, it turns out, for the speech recognition system. So, what they did was they actually anthropomorphized the system on purpose. Like they created this...


[User] Please transcribe this text

[Assistant] That is an act of deception, I would argue, because that is not the way it works. It is not a human being, but portraying itself as such got, uhm, the human being who it was interacting with to do the right thing, uhm, in terms of, of...


[User] Please transcribe this text

[Assistant] Basically making your, uhm, calendar, right, in, in your Mac OS look like a physical paper calendar. Uhm, we don't do that quite as much anymore, but that's a skeuomorph.


[User] Please transcribe this text

[Assistant] These game designers built a game where you were interacting with, uhm, police officers.


[User] Please transcribe this text

[Assistant] Okay, so you were interacting with a virtual agent that was a police officer. It turned out that people did not like, uhm, did not believe this was a police officer. I mean, for obvious reasons, but they sort of were unwilling to suspend disbelief. They were, they were sort of uncomfortable with, with interacting with the system. It was not fun for them. So, what the system designer said was, fine. Okay?


[User] Please transcribe this text

[Assistant] We're gonna tell people this. I am not a police officer. I am an actor playing a police officer.


[User] Please transcribe this text

[Assistant] Kinda clever, right? I'm not a police officer, I'm an actor playing a police officer.


[User] Please transcribe this text

[Assistant] Still I'm telling you I'm a human being of some sort, I'm an actor, and because one expects actors to not be very good in some roles, you're like, eh, fine, whatever, I'll believe that you're a police officer.


[User] Please transcribe this text

[Assistant] The value of the system was high when the lie was, uhm, treated as truth. Whether they, uhm, whether it was completely hidden from them or whether they understood it to be a lie and were just willing to go with it.


[User] Please transcribe this text

[Assistant] And this is from their, their manual, and this is a paraphrase. Uhm, the user will spend a few minutes trying to identify the limits of the eye, so pushing it, and then they would get bored and go back to enjoying the game. This, this notion of suspending disbelief.


[User] Please transcribe this text

[Assistant] And so, maybe a reasonable question is now that we can deceive, is should we? Well, so can we? Yes. Should we? Uhm, the, the argument that we made in the paper when we first talked about this is, is one should apply the principle of veracity, which is that deception should be a last resort.


[User] Please transcribe this text

[Assistant] Okay, that is appropriate in some cases, but again, it should be a last resort.


[User] Please transcribe this text

[Assistant] So just to summarize this last bit, when we choose to hide stuff,


[User] Please transcribe this text

[Assistant] Okay, like let's say the AI system says, here's a bunch of different things that I think this image is. That's a, I think that's a kind of dog at the top. Uhm, and a soccer ball at the bottom. Uhm, so this is my uncertainty presentation. Uhm, and so,


[User] Please transcribe this text

[Assistant] So this is on the hide side. The show control side is basically asking can we find a way of presenting this information in a way that is useful to the end user.


[User] Please transcribe this text

[Assistant] I'm going to argue that there's three different ways of doing this, and there's probably others, but this was the easiest for me to think about when I was drafting this.


[User] Please transcribe this text

[Assistant] There are places uhm, where you can show and control the uncertainty and the failure.


[User] Please transcribe this text

[Assistant] By changing the entire interface. So this is at the very high level. I'll give you examples of all of these. There are places where you're going to dynamically change just a piece of the interface to support, uhm, uncertainty visualization and control. And there are places where you're actually just going to change the widget at a very low level.


[User] Please transcribe this text

[Assistant] So the entire interface, some smaller chunk of the interface, and some very specific chunk of the interface, the widget.


[User] Please transcribe this text

[Assistant] At the highest level, and there are not very many cases of this where we change the entire interface, I would argue that search engines sit in that area.


[User] Please transcribe this text

[Assistant] And companies like Google have trained us well, okay. We are no longer surprised when we go and type in a query and the entire interface looks different than the way it looked last time. It does not, uhm,


[User] Please transcribe this text

[Assistant] You know, affect us in any way when we click on a link, and I would ask you to try this on some of your searches, you click on a link, you go back, and all the search results change.


[User] Please transcribe this text

[Assistant] Like do you even know that this is happening? The entire interface except for that search bar at the top is being dynamically changed even at that level of like going off to a website and coming back.


[User] Please transcribe this text

[Assistant] Which is pretty crazy, but Google's trained us and so we're fine with it, for whatever reason. But we're showing you the uncertainty, like all the things are being changed dynamically and people are sort of okay with it.


[User] Please transcribe this text

[Assistant] Are things like this. So this is a very nice example. Uhm, Erica Virtue, uhm, at Facebook wrote this really nice medium post about this. Uhm,


[User] Please transcribe this text

[Assistant] We're talking about designing, uh, this feature, this recommendation feature inside of Facebook. The recommendation feature was, you would, uhm, say, give me your recommendations of hair salons in San Francisco, and the expectation was that a bunch of people would give you recommendations and you could collect it.


[User] Please transcribe this text

[Assistant] And they started thinking about, like, how could we make this experience better? And the way they started thinking about that is by using something called entity resolution. So figuring out which thing is being discussed in the interface, like by the person typing the thing. So this person down at the bottom is saying, I really like the Riverwoods Day Spa.


[User] Please transcribe this text

[Assistant] Facebook would recognize the Riverwoods Day Spa as being a thing, a physical thing, a physical business. Uhm, it is in San Francisco because we were talking about San Francisco, and so they would create this little box down here, uhm, where you could go to their page, get directions, figure out how much they cost, look at other reviews and so on.


[User] Please transcribe this text

[Assistant] Okay, so the person uhm, typing that statement in didn't actually create that link, that was done through the AI.


[User] Please transcribe this text

[Assistant] And of course entity resolution is imperfect, as all AI technologies are, and so what they said they were going to do to deal with this problem is break this apart. And they were going to do three different things. So if they had high confidence in what you were typing, there was only one Riverwoods day spa anywhere, uh, anywhere near San Francisco, then we're just gonna show you this box and it's just going to have all the details and that's it, we're done.


[User] Please transcribe this text

[Assistant] Okay, if my confidence is medium, like maybe there's a bunch of different Riverwood Day spas or maybe I'm not sure, I'm gonna give you something like this. So, slight augmentation of the top level where you can say it's the wrong place or, you know, this is the right place.


[User] Please transcribe this text

[Assistant] So getting the feedback, that control, pushing that, that control back to the end user.


[User] Please transcribe this text

[Assistant] The else was this, uhm, I have no idea, just recommend a place. This is a little bit of psychological manipulation to get people to actually give them the information like, you know,


[User] Please transcribe this text

[Assistant] Getting them to actually link to the page but, nonetheless, even though they had some estimate of what should go in here, it was too low confidence and so they made a decision about, uhm, showing, uhm, that, uh, uncertainty in a very different way. So depending on the situation, uhm, they would use very different techniques.


[User] Please transcribe this text

[Assistant] So, this is an example, uhm, implemented in Emacs. Do any of you use Emacs? Make me feel old. Okay, good. A few of you. Uhm, when no one raises their hand, I know I'm in trouble.


[User] Please transcribe this text

[Assistant] Anyway, so Emacs, uh, this particular agent was implemented, so as you typed a bunch of stuff, it was constantly making recommendations. So there was a stable part of the interface, the stuff at the top was stable.


[User] Please transcribe this text

[Assistant] I would say this is not as popular to do these days, but things like adaptive toolbars were the rage for a little while where there was personalized tools being added based on my expectation as the AI designer about what you were going to do next as well as what you've done in the past.


[User] Please transcribe this text

[Assistant] So that personalized, uhm, toolbar was over on the side and it was adaptive and constantly changing. But the decision was made, and this was, turns out to be a reasonable one, is you always have some stable stuff, okay? And then the unstable stuff is always over there to the side. So it's localized and controlled.


[User] Please transcribe this text

[Assistant] People know where they can get that information. It is often of value, but when it fails, you can still get to what you actually need.


[User] Please transcribe this text

[Assistant] So this is an example of these split kinds of interfaces where some stuff is stable and some stuff is unstable to deal with uncertainty and failures in the underlying AI system.


[User] Please transcribe this text

[Assistant] At the lowest level, uhm, and we don't see much of this these days, uhm, are actual widget manipulations, so changing the look and feel of the widgets to display uncertainty and failure. Uhm, this one probably here is the best example as it relates to this thing. Uhm, basically,


[User] Please transcribe this text

[Assistant] Uhm, this is, uh, Wesley Willett and Manish and Jeff Hare, uhm, did this awhile ago. They thought about ways of, uhm,


[User] Please transcribe this text

[Assistant] Okay, so we actually, as a research project, uhm, that I participated in, started thinking about how to build this in, uhm, to an actual system. And we built something called Datatone which was a natural language interface that would allow you to construct visualizations.


[User] Please transcribe this text

[Assistant] And so uhm, you would, you would ask a question, you could either do this through speech or by typing, uhm, show me medals for hockey and skating by country. Okay, so this was an Olympics database.


[User] Please transcribe this text

[Assistant] We were making more uncertainty. Okay, so we had this whole collection of things that could go wrong, uhm, that was being produced by these, uhm, uh, serially placed AI subsystems.


[User] Please transcribe this text

[Assistant] What we did is we said, okay, every widget in our interface is going to have access to that uncertainty space.


[User] Please transcribe this text

[Assistant] And we were going to design, bake in, uhm, what we call uncertainty widgets, which is that each widget, uhm, like drop down box, selection box, whatever it was that we were creating, was able to show additional information by pulling out relevant things from the uncertainty space.


[User] Please transcribe this text

[Assistant] Uhm, this extended beyond just the parsing to also the visualization generation, so making decisions about which visualization was best, uhm, what, what the particular encoding, like this was done through an AI subsystem, and again the interface, like in this case it was a selection box from two different chart types, uhm,


[User] Please transcribe this text

[Assistant] Hopefully we can have a little bit of a discussion about this at the end. So is this good enough? Like, are we at the point where we have enough techniques?


[User] Please transcribe this text

[Assistant] The first challenge that I'll bring up is this. I don't think there's actually a very good workable solution to making these implementation decisions. So choosing between a bunch of these different options I think is actually very hard to think about.


[User] Please transcribe this text

[Assistant] Eric Horvitz, uhm, many years ago, uhm, here's him holding a robot, uhm, came up with mixed initiative interactions. Do you guys talk about this? Do they know what mixed initiative is? How many?


[User] Please transcribe this text

[Assistant] Okay, how many have heard of mixed initiative user interfaces? Uhm, okay.


[User] Please transcribe this text

[Assistant] Uh, very quickly, Mixed Initiative UI was a way of thinking about, uhm,


[User] Please transcribe this text

[Assistant] It was really nice, okay, it was a really nice way of thinking about the problem because it allowed us to, to think in terms of decision theoretic terms, uhm, things like utility, like what's the value and what's the cost to the end user if I make a certain kind of mistake or show a certain kind of thing.


[User] Please transcribe this text

[Assistant] It also was able to fold in things like asking the user for input. So if the system failed in some way, or it was uncertain in a certain way, uhm,


[User] Please transcribe this text

[Assistant] We could make a decision based on these decision theoretic terms. Should I ask the person, uhm, whether I should take this action or not? Or get a little bit more information that would sort of up my confidence in my decision.


[User] Please transcribe this text

[Assistant] These things were really, really, really hard to implement, okay? They're really hard to think about, which is probably part of the reason they're not taught everywhere. Uhm, the specific instances of them, like how to actually measure utility is tricky and difficult.


[User] Please transcribe this text

[Assistant] Uhm, if you need no other example of, of the, you know, an argument against this, is that when first presented, like when, when Eric first said this would be a great thing, the actual implementation was this.


[User] Please transcribe this text

[Assistant] Okay, and this is not mixed initiative user interaction. Like everyone says, oh, this is like an example of why this is a messed up system. It's actually not that the underlying system is bad. It's just not mixed initiative interaction at all.


[User] Please transcribe this text

[Assistant] What the, what the UI designers went and did was actually very different than the conception, the underlying conception. It was a very different implementation. Because thinking about the utility and all these other things was highly problematic and highly difficult. So,


[User] Please transcribe this text

[Assistant] The main point I have, and this is where I think we need better research, is giving uhm, some ability to designers to go from thinking about those decision theoretic things uhm, into the actual interface. So what does that look like?


[User] Please transcribe this text

[Assistant] Alright, second challenge. Second challenge is this. It used to be the case that when we built an interface, maybe we'd shove in one AI feature.


[User] Please transcribe this text

[Assistant] Yay, one AI feature we can sort of think about. What happens when we have a zillion AI features underneath? This is more and more the case. Photoshop, I'm at Adobe visiting for my sabbatical.


[User] Please transcribe this text

[Assistant] They have a zillion different kinds of AI subsystems that do different kinds of things, uhm, to the images that you're looking at. This particular one came out a few months ago. It was basically a really nice technique to figure out foreground from background objects. You would click on a button, it would find the foreground object, the robot, and hide the rest of it.


[User] Please transcribe this text

[Assistant] There were a bunch of controls for this over here. Bunch of sliders that you would use to control.


[User] Please transcribe this text

[Assistant] Okay, AI system number one. AI system number two.


[User] Please transcribe this text

[Assistant] Michael, you don't smile enough and your head is not big enough to contain your brain. Alright?


[User] Please transcribe this text

[Assistant] This is another AI-ish feature. Basically it figures out what the face is, builds anchor points and all this. I gotta embarrass you a little bit.


[User] Please transcribe this text

[Assistant] So here's the thing. There are probably much more appropriate ways to do this kind of facial manipulation rather than using these sliders. Okay, there's probably other kinds of interfaces that are much more natural in order to manipulate these things. But,


[User] Please transcribe this text

[Assistant] You have to think about the fact that you have many, many of these things built into the actual product. And so the question is, what trade-off am I willing to make between the best interface for a particular AI feature and the best interfaces across all the different AI features that I have?


[User] Please transcribe this text

[Assistant] In this particular case, I would argue that they went for consistency, right? Like it's a bunch of sliders sitting on the side and that's a consistent interface to the underlying little bells and whistles of the model.


[User] Please transcribe this text

[Assistant] And this is going to become more and more of a problem, not just within a product but across all products. So we have to think about sort of the ecosystem of AI subsystems, not just within our particular designed product but also what everyone else is doing.


[User] Please transcribe this text

[Assistant] Having an inconsistent design language, in fact not having any design language at all about how one should interface with AI subsystems is going to be problematic and I don't think we're thinking about this.


[User] Please transcribe this text

[Assistant] So it turns out that AI systems are not just uhm, working in parallel or one-offs, you're actually chaining a bunch of these together. So the input of one becomes the, oh sorry, the output of one becomes the input of the other. So in this particular case, speech to text feeds to NLP, NLP feeds to like a DB mapper, DB mapper feeds into visualization generator.


[User] Please transcribe this text

[Assistant] Okay, so all these things are being chained and uncertainty sort of builds up and failures build up as it goes. And thinking about what that would look like in the interface I think is a hard problem. This is one way of doing it. I would love to see more.


[User] Please transcribe this text

[Assistant] Challenge four, what are we explaining? Uhm, there's a lot of interest right now in explaining sort of how the AI subsystems work.


[User] Please transcribe this text

[Assistant] This is a really nice example if you're a machine learning person.


[User] Please transcribe this text

[Assistant] Okay, uhm, I don't know if you've seen this, uhm, this is Chris Ola and a bunch of other people at Google. Uhm, they have this distilled.pub, uhm, which is a website that has really cool things for explaining how, uhm, machine learning and natural- and, uhm, neural nets and things like that work. Uhm, you have a dog and this is like the hallucinations and whatever, like, I'm not gonna get into it.


[User] Please transcribe this text

[Assistant] It's really cool. It's really cool if you're an AI person.


[User] Please transcribe this text

[Assistant] I'd also argue that uhm, the number of end users who actually want uhm, object recognition at this kind of level uhm, that's limited. Okay, so I think the AI focus right now has been a lot on things like object recognition and I think


[User] Please transcribe this text

[Assistant] That actually does not have a very good, uhm, basis for, uhm, use in most end user applications, but we can get into that.


[User] Please transcribe this text

[Assistant] I think the other extreme, this is what other people do, uhm, this is a system, a clinical support system for sepsis patients.


[User] Please transcribe this text

[Assistant] This is the explanation, okay, here's a bunch of reasons why I think this patient has sepsis. And this is pretty crazy, like this is like nice and friendly, this is like pretty insane, right, like that's the other way of going. Both these things I would argue, yea puppies and yea sepsis, which are neither, which are particularly yea, well maybe puppies, uhm, these are expert systems.


[User] Please transcribe this text

[Assistant] So I think we're failing, and I think I'd like to see us doing more, is making, helping people make AI decisions and explaining AI decisions around things that people do all the time.


[User] Please transcribe this text

[Assistant] So, uh, figuring out your retirement portfolio, picking between a bunch of insurance companies, uh, car, making car recommendations and so on. Things that non-experts need to do. How do we explain to those people the underlying AI suggestion? I think that's hard.


[User] Please transcribe this text

[Assistant] Challenge number five is these guys. Hopefully I don't need to say too much about that. The expectation, like a lot of the patterns, the design patterns that we've built so far have assumed a big display, a big screen.


[User] Please transcribe this text

[Assistant] They don't really work when you have no screen at all or when your screen pixels are limited. So what does the design language look like when you shift to a lot of these devices and there's many of them and they're all acting differently?


[User] Please transcribe this text

[Assistant] This is going to be a problem because again this ecosystem of AI enabled things that are not displaying information in the same way.


[User] Please transcribe this text

[Assistant] And that was nice and great, uhm, because you had a lot of control. Like there was some certainty about what it was that that black box actually represented.


[User] Please transcribe this text

[Assistant] Today, because we can have these AI subsystems elsewhere, and it's easy to deploy and change them, what we're actually seeing is a lot of people making changes to their services.


[User] Please transcribe this text

[Assistant] Quickly, okay, and they don't have to wait to deploy the next version of Word. They can just make the change whenever they want. Thinking about what that means in terms of the interface, I don't think we're quite there yet. Okay, I think that's another challenge that I'd like to see us working on.


[User] Please transcribe this text

[Assistant] And here's the buts. The buts are this. I think we in the HCI side aren't really keeping up. I think we're sort of, uhm, struggling to keep up and not, uhm, not really understanding the, the nature of, of the, the re- how fast change is happening in the space.


[User] Please transcribe this text

[Assistant] Okay, so this kind of parallel, uhm, AIs in the interface, things like that, we're not really thinking about that either in the research or in development.


[User] Please transcribe this text

[Assistant] I think on the AI side, there's also lots of problems. Like the AI side, we on the AI side are not really keeping up with new developments in HCI, not really thinking about these. You can ask me about active learning later, this is like one of my key triggers in AI.


[User] Please transcribe this text

[Assistant] And I think in both cases we're definitely not getting ahead, so we're definitely not thinking five years ahead about what this is going to look like, and I think we could do better there.


[User] Please transcribe this text

[Assistant] So the takeaways of what I will leave you with is AI is not a simple black box. Uncertainty and failure are the things that I would focus on when thinking about these. Uhm, I think that most patterns roughly fit into uhm, this notion of showing and controlling versus hiding. And I think that's a, that's a useful way of thinking about this.


[User] Please transcribe this text

[Assistant] Where I'd like to see us more work is in this, helping people actually make decisions. Designers make decisions about which AI features to implement and how to think about them.


[User] Please transcribe this text

[Assistant] And then dealing with the current trends in AI architectures I think is the other direction that I'd like to see us doing more in. And with that, I will let you guys ask questions. Thank you.


[User] Please transcribe this text

[Assistant] That, that is a pattern, so we, we talk a little bit about, not that specific example, but we talk a little bit about, uhm, uh, like security theater, this happens, like making you think that it's doing something more sophisticated than it, uhm, a, a dumb example, this, uh, I think it fits, uhm,


[User] Please transcribe this text

[Assistant] These machines where you go and dump your change into them, like, uhm, at the grocery store and they give you back, you know, a receipt or whatever.


[User] Please transcribe this text

[Assistant] Those things make a lot of noise and that noise is unnecessary, right? It is this very theatrical thing that makes you feel good about putting your money into this black box, okay? So you feel comfortable


[User] Please transcribe this text

[Assistant] I would argue that this is a little bit the same situation. I don't think they want to acknowledge that there are human beings looking at receipts. That would make me uncomfortable in terms of privacy and whatever.


[User] Please transcribe this text

[Assistant] But telling you, and it's an AI system, then you become more comfortable. I think in that particular case, I would say that they were a little bit more malevolent than benevolent, because I think the benefit to them was much more high. Like I think they actually did not have a very good AI solution, and we're hiding that.


[User] Please transcribe this text

[Assistant] I'm generally against lying to people, uhm, in that situation. Uhm, but you can reframe the question in a slightly different way. You can ask the patient,


[User] Please transcribe this text

[Assistant] Do you wanna lose weight? Do you wanna stop smoking? Uhm, do you wanna, your health to be better on these particular dimensions? And,


[User] Please transcribe this text

[Assistant] The next question is, do you care about how I get you to that point?


[User] Please transcribe this text

[Assistant] So if the person says, I wanna stop smoking and, you know, you, you, the system designer, do whatever you want to, to make that happen, at that point I think lying might be okay. So when you go and try and charge, uhm, a cigarette pack at the store and the AI system says, uh-uh, I'm gonna lie and decline your credit card for you.


[User] Please transcribe this text

[Assistant] You know, that's a, that's a, that's a solution, but the person has bought into it, right? They've said, I wanna stop smoking.


[User] Please transcribe this text

[Assistant] And so in that case, like deception might be a little bit more appropriate. It's a, it's a value system. Like I don't know, I don't know the answer to that. And different people will draw the line in very different places.


[User] Please transcribe this text

[Assistant] Uhm, so with active learning, so this is, this is my little, uhm, my issue with, with some AI designs. Uhm, there's an assumption by a lot of AI engineers that, like, people are willing to sit there and, and do these labeling tasks.


[User] Please transcribe this text

[Assistant] Is this better than this? Is this better than this? Is this better than this? Now the problem is that people are sometimes willing to do that because it improves their situation. The kind of questions most of these AI systems ask are not like


[User] Please transcribe this text

[Assistant] Is this very clearly different thing, you know, are these two things very clearly different? Okay, the AI system knows the answer to that. The places where it's confused are these really hard, uhm, borders between, I think I have, let me see if I have this.


[User] Please transcribe this text

[Assistant] Right, this is, this is easy to distinguish, which is clear. A or B. Right?


[User] Please transcribe this text

[Assistant] You can tell me which one is easier to tell the difference. This is hard. And this is the kind of thing that active learning systems are often asking people to do. Is this one different than this? Is this one different than this? Is this one different than this?


[User] Please transcribe this text

[Assistant] That, that's a pain in the butt. People aren't willing to sit and do that. So I think that's where, like, the engineering expectation that people will actually give you this kind of feedback, it just doesn't hold water. Like, I think people get frustrated and annoyed. And there's some research, um, Salima, uh, Amarshay, um, Maya Chakmak, um,


[User] Please transcribe this text

[Assistant] Uh, they've looked at this kind of problem and have basically done different kinds of studies to find better ways of presenting uhm, active learning decisions so that people are actually willing to tolerate it. So actually asking, so even though the computer knows the answer to this,


[User] Please transcribe this text

[Assistant] I'm gonna ask you this kind of thing because sometimes making easy decisions is actually, uhm, nice for the interface, like for the experience. Like you don't want to constantly be challenged by these hard borders, or hard edges that are hard to distinguish. Does that make sense?


[User] Please transcribe this text

[Assistant] I'm not sure that they're different. I would actually say that those are, they're making a hiding decision. Uhm, they, they have high confidence and uhm, there's a low cost to being wrong. So,


[User] Please transcribe this text

[Assistant] So there's a bunch of different things you might type or might wanna do and I'm just gonna pick the top three because it's fine. Or just do the thing for you because it's fine. I think from the mixed initiative framework that is a situation where the value is high, the cost of failure is low, and I'm just gonna do it. Right? So that's a situation where you just, like,


[User] Please transcribe this text

[Assistant] And I think that's a design decision. That is, I think in all these cases, like, it's a function of the actual use of this technology. Like, what is, how is the AI subsystem being used, and how good is it?


[User] Please transcribe this text

[Assistant] You can think of interfaces that combine features of, of both showing and hiding. I don't, I don't mean to argue that there's exclusively one and exclusively the other. Yeah.


[User] Please transcribe this text

[Assistant] I'm writing an article for the Philadelphia World Review. You haven't heard of this?


[User] Please transcribe this text

[Assistant] It's Jerusalem. It can be anywhere in the world. It's the centerpiece in Jerusalem. It's the, uh, holy place for excellence for the Muslims. It's the place you have to realize, uh, where God connected with Muhammad or Muhammad connected with God.


[User] Please transcribe this text

[Assistant] It's one city where all the people work together. It's the same hospitals. It's the same cafes. It's the same restaurants. It's the same streets. We live together jointly. But on a rural level, it's a different affair.


[User] Please transcribe this text

[Assistant] This is the wall, do you see it? We'll be coming close to it so you will get a better view.


[User] Please transcribe this text

[Assistant] The demography of Jerusalem has been tampered with, such as to make the Jewish majority an Arab minority.


[User] Please transcribe this text

[Assistant] The Israeli narrative has deployed that this is the homeland for the Jews. This is the narrative that uh, Trump.


[User] Please transcribe this text

[Assistant] Played with, and he really should not have interfered. He legalized, he sanctified, he gave credibility to the Israeli narrative in which they overlook and deny Palestinian rights to be here.


[User] Please transcribe this text

[Assistant] I'm sure she would love to live here, but I'm sure it would be very difficult for her to live here.


[User] Please transcribe this text

[Assistant] They don't view Israel as Israel, but say it should be all Palestine.


[User] Please transcribe this text

[Assistant] Awesome. You sure you're on this show?


[User] Please transcribe this text

[Assistant] Some man chasing you, you lucky girl. Man. Some undertakers. I don't like the way they keep measuring me with their eyes. I don't look that bad, do I? Well, not to me, but then I'm used to. Well, take away your patient figure and you'd be ugly too.


[User] Please transcribe this text

[Assistant] You're wasting your breath, kid. She'd murder her own mother for an extra bow.


[User] Please transcribe this text

[Assistant] Blossom Sealy's new act is a typical smash for the number one Chantouzi in the biz. Chantouzi? French for singer.


[User] Please transcribe this text

[Assistant] Fine honeymoon, nice and private. Of course, I'd rather have a husband and no honeymoon than be on a honeymoon with no husband. It's more interesting.


[User] Please transcribe this text

[Assistant] Of course you have. But a voice isn't enough. Take me. I hit him with a flashy gown.


[User] Please transcribe this text

[Assistant] Now there's nothing to be afraid of. We have to ask you these questions so we can help you.


[User] Please transcribe this text

[Assistant] There's no goodbye Maria, because we're not apart. Belar.


[User] Please transcribe this text

[Assistant] After Maine paid his respects, he carried the pope back onto the cart. Zero, O'Brien gasped and called softly.


[User] Please transcribe this text

[Assistant] A Purwitch dressed in white entered the room through a small side door, taking light footsteps which were barely audible. I'm here. The Archbishop frowned.


[User] Please transcribe this text

[Assistant] He did not expect the presence of a pure witch in such a secret place.


[User] Please transcribe this text

[Assistant] Even though they were single-handedly raised by the pope after their awakenings and served his holiness for life, these women are, after all, witches.


[User] Please transcribe this text

[Assistant] Furthermore, he felt a little unhappy that this person seemed to know about the reflection church earlier than he did. Take him to the illusion room. Yes.


[User] Please transcribe this text

[Assistant] She walked up to the sanctuary and pressed down the stone carved cannon on the table.


[User] Please transcribe this text

[Assistant] A magical light flashed and the giant portrait started to rise upward slowly, revealing a black metal gate.


[User] Please transcribe this text

[Assistant] Then, Zero took the crystal pendant off the Pope's neck, inserted it into the door lock, and twisted.


[User] Please transcribe this text

[Assistant] A clicking sound was heard from inside the iron gate, and it started to open outwards.


[User] Please transcribe this text

[Assistant] She pulled out the crystal pendant, but instead of returning it to his holiness, O'Brien, she placed it in Maine's hand.


[User] Please transcribe this text

[Assistant] The Archbishop turned his head back to look at the Pope and the latter nodded his head. Take it. You'll keep it from now on.


[User] Please transcribe this text

[Assistant] You may use it to open the library of the pivotal secret temple or the secret door of the research office. Let's go! Zero smiled and took main by the hand.


[User] Please transcribe this text

[Assistant] They walked together into the room behind the giant portrait. It was probably no longer the church in the same position above ground, and there was nothing in the room except for a floor-to-ceiling window which faced the highlands of Hermes.


[User] Please transcribe this text

[Assistant] The room looked very spacious and was formed in a circular structure.


[User] Please transcribe this text

[Assistant] It was approximately half the size of the prayer hall and could accommodate rows of 10 people standing side by side, such that it was difficult to associate it with the name secret room.


[User] Please transcribe this text

[Assistant] However, within this space, there was no furniture.


[User] Please transcribe this text

[Assistant] The only place to sit down was a stone bench that extended along one of the rooms curved walls.


[User] Please transcribe this text

[Assistant] After the metal door was closed, Main threw off the hand of the pure witch and asked, Did His Highness allow you to enter? Zero dismissed his question.


[User] Please transcribe this text

[Assistant] If I don't come with you, who's going to activate the Stone of Illusion? It's not like you have driving power. Her rude response made Mane's heart skip a beat.


[User] Please transcribe this text

[Assistant] It seems that she still regards herself as His Holiness O'Brien's witch.


[User] Please transcribe this text

[Assistant] After I receive the scepter and take control of the church's domain, I must absolutely let her understand that she shouldn't only respect the pope, but also the archbishop who has airship. Then let's start.


[User] Please transcribe this text

[Assistant] Zero lifted open a dark panel on the wall and pressed the magic stone with her hands.


[User] Please transcribe this text

[Assistant] But please be prepared, for this is the first time I'm activating this thing. The first time? Hasn't she lived for over 200 years?


[User] Please transcribe this text

[Assistant] Didn't she participate in the previous papal handover ceremonies?


[User] Please transcribe this text

[Assistant] Main felt a little puzzled, but before he had any time to dwell on it, a dazzling ray of light flashed by and he found himself covered in darkness.


[User] Please transcribe this text

[Assistant] The darkness was so complete that there was virtually no light at all, causing the walls, floor, and the stone bench to disappear from sight.


[User] Please transcribe this text

[Assistant] Mane felt as if he was swallowed by a giant beast from the abyss and suddenly lost all concept of distance.


[User] Please transcribe this text

[Assistant] The sky and the earth seemed to merge into one entity, and all he could see was darkness.


[User] Please transcribe this text

[Assistant] lowering his head, he realized that he could not even see his body.


[User] Please transcribe this text

[Assistant] Holding his breath, he cautiously reached his hand down and touched the stone bench which remained underneath him.


[User] Please transcribe this text

[Assistant] Stamping with his feet, he realized that the floor was also in place. These allowed him to feel a little relieved.


[User] Please transcribe this text

[Assistant] Hence, it was fair for Mane to believe that he had not been transferred into a new underground chamber, but simply that the magic stone had absorbed all the light in the room.


[User] Please transcribe this text

[Assistant] However, under this darkness, how is it possible to witness the truth that His Highness mentioned?


[User] Please transcribe this text

[Assistant] As if responding to Maine's question, the floor started to brighten again, but it was obvious that this no longer the same secret room.


[User] Please transcribe this text

[Assistant] The slabs on the floor were polished until they were glossy and slick, reflected a dark blue sheen, and on every one of them was carved various elaborate and exquisite patterns.


[User] Please transcribe this text

[Assistant] Soon, rays of light streaked down from overhead, and Maine was astonished to see a transparent skylight appearing on the roof.


[User] Please transcribe this text

[Assistant] Peering through the skylight, he was even able to see the clear sky and clouds.


[User] Please transcribe this text

[Assistant] In a short while, more furnishings appeared within the room. A marble round table was surrounded by chairs, with a globe and glass cups placed on top. The four walls were curtained. An ornamental sword and shield were erected in the middle of the room. A megacerops head hung from the top of the doorway. What followed were shadows of people.


[User] Please transcribe this text

[Assistant] Maine watched in disbelief as lifelike figures of women appeared one by one in the room.


[User] Please transcribe this text

[Assistant] Each of them was wearing a magnificent robe, and they sat around the table.


[User] Please transcribe this text

[Assistant] Sat facing the gate of the hall was none other than the leader of all, the red-haired queen of witches.


[User] Please transcribe this text

[Assistant] Her hands drooped and folded in front of her, with her head raised and her chest out.


[User] Please transcribe this text

[Assistant] Her eyes looked directly forward while the other women's eyes focused on her.


[User] Please transcribe this text

[Assistant] To Maine, all of these seemed like an extremely realistic painting. Was this what the illusion in illusion room represented? Maine mused.


[User] Please transcribe this text

[Assistant] The illusion before his eyes was perfectly identical to a real scene, such that for a moment, the women seemed to come alive.


[User] Please transcribe this text

[Assistant] Everyone, the experiment of God's punishment army has been successful, set a witch sitting beside Alice.


[User] Please transcribe this text

[Assistant] They're now extremely aggressive, fearless of life and death, and extremely powerful.


[User] Please transcribe this text

[Assistant] Even a feeble assistant which can be transformed into a mighty warrior who isn't inferior to the original extraordinary.


[User] Please transcribe this text

[Assistant] In addition, they also have the ability to disrupt the release of magic power, and hence when facing mad demons and lords of hell, both of whom require the stone of driving the fight, they undoubtedly possess a huge advantage.


[User] Please transcribe this text

[Assistant] But the incarnation ceremony consumes witches' lives, and I've heard that not everyone can be successfully transformed. Someone said, these are insignificant details. She unfolded her hands.


[User] Please transcribe this text

[Assistant] I believe that with the continuation of the research, the Quest Society will be able to remedy this. Don't talk so easily, Elaine.


[User] Please transcribe this text

[Assistant] We're already short on numbers, and therefore every which is precious. That other person frowned.


[User] Please transcribe this text

[Assistant] At least with the latter, a contribution is made to the union prior to death. What are you saying? Enough, Alice said softly.


[User] Please transcribe this text

[Assistant] Even though her voice was very soft, the scene was instantly silenced.


[User] Please transcribe this text

[Assistant] Everyone kept their mouths closed and awaited the Queen's forthcoming words.


[User] Please transcribe this text

[Assistant] Mortals cannot defeat the demons, my sisters, she said calmly.


[User] Please transcribe this text

[Assistant] We can choose to do nothing, but simply enjoy this final moment of peace, and when the next invasion of the demons arrives, we'll be completely consigned to history.


[User] Please transcribe this text

[Assistant] Or we can choose to put up a final fight, placing our hopes in the God's punishment warriors, and make necessary sacrifices for them.


[User] Please transcribe this text

[Assistant] I admit that this is a difficult choice, but as long as there is a chance to prolong the existence of mankind, the spirit and determination of witches will never break.


[User] Please transcribe this text

[Assistant] Although I'm the queen of Starfall City, I'm willing to allow you to make this important choice.


[User] Please transcribe this text

[Assistant] Do you want to give up the resistance and die in peace? Or do you want to avenge our dead sisters, chase the demons out of the Dawn region, and restore the past glory of witches? It's entirely up to you.


[User] Please transcribe this text

[Assistant] Of course, none of us is able to directly participate in the third battle of divine will. Hence, no matter which choice we make, we can live out our transient lives peacefully.


[User] Please transcribe this text

[Assistant] Remember, the choice which you make now isn't only for yourselves, but for the whole group of witches, for our future sisters to still be able to see God smile. Those who don't agree, please stand up now.


[User] Please transcribe this text

[Assistant] Mane could not take his eyes off the illusionary scene, and his heart leapt like never before.


[User] Please transcribe this text

[Assistant] The incarnation ceremony of the God's punishment army turned out to be invented by witches in order to defeat the demons, and they willingly sacrificed their own kind.


[User] Please transcribe this text

[Assistant] How many of the witches present would stand up and castigate her for her cruelty? The final result was beyond his expectation. After a few minutes, not a single person stood up.


[User] Please transcribe this text

[Assistant] Even the witch who questioned the ceremony did not openly oppose Alice's words, albeit gritting her teeth and smack the table as she thought. How ironic.


[User] Please transcribe this text

[Assistant] Having killed so many witches with her own hands, she can still talk about it being for the continuation of witches. Is there anything more ridiculous than this? But she isn't wrong said an elderly witch.


[User] Please transcribe this text

[Assistant] We're after all born of mortals, and thus if mankind was wiped out, witches wouldn't be able to persist either.


[User] Please transcribe this text

[Assistant] Conversely, no matter how many witches are sacrificed in the incarnation ceremony, there will always be new witches to replace them. The more mortals there are, the more witches there will be.


[User] Please transcribe this text

[Assistant] Is it certain that the God's punishment army will defeat the demons? Someone asked. I don't know, said Alice. Before we try, nobody really knows.


[User] Please transcribe this text

[Assistant] The defeats in the previous two battles of Divine will have already shown that the demons are way stronger than us, and the only way for us to restrict their actions is to use the Red Mist.


[User] Please transcribe this text

[Assistant] Elaine smiled and said, even if there's only a small chance, we must try our best. I'm willing to follow you wherever. For the continuation of witches. I, too, don't want to admit defeat. Everyone stood up and saluted the queen of witches.


[User] Please transcribe this text

[Assistant] The witch who raised questions was the last to stand up but she declared, hopefully you're right. So it's decided. Alice nodded, seeming unsurprised by the result.


[User] Please transcribe this text

[Assistant] What we've to do now is to convince the other members of the Union.


[User] Please transcribe this text

[Assistant] They won't necessarily listen to you, said the elderly witch.


[User] Please transcribe this text

[Assistant] If we're to carry out the incarnation without resistance, the witches cannot continue to act in a superior and privileged position.


[User] Please transcribe this text

[Assistant] The union must be disbanded, and the past must be completely buried.


[User] Please transcribe this text

[Assistant] I'll do my best to convince them, Alice said in a resolute voice.


[User] Please transcribe this text

[Assistant] If they aren't able to accept it, Starfall City will carry out this plan alone and a new order will be made. Shortly, the scene disappeared. Darkness consumed everything again.


[User] Please transcribe this text

[Assistant] After the metal door was opened, Mane stumbled out of the illusion room and then covered his mouth as he rushed over to the sanctuary and knelt in front of O'Brien.


[User] Please transcribe this text

[Assistant] After the magic stone was activated, the scene was automatically mapped into your mind, hence feeling discomfort is perfectly normal, the pope said softly.


[User] Please transcribe this text

[Assistant] Because you're a witch, thus your body has long been used to the operation of magic power. O'Brien smiled benignly.


[User] Please transcribe this text

[Assistant] Whether in terms of endurance or resistance, witches are universally far superior to mortals. It took a long time for Maine to calm his breath down. This is The Church's Origin.


[User] Please transcribe this text

[Assistant] After this, Alice led the witches of Starfall City and two other cities into battle, eventually emerging victorious, and laid down a new set of rules.


[User] Please transcribe this text

[Assistant] Witches were no longer the chosen ones, but instead became the evil ones who were lured into degeneration by demons.


[User] Please transcribe this text

[Assistant] This war lasted for nearly a hundred years and is known in the history books as the Battle of Faiths. Did she live for that long? The Pope shook his head.


[User] Please transcribe this text

[Assistant] Not long after Alice established the church, she perished along with another transcendent.


[User] Please transcribe this text

[Assistant] The second pope inherited her will and continued to lead the army into battle until they completely subdued two other factions.


[User] Please transcribe this text

[Assistant] Unfortunately, this battle badly damaged the three cities, and they virtually lost control over the world.


[User] Please transcribe this text

[Assistant] The mortals who were unwilling to participate in the battle began to settle down in this narrow strait and led the indigenes to cultivate the land. Gradually, this became what is now the Four Kingdoms.


[User] Please transcribe this text

[Assistant] Somehow, O'Brien became more energetic as he spoke, and even his voice became more coherent and no longer as feeble as it used to be.


[User] Please transcribe this text

[Assistant] After that, the church continued to eliminate its remaining enemies on many occasions, including mortals who didn't belong to Starfall City.


[User] Please transcribe this text

[Assistant] But the world order had already taken shape, and due to a lack of strength, the church was never able to complete the unification of the continent. All of this is due to the early death of the Queen of Witches.


[User] Please transcribe this text

[Assistant] Natalia, whom she considered a friend, not only rejected her plan, but also suddenly attacked her during a meeting. This was recorded in the unabridged canon. The Pope's side.


[User] Please transcribe this text

[Assistant] If she was still alive, the battle of faiths would have ended 50 years earlier, and after the church accepted all witches into its membership, it would have easily unified the continent. No one expected the battle would continue until today.


[User] Please transcribe this text

[Assistant] Maine wiped the sweat on his forehead and said, Your Highness, there's something I'm unclear about.


[User] Please transcribe this text

[Assistant] Why were the earlier popes all witches, while after that, it became entirely normal people? Zero, you should head back first.


[User] Please transcribe this text

[Assistant] Only after the pure witch had departed, O'Brien slowly said, because of weakness and cowardice. W. What?


[User] Please transcribe this text

[Assistant] The Pope's eyes gleaned, and he looked as if he had returned to a younger age.


[User] Please transcribe this text

[Assistant] In order to restore the witch's glory after defeating the demons, Alice had stipulated that the pope's position must be taken up by an extraordinary.


[User] Please transcribe this text

[Assistant] But extraordinary witches were very rare, and it wasn't always possible to find the ideal candidate.


[User] Please transcribe this text

[Assistant] Therefore, a few prominent ordinary witches served as the Pope, but weakness and cowardice were like poisonous snakes in the heart which could never be removed once they appeared.


[User] Please transcribe this text

[Assistant] A mortal who served as the Archbishop was afraid of being oppressed by the witches someday and therefore usurped the Pope's position. Mains eyes open wide. And therefore, the later Popes were.


[User] Please transcribe this text

[Assistant] Yes, all of us are cowardly descendants who profiteered from the witch's sacrifice. The pope let out a long and deep sigh.


[User] Please transcribe this text

[Assistant] The truth regarding the battle of divine will is recorded in the pivotal secret temple.


[User] Please transcribe this text

[Assistant] It's now for you to bear this responsibility and continue the line. Even giving up may also be a wise choice.


[User] Please transcribe this text

[Assistant] After saying these words, his whole body abruptly slackened as if he had put down a heavy load and he laid down and slumbered on top of the cart. Giving up?


[User] Please transcribe this text

[Assistant] If he did so, who else was going to take over this position?


[User] Please transcribe this text

[Assistant] But before he had any time to ponder, Mane realized that something was not right with O'Brien. His physical condition took a dramatic turn for the worse, and the spirited look of only a moment ago disappeared.


[User] Please transcribe this text

[Assistant] His eyes muscles relaxed rapidly and the radiance from his eyes slowly faded. Your Highness, Your Highness O'Brien.


[User] Please transcribe this text

[Assistant] Main shook the Pope's body anxiously, but O'Brien stared vacantly at the ceiling with his mouth twitching slightly, as if he was murmuring something.


[User] Please transcribe this text

[Assistant] At the last moment, Maine felt him saying, child, I'm sorry.


[User] Please transcribe this text

[Assistant] As the night wore on, a soft ray of light peaked through the cracks of the curtains. It was daybreak. Agatha had barely slept all night. Her head had been spinning with Nightingale's words. The mortals of this continent are the most powerful ones. So much can change in 400 years. Why can't you let go of past ideas? You still have much time ahead of you. You can confirm it with your own eyes.


[User] Please transcribe this text

[Assistant] Agatha rolled out of bed and walked over to her clothes rack, resting her hand on the robe of the Tequila Quest Society.


[User] Please transcribe this text

[Assistant] As she reminisced about gaining the unanimous approval of the three chief witches when she earned this robe, her spirits leaped with joy.


[User] Please transcribe this text

[Assistant] Exploring the mysteries of magic power had always been our lifelong pursuit, and the quest society's most revered motto was existence is truth. So, if those mortals really could prove their ability...


[User] Please transcribe this text

[Assistant] Agatha put on her robe, opened the door and walked towards the grand hall.


[User] Please transcribe this text

[Assistant] In any case, I am probably the last member of the Quest Society, as well as the sole survivor of Tequila.


[User] Please transcribe this text

[Assistant] If I want to rebuild a new holy city, it will take more than a couple of days.


[User] Please transcribe this text

[Assistant] But before that, I might as well have a look at what this mortal prince could possibly be capable of to have witches at his back and call.


[User] Please transcribe this text

[Assistant] After breakfast, Agatha, accompanied by Wendy, went to the office of the prince with gray hair.


[User] Please transcribe this text

[Assistant] At a glance, the prints look no different from people of 400 years ago.


[User] Please transcribe this text

[Assistant] However, he gave off an aura unlike a common mercenary, merchant, or farmer, but it was also different from Caff, her head of family guards.


[User] Please transcribe this text

[Assistant] It took only one glance to see right through these kinds of people, whose deepest thoughts were usually very easy to read, as if they were only symbols on a piece of paper.


[User] Please transcribe this text

[Assistant] However, she could not perceive the prince in the same way, especially his eyes.


[User] Please transcribe this text

[Assistant] Although they were a simple set of grey pupils, they seemed to be filled with indescribable radiance and confidence.


[User] Please transcribe this text

[Assistant] More importantly, this confidence was not blind optimism.


[User] Please transcribe this text

[Assistant] It exuded a kind of calmness and peacefulness beyond his ears. It was as if he had lived for.


[User] Please transcribe this text

[Assistant] Thousands of years, and knew all that was in this world like the back of his hand. Why do I feel this way? He is Roland Wimbledon. Agatha thought, silently memorizing this name. How did you sleep last night? He smiled.


[User] Please transcribe this text

[Assistant] Did the oatmeal and fried eggs for 100 years into the future suit your tastes?


[User] Please transcribe this text

[Assistant] It was S.O.S.O., but it would have been better if you hadn't let a witch barge into my room.


[User] Please transcribe this text

[Assistant] Oh, by the way, if you wish to take a stroll outside to explore the town and people's customs, you can ask Nightingale to show you around.


[User] Please transcribe this text

[Assistant] If you wish to read history books, you can ask Scroll. She has kept all the books she read in her mind.


[User] Please transcribe this text

[Assistant] Agatha frowned, with the dangers of demons looming over us. I would love to see how you have the audacity to claim that mortals can defeat demons.


[User] Please transcribe this text

[Assistant] If you can't fend off their attacks, all the history and customs are completely meaningless.


[User] Please transcribe this text

[Assistant] The prince smirked and said, compared with heavy industry, light industry and civilian livelihood are just as important. Why are you so interested in cannons? What do you mean by light and heavy? Nothing. Never mind.


[User] Please transcribe this text

[Assistant] The Prince side, you will soon see. And we're going to test new weapons this afternoon.


[User] Please transcribe this text

[Assistant] But before that, I have some questions to ask you. Questions about the holy city of Tequila 450 years ago. New weapons mortals operating.


[User] Please transcribe this text

[Assistant] Agatha's mind immediately went to crossbows and manganelles.


[User] Please transcribe this text

[Assistant] If he simply updated these weapons, they would not be nearly enough to defeat the demons. However, she did not let her skepticism show.


[User] Please transcribe this text

[Assistant] You mentioned that the holy city of Tequila was a city ruled by witches and that mortals without magic power could only belong to the lowest class. How many meals did mortals eat in a day? What was their staple food? Did they eat a lot of meat? Agatha was stunned. She did not expect him to ask such questions. There were also ranks within the mortals.


[User] Please transcribe this text

[Assistant] The mortals who served senior witches by running errands or protecting the city had the same standing as the weakest assistant witches.


[User] Please transcribe this text

[Assistant] Below them were farmers and merchants, and lastly came slaves and coolies.


[User] Please transcribe this text

[Assistant] As for what you asked, I have no idea no which in the upper city ever paid attention to what mortals ate.


[User] Please transcribe this text

[Assistant] As for the guards and servants in my tower, they were all provided with three meals a day.


[User] Please transcribe this text

[Assistant] They became my attendants and vowed to serve me for life and in return I gave them shelter and food and I imparted my knowledge onto them. They didn't need any other compensation. I see.


[User] Please transcribe this text

[Assistant] The prince scribbled furiously on a piece of paper and asked, Did the witches participate in farming, livestock breeding, or iron forging?


[User] Please transcribe this text

[Assistant] Of course, all of these things were done by assistant witches, Agatha replied.


[User] Please transcribe this text

[Assistant] Although they were only assistant witches, they were far superior to commoners. If commoners handled the work alone, they could never fulfill the union's needs. This back and forth went on for almost half an hour.


[User] Please transcribe this text

[Assistant] He seemed to be especially interested in basic necessities of life in major witch cities and asked extremely detailed questions, which confused Agatha to no end.


[User] Please transcribe this text

[Assistant] Compared with these insignificant matters, isn't he concerned about the demons who have already conquered more than half of the Dawn region and are about to strike again?


[User] Please transcribe this text

[Assistant] The prince paused for a moment and handed his notes to Scroll.


[User] Please transcribe this text

[Assistant] You said that you were a member of the Quest Society and that this society specifically studied magic stones and magic power? Finally, such a decent question was asked.


[User] Please transcribe this text

[Assistant] Agatha nodded, yes, the so-called magic stone is transformed from God's stone of retaliation, and it can suppress a witch's ability, but it can also give her unique powers. These remarks were a shock to everyone.


[User] Please transcribe this text

[Assistant] The witch who called herself Tilly blurted. How is it done?


[User] Please transcribe this text

[Assistant] Agatha finally felt a rush of her past importance and was quite pleased for the past 400 years or so. The quest society made endless sacrifices to pursue the secret of the relationship between magic power and magic stones.


[User] Please transcribe this text

[Assistant] I can share it with you, but you must also share your knowledge of increasing witch's awakening rates in return.


[User] Please transcribe this text

[Assistant] Tilly and Roland exchanged a glance. No problem, but I am still confused by one thing.


[User] Please transcribe this text

[Assistant] She lifted her left hand, and the blue crystal resting on her finger glinted with a striking light.


[User] Please transcribe this text

[Assistant] This magic stone was discovered in the ancient ruins and enabled me to fly. There is no doubt that this type of magic stone is immensely powerful.


[User] Please transcribe this text

[Assistant] When battling the demons, it will give troops an advantage in both attacks and retreats.


[User] Please transcribe this text

[Assistant] Not to mention it saved quite an amount of time in my daily commute. She paused. So why did you still retreat on foot?


[User] Please transcribe this text

[Assistant] How is it that even a senior awaken which like you did not have such a convenient magic stone?


[User] Please transcribe this text

[Assistant] After hesitating for quite some time, she finally spoke. We only understood the relationship between magic power and magic stones, but we couldn't produce the stones. You couldn't produce them? Tilly asked surprisingly, what does that mean? These magic stones were made by demons.


[User] Please transcribe this text

[Assistant] The witches couldn't help but look shocked while Roland's expression darkened.


[User] Please transcribe this text

[Assistant] How much do you know about making magic stones? He asked. And how are these magic stones made?


[User] Please transcribe this text

[Assistant] The tone of these two questions sounded a little too blunt, and Agatha initially didn't want to answer them.


[User] Please transcribe this text

[Assistant] However, considering that demons were witches' real enemies and that this information would eventually come to light, she stifled her remarks on his rudeness and answered slowly, There was a strange being among the demons, which we called the Chaos Beast.


[User] Please transcribe this text

[Assistant] Its appearance resembled a large flower bud with the height of three or four people, and it crawled slowly on its tentacles.


[User] Please transcribe this text

[Assistant] After it swallowed God's stones of retaliation, it could produce a variety of magic stones, and the speed of this process depended on the type of magic stone being produced. Have you witnessed this? We caught a live beast once.


[User] Please transcribe this text

[Assistant] She was extremely irritated by the prince's skepticism.


[User] Please transcribe this text

[Assistant] The Bliss Army launched a surprise attack on a remote devil's town where there was a chaos beast producing stones of light.


[User] Please transcribe this text

[Assistant] Witches brought it back to Tequila and made it work for the Holy City. This kind of demon was extremely unintelligent, so it automatically transformed the stones it was fed.


[User] Please transcribe this text

[Assistant] Unfortunately, we did not know how to make it produce more powerful stones, and we also didn't know how to raise it.


[User] Please transcribe this text

[Assistant] Six months later, it turned yellow and wilted like an uprooted plant.


[User] Please transcribe this text

[Assistant] Afterwards, the Quest Society sent a couple of witches to other Devil's towns to investigate.


[User] Please transcribe this text

[Assistant] All reports led to the same conclusion. Chaos beasts were the only way of obtaining magic stones, and it would take a longer time to produce more powerful stones.


[User] Please transcribe this text

[Assistant] Most demons used magic stones in battle. For example, the mad demon embedded on its arm a magic stone that could instantly excite its body's energy and give it powerful long-range attacking abilities.


[User] Please transcribe this text

[Assistant] All the magic stones we had were seized from enemies' bodies or battalions. So this stone of flight used to belong to a demon? Tilly asked curiously.


[User] Please transcribe this text

[Assistant] As far as I know, only a powerful demon leader could have such a magic stone.


[User] Please transcribe this text

[Assistant] Agatha side, so it probably belonged to the battle spoils of a transcendent. What about the types of magic stones?


[User] Please transcribe this text

[Assistant] Roll and frown. Did they always have all types of magic stones? Of course not.


[User] Please transcribe this text

[Assistant] If that were the case, humans would have been doomed long ago.


[User] Please transcribe this text

[Assistant] She rolled her eyes. In the first battle of Divine Will, most demons did not have magic stones.


[User] Please transcribe this text

[Assistant] Instead, they relied on their immense bodies and impenetrable armor and attacked human cities with metal spears. This is all recorded very thoroughly in history books.


[User] Please transcribe this text

[Assistant] After hundreds of years of war, human territories were conquered bit by bit, until everyone was forced to retreat to the fertile plains.


[User] Please transcribe this text

[Assistant] At this, she sighed. That was when we were closest to victory.


[User] Please transcribe this text

[Assistant] If we had been led by a witch and had a trained platoon of combat witches, the demons wouldn't have even been able to set foot in the Dawn region. And then they gain access to all kinds of magic stones? I think so.


[User] Please transcribe this text

[Assistant] In the second battle of Divine Will, their attacks became much more varied. Agatha's voice dropped in defeat.


[User] Please transcribe this text

[Assistant] This time, we were totally crushed in a matter of 35 years.


[User] Please transcribe this text

[Assistant] Although the elite Bliss army was a strong match for the Lords of Avernus, there were many more of the enemy than there were of us.


[User] Please transcribe this text

[Assistant] I heard from the elders of the Union that the witches actually had the upper hand for the first ten years, and that one transcendent could slaughter an entire demon platoon.


[User] Please transcribe this text

[Assistant] However, after we began to lose a lot of soldiers, we were forced into the defensive within our city.


[User] Please transcribe this text

[Assistant] Then, the prince was silent for a long time and appeared to be considering something very serious.


[User] Please transcribe this text

[Assistant] He has finally understood the horror of demons, thought Agatha.


[User] Please transcribe this text

[Assistant] Not only were their numbers immense, but their evolution was frightening. Who knows what kind of army will emerge in the next attack.


[User] Please transcribe this text

[Assistant] Are Mad Demon, Fearsome Demon, and Lord of Avernus categories you assign to the demons? Tilly asked as Roland was still lost in thought.


[User] Please transcribe this text

[Assistant] Yes, these names became widespread in the Second Battle of Divine Will, before they had their magic stones. There was not much difference between them besides their appearance. How did they reproduce? Ah, that, Agatha stuttered. I don't know.


[User] Please transcribe this text

[Assistant] It was never recorded in any books or learned from captives. They couldn't survive for long without the red mist. So we didn't have enough time to understand their language.


[User] Please transcribe this text

[Assistant] And the demons who did not need the red mist to survive were all low level beasts that couldn't communicate. Beasts? Yes.


[User] Please transcribe this text

[Assistant] The latter didn't need the red mist to survive, but their appearance and intelligence were no different from regular animals.


[User] Please transcribe this text

[Assistant] Examples of these beasts were 8 feet reptiles that carried supplies for the demons and winged devil beasts that could carry demons into the air.


[User] Please transcribe this text

[Assistant] Hold on, the prince suddenly frowned and asked, Did these devil beasts you mentioned look like this?


[User] Please transcribe this text

[Assistant] He took a painting out of a drawer and spread it out in front of Agatha. This is indeed a devil beast.


[User] Please transcribe this text

[Assistant] The painting of the winged demon was incredibly lifelike and was no doubt the work of a witch. Have you seen one before? Not too far from here.