[User] Please transcribe this text

[Assistant] Most of these in this talk here, but for this first part, I'm actually going to just focus on the interaction accessibility and a device accessibility as it relates primarily to people with limited mobility.


[User] Please transcribe this text

[Assistant] So first is device accessibility.


[User] Please transcribe this text

[Assistant] Like I mentioned before, make a lot of assumptions about users and their abilities and this facilitates the needs for more customizable hardware, but also more just accessible hardware in terms of looking at things that people can do already. So like what devices do people already use and then how can we integrate that


[User] Please transcribe this text

[Assistant] There's this, or when we think about VR in general, there's this really...


[User] Please transcribe this text

[Assistant] Uhm, so there's been some work that's looked at this.


[User] Please transcribe this text

[Assistant] For people who are blind, for example.


[User] Please transcribe this text

[Assistant] Uh, caught the cane troller that was done at Microsoft research, um, a few years ago that was led by an intern.


[User] Please transcribe this text

[Assistant] Uhm, Yuan, who's now a professor at University of Wisconsin, essentially we're looking at, well, how can you make interactions inside of VR more accessible to people who are blind? Well, you can actually create custom hardware to allow people to have these kind of tactile sensations or feelings when they're actually interacting inside of a virtual world. So this is a great


[User] Please transcribe this text

[Assistant] And actually worked on kind of like a follow up to this project and actually made some really great improvements. So this is like one really interesting and really useful way of thinking about device accessibility. It's like, well, how can we create new things that allow people to have new experiences?


[User] Please transcribe this text

[Assistant] I, when I joined the team, I, you know, like I mentioned before for my dissertation research, I was really looking at things from.


[User] Please transcribe this text

[Assistant] So to help us answer these questions, we ran this interview study. We had 16 participants with limited mobility. We had a mix of in-person and remote interviews, and we had a couple of people with


[User] Please transcribe this text

[Assistant] Uhm, so for the people who were actually in person that came in and did the study and the people didn't have prior VR experience, we actually gave them an opportunity


[User] Please transcribe this text

[Assistant] Uhm, iCloud's first contact, you could think of it as like a tutorial type of thing to teach you how to use hand tracked controllers and things. And then Google Earth VR, which we just thought could be an interesting application for people to play around with to get a sense of how they like the VR experience.


[User] Please transcribe this text

[Assistant] Uhm, so from this paper that we published at Assets, uhm, just last year, uhm, it was, the conference was in October, we essentially identified what we call these like seven VR accessibility barriers. So things like setting up a VR system,


[User] Please transcribe this text

[Assistant] Putting on and taking off an HMD, adjusting the HMD head strap, uhm, cord management, manipulating dual motion controllers, inaccessible buttons and maintaining view.


[User] Please transcribe this text

[Assistant] Other controllers were things that we identified as just, hey, if somebody wants to be able to interact with a VR system.


[User] Please transcribe this text

[Assistant] Just kind of like once they open the box, here are some of the issues that they're going to have to deal with when trying to think about somebody's kind of concerns. So I'm going to talk not about all of these, but I just kind of want to highlight a couple and some of these you might look at and might be like, okay, yeah, that makes sense. But why is that a concern or why is that like an HCI problem?


[User] Please transcribe this text

[Assistant] So for example, we can just look at things like, you know, adjusting the H&B headstrap or, and you can kind of tie this in more broadly with putting on and taking off the H&B.


[User] Please transcribe this text

[Assistant] How you would design this so that it will work well for someone who's seated in the wheelchair can actually make it, in their case, difficult for them to move their head and have these other interactions.


[User] Please transcribe this text

[Assistant] But it also kind of just speaks to this broader question that you can have of, well, like, hey, how do we create more accessible type of like,


[User] Please transcribe this text

[Assistant] So if we have these technologies that presume a certain level of control and certain level of autonomy, how does that break down then if people then have to understand how people can't use this in a way that they


[User] Please transcribe this text

[Assistant] So this other one that is more relevant to some of the research I'm going to talk about after this is, so this thought about manipulating dual motion controllers, so unlike in a traditional


[User] Please transcribe this text

[Assistant] desktop computing sense where it's kind of like, you know, you kind of have one pointing implement is the main focus of control. So you either have like a mouse or a track ball, or you're using your finger on a touchscreen.


[User] Please transcribe this text

[Assistant] Uhm, in a VR environment, you could have two controllers and use both hands to do different things at different times.


[User] Please transcribe this text

[Assistant] Uhm, and then another participant P2 here that said like, you know, I just wouldn't use it too much or requires coordination. It could be really frustrating.


[User] Please transcribe this text

[Assistant] Hey, these uhm, these controllers themselves aren't just used for like their motion tracking capabilities. Uh, people also have to interact with the individual buttons and things on them to actually, you know,


[User] Please transcribe this text

[Assistant] So we have one participant here, P6, and this is kind of represented in the image here. It's saying it's like, yeah, I think walking was pushing down, so this is walking in a virtual environment.


[User] Please transcribe this text

[Assistant] Uhm, so you push one of the joysticks down and then to do that, I had to use two hands and leave the other controller to do it. As soon as I heard that you have to push it down, I was like, yep, nope, that's not happening.


[User] Please transcribe this text

[Assistant] And that's primarily because people might have low strength, for example, in their hands or low dexterity. So if you're expecting people to be able to hold and manipulate multiple buttons at the same time while still holding this controller, like participant six said, that might not be possible for them. They might actually have to like stop what they're doing, put another controller down and then use both hands to just press down the joystick, right?


[User] Please transcribe this text

[Assistant] So when we ask people kind of like, well, what would be, what would be some things that you might want to see? Like what would make these experiences more accessible to you? Uh, so some people mentioned things like, well, just like alternative input controls, right? So if we can't make this assumption that everyone who's going to use a virtual reality system is going to have kind of the,


[User] Please transcribe this text

[Assistant] How could we do these interactions with devices that people already use? So if someone uses a quad stick or they use a series of switches or a trackball, what's the possibility for that?


[User] Please transcribe this text

[Assistant] And then other ones is like, well, how could we use other different types of input modalities? So how could we use gaze input or voice input to allow more accessible, uh, VR manipulations? But even with gaze and voice, we're still not there yet in terms of understanding how would somebody, for example,


[User] Please transcribe this text

[Assistant] Interactions accessible given the more limited input modalities that might be available to some people.


[User] Please transcribe this text

[Assistant] So one possibility, and I think that's really good, and I know Patrick Carrington was here earlier in the quarter, is to think about shareable computing. So this is this idea that, you know, shareable computing is really this technology that takes advantage of the space on or around a wheelchair.


[User] Please transcribe this text

[Assistant] One thing that, uh, we were looking at here, uh, the title of the slide is wrong, so, uh,


[User] Please transcribe this text

[Assistant] Forget the accessible by manual input part, that's coming later. Uhm, the question we're actually looking at here is this, uhm, well, how do we leverage more traditional 2D input devices, things that could fit more comfortably on the form factor of a wheelchair to, you know, complete more complex spatial interactions?


[User] Please transcribe this text

[Assistant] So one thing we looked at and we thought was actually a good example of this is like virtual hand manipulations. So in a VR environment, you could use these touch controllers to perform different type of hand manipulation. So you could do things like


[User] Please transcribe this text

[Assistant] Make a thumbs up or grasp a cube and use a controller to like throw it or just drop it, toss it in different directions.


[User] Please transcribe this text

[Assistant] So this is something in which there's no really good design schemes right now to say well how would I use a commodity input device like a joystick and a switch to be able to perform all these various types of virtual hand manipulations. It's something that


[User] Please transcribe this text

[Assistant] Uhm, so for example, for this, you know, we did like a user elicitation study where we, we got 21 participants for limited mobility.


[User] Please transcribe this text

[Assistant] Once again, this is happening in COVID time, so we weren't able to actually do this in person, but what we actually did was use the crowd elicit platform that Abdallah Ali and others created to actually be able to do this like remote user elicitation study. So we created 16 hand manipulations as reference, and what we did was we showed them


[User] Please transcribe this text

[Assistant] Use different types of input devices to be able to perform this gesture, for example. So in our study that I'm going to talk about now, it's like, you know, we asked people, it's like, hey, if you use a touch pad, you know, it can sense multiple concurrent touch points. How would you use that to perform these types of gestures? And some of the things that we found initially, oh.


[User] Please transcribe this text

[Assistant] Well, to analyze, before we get to that, to analyze this, what we did was we looked at the taxonomy of surface gestures, uhm, by Robrock et al. that looked at, okay, hey, we can analyze different gestures that people make on the touchscreen to understand kind of how people's mental models of these gestures actually work. Uhm, so,


[User] Please transcribe this text

[Assistant] Some initial findings, for example, that we have around this that speaks to these notions of mental models, for example, are, we looked at like discrete hand gestures. So we had things in our set of 16 total that were like four discrete hand gestures. So this kind of like,


[User] Please transcribe this text

[Assistant] And what we found was that when we were analyzing people's answers for how they would perform these gestures, 41%, which was the most popular


[User] Please transcribe this text

[Assistant] Uhm, or the most common results were abstract gestures. So abstract gestures are ones in which there's really no mapping between what the person does with the gesture and how that relates


[User] Please transcribe this text

[Assistant] 57% of those are metaphorical, so meaning that the gesture actually is more...


[User] Please transcribe this text

[Assistant] So an example of that, for example, is we can look at these dichotomous reference. So meaning that there's the same reference, but kind of like in reverse in this, in some situation. So for example, we can have an action that's just like, well, how would you grasp an object using this virtual hand? And then we can ask, well, how would you release it?


[User] Please transcribe this text

[Assistant] And this speaks to this kind of metaphorical gestures that people were referring to, so uhm,


[User] Please transcribe this text

[Assistant] So somebody, P16 for example, would say, tell the grass, the cube, they said they would squeeze their fingers on the touchpad together to grab the object. And then to release, they would kind of just do the opposite or the reverse of the grass gesture, which in this case would just be to extend their fingers, uhm, to release.


[User] Please transcribe this text

[Assistant] This is a study that we've actually been really interested in and it's something that we think is really useful. Uhm, here's kind of a diagram that shows all the 16 different


[User] Please transcribe this text

[Assistant] Uh, so for example, what we found was that these translation ones, so like move forward, move backwards, people were able to come up with really


[User] Please transcribe this text

[Assistant] Easy gestures for those because they might say just directional swipes, for example, might be good enough for people to understand like, okay, if I want to move something backwards or forwards, I might just swipe up and down for that interaction.


[User] Please transcribe this text

[Assistant] But like I mentioned before, these kind of more, um, discrete gesture type of things in which there's not really great mental models for like, how do you do a thumbs up on a touch pad? Uhm, we might have to think about ways that, uh, create more accessible mappings for those different types of interactions.


[User] Please transcribe this text

[Assistant] So this is like one example of ways that we need to think about how commodity input devices can be used in a way to create much more complex, expressive interactions when we actually get to VR and different types of augmented reality interfaces. So some of the takeaways that we had from


[User] Please transcribe this text

[Assistant] Not just the study that I just mentioned, but the assets paper, uhm, this actually comes from the assets paper, so,


[User] Please transcribe this text

[Assistant] Just a couple of things in terms of design. So one thing that's really important is this notion like designing for interdependence. So interdependence is kind of a design framework in which you don't think about how a person would do something independently. You actually try to uhm,


[User] Please transcribe this text

[Assistant] Uhm, for lots of different interactions and what we heard from our participants was that, hey, when thinking about how to create more accessible VR systems, just don't think about the one user, the one person. Think about the ecosystem that that person is in. How could you help or how could you involve


[User] Please transcribe this text

[Assistant] Uhm, design for customization really comes to this question of saying, you know, we really need input controls and interaction methods that people can really take advantage of, create more bespoke solutions. But at the development level, we really need to think about how to enable that level of flexibility and control.


[User] Please transcribe this text

[Assistant] Uhm, design for accessibility. So this is really just meaning more towards like ability, right? So if you're thinking about different ways that people might use a system, you really want to put the ability part first and really, uhm,


[User] Please transcribe this text

[Assistant] You know, like, why would you want to be a part of this experience? This is an experience for people who can walk or experience for people who can throw. And if you can't walk or throw, then you shouldn't want to be able to engage in these experiences. But we found this that like, you know, people want to interact with VR systems in all types of ways. So don't kind of make the assumption that like, Hey, what?


[User] Please transcribe this text

[Assistant] What matters the most is just immersion or what matters the most is embodiment in these certain situations. Really consider what people want and how they're gonna interact in these different types of situations.


[User] Please transcribe this text

[Assistant] Like I mentioned, when we talked about device accessibility, VR is much more than these traditional systems. Think about physical input and output.


[User] Please transcribe this text

[Assistant] So if there is an action or something that was hard for you to do in the virtual world, but in the real world, but now the virtual world expects you to perform that action the same exact way, we're just now making the virtual world just as inaccessible as the real world.


[User] Please transcribe this text

[Assistant] Uhm, so we need to really think about these inaccessible methods and user interfaces that can work well for these more accessible devices that we also create.


[User] Please transcribe this text

[Assistant] Uh, so another example of this, uhm, for people with blind or low vision is this system called Seeing VR, which is essentially this, uhm, collection of enhancements, uhm, that will allow people to


[User] Please transcribe this text

[Assistant] Uhm, and again, where I'm really looking forward is how does this work for people of limited mobility? So uhm, I was really interested in this question of accessible by manual input and one of the participants actually has this quote, so uhm,


[User] Please transcribe this text

[Assistant] In talking about using two hands to perform actions in VR, they said, I'm not sure that that would be possible for me to do them simultaneously. If I had to do, I could do a one-handed, but if I had to do both hands at the same time, that's not going to work.


[User] Please transcribe this text

[Assistant] This is the question we asked, so I have the great intern, Yamagami, who's a


[User] Please transcribe this text

[Assistant] Uh, PhD candidate at University of Washington who was an intern with me last summer and we kind of explored some of these areas. So once again, as you know, how can we allow


[User] Please transcribe this text

[Assistant] So what we did is we started off with this kind of popular framework, the Giard framework for this asymmetric division of labor. So what this really tells us is that like, hey, we can really think about it as like, there's only really a few


[User] Please transcribe this text

[Assistant] So there's symmetric interface in which both hands do the same role at the same time. So you could think of that as like, um, jump roping, both hands are kind of like working.


[User] Please transcribe this text

[Assistant] Asymmetric coordinated is when both hands are working together on like a single object. So you can think of that as something like, you know, you're cutting an apple, one hand holds the apple while the other hand is kind of actually doing the slicing. And then you can think of asymmetric uncoordinated as two hands are just doing


[User] Please transcribe this text

[Assistant] So if you're really thinking about like, well, how can we create interaction techniques for this space? Well, you can maybe think about it along the spectrum of like, well, how much autonomy do you want the user to have? Right? So you could really think it's like, okay, if I have a spectrum between less autonomy to more autonomy, what can I do with that? So with less autonomy in terms of


[User] Please transcribe this text

[Assistant] The user actually having some input in what's going to happen. Maybe you say, hey, if the user encounters a bimanual task, we just automatically perform a task for them or we give them the option to automatically perform the task.


[User] Please transcribe this text

[Assistant] So that could be something like, you know, hey, you encounter a ladder, you could just automatically climb the ladder. We're not going to have you do the two-handed interaction piece. Similarly, if you want something that's more in the middle, you can maybe do something where you actually infer the output of the two hands given input from the one hand. So if I know what one hand


[User] Please transcribe this text

[Assistant] What one hand is doing, could I then infer what the other hand should be doing based off the input that I'm getting from the one hand? And then if you really want to give people a lot of autonomy, maybe you could do something like use an alternative input method. So maybe you could use a headset or eye gaze as something to compensate for not having the other hand.


[User] Please transcribe this text

[Assistant] Uh, so what you could do then is, you know, whatever designs or interaction techniques you come up with on a spectrum from less to more autonomy, then you can kind of overlay those with, you know, what's happening with these different types of interaction methods, so.


[User] Please transcribe this text

[Assistant] In this case, I'm going to go through an example of saying like, okay, well, if I have automatic


[User] Please transcribe this text

[Assistant] And frying or alternative, what does that look like for symmetric out of phase interactions?


[User] Please transcribe this text

[Assistant] So for example, if I wanted to infer the location of a virtual hand in a symmetric out of phase thing, perhaps what I'll do is I'll create a system in which the virtual hand kind of always just mimics the direction, speed, and distance of what the real hand is doing. So if the real hand performs some movement


[User] Please transcribe this text

[Assistant] And then similarly, maybe for the alternative input, you say things like, hey, well, the real hand is going to operate this, but let's say I want to use like the actual headset to


[User] Please transcribe this text

[Assistant] What she's doing is using kind of the location of the, the HMD, kind of like the gaze offset of the, or the head tracked offset of the HMD.


[User] Please transcribe this text

[Assistant] So this could be like a really great way to say like, hey, if you only have one hand available to you, great. We could just, you know, use some other different type of input device and use that as a way to then control what the other hand should be doing.


[User] Please transcribe this text

[Assistant] I'm going to skip that because I'm running low on time here and I want to get to questions and answers. So I just kind of want to talk about future opportunities that the part of the, uh, the


[User] Please transcribe this text

[Assistant] Accessible by design paper that I really have a chance to get into. So just really briefly, so that everyone's on the list for content accessibility. So this is really looking at ways to say like, hey, we don't really have agreed upon methods to make content accessible in VR. And we don't, so, you know, and there's trickle down effects to that. So we don't know what type of metadata is necessary. We're not sure what standards we should have. And then we're also not sure how we can allow.


[User] Please transcribe this text

[Assistant] So these are all things in terms of like, well how do we actually make the content


[User] Please transcribe this text

[Assistant] There's this other thing here about inclusive representations, which is really important. So this idea that, you know,


[User] Please transcribe this text

[Assistant] VR has a lot of embodied experiences and the people usually embody these types of virtual avatars. So what's the question then of how, how do people with disabilities want to be


[User] Please transcribe this text

[Assistant] So if you're a person who uses a wheelchair and doesn't walk, but you have to embody an avatar that does walk and has use of both hands and things like that, is that actually an immersive experience for that person? Does that actually create something that they find enjoyable or would they actually prefer different types of mappings, different types of representations?


[User] Please transcribe this text

[Assistant] Um, and then the last thing here is really about like application diversity. So VR provides these opportunities to kind of do different types of application types. So you could do things like virtual orientation and mobility training for people who are blind or low vision. You could like, you know, recreate a virtual city and have them actually, you know, practice navigating walking.


[User] Please transcribe this text

[Assistant] Really, you know, kind of what I wanted to talk today about and demonstrate was just that, you know, people with limited mobility are going to encounter a lot of accessibility barriers when interacting with VR systems and it's really important to really be mindful and think about


[User] Please transcribe this text

[Assistant] So that's it for me, so I'll be happy to answer any questions.


[User] Please transcribe this text

[Assistant] How can we take input devices that people already have, already have familiarity with, they have already experts in that, and how can we leverage those to the best, the biggest extent possible? So for example, there's people who use like quad sticks, like so people might be paralyzed from the neck down, who might use quad sticks, sip and puff devices, things like that. And the question would be, okay, well we have these kind of


[User] Please transcribe this text

[Assistant] How can we democratize the availability of input devices to the greatest extent possible?


[User] Please transcribe this text

[Assistant] Once we understand what the possible range of input devices are, how can we then start creating more accessible interaction techniques and things like that on top of those to take advantage of them to the best extent possible?


[User] Please transcribe this text

[Assistant] Yeah, so that's a great question. So uhm, so I haven't done a lot of explicit investigation on this. Mostly like I said before, the COVID pandemic has really hampered our ability to actually be able to


[User] Please transcribe this text

[Assistant] Go to, go to people with disabilities and actually understand their preferences and, you know, what things they do in real life. Um, but I will say like, for example, the difference, one of the differences is that, you know, in the virtual reality, virtual reality environment, the developers in some sense control everything.


[User] Please transcribe this text

[Assistant] Every pixel is a Sunway generated for some action. So we can have a lot more control over a virtual reality environment where as an augmented reality, there's a lot more, there's a lot of, there's a


[User] Please transcribe this text

[Assistant] All of these different ways to kind of like overcome that, right? We can make people's like arms stretch really far or we can like really just like expand the screen or rotate the environment around them. But if I'm here and there's really something over there that I need access to.


[User] Please transcribe this text

[Assistant] That's gonna be a problem. And I still think that there's this broader notion of spatial computing or spatial input in which, you know, if we're expecting


[User] Please transcribe this text

[Assistant] What if you're a person who doesn't have the fine motor control in your hands to be able to perform these interactions? Then what's the opportunity, right? Like, how do we then make those interactions accessible to you, even though the kind of default


[User] Please transcribe this text

[Assistant] This thing works really well for me. If I could use this thing like all the time for all these different situations, I would prefer to do that. So part of it is like understanding like hey, if a person uses a quad stick or a sip and puff device and we're just like,


[User] Please transcribe this text

[Assistant] Use that as like just a constraint and say like, hey, a person who's paralyzed from their neck down uses this device implement and they still want to be able to have a great VR experience. Like what does that look like? How, like, how can we really take this like device of one or user of one type of scenario and then really start to think about how to design those type of.


[User] Please transcribe this text

[Assistant] I think for example, if you're interested in working with solutions for people who are like blind or have low vision, I think there's still these like really open questions around like, how do you create like, uh, what is like the screen reader equivalent for like a VR environment? Like what's the voiceover that's going to exist when people start to think about


[User] Please transcribe this text

[Assistant] So I'm not, I wouldn't necessarily say these are like low hanging fruit in terms of like the, how you would actually get to solutions for these, but I think that in terms of just like the problem space, I think kind of like trying to identify how we can best leverage what people can use right now and then try to understand


[User] Please transcribe this text

[Assistant] And then I think once we kind of have that understanding, then I think we can actually start building on top of that and actually then start to think about different ways to


[User] Please transcribe this text

[Assistant] So there's a platform that they call JackDAC, which is essentially this hardware platform that allows people to be able to plug just any different type of input module into it. So the idea could be, for example, you know, how could you, how could we democratize what we think about a, what is a controller really? And if we think about as a controller, it's just like this.


[User] Please transcribe this text

[Assistant] Expand our role of what we think the input ecology can look like, uhm, to be more accessible and inclusive to people I think could be really powerful.


[User] Please transcribe this text

[Assistant] Yeah, I think they're both, I think they're both tricky. So I think like, so for example, for somebody's kind of like output kind of questions, you know, for example, with people with different sensory disabilities. So for example,


[User] Please transcribe this text

[Assistant] Uhm, if you take sound, for example, and one way we might think about, well, how do we make sound more accessible in a virtual reality context could be doing things like, oh, we can actually have these kind of like different types of uhm,


[User] Please transcribe this text

[Assistant] If a person's deaf or hard of hearing and we want to make the sound that's available in a VR environment more accessible to them, well, how can we take that input, that output channel and then transfer that to something that's


[User] Please transcribe this text

[Assistant] More tactile or haptic or how can we turn it into something that's more visual but not in a way that like


[User] Please transcribe this text

[Assistant] Is too overwhelming or overloading to the person on the screen and be able to do these different things.


[User] Please transcribe this text

[Assistant] How do you describe a complex virtual scene to someone who's blind? Like, what's the, what's the nature of that interaction? I think right now with 2D interfaces, we have this kind of more like scanning type of mental model of it where it's like, okay, you started kind of like the top of the document that you kind of


[User] Please transcribe this text

[Assistant] Go down or you go across lists or tables or things like that, but if I just have an entire virtual world ahead of me,


[User] Please transcribe this text

[Assistant] Uh, the people around them and how that's going to influence what they can do. So it's not something that I myself, I'm more interested in like the input side and just in terms of the research I do, but I know a lot of people are thinking about these kind of like more like.


[User] Please transcribe this text

[Assistant] Output questions that you ask and like for people with different sensory disabilities, what can we do in that space? So I think it's something that people are working on, but I think there's still a lot of open questions there.


[User] Please transcribe this text

[Assistant] When Mortimer spoke these words, I knew the case was ours.


[User] Please transcribe this text

[Assistant] Holmes sat up in his chair and his eyes showed he was very interested in the mystery.


[User] Please transcribe this text

[Assistant] Why did nobody else see this hound's footprints? he asked.


[User] Please transcribe this text

[Assistant] The footprints were about 20 meters from the body.


[User] Please transcribe this text

[Assistant] Nobody thought to look so far away, Mortimer replied.


[User] Please transcribe this text

[Assistant] Are there many sheepdogs on the moor? asked Holmes. Yes, there are.


[User] Please transcribe this text

[Assistant] But this was no sheepdog. I know what a sheepdog's feet look like.


[User] Please transcribe this text

[Assistant] These footprints were very large indeed. Enormous! Mortimer answered.


[User] Please transcribe this text

[Assistant] But whatever the animal was, it had not gone near Sir Charles, Holmes said. That's right.


[User] Please transcribe this text

[Assistant] What was the weather like on that night? Holmes asked.


[User] Please transcribe this text

[Assistant] It was wet and cold, although it wasn't raining.


[User] Please transcribe this text

[Assistant] Describe the path of trees that Sir Charles walked along to me.


[User] Please transcribe this text

[Assistant] It is the famous U Alley of Baskerville Hall. It is quite peaceful and beautiful.


[User] Please transcribe this text

[Assistant] There is a path between two long lines of yew trees. These trees were planted very close together. Down the middle is a path of small stones.


[User] Please transcribe this text

[Assistant] The path has grass on each side of it, said the doctor.


[User] Please transcribe this text

[Assistant] I understand that in one place there is a gate that leaves the path, said Holmes.


[User] Please transcribe this text

[Assistant] Yes, there is a small gate which leads to the moor.


[User] Please transcribe this text

[Assistant] Is there any other opening through the line of trees? No?


[User] Please transcribe this text

[Assistant] There is a way out through a summer house at the end of the path. Had Sir Charles reached the summer house? No.


[User] Please transcribe this text

[Assistant] He lay about fifty meters from it, said Mortimer.


[User] Please transcribe this text

[Assistant] Now, Dr. Mortimer, this is important.


[User] Please transcribe this text

[Assistant] You say that the footprints you saw were on the path and not on the grass.


[User] Please transcribe this text

[Assistant] No footprints could show on the grass, said Mortimer.


[User] Please transcribe this text

[Assistant] Another question. Was the gate to the moor closed? Yes, it was closed and locked. How high is the gate? asked Holmes. It is very small. So a person could climb over it.


[User] Please transcribe this text

[Assistant] Tell me about the footprints you saw by the moorgate.


[User] Please transcribe this text

[Assistant] Sir Charles seems to have stood there for five or ten minutes said Mortimer. I know that because the footprints were very deep.


[User] Please transcribe this text

[Assistant] Excellent, said Holmes. This man is a very good detective, Watson.


[User] Please transcribe this text

[Assistant] Sir Charles had left his footprints all over that little bit of the path where he was standing. I could not see any other prints. Sherlock Holmes smiled.


[User] Please transcribe this text

[Assistant] I like to look closely at these things myself, he said.


[User] Please transcribe this text

[Assistant] Dr. Mortimer, you should have come to me immediately. Mr. Holmes.


[User] Please transcribe this text

[Assistant] The best detective in the world cannot help with some things, the doctor said.


[User] Please transcribe this text

[Assistant] Before Sir Charles died, people have told me some very serious things.


[User] Please transcribe this text

[Assistant] Several people have seen an animal running on the moor at night. It looks like an enormous hound.


[User] Please transcribe this text

[Assistant] Everyone agrees that it was a huge creature, larger than any dog could be. And Mr. Holmes...


[User] Please transcribe this text

[Assistant] The animal shone with a strange light, like a ghost. I have questioned these people carefully. None of them are crazy or drink too much. They all tell exactly the same story.


[User] Please transcribe this text

[Assistant] Although these people have not seen the thing up close, it is exactly like the hellhound of the Baskerville story.


[User] Please transcribe this text

[Assistant] The people are very frightened and only the bravest man will cross the moor at night.


[User] Please transcribe this text

[Assistant] But surely, doctor, you must think that the footprints were made by a living animal, not a ghost.


[User] Please transcribe this text

[Assistant] When the hound first appeared two hundred and fifty years ago, it was real enough to tear out Sir Hugo's throat and kill him.


[User] Please transcribe this text

[Assistant] If you and the people of Devonshire really believe that Sir Charles's death was caused by a ghost, my detective work cannot help you, Holmes said seriously. Maybe this is true, said Mortimer.


[User] Please transcribe this text

[Assistant] But you can help by advising me what to do for Sir Henry Baskerville, even if you cannot help more. Dr. Mortimer looked at his watch.


[User] Please transcribe this text

[Assistant] He arrives in London on the train in little over an hour.


[User] Please transcribe this text

[Assistant] Sir Henry is now the head of the Baskerville family, asked Holmes.


[User] Please transcribe this text

[Assistant] Yes, said Dr. Mortimer. He is the last of the Baskervilles.


[User] Please transcribe this text

[Assistant] Why doesn't he just go to the family home? asked Holmes.


[User] Please transcribe this text

[Assistant] Because, Mr. Holmes, too many Baskervilles have died horrible deaths there.


[User] Please transcribe this text

[Assistant] No matter what the reason is, this is a plain fact.


[User] Please transcribe this text

[Assistant] Perhaps we would be putting Sir Henry in too much danger. But Sir Charles's good work must go on.


[User] Please transcribe this text

[Assistant] If it does not, then all the people on the Baskerville lands will be much poorer.


[User] Please transcribe this text

[Assistant] If the Baskerville family leaves the hall, the whole area will be ruined. I don't know what to do. That is why I came to you for advice. Holmes thought for a few moments. Then he said,


[User] Please transcribe this text

[Assistant] You think it is too dangerous for any Baskerville to live at the hall because of this terrible hound.


[User] Please transcribe this text

[Assistant] I think you should go and meet Sir Henry Baskerville at the train station. Do not tell him anything about our discussion. In 24 hours I will tell you what I think you should do.


[User] Please transcribe this text

[Assistant] At 10 o'clock tomorrow morning, I would like you to bring Sir Henry Baskerville here.


[User] Please transcribe this text

[Assistant] Dr. Mortimer got up from his chair and shook hands with both of us.


[User] Please transcribe this text

[Assistant] As he was leaving the room, Holmes said, One more question, Dr. Mortimer.


[User] Please transcribe this text

[Assistant] You said that before Sir Charles's death there were other people who saw this strange animal on the moor. Yes, said Mortimer.


[User] Please transcribe this text

[Assistant] Since Sir Charles' death, has anyone seen the animal?


[User] Please transcribe this text

[Assistant] I do not know of anyone who has seen it. Thank you, Dr. Mortimer. Good morning.


[User] Please transcribe this text

[Assistant] After Mortimer had left us, Holmes sat down in his chair.


[User] Please transcribe this text

[Assistant] He looked pleased and immediately asked for his pipe.


[User] Please transcribe this text

[Assistant] He always acted this way when a case interested him.


[User] Please transcribe this text

[Assistant] Have you solved the mystery? I asked him, smiling. It is hard to say.


[User] Please transcribe this text

[Assistant] Take for example the change in the footprints.


[User] Please transcribe this text

[Assistant] Did Sir Charles walk on his toes down the path of trees? Why would he do that? That would be ridiculous. The truth is he was running. Running for his life. He ran until his heart stopped from fear, Watson. He was frightened to death. What was he running from, I asked?


[User] Please transcribe this text

[Assistant] That is the difficult question, said Holmes.


[User] Please transcribe this text

[Assistant] I think he was frightened by something and began to run.


[User] Please transcribe this text

[Assistant] He was so scared that he didn't know what he was doing.


[User] Please transcribe this text

[Assistant] That explains why he ran away from the house instead of towards it, where he would be safe. Then there is another question. Who was he waiting for that night?


[User] Please transcribe this text

[Assistant] And why would he meet this person in the path of trees, the yew alley, and not in the house like a normal person? You think he was waiting for someone? Sir Charles was old and rather sick.


[User] Please transcribe this text

[Assistant] We know why he took a walk each evening. It was for his health.


[User] Please transcribe this text

[Assistant] But why did he stand in the cold on wet ground for five or ten minutes?


[User] Please transcribe this text

[Assistant] Dr. Mortimer cleverly saw the cigar ash on the ground, so we know how long Sir Charles stood there.


[User] Please transcribe this text

[Assistant] And Watson, we know that he was deathly afraid of being outside at night because of his belief in the Hound of the Baskervilles.


[User] Please transcribe this text

[Assistant] So it is extremely unlikely that he waited at the moor gate every evening. He must have been there for a special reason.


[User] Please transcribe this text

[Assistant] I am beginning to understand some things, Watson.


[User] Please transcribe this text

[Assistant] Now he needed more details of the case to help him.


[User] Please transcribe this text

[Assistant] Chapter 3 The Death of Sir Charles Baskerville


[User] Please transcribe this text

[Assistant] When Dr. Mortimer had finished reading this strange story, he looked across at Sherlock Holmes.


[User] Please transcribe this text

[Assistant] I was surprised to see that Holmes looked bored, but also looked as if he thought the letter was funny.


[User] Please transcribe this text

[Assistant] Do you think this is an interesting story, Mr. Holmes? asked Dr. Mortimer. Holmes laughed. My dear sir!


[User] Please transcribe this text

[Assistant] Only a storyteller or a child would find this story interesting. Dr. Mortimer did not answer him.


[User] Please transcribe this text

[Assistant] Instead he took a newspaper from another pocket in his coat.


[User] Please transcribe this text

[Assistant] Now, Mr. Holmes, let me read you something else.


[User] Please transcribe this text

[Assistant] This was written only three months ago. It is from the Devonshire County newspaper. And it is about the death of Sir Charles Baskerville. Holmes looked more interested. Dr. Mortimer began to read.


[User] Please transcribe this text

[Assistant] People in Devonshire County feel much sadness at the terrible death of Sir Charles Baskerville. Sir Charles was an extremely good man.


[User] Please transcribe this text

[Assistant] Although he had lived at Baskerville Hall for only two years, everyone in the area liked him very much.


[User] Please transcribe this text

[Assistant] Because of the great help he gave to the people of Devonshire.


[User] Please transcribe this text

[Assistant] Sir Charles had lived in a foreign land and made his money there.


[User] Please transcribe this text

[Assistant] He returned to England to repair Baskerville Hall and the villages around it.


[User] Please transcribe this text

[Assistant] These places were old and in very bad condition, but with Sir Charles's help, they became beautiful once again.


[User] Please transcribe this text

[Assistant] He was a friendly and kind man who gave much help to the poor.


[User] Please transcribe this text

[Assistant] It is unfortunate that the police report on Sir Charles' death does not tell us everything that happened.


[User] Please transcribe this text

[Assistant] However, we can be sure that he was not murdered.


[User] Please transcribe this text

[Assistant] The strange stories the village people tell about his death are not true.


[User] Please transcribe this text

[Assistant] His friend and doctor, Dr. James Mortimer,


[User] Please transcribe this text

[Assistant] Sir Charles went for a walk in the gardens of Baskerville Hall. His favorite place to walk was down a path between two rows of trees.


[User] Please transcribe this text

[Assistant] This path is the famous U Alley of Baskerville Hall. On the night of June 4th,


[User] Please transcribe this text

[Assistant] He went out to walk and to smoke his cigar as he often did. Sir Charles was going to London the next morning.


[User] Please transcribe this text

[Assistant] And Barrymore, his head servant, was inside the house packing his suitcases.


[User] Please transcribe this text

[Assistant] But by midnight Barrymore realized that Sir Charles had not returned. So he went to look for him. He saw that the door of the house was open. That day had been rainy and wet.


[User] Please transcribe this text

[Assistant] So Barrymore could see the footprints left by Sir Charles's shoes as he had walked down the path of trees.


[User] Please transcribe this text

[Assistant] Halfway down this path is a gate, which leads to the moor.


[User] Please transcribe this text

[Assistant] There were signs that Sir Charles had stood at this gate for some time before continuing his walk.


[User] Please transcribe this text

[Assistant] Barrymore followed the footprints to the end of the path, and there he found Sir Charles's body.


[User] Please transcribe this text

[Assistant] Barrymore told the police an interesting detail about these footprints.


[User] Please transcribe this text

[Assistant] He said that they changed between the gate leading to the moor and the end of the path of trees.


[User] Please transcribe this text

[Assistant] As far as the Moore gate, all the footprints were normal.


[User] Please transcribe this text

[Assistant] But Barrymore saw only toe prints in the ground after Sir Charles passed this gate.


[User] Please transcribe this text

[Assistant] Barrymore thought that Sir Charles had walked on his toes.


[User] Please transcribe this text

[Assistant] A man called Murphy, who buys and sells horses, was close to Baskerville Hall when Sir Charles died.


[User] Please transcribe this text

[Assistant] This man is known in the town for his drunkenness


[User] Please transcribe this text

[Assistant] And that night he had been drinking a lot of beer as usual. However, he swore that he heard someone shouting, as if they were calling for help.


[User] Please transcribe this text

[Assistant] The shouts came from the direction of Baskerville Hall.


[User] Please transcribe this text

[Assistant] Doctor James Mortimer was asked to come and look at Sir Charles' body. After he examined the body, Doctor Mortimer decided that Sir Charles had not been murdered.


[User] Please transcribe this text

[Assistant] However, Doctor Mortimer told police that Sir Charles had a look on his face of surprise and fear.


[User] Please transcribe this text

[Assistant] But Dr. Mortimer decided this was normal.


[User] Please transcribe this text

[Assistant] Sir Charles's weak heart had failed suddenly, and this had caused his death.


[User] Please transcribe this text

[Assistant] Now that Sir Charles is dead, there will be a new head of the Baskerville family. He will own Baskerville Hall.


[User] Please transcribe this text

[Assistant] The people of Devon sure hope very much that this person will move into Baskerville Hall as soon as possible.


[User] Please transcribe this text

[Assistant] The people would like Sir Charles's good work in this area to continue.


[User] Please transcribe this text

[Assistant] The new head of the Baskerville family is a man named Sir Henry Baskerville.


[User] Please transcribe this text

[Assistant] At this time, we do not know if Sir Henry is still alive or if the lawyers can find him.


[User] Please transcribe this text

[Assistant] He is the son of Sir Charles Baskerville's younger brother, who died some years ago.


[User] Please transcribe this text

[Assistant] This young man has been living in the United States of America.


[User] Please transcribe this text

[Assistant] At this time, the Baskerville lawyers are trying to find him to tell him of his inheritance. The article was finished.


[User] Please transcribe this text

[Assistant] Dr. Mortimer put the newspaper back into his pocket.


[User] Please transcribe this text

[Assistant] Well, Mr. Holmes, those are the facts about the death of Sir Charles. Everyone knows these things, he said.


[User] Please transcribe this text

[Assistant] Thank you for reading me this paper, Dr. Mortimer.


[User] Please transcribe this text

[Assistant] It is very interesting to me, om said.


[User] Please transcribe this text

[Assistant] I read about Sir Charles' death in our London newspaper, but I did not know the details.


[User] Please transcribe this text

[Assistant] The newspaper gives us the facts that everybody knows.


[User] Please transcribe this text

[Assistant] Now I want you to tell me all the other facts that only you know.


[User] Please transcribe this text

[Assistant] I haven't told anyone what I really think, he said. I am a man of science, as you know.


[User] Please transcribe this text

[Assistant] I have always believed that there is a good reason for everything.


[User] Please transcribe this text

[Assistant] And certainly, I did not want to say anything that might stop Sir Henry from going to Baskerville Hall.


[User] Please transcribe this text

[Assistant] But I will tell you some things that were not in the report.


[User] Please transcribe this text

[Assistant] In the months before his death, Dr. Mortimer went on, Sir Charles was extremely worried and nervous.


[User] Please transcribe this text

[Assistant] His body was near to breaking down, I think. You see...


[User] Please transcribe this text

[Assistant] No matter what I told him, he believed the story of the Hound of the Baskervilles.


[User] Please transcribe this text

[Assistant] He was certain that the hound would find and kill him, as it had killed Sir Hugo.


[User] Please transcribe this text

[Assistant] He believed the Baskerville family was cursed because of Sir Hugo's evil acts. He would not go out on the moor at night.


[User] Please transcribe this text

[Assistant] Often he would ask me if I had heard the howl of a hound on the moor at night.


[User] Please transcribe this text

[Assistant] He was always very excited and nervous when he asked this question.


[User] Please transcribe this text

[Assistant] About three weeks before Sir Charles died, I went to Baskerville Hall.


[User] Please transcribe this text

[Assistant] As I drove up to the house, Sir Charles was standing at the door as if he was meeting me.


[User] Please transcribe this text

[Assistant] I went up to him, but he did not say hello. There was a look of horror on his face.


[User] Please transcribe this text

[Assistant] And he was staring at something behind me.


[User] Please transcribe this text

[Assistant] I turned around quickly and saw something moving between the trees. It looked like a small black cow.


[User] Please transcribe this text

[Assistant] Sir Charles was so frightened that he looked as if he would die that minute.


[User] Please transcribe this text

[Assistant] I went to look for the animal to make him feel better. However, it had disappeared. I stayed with Sir Charles all that evening.


[User] Please transcribe this text

[Assistant] It was then he gave me the old papers I have just read to you.


[User] Please transcribe this text

[Assistant] Mr. Holmes, Sir Charles behavior that evening may be important when you think about what happened on the night of his death.


[User] Please transcribe this text

[Assistant] When Barrymore, the head servant, found Sir Charles's body, he sent someone to get me. I examined Sir Charles's body. No one had killed him.


[User] Please transcribe this text

[Assistant] All the things in the newspaper I have just read to you are true. But Barrymore made a mistake.


[User] Please transcribe this text

[Assistant] He said that there were no other footprints on the ground around Sir Charles's body. He did not notice any. But I did. These prints were not close to the body, but they were large and very clear.


[User] Please transcribe this text

[Assistant] Were they a man's or a woman's footprints? asked Holmes. Doctor Mortimer looked at us for a moment.


[User] Please transcribe this text

[Assistant] Mr. Holmes, they were the footprints of a huge hound.


[User] Please transcribe this text

[Assistant] The she-wolf. Written by Jack London. One day, while the she-wolf was out hunting,


[User] Please transcribe this text

[Assistant] The gray wolf cub went to the mouth of the cave. He looked out at the world for the first time. The light was very bright, but he saw trees in the river. He saw the mountain and the sky above it. At first he was afraid and the hairs on his back stood up, but nothing happened. And after a time he moved outside.


[User] Please transcribe this text

[Assistant] Over and over he went until at last he stopped. For a minute or two, the grey cub was afraid to move. He sat and looked around him, then he got up and began walking. He walked into things or fell over them, and he heard his feet on stones and his head on trees. He came to his dream and looked into the river.


[User] Please transcribe this text

[Assistant] Have you ever wondered why we vote on Tuesdays? Everyone knows election day is always Tuesday, but we wondered why. And it's not just newscasters.


[User] Please transcribe this text

[Assistant] See, look at those ducks. What I wouldn't give for one of those fat ones. It's been almost two weeks since I've had a solid meal.


[User] Please transcribe this text

[Assistant] I've starved. All I do is stand in front of windows like this and drool. And all night long I have food nightmares. Oh, last night's dream was a pip.


[User] Please transcribe this text

[Assistant] I dreamed that it was raining orange sauce, and thousands of roast duck carrying sliced pineapple for umbrellas walked by. And then one big fat duck slipped in the thick French dressing. And there was a pile up.


[User] Please transcribe this text

[Assistant] What about the growners? You know, the fellows that croon the words to the music?


[User] Please transcribe this text

[Assistant] I haven't. I'm on a diet. The barbie socks make us lean and hungry.


[User] Please transcribe this text

[Assistant] I gotta hand it to him though. 20 minutes ago he was a crude exhibitionist. Just a little bit of howling and you joined the pack.


[User] Please transcribe this text

[Assistant] Oh, sure. My name is Buffalo Bill. Would you like to dance?


[User] Please transcribe this text

[Assistant] Well, not just any woman, Kitten. But I mean, it'd have to be somebody pretty special. Like you.


[User] Please transcribe this text

[Assistant] You're wonderful. If you don't believe me, just look in your mirror. Somebody else agreed with me this evening, too. That look in Barry Clayton's eye, old brother.


[User] Please transcribe this text

[Assistant] Oh, kiddin', be reasonable. After all, you've gotta eat sometime, somewhere, and, well, it might as well be with me.


[User] Please transcribe this text

[Assistant] Well, I'd be surprised if you weren't. But that's no reason to starve. After all, don't engage women eat. I can only understand one thing.


[User] Please transcribe this text

[Assistant] And could it be a taxi you'll be wanting, ma'am? Oh, it's too nice a night for a taxi anyway. Let's walk. I know a wonderful restaurant right around the corner.


[User] Please transcribe this text

[Assistant] Don't gaze at those stars that shine so brightly. Just give them the brush off. You know, the once over lightly.


[User] Please transcribe this text

[Assistant] Hello Earl, how's everything? Fine. I'll miss you, Mr. Williams.


[User] Please transcribe this text

[Assistant] Well, I didn't expect to see you here tonight, Jimmy. Oh, it was Miss Hilliard's idea. You know, that trumpet makes my ears burn. Well, I always knew you were a good sport, man.


[User] Please transcribe this text

[Assistant] I don't know, but it sounds like dirt to me. Wait right here a minute, will you kid? And I'll be right back.


[User] Please transcribe this text

[Assistant] Oh, what I know is enough. It all adds up, every little trick. Sitting with your back toward him that night so that he'd see that you were with me.


[User] Please transcribe this text

[Assistant] That's great, pal. Just what Kit and I were talking about. I'm tickled to death. She knew what she wanted, and she knew how to get it. Why don't you jam a couple of choruses at a wedding march? You might as well double-cross Mendelssohn, too.


[User] Please transcribe this text

[Assistant] And AI can also help practitioners make more informed decisions. But these benefits do not come without risks.


[User] Please transcribe this text

[Assistant] They lack many of the common sense or pro-social reasoning abilities that humans may take for granted. So they can also fail in ways that a human would not.


[User] Please transcribe this text

[Assistant] A long line of literature suggests that in many domains, humans and AI systems have complementary abilities and limitations. While the gap may narrow over time,


[User] Please transcribe this text

[Assistant] In my research, I explore how we might design systems that draw upon the complementary strengths and


[User] Please transcribe this text

[Assistant] This can improve decision making compared with either humans or AI making decisions alone.


[User] Please transcribe this text

[Assistant] And so for example, successful partnerships have been demonstrated between human experts and AI systems in radiology as well as other domains.


[User] Please transcribe this text

[Assistant] So why does some human AI partnerships succeed while others fail? Oftentimes, it's a simple lack of human-centered design.


[User] Please transcribe this text

[Assistant] Uh, so for example, many null or negative results in this area have come from studies on crowdsourcing platforms like Amazon's Mechanical Turk.


[User] Please transcribe this text

[Assistant] Where crowd workers might be assigned to assist an AI on tasks that truly require expert level domain knowledge, which we wouldn't expect an average crowd worker to have.


[User] Please transcribe this text

[Assistant] In my lab, as well as based on the conversations I've had today, uhm, among various people at Stanford currently.


[User] Please transcribe this text

[Assistant] We then work to engage these stakeholders throughout the entire design and development life cycle for new technologies, ranging from initial need finding and ideation to design decisions about the underlying algorithms and


[User] Please transcribe this text

[Assistant] For example, to help stakeholders understand and shape algorithmic elements of a new systems design.


[User] Please transcribe this text

[Assistant] First, I've explored how to co-design more effective human AI partnerships. And second,


[User] Please transcribe this text

[Assistant] And for the sake of time, today I'll share just one prior project under each of these broad themes.


[User] Please transcribe this text

[Assistant] In which students work at their own pace while the teacher's role is transformed from that of a lecturer at the front of the class to more of an active facilitator of students learning.


[User] Please transcribe this text

[Assistant] To enable more personalized instruction, we're seeing greater adoption of AI-based educational software.


[User] Please transcribe this text

[Assistant] So as an example, intelligent tutoring systems provide students


[User] Please transcribe this text

[Assistant] AI tutors and human teachers really work together to help students learn.


[User] Please transcribe this text

[Assistant] But the role of the teacher in these AI-supported classrooms has historically been understudied and this partnership arguably isn't working as well as it could.


[User] Please transcribe this text

[Assistant] So for instance, it's difficult for teachers to monitor classes of 15 to 30 students who are all working at their own pace with each student potentially working on entirely different material within the software.


[User] Please transcribe this text

[Assistant] Many of the teachers I've worked with have noted feeling left out of the loop in their own classrooms during class sessions using these AI tutors.


[User] Please transcribe this text

[Assistant] Yeah, absolutely. So a range of software, um, Carnegie Learning, uh, Assistments, um, in some cases, Alex.


[User] Please transcribe this text

[Assistant] The majority of the teachers, although only a slight majority, were using Carnegie Learning, so cognitive tutors.


[User] Please transcribe this text

[Assistant] Various researchers in this area had begun pointing out that the choices of analytics, uhm, in existing teacher facing tools.


[User] Please transcribe this text

[Assistant] Including field observations in K-12 classrooms, directed storytelling to understand teachers' past experiences using AI in the classroom, as well as card sorting and speed dating studies. And so I won't go into, uhm,


[User] Please transcribe this text

[Assistant] If teachers could have any information they wanted about their students in real time, what would this be?


[User] Please transcribe this text

[Assistant] We asked teachers what superpowers they would want in these AI supported classrooms in order to help them better do their jobs.


[User] Please transcribe this text

[Assistant] Among several other superpowers, we found that teachers consistently wanted to be able to instantly see when students were unproductively struggling and might truly need a teacher's help.


[User] Please transcribe this text

[Assistant] And also to see students thought processes unfolding in real time as they work through math problems, for instance.


[User] Please transcribe this text

[Assistant] Since so much of the rich information the teachers gain is from actively looking at their students. So their faces, their body language.


[User] Please transcribe this text

[Assistant] So for example, this early concept sketch shows the AI software essentially raising a student's hand for them when it detects that the student currently seems to need help, perhaps for a, uh,


[User] Please transcribe this text

[Assistant] For these sorts of teacher analytics tools to have a positive impact in the classroom.


[User] Please transcribe this text

[Assistant] So as one example, although most teachers believe that they prioritize students in their classes who need their help the most while using the software,


[User] Please transcribe this text

[Assistant] So moving towards designing an actual system, uh, based on these insights,


[User] Please transcribe this text

[Assistant] And in this process, we also decided to explore this idea of teacher smart glasses further.


[User] Please transcribe this text

[Assistant] So the use of wearable devices like smart glasses for real time teacher support, uhm, at this point was only just beginning to be explored.


[User] Please transcribe this text

[Assistant] And prior work had rarely made the link from, uhm, choices of analytics to


[User] Please transcribe this text

[Assistant] So to explore some of these questions further, we conducted an iterative series of design and prototyping studies.


[User] Please transcribe this text

[Assistant] Working with teachers in their own classrooms at their schools.


[User] Please transcribe this text

[Assistant] In low fidelity experience prototyping sessions, teachers would put on plastic eyeglass frames.


[User] Please transcribe this text

[Assistant] And would look at a scene of a classroom on a computer screen. When teachers put the glasses on, we would Wizard of Oz Analytics


[User] Please transcribe this text

[Assistant] Uh, overlaid on the classroom image. And these analytics were based directly on the sorts of superpowers that teachers had requested previously.


[User] Please transcribe this text

[Assistant] So whenever a teacher would generate an idea for new information, we would press them to explain how they envisioned actually using that information.


[User] Please transcribe this text

[Assistant] And we found that asking that question would often lead teachers to refine their ideas, which were then incorporated into the version shown to the next teacher.


[User] Please transcribe this text

[Assistant] After encouraging feedback from these prototyping sessions, we next moved to higher fidelity experience prototyping using the Microsoft HoloLens.


[User] Please transcribe this text

[Assistant] During each study session, we placed holograms throughout a teacher's physical classroom and we were able to modify these holograms in near real time based on a teacher's live design feedback.


[User] Please transcribe this text

[Assistant] By editing images that were synced with the app, uh, while the teachers viewed these through the HoloLens.


[User] Please transcribe this text

[Assistant] By this point in the process, uh, out of a very large initial design space,


[User] Please transcribe this text

[Assistant] We'd converged to a compact set of real-time indicators that appeared to be most useful to teachers.


[User] Please transcribe this text

[Assistant] And these included what teachers believed were key moments for them to step in and intervene.


[User] Please transcribe this text

[Assistant] We also settled on two displays that would pop up when a teacher clicks on an individual student's indicator.


[User] Please transcribe this text

[Assistant] So for example, if the AI tutor diagnoses that a student is struggling with particular skills, the teacher's classes would display both the skill


[User] Please transcribe this text

[Assistant] And these concrete examples were really important to teachers. So teachers would often use them to second-guess the AI tutor's diagnoses.


[User] Please transcribe this text

[Assistant] So we next began developing a fully functional prototype called LiMelo.


[User] Please transcribe this text

[Assistant] Now, analytics would be streamed live from AI tutors to the teachers' glasses.


[User] Please transcribe this text

[Assistant] And here you can see, uh, Lumilo in action with an off task student.


[User] Please transcribe this text

[Assistant] Before bringing this prototype into actual K-12 classrooms, many open design questions remained.


[User] Please transcribe this text

[Assistant] But also in helping to shape the specific algorithms that were used to drive Lumilow's analytics.


[User] Please transcribe this text

[Assistant] Based on replays of actual data from field contexts.


[User] Please transcribe this text

[Assistant] In order to help make the consequences of particular algorithm design decisions more tangible to stakeholders, even if they know very little about AI.


[User] Please transcribe this text

[Assistant] During a session, a human wizard makes live changes to algorithmic elements of a system's design based on stakeholder feedback.


[User] Please transcribe this text

[Assistant] And there were no students present, but instead we simulated a class session from historical data so that if the teacher looked at the computer screens, it looked as if there were students working, although go students in this case.


[User] Please transcribe this text

[Assistant] And teachers could then experience the consequences of particular design decisions they'd asked for while wearing Lumilo and moving throughout the lab space.


[User] Please transcribe this text

[Assistant] If you're interested in seeing some actual clips from some of these sessions, you can find them at this link.


[User] Please transcribe this text

[Assistant] Replay enactments is one response to recent calls in the HCI and design communities for new kinds of prototyping methods for data-driven AI systems.


[User] Please transcribe this text

[Assistant] By using authentic data and algorithms, replay enactments can help reveal critical issues that conventional Wizard of Oz studies can't surface. For example,


[User] Please transcribe this text

[Assistant] The no glasses condition was business as usual for one of these AI supported classrooms.


[User] Please transcribe this text

[Assistant] Where teachers would circulate throughout the room without any form of support or augmentation.


[User] Please transcribe this text

[Assistant] Uh, part of the benefit of a teacher's use of Lumilo was due to their use of the glasses, even without any advanced analytics.


[User] Please transcribe this text

[Assistant] Uh, I looked at how teachers allocation of time across students are varying prior knowledge.


[User] Please transcribe this text

[Assistant] And I found that compared with the two other conditions, when teachers had access to real-time analytics about their students, they tended to spend much more of their time working with students who had come in with lower prior knowledge.


[User] Please transcribe this text

[Assistant] In turn, I found that providing teachers with real-time analytics


[User] Please transcribe this text

[Assistant] Serve to shrink achievement gaps within the classroom compared with business as usual for an AI supported classroom.


[User] Please transcribe this text

[Assistant] For students in both the glasses and no glasses conditions.


[User] Please transcribe this text

[Assistant] Students coming in with higher prior ability tend to benefit more from working with the tutoring software.


[User] Please transcribe this text

[Assistant] However, for students in the glasses plus analytics condition, the post-test by pre-test curve was flatter than in the other two conditions.


[User] Please transcribe this text

[Assistant] Meaning that when teachers had access to real-time analytics about their students, lower pretest students learned more.


[User] Please transcribe this text

[Assistant] I looked at the very same student behaviors that Lee Maloe tracks.


[User] Please transcribe this text

[Assistant] So a plausible explanation for this is that, uh, when students believe that their teacher has magic eyeglasses that let them see everything they're doing in the software, the frequency of the very behaviors that Lumilo was designed to detect goes down.


[User] Please transcribe this text

[Assistant] So one key takeaway of this is that future studies on teacher analytics tools like Lumilow should try to tease apart the effects of the analytics themselves from other potential impacts of a student knowing that they're being monitored, as we did here.


[User] Please transcribe this text

[Assistant] I found that when teachers and AI work together using Lumilo, students learn significantly more than they would working with an AI tutoring system alone.


[User] Please transcribe this text

[Assistant] Presenting teachers with real time analytics about student learning, metacognition, and behavior at a glance.


[User] Please transcribe this text

[Assistant] Served as a sort of equalizing force in the classroom, helping to narrow the gap in learning outcomes between students who were coming in with higher versus lower prior knowledge.


[User] Please transcribe this text

[Assistant] And although there's been a lot of interest in teacher analytics tools, this is actually the first experimental study to demonstrate that real time teacher analytics can enhance students learning outcomes, which is really exciting.


[User] Please transcribe this text

[Assistant] So Lumilo is an example of a successful human AI partnership, uh, achieved through co-design with practitioners in a real world context.


[User] Please transcribe this text

[Assistant] Supporting humans and AI systems in drawing upon each other's complementary strengths to make fairer decisions.


[User] Please transcribe this text

[Assistant] I've talked a little bit about how AI can help humans make fair decisions.


[User] Please transcribe this text

[Assistant] For example, Lumilow helped teachers distribute their time more equitably across students.


[User] Please transcribe this text

[Assistant] So for instance, real-time analytics in a teacher tool.


[User] Please transcribe this text

[Assistant] Might have greater accuracy for some populations of students, while producing misleading outputs for students from minority groups. Or an automated essay scoring system.


[User] Please transcribe this text

[Assistant] So next, I'll share some work I've done towards helping humans improve fairness in AI systems.


[User] Please transcribe this text

[Assistant] This was a collaboration with a group of machine learning researchers at Microsoft Research investigating industry product teams needs for support in creating fairer machine learning systems.


[User] Please transcribe this text

[Assistant] By unfairness in machine learning, I mean very broadly cases where systems favor certain individuals or groups over others in systemic, undesirable ways. So we've already seen some examples in education.


[User] Please transcribe this text

[Assistant] And beyond these calls to action, a lot of machine learning research has focused on the development of algorithmic methods to


[User] Please transcribe this text

[Assistant] Where developers can input their data and models and the toolkit will generate a bias report based on fairness metrics that exist in the literature.


[User] Please transcribe this text

[Assistant] So these toolkits take existing algorithmic fairness methods from machine learning and try to push them into use in the real world.


[User] Please transcribe this text

[Assistant] But if we want real world adoption of fairness research, it's crucial that this work is informed by an understanding of actual needs for support of the practitioners who will ultimately use them.


[User] Please transcribe this text

[Assistant] So in this work, we conducted the first systematic investigation in the literature.


[User] Please transcribe this text

[Assistant] of industry product teams needs for support in developing fairer machine learning systems.


[User] Please transcribe this text

[Assistant] We wanted to investigate what went wrong in the kinds of disaster stories that I mentioned earlier.


[User] Please transcribe this text

[Assistant] So in some sense, they had every reason not to talk openly with us.


[User] Please transcribe this text

[Assistant] Key to developing trust in this context, just as it was in my prior work with K-12 teachers.


[User] Please transcribe this text

[Assistant] For each major decision point, we asked participants to recall specific challenges they'd faced and to describe how they'd handled them at the time.


[User] Please transcribe this text

[Assistant] Who work on ML based products in a range of roles.


[User] Please transcribe this text

[Assistant] So our, our meta takeaway from this work was that there were indeed major disconnects between the research literature on fairness and machine learning and industry teams' actual needs for support.


[User] Please transcribe this text

[Assistant] So given limited time, I'll, uh, summarize just a few of the themes that emerge from this work.


[User] Please transcribe this text

[Assistant] But rather that fair machine learning research addressed different problems than the ones they were facing.


[User] Please transcribe this text

[Assistant] As one example of this, although most existing FairML methods presume access to demographic data at an individual level,


[User] Please transcribe this text

[Assistant] Several of the teams we spoke with are only able to collect demographics at coarser levels, such as neighborhood or organizational level statistics. And this is for legal reasons in some cases.


[User] Please transcribe this text

[Assistant] In contrast, industry practitioners often have some control over their datasets, especially at larger companies.


[User] Please transcribe this text

[Assistant] And this is the first place they're likely to turn when attempting to address fairness issues.


[User] Please transcribe this text

[Assistant] African American students rarely receive high scores from teachers. And in turn, their models end up replicating these biases in subtle ways.


[User] Please transcribe this text

[Assistant] As a third major theme, interviewees often express anxieties about their own blind spots as individuals and as teams.


[User] Please transcribe this text

[Assistant] So, for example, the fair machine learning literature will typically assume that the subpopulations we want to be fair towards are known or even defined by law.


[User] Please transcribe this text

[Assistant] And express needs for support in identifying which subpopulations to consider in the first place.


[User] Please transcribe this text

[Assistant] Some teams reported having a practice of getting together with a whiteboard and trying to imagine everything that could possibly go wrong with a particular system so that they could make sure to monitor for those issues in advance.


[User] Please transcribe this text

[Assistant] So for these reasons, participants noted that it would be nice to somehow pool knowledge of potential pitfalls across teams with different cultural backgrounds who may have complementary knowledge as well as complementary blind spots.


[User] Please transcribe this text

[Assistant] And one way to facilitate this might be to create mechanisms for greater sharing of test sets across organizational boundaries, which encode nuanced knowledge that no single team is likely to have.


[User] Please transcribe this text

[Assistant] The team received many complaints from non-US-based users, since the system seemed to work well for US celebrities while misidentifying or misgendering celebrities from their countries.


[User] Please transcribe this text

[Assistant] And as a result, they often made errors when curating the datasets.


[User] Please transcribe this text

[Assistant] One promising direction for future research in this space may be to support teams in the ad hoc recruitment of diverse team external human experts for particular tasks.


[User] Please transcribe this text

[Assistant] Where in some cases here, relevant expertise may simply mean being immersed in a particular cultural context at a particular time.


[User] Please transcribe this text

[Assistant] So, while I was only able to summarize a handful of findings from this work today, many more are covered in our CHI paper.


[User] Please transcribe this text

[Assistant] When we started this project, participatory approaches to work on fairness in machine learning were rare.


[User] Please transcribe this text

[Assistant] The practitioners we worked with expressed appreciation for having their voices heard rather than simply having researchers and the media critique their products from the outside.


[User] Please transcribe this text

[Assistant] Working alongside product teams and really engaging with the particulars of specific products can raise interesting design and technical challenges.


[User] Please transcribe this text

[Assistant] And in turn, this can productively guide more basic research and theoretical work in machine learning.


[User] Please transcribe this text

[Assistant] Uh, building on our findings from this work, we developed a research agenda for future collaborations between HCI and machine learning researchers.


[User] Please transcribe this text

[Assistant] Uhm, this research agenda has also directly informed a lot of the ongoing work in my group.


[User] Please transcribe this text

[Assistant] So on that note, before we open for discussion, I thought I'd kind of briefly share some of the directions we're working on currently.


[User] Please transcribe this text

[Assistant] Uh, focuses broadly on studying and designing new forms of human AI collaboration.


[User] Please transcribe this text

[Assistant] Uhm, in particular, a lot of our projects focus on care-based social and creative forms of work.


[User] Please transcribe this text

[Assistant] We're interested in helping humans and AI systems better assess, inform, and potentially contest or disagree with each other's decisions. And to do this work,


[User] Please transcribe this text

[Assistant] We're working across various real world contexts where AI systems are already being used to guide really high stakes and impactful positions.


[User] Please transcribe this text

[Assistant] Such as social work, education, mental healthcare, and software design and development.


[User] Please transcribe this text

[Assistant] We're particularly interested in designing tools to empower workers in contexts where AI has the potential to help, but which fundamentally resists full automation. And these include many care-based professions.


[User] Please transcribe this text

[Assistant] Where human empathy and relationship building are central. But despite resistance to automation, care-based labor remains chronically undervalued.


[User] Please transcribe this text

[Assistant] We can see this with underpaid and overworked social workers, school teachers in K-12, and caregivers for children, the elderly, and people with disabilities.


[User] Please transcribe this text

[Assistant] Uh, one current project we're working on under this theme focuses on co-designing algorithmic systems and platforms to empower in-home caregivers, which is currently one of the fastest growing segments of the American workforce.


[User] Please transcribe this text

[Assistant] Uhm, increasingly we're seeing companies introducing new algorithmic systems to transform what in-home care work looks like. But many of these systems tend to be designed in ways that


[User] Please transcribe this text

[Assistant] Center the needs of clients and employers while further disempowering the caregivers themselves.


[User] Please transcribe this text

[Assistant] So in this project we're working with caregivers to co-design new worker-centered prototypes.


[User] Please transcribe this text

[Assistant] Another set of projects we're working on focuses on the design and empirical evaluation of new tools to help machine learning practitioners assess and address unfairness in the systems they develop.


[User] Please transcribe this text

[Assistant] So following up on the research agenda that I mentioned previously.


[User] Please transcribe this text

[Assistant] For example, in one ongoing project, we're designing interventions to address problems rooted in organizational and team behavior at companies.


[User] Please transcribe this text

[Assistant] So these toolkits are commonly tailored for use by machine learning developers, and their designs provide little opportunity for input or collaboration with other team members.


[User] Please transcribe this text

[Assistant] So we're exploring design opportunities to instead promote more collaborative work practices where UX researchers, social scientists, and others with relevant expertise for thinking about fairness have a seat at the table.


[User] Please transcribe this text

[Assistant] In one more ongoing project, I'll mention and then I'll wrap up, we're creating and empirically evaluating a new collective auditing platform.


[User] Please transcribe this text

[Assistant] So our goal in this project is to empower everyday users of algorithmic systems in addition to crowd workers.


[User] Please transcribe this text

[Assistant] To surface unfair or harmful algorithmic behaviors as early as possible, uh, whether to product teams or to advocacy groups or so forth.


[User] Please transcribe this text

[Assistant] And this project aims to overcome limitations that we've observed in our prior work.


[User] Please transcribe this text

[Assistant] With product teams existing practices for user testing or auditing their own systems.


[User] Please transcribe this text

[Assistant] So that was just kind of a quick overview of some directions we're pursuing now.


[User] Please transcribe this text

[Assistant] Also, beyond today's discussion, feel free to reach out via my email list here as well as my Twitter handle. So, thank you.


[User] Please transcribe this text

[Assistant] It was an iterative process of, uhm, learning the wrong ways to do it. Uh, over time we essentially, uhm,


[User] Please transcribe this text

[Assistant] Uhm, that our goal here is not to write a call out paper. We are not going to work with you and then write a CHI paper or some other conference paper in which we're talking about all of the problems in a particular product. We are trying to actually understand your processes to improve those processes.


[User] Please transcribe this text

[Assistant] That was one important piece. The second was definitely, uhm,


[User] Please transcribe this text

[Assistant] In papers and to review the ways in which we were de-identifying particular companies or particular teams within companies.


[User] Please transcribe this text

[Assistant] Yeah, absolutely. Yeah, that, that is a real tension in some domains, definitely more than others.


[User] Please transcribe this text

[Assistant] And, uh, in projects we're kind of getting off the ground now, uh, I mentioned one of the ongoing projects is


[User] Please transcribe this text

[Assistant] Working within a team, so looking at interactions between, say, UX researchers and developers and product managers.


[User] Please transcribe this text

[Assistant] In many cases, competing objectives in the fairness space when working on actual products.


[User] Please transcribe this text

[Assistant] I would say by definition, the analytics are limited to quantifiable situations.


[User] Please transcribe this text

[Assistant] Or actually, maybe I'm misinterpreting the question a bit, so, so.


[User] Please transcribe this text

[Assistant] If I understand correctly, I think you might be thinking about kind of asking about whether


[User] Please transcribe this text

[Assistant] It tends to be the case that our best models are pretty bad compared to human performance, at least if the human has, uhm, has certain stimuli available to them that they would have in a classroom environment and is attending to a student in the moment, which are strong assumptions.


[User] Please transcribe this text

[Assistant] So I, yeah, I think the short answer, I'm not sure if I'm completely answering the question, is absolutely, it tends to be the case that uhm, and there's a broader discussion within the...


[User] Please transcribe this text

[Assistant] And thinking about the ethical implications of that potential narrowing. So that's something that I think about a lot. Not just in education, honestly, but in every domain that I work in, including social work, any projects in healthcare, this is really a broader problem.


[User] Please transcribe this text

[Assistant] Yeah, that's, that's a great question. I'm gonna actually bring us back to this slide. So we, uhm,


[User] Please transcribe this text

[Assistant] That I typically do now in projects for the most part. Typically now I think if I were doing this project I would.


[User] Please transcribe this text

[Assistant] And so we were interested in filling this gap and hearing teacher perspectives.


[User] Please transcribe this text

[Assistant] The slide I'm showing here is a piece of the story that I didn't really get to talk about today, which is the co-design with students.


[User] Please transcribe this text

[Assistant] And again, I feel like in retrospect, we brought this in a little bit too late, but we learned a lot through this process. And it was actually one of my favorite parts of this project from a participatory design methods perspective in which we were iteratively kind of facilitating these iterative negotiations between teachers and students.


[User] Please transcribe this text

[Assistant] And so uhm, in this paper, shown at the bottom, we talk about a lot of the concerns that students had, uhm, the reasons why teachers felt strongly about having some of those designs. Ultimately, we were, for the most part, able to get to a place where uhm,


[User] Please transcribe this text

[Assistant] We would alleviate any major concerns students had, for example, venturing into the territory of affect detection, uhm, really, really went into the creepiness territory for students, whereas other analytics that we had, students would actually agree, oh, seems helpful for my teacher to know that.


[User] Please transcribe this text

[Assistant] Uhm, so we were exploring some of those tensions, but by the end of this process, uh, we got to a place where, uh,


[User] Please transcribe this text

[Assistant] Even though they might be disappointed that some features they wanted were not in there, uhm, we're still, we're still overall happy with the design.


[User] Please transcribe this text

[Assistant] Not all of these changes, the majority of these changes were not reflected in the version that we tried in the classroom study. And that's where some of my regrets come in. Have we started this process earlier? More of these design changes would have, we would have been reflecting student and teacher needs a bit more in the experimental study that I covered.


[User] Please transcribe this text

[Assistant] Well, thanks. Thanks, Michael. I'm super happy to be here, excited to be able to share this work with you.


[User] Please transcribe this text

[Assistant] I'm going to be talking about my work building uh, chairables and thinking and talking and thinking about designing technology for both accessibility and engagement.


[User] Please transcribe this text

[Assistant] Uhm, so I, as Michael said, I'm an assistant professor in the HCII, uhm, and director of my lab, which, uh, I call the AXL Lab.


[User] Please transcribe this text

[Assistant] Uhm, so just to start out to give some context to this, because I think a lot of people have different definitions for what accessibility means and what engagement might mean, so choose your own definition, but here is how I think about these things.


[User] Please transcribe this text

[Assistant] My research, like I said, is focused on accessibility. So thinking about people with diverse abilities, uh, thinking about people with disabilities and how they interact with technologies, how technology can better support and break down barriers that, um, either society has presented or different technologies have, um, uh, created.


[User] Please transcribe this text

[Assistant] If we think about this in terms of a research question, uhm, this is probably a very common question that you might find for accessibility research. So how can we design


[User] Please transcribe this text

[Assistant] So I've been my generally my research on charitable computing has been focused around this question related to how we use technology and information technology and communication technologies to support wheelchair users for charitable. So I'll start.


[User] Please transcribe this text

[Assistant] So in this, in just this image, there's four images captured from one of my studies showing different hand poses of someone interacting with a touch sensitive surface.


[User] Please transcribe this text

[Assistant] An interactive surface, so you just have people with these different kinds of hand poses, different kinds of fists, uh, postures.


[User] Please transcribe this text

[Assistant] Furthermore, uh, we sometimes fail to consider other devices or other assistive devices that people might be using while interacting with these, uh, with these technologies like wheelchairs. So wheelchairs in particular, we can think of them as both creating and alleviating challenges of interacting with a computing device. On the one hand, the wheelchair


[User] Please transcribe this text

[Assistant] On the other hand, in terms of alleviating some of the challenges created, a wheelchair also has spaces to carry these devices or store these devices, being able to carry and store multiple computing devices or hide wires without actually burdening the user, as we've seen with some early innovations in wearable technologies.


[User] Please transcribe this text

[Assistant] Terribles are designed or devices are designed to fit the wheelchair's form factor while maintaining its shape and considering the abilities and preferences of the user.


[User] Please transcribe this text

[Assistant] Uhm, thinking about kind of the first chairable that we built, the JestRest, which was a flexible input device for power wheelchair users, and then talking about more of the work, uh, getting into chairables for athletes and some of the more current work that I've been doing with wearables and spokesense.


[User] Please transcribe this text

[Assistant] And then I'll finish up with some of the current and new directions that we're taking in the lab and further research.


[User] Please transcribe this text

[Assistant] We'll start with, uh, how we've conceptualized design, thinking about leveraging the wheelchair in design. So I like to think of this as participatory ideation and design, uh, to get to the goals. I'll preface this with kind of the problem space.


[User] Please transcribe this text

[Assistant] Uh, motor impairment, so thinking about motor impairments and how they can affect multiple parts of the body. So in addition to using a wheelchair because of some ambulatory ability where you might have difficulty or an impairment to the legs, um, they may also have difficulty or an impairment to, uh, the head, neck, arms, and hands. And so sometimes these devices or the devices that, uh, these


[User] Please transcribe this text

[Assistant] And the use of the wheelchair, as I mentioned, can sometimes exacerbate these existing challenges. And I'll give you a couple of examples as we move forward.


[User] Please transcribe this text

[Assistant] And again, we can go back to that kind of original question of how to design devices and interfaces that better support people with diverse abilities. So how do we support people using wheelchairs and these devices together?


[User] Please transcribe this text

[Assistant] Developing solutions if an existing solution can't be adapted or modified to meet that need and then evaluating those solutions. In this case, the more the emphasis has been on was on the design side of how we should approach design. How should the


[User] Please transcribe this text

[Assistant] Users, how do the users actually approach this challenge and how can we involve, uh, the, our target population in the process of actually coming up with these solutions?


[User] Please transcribe this text

[Assistant] And so in this first study, our goal was to understand existing challenges with technology and mobile devices and to ideate on what ideal, desirable solutions might be that address the needs that these users have in relation to that.


[User] Please transcribe this text

[Assistant] He uses a head array or a set of buttons that are mounted around the head rest of his wheelchair to actually control it, drive it around and control various functions. He was also very fortunate to have


[User] Please transcribe this text

[Assistant] Could he use a switch or a button that was attached near his elbow to control different devices or at least to have that kind of input? In addition to the header array.


[User] Please transcribe this text

[Assistant] So this idea of always available interfaces, which is common to wearable technologies where you have the access to this device at all times whenever you need it.


[User] Please transcribe this text

[Assistant] But also tailoring to the specific abilities of the user, which is another common accessibility approach. So combining those two and then adding in that this idea of maintaining the silhouette or the shape of the user and their wheelchair together. So maintaining their body shape in these ways is this kind of illustration indicates.


[User] Please transcribe this text

[Assistant] So we spent four days doing a series of focus groups with these clinicians and a few wheelchair users.


[User] Please transcribe this text

[Assistant] Uhm, to ideate on what a solution might be for a phone dialer, a game controller. In the first case, we focused on the specific applications and then from kind of examining the results of that, we moved into a more general space of


[User] Please transcribe this text

[Assistant] Focusing on how people can input information into a system and then how can people interact through different kinds of innovative outputs.


[User] Please transcribe this text

[Assistant] We conducted these on four consecutive days with a series of people in and out. After we came out of those focus groups, we took those results into individual design sessions focused around the different inputs and output techniques that arose from that study.


[User] Please transcribe this text

[Assistant] Uhm, and we asked individual users who once again think about uhm, how they would configure an ideal solution for them to make their wheelchairs interactive.