Holographic Programming: Exploring the HoloLens
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Klint, Lars
Virtual reality and augmented reality are terms most developers and technical enthusiasts recognise. What about “Mixed reality”? A reality that is part real world, part digital world, a reality that is enhanced with Holograms. Microsoft’s HoloLens introduces users to an entirely new way of thinking about computing. Learn how to use the Holographic Development Kit (HDK) to build completely new experiences that will blow your mind (only figuratively, I hope), and get a sense for what is possible. You will be challenged to think of how to solve problems in an entirely new way that leverage holographic projections, to provide intuitive natural interactions with the digital world.
 Good morning. How cool was that press play to untape, what was it called? Press play on tape last night? Holy crap. It made me feel really old because I knew all the games. Thanks for coming. I just want to sort of start with a little bit of disclaimer. Did anyone yesterday go to Renee's talk? Just to show hands please. It's a few. Alright, so there was a HoloLens talk yesterday and Renee actually has a HoloLens and I don't. See, no. No, it's good for you because I'm guessing you don't have a HoloLens either. Does anyone have a HoloLens? Jimmy, hands down. No, and that's the whole idea, right? There are really none of them around. There are, I think, so Jimmy has one. I'm pointing it out. No, Jimmy has access to one and it's only because he gave up his first born, right? And really hard to get. It's almost impossible. So therefore, I'm here to tell you how you can develop for HoloLens anyway. And that's why it's cool. So I don't need a HoloLens. So if you thought I should have a HoloLens, you can give me one. No? Okay. So why am I here? My name is Lars and I am actually Danish, but I have lived in the land of fruit and nuts. No, not California. Australia for a very long time, which is kind of why my accent is like this. So I'm drink the Kool-Aid. I talk a lot about Microsoft. Windows platform, MVP categories where HoloLens belongs for the moment. So that's kind of cool because I fit in there. I do some PluralSight courses, which have been a bit of a mix. Kids' Cause, some Windows phone stuff. Yes, I am the guy that uses Windows phone in Australia. And I've been Troy's test subject on a couple as well. So if you don't know what PluralSight is or if you would like to try, have some free passes. So feel free to grab one after the talk. I'll write a bit about different things here. Also about HoloLens, although I haven't written much because it's so new. And I just want to mention this. Did you know that NDC is going to Sydney? Yeah? So yes, says the Australian. So if you want to have another go and a trip to Australia, because we have really, really nice beaches. We have beautiful fauna and we welcome everybody equally. We really like you to come and obviously someone else likes would like that too. This is generally the perception of Australia. It is not true. We are quite a civilized bunch as long as you provide us with beer. Now, I have this experience from last year at NDC and I had a Norwegian audience. That's a thing. Did you know that? It's like we use in the world. It's like, did you get Norwegian audience? Oh no. Please don't be like this. Please don't be these guys. The best way to learn is to do this. Ask questions, be interactive. I don't mind. I thrive on questions. And throughout the talk, if you follow this hashtag, I'm going to tweet whatever is relevant in the talk, hopefully at the same time as I talk about it. So that's just a bit of context if you miss out on a thing or a link or a video or whatever it might be. So enough about me. Let's talk a bit about what we do every day. So the current state of software is that we develop things for a screen. Is that right? How many here are software developers? Okay, everybody else? Anyone here is a manager or something? No? Good. Out. No. This is for everyone. But the thing is we all live in this software world and we more so than everybody else. We have a duty to provide good software to users. The problem is though that we are limited by what we can display it on. So if you're a mobile developer, you develop for a flat screen. It's in someone's hand. Granted, you have lots of accelerometers and GPS and all sorts of sensors that you can use, but you are limited to that screen. It's the same if you're a desktop software developer. You can only develop for whatever people can show on their screen. And it goes on. If you do games, does anyone here do game development, by the way? Do we have any game developers? It's like one in a really, really long arm. He made Donkey Kong. Because that's an advantage in holographic programming. No question, especially because of the 3D aspect. But with games, you still, you sit down. Unless you are using a Kinect or a Wii, which I haven't seen anyone do without injury, then you sit down, right? And you limited to the screen again. And the same collaboration. And this is a really cool point about HoloLens, collaboration. But you normally just have, I don't know if you sit in point at a screen. That seems a bit odd. But there are very, very physical limitations to what you can do. Yeah, come on in, guys. If you sit up the front, I'll point now. I won't point you out. But the gist of all this is that when you've been doing it for 20 plus years, which is why I knew all the games yesterday, oh, God. It kind of, you kind of become a little bit of a boring duck. It all becomes a little bit the same, right? And you just sort of go, another one. And you go, oh, this one, this thing, I can use GPS. Yes. And it's sort of you try to start focusing on the little things that make it worthwhile, right? So I'm in the same boat. I do mainly web development. I'd love to do more Windows apps, but no one gives a shit. Unfortunately. So that's sort of the current state. And I guess I'm here to try and tell you that it can be really, really different. And you can start right now, even though we don't have these beautiful devices. So let's just, I know, so just to show hands again, because we like like double of book before, if anyone were at Renee's talk yesterday, please just show it. Yeah, there's a few there. So, and that's okay. I'm going to try and do a different take on it. There will be some mobile app, but I don't have a HoloLens. So I have to do it differently, right? Okay. Just so you know what you're in for. So if you don't like that out, no, you can say. The first thing is virtual reality. So we have this concept of virtual reality, which has been around for God knows how long I was, I think it was eight or nine when I did virtual reality for the first time. And it was in, I don't know, in this contraption thing, and you're holding a, like a trigger that would make you move. And then you're supposed to shoot the other guy. And it was all eight bit and it was all, it was kind of like the demo yesterday. It was all squares and spheres. And that was it, right? So it's been around for a long time. But it is a complete virtual world. So everything you see is replaced. Everything you see in this world is made by someone else. It has very limited physical movement, obviously, because if I had an Oculus Rift on now, I would walk off the stage, right? So you can't really move around because you can't see where you're going. But it's great for prototyping. If you're trying to show someone what a new building is going to be like, there's nothing better than giving them on a virtual reality and they can look inside the building, right? So there are really, really good use cases for it. Next thing is augmented reality. So you probably have a smartphone. Yes, anyone doesn't have a smartphone? Because I want to see it. Now, it's so prevalent, right? We all have these devices in our pockets, which are mini computers. Augmented is kind of like a tiny bit of what HoloLens can do, I guess is the best way of explaining it. It's usually triggered by some sort of object. And your phone will pick it up with a camera and it'll show something, it'll augment the reality. Usually what I've seen is brochure here. So you have, I want to buy a boat. I point my smartphone app at this boat and the boat comes alive on the screen, right? It comes 3D. Very cool. It's really powerful. And then because HoloLens didn't really fit into any of these sort of digital realities, as I like to call them, they had to come up with their own. So it's called mixed reality. And I'm sure this may change in the future as people find out more about what it can do. But right now the term is mixed reality. And these are all digital realities, just different ways that we are interacting with a digital reality. So the HoloLens, the whole idea is you have seamless integration of real and digital. If you went to Renée's talk, you saw this. You saw that it'll interact with whatever physical devices you have next to you. The whole idea is to have natural interactions, right? So whatever you do normally, it's still getting there. I wouldn't say it's still normal to do this all the time. But we're getting there. It's an interaction that you can do. It's a physical movement that you could do in real life. And then the whole idea is you can use any room, any surface, anywhere. So normally when you look at virtual reality, and Troy kind of touched on this in his keynote, you get this feeling, right? Like this is usually the reaction people get. It's like, holy shit! Because it replaces everything. And you don't have your in-area balance necessarily. You don't know what's coming. You can't escape it other than just scream, right? And it does replace your entire physical reality to something that is much more real, I guess. I love that one. So that's what Sonic looks like in real life. But HoloLens to me is why we're here. And it is just absolutely pure magic. I tried it on. I have used one. And it does blow your mind. Even though you've read everything about it, and you've seen all the videos, it's just that whole change of what your world is, is quite impressive. One important thing that I have come across though, the more I read about this, the more I use the tooling, the more I try and come up with ideas that are really going to blow people's mind, as we always say. We need to understand what is the difference between HoloLens app and a virtual reality or augmented reality app? And it's an important difference because I don't want to take any of Renee's thunder away because the flight, the Holo was a Holo flight yesterday, was absolutely impressive, right? But to me it wasn't a HoloLens app. And I might be a bastard for saying that, but it just wasn't. Because it wasn't interacting with the real world. The planes, if they could crash maybe into the wall or whatever, it might be, but that would just be morbid. It wasn't a proper HoloLens app. It was impressive and it was really, really polished and well done. No question about it. But something like another test or not test, but demo app that Microsoft has is this game. And in this game, which I forget what was called, what was it called? Fragments, which is fragments of memory for this boy and trying to solve a mystery. What happens to the boy? And you look around the room and things get replaced depending on what room you're in. And people will come into your reality, they'll sit down on your couch and talk to you. Like that to me is a holographic app. It's HoloLens app. It changes what your physical world looks like. Important distinction at least to me. Anyway, okay. Let's have a look at some of the hardware specs. I love this picture because all we've seen is on the right. We've all we've seen is that HoloLens, right? And it looks pretty good. Looks a little bit like you're from X-Men, but it looks pretty good. But it came from the other one. It came from that prototype. And that's the earliest prototype I could find any pictures of. And the thing he's got on his head, that black bit is actually a Kinect. So he's got a Kinect on his face along with all the other gadgets. So just bear in mind, we've come a really long way. I'm not sure how long it's taken. I think it's about five years so far, five, six years project. So it's not simple. So the specs for this. They are important as much as we like building software. I'm not a hardware guy, really, as long as my laptop works and it doesn't hold up. I'm pretty good. But it's important to remember what is inside this thing. So that's why I say hardware and then I'm going to say Windows 10. It's not hardware. But it's important to know that Windows 10 runs on this device. And as Renee touched on yesterday as well, UWP is how you develop apps for this. So you use the universal Windows platform. And you can use, if you're a masochist, C++. I'm not man enough to use C++. So I use Cshare. Just because it works everywhere and it does a lot of the heavy lifting for me. But there are scenarios where you have to use C++ just because of the performance. So I'll just walk over here because I like you guys too. It's an Intel processor, so it's 32 bits. And people are, oh, it's not 64. I don't think it matters. Often 64 just means that it takes a little bit longer to process because the memory space is bigger and you need a more powerful processor. There's lots of drawbacks as well. So I'm not too worried about that. This is cool though. HPU. Anyone heard of HPU before? No? A holographic processing unit. Yeah, now we're talking X-Ment, right? How cool is that? But it is the thing that makes holograms real. It is the thing that calculates all these millions of polygons that fast. And I don't know, there hasn't been any specs officially announced about it, but rumor says it's about a terabyte a second of data that it processes. So I, you know, it's a pretty serious thing. Two gigs of RAM, which doesn't seem like a lot. I thought it would be like with today's RAM, we're like 200 or something. But two gigs of RAM, 64 gigabytes of storage is where you install your apps. So it still is a fully self-contained computing device, right? Has four mics. So the mics above your head, essentially, if you saw on the device, and it's really good. They're really, really good. I'm very impressed with the speech recognition in Katana. It's Katana using it, but in the HoloLens. And then four speakers, which sits around you, but I'll get back to that later. And then Bluetooth, Wi-Fi as well. But the one thing that is the most important thing, take away all this googly-gar here, is the fact that this thing is untethered. Like, there is no wires off it. It is a completely self-contained unit. And that to me is, it blows everything away. Because you can walk, you can put a hologram here on the wall. So that's in your living room. And that might be, you know, Netflix streaming or something. And you say to it, Netflix, stop. And it'll stop. And then you walk in your kitchen, you make a cup of coffee, you come back, and it's exactly where you left it. Like, you don't have to remove the whole, and you can take it off and on again if you want. But things are exactly where you left them, if you're a good developer. Because it's not that easy. There are lots and lots of abstractions and tools to let you do it. But it can be a little bit, let's say daunting to read all the documentations on all the various things that make a hologram stick. But I won't go into that, because it's boring. This is normally what you see. Have anyone here followed HoloLens sort of marketing blurb and tweets and Facebook stuff and seen the videos and all that sort of stuff? Yeah? Yeah, a few nods? Please don't be Norwegian. Come on. So you've seen this. And this is normally what you get. You say, oh, this guy's staying there. And look, he's got the ocean and it must be Australia because the shark's about to eat someone. And this is actually not how it's going to look. Because the feel of you and this thing is very, very narrow. It's more like this. So it is like a box in front of you, right? And that's why we're talking about sound later, because sound becomes really important. You can't see much. And that's why every single demo you see, they kind of look like this, because they can't see anything, right? They're trying to look, even the whole hologram can't fit inside that field of view sometimes. And it takes a little while to get used to, maybe an hour or two. But it is a limitation for sure. It is something that you need to be aware of, because people can't necessarily see everything they do. And the other thing is you put a hologram over there, and then you saw Renee yesterday, he just fired off 100 of these meatballs. If you had put 100 things different, how are you going to remember where they are? Like you have no idea, right? So those that as well, you can't see necessarily, you forget where you put things. All right. Just to blow your mind a little bit, let's have a look at this video. It's from E3, if I can get over here. Which is a gaming conference. Oops, I'm not putting sound on. Sound please. Hello. Is there sound on? I'm not sending any sound. Oh no. All right. I'm going to do that again. There are more guards. Still nothing. Oh, that's really disappointing. That worked about five minutes ago. Hang on. No. It says it's plugged in. I got a notification. Nothing. All right. Bear with me guys. Because it's important. I do need sound later. So let me just get this sorted. No. Yeah, that's what I did. That one. Thank you. See, I'm not a hardware guy. Thank you. Okay. Right, Clay. I really don't know what I'm doing here. Yay. Uh oh. Problem. All right. Let's try again. I didn't enable the other one. I'm not going to do it again. Really? All right. That's disabled. Okay. Touching. Welcome to my talk on hardware configuration 101. All right. Let's keep going. So this is from E3 and this was one of the earlier videos or demos that Microsoft did on stage. It wasognin e3a, since on your Minecraft world. It's awesome to play with the controller, but could we show them something new, Sax? Sure. Let's take our experience off the wall and then put it on the table over here. Create World. I see Lydia way up there. As I run around and play, Sax can easily navigate and manipulate the world using his voice and his hands. He can walk around the hologram, pan around for different viewpoints, and even look inside. How cool is that? Holy crap. And that's why they use Minecraft, right? Because we all know it, and we all know it as a thing on a screen. And then suddenly they make it 3D in real life. So this was an early example. Things have moved on since then. But I still like this because it really showcases what is possible in a way that we can relate to. So let's have a look now. Is there any questions, by the way? I'm going to keep asking. OK, building blocks of how do you build a HoloLens app? Because there are elements that are really important. There's four in particular that you need to pay attention to. So we've got Gaze. Gaze is what you're looking at. Gaze is essentially how you select things as with a mouse, with a mouse. So it's your mouse cursor. And it's not your eyes. So you don't have to program for people looking. It's the center of the HoloLens as a ray. So it's usually called Raycast. And that's because the object that you use in programming, as we'll see in a second, is called Raycast. And the Gaze is your mouse pointer for whatever object you want to interact with. So it's important. It's important that you know what people are wanting to interact with. So let's have a look. Just quickly. So I've got, here's something I prepared earlier. Can you all see that? All right? Yeah? I know it's a little bit tricky in this room with the middle. I mean, I feel like I'm in a tennis match, right? I'm doing this all the time. But you guys just try and see if you can see the screen, OK? So all HoloLens objects are built in Unity. Actually, we can start in Unity just for good measure. So you set up a scene, which is essentially your app. So an app has at least one scene. And I have some 3D objects here. So this is my cursor here. So if I double click on that, you get lots of information about it. And if you don't know Unity, don't be scared. I had, like six months ago, I had no idea what it is meant. No clue. It is not that scary. Most of these things, you never have to touch, or have to touch once. But you set everything up in Unity. In this case, we have this cursor, which is just a 3D object of a ring. And that ring, we want that to be where people are looking, right? We want that to be the indication that you're looking at something and where is this placed. So the way you do this is you create scripts. So down here, I've got folders. And I have some scripts in these folders. And they are just C-sharp scripts. If you notice, C-sharp, it really is as simple as that. We create a script by going through all the far new script, blah, blah, blah, blah. And then we drag and drop that script onto the cursor. In this case, it's the world cursor, right? I just want to show you the code, because otherwise we can spend really long time just going through all this. Every single script that wants that you get attached to objects have a start and an update. Start is basically the way you set things up that runs once on initialization of the app. An update runs every frame. So there's 60 frames a second on HoloLens, without exception. Everything has 60 frames a second, because that's the way you make people not throw up. So it's really important that you have really, really good frame rate. And there's lots of tooling to help you do that. But all this does is that we have a mesh renderer. Actually, that's the mesh for, yep. And in here, we have the transform. So that's where the camera, or HoloLens, is facing, the position. So that's your head position, whether you do this. And then where forward. So that's where you're looking. What's the raycast, right? And then you use this physics.raycast, which is a boolean. Give it the position of where you're looking. And if that's true, then you can use this hit info, which is a raycast hit. So that's what am I looking at. This turns false if you're not looking at anything, or if the little ring doesn't hit anything, right? Which is why we have enabled false. Pretty simple. If we do hit something, we enabled the cursor, the mesh renderer. And then we set the hit info. So the hit info of the raycast and the point of that is what we set the transform position to be. So that's where we're looking. And then we can rotate it. So the position of the mesh, so this is complicated. The position of the mesh compared to where we're looking, right? So if we're looking, as we've seen in a minute, at a ball, we'll put that on the ball. And then we have the rotation here, which we're saying, well, let's make it hug the surface. So it looks a bit more real. So you don't just have this flat ring that sort of sits not in real life. So the way that looks is, if I just go in here, we'll just debug it without debugging. So this is the emulator, right? And that's a really, really cool tool. It's actually incredibly well done for a first-gen device. The emulator does everything the HoloLens does, more or less, obviously without the physical experience of it. But it gives you 3D space. It gives you all of the, I'll just do this. Here we go. Reactivate it. Yeah. Has all the tooling goes with it so we can see where everything is. Let's move it a little bit more. So we have all the coordinates of everything that you would be looking at. So this is your head coordinates. Why is this not starting? Come on. So that's the start menu. I'll just press Start. And I'm just going to go over here. And I'm going to select my origami. So the tap is what I was doing. That's just the Enter key. So it's as simple as that. You just have Enter, and you have Start. And that's pretty much it. So here's our little app. This is our, and I'm using the arrow keys and the gamer keys, WASD. And I'm going to move around this. And as you can see, when I go over the surface here, hey, let's just zoom in a bit. I'll move in. That's our ring, right? So we're hugging the surface as we go around. And that's it. That's all you do. Well, obviously, you can have this cursor be anything. You can be your company logo. It can be your finger. It could be whatever you want it to be, right? So that's gaze. It is really as simple as that. OK. The next thing, which is probably a little bit more interesting, is, what's that? Here we go. Gestures, right? So we want to be able to interact with these elements. I love that one. We want to be able to interact with the objects that we're looking at, and that's where we use gestures. The most common one of the, hello? Oh, you're still here. Now, the most common one of these is the tap. And there's what's called a gesture field of view, or gesture frame. And that's essentially where the whole lens is looking for your finger. So you can have actions based on whether it can see your finger or not. But essentially, it's all this, this. You just do this all the time. There's a whole lot of this. And that's mouse clicks. And it is meant for you to do this. So if you move your head, it can't see your finger, right? So you start becoming a robot because you've got to sort of move your hand with everything. It's a bit mechanical, but it works really well. It does pick it up really, really well. OK. So that's the tap gesture. It also comes as tap and hold, or tap and hold and move in 3D space so you can move things around and manipulate them. And then you have the bloom, which is just that. That's just start menu. That will always work. You can always get the start menu up. And again, see, there's again more marketing shots. No field of view displayed at all. So let's just try and make a few gestures, shall we? OK. So the way that you do gestures is you have a gaze gesture manager. So it's built in the SDK. So all of these tooling, by the way, are free. And if you follow the last Twitter handle, I will eventually tweet a link to where you can get all the tooling and how to set it up. And I wrote an article yesterday on the whole introduction to building HoloLens apps. So we have a game as a gesture recognizer, which we actually recognize the gestures. And we then, on start, which is our initialization, we attach these events, event handlers. So tapped event comes for free. We're going to do that a lot. And all that does is that if it detects a tapped event, then we say, well, if we're actually looking at something, then we send it an on select action. And then we just start capturing gestures. And that shall start. And then every frame, we do a raycast again. And we set the focused object. So every frame, we get the raycast. And we look at, OK, are you looking at something? No. Yes. No. Bull, bull, bull, bull, bull, bull. Hold on. 60 times a second. Which means that when we actually look at something, we tap, it knows whether we focused on something or not. And that is really as simple as that. And then we cancel gestures and start capturing them. Bit of optimization there. So we get our right frame rate. And all that does, so again, we up here. I'll just click on my emulator. So I can move around here. So when I, oh sorry, I forgot to show you what actually happens. So here, on select, we click on the right frame rate. On select, that's that method here. So that's attached to the object itself. The thing that I can select, the thing that I tap on, we'll get this. So if you tap on something that doesn't have the on select, well, nothing happens. It's just an event. Does this look familiar? C-sharp developers? This is something you could do, right? Well, you can read this. It's not rocket science. It really is quite straightforward. Because initially, when I thought a HoloLens app, oh, God, it's going to be tricky because there's maths and there's 3D space in there. But no, it is initially very simple like this. As Rene said, you do get into situations sometimes that are not. All this does is it says, if you don't have a rigid body, that essentially just means, do I have physics? So does the 3D object have physics? If not, you give it physics. So what happens is that when you tap on it, so I'm just going to hit the Enter key here, falls down. Because I'm in the room. Actually, I'll show the room. And they're suspended in mid-air because they don't have physics. They're like magic, right? One thing I just want to show you now, we're looking at the emulator. So there's all these things here. There's all the body was that moved. If I tilt the head, so if I move here, so Q and E will tilt the head. So you see my head can roll. Because you could do this when you have HoloLens on. So you get that information as well. You can select what room you're in. So their room's preloaded. If you have a HoloLens, you can record your own room and put it into the emulator. Makes it even more useful. But all this is that has this device portal, which is a website. So this is now connected to the emulator. And if you had a physical device, you would get the same information just for the physical device. And the really cool part of this is this 3D view. So this is my plane. So I just used the mouse key here to go in and look. I mean, obviously, this doesn't look that impressive, because it's just a bunch of squares. If I update my surface here, it builds the room for me that I'm in. So I can fully completely see where this room is. Just a little bit like that. Move it over here. And then if I open the emulator, as I move around, see it moves over there as well. So obviously, I would have this on a different screen. I normally work with three screens, just because there are so many tools that you want to see at the same time. So you get a fully, I don't want to say immersive, but a full experience of what you would experience in a HoloLens. And there's many more tools in this, especially around performance, which is a complete talk on its own, because it's a reasonably complex topic if you want to have lots of holograms, especially at the same time, even if they need to interact as well. So still no questions. Is this? Yes? Yes? I have a question concerning, I think this is the coolest thing that's happened in the last couple of years. So do you know anything about the plans of rolling out consumer versions? Do I know anything about rolling out consumer versions? Is anyone here from Microsoft? No, I don't know a thing. I have no idea. No, I actually don't know. There's been nothing I've heard at all. Right now it's dev kits. And the one I tried is very much a dev kit. It's like a prototype, essentially. They're quite fragile. They do work fully as you see in the videos. There is no magic there in terms of Photoshopping. But it is, the field of view and the way that it feels is still very much a prototype. But consumer versions, I don't know. I can guess. Two years, three years? I don't know. I think it's probably about that far. But I hope it won't be $3,000 for one. But no, I don't. I don't know when the consumer version would be released. Yeah? You said field of view. Do you think there is any plan of expanding the field of view? So is there any plans of expanding the field of view? I really hope so. That's the only I can say. There was a job at someone posted that was announced from Microsoft of an Optics specialist or something for the something devices unit or something. I mean, hopefully that's a good thing. But I would have thought so. I think it has to do with processing power. So once they get their hardware better, they might be able to do more. But I actually don't know. Again, I really hope so. Because it would make it a lot more immersive, for sure. OK. Let's go back and look at the thing that I really hope is going to work. Bit temperamental sometimes. So voice is quite important because you can't keep clicking on things, right? You can't keep doing this because it gets boring and you get tired. So you can talk to the device and it'll understand everything you say. I don't know what I'm saying. I'm just saying it's not a big deal. I mean, but in English, it almost doesn't matter what accent you have. I think if you have really like strong voice, you can't really hear it. It almost doesn't matter what accent you have. I think if you have really strong Scottish eyes or something, it might be a problem. But it actually does work really well. OK. I'll get rid of that guy for you. Sorry. There's a command which is select. So you can always look at something and say select. If it is selectable, something will happen. And that says you can't override that. It's a reserved word. Select will always work. And of course, Katana works, right? Yeah, Katana. Because it's going to bleep everything over here. But you can say, you know, Katana. And she'll come up and she'll start working on your device as well. So yeah, it's internet connected. And then there's custom commands, which is by far the most useful thing. And there's lots and lots of theory around this that I won't go into as such. But just bear in mind that speech design is by far the hardest part of this. Getting someone to do something when you say something is not hard. But having a whole catalog of commands that you want your users to use is by far the hardest. Because you can't have words that are too similar. You can't have sentences that are too long because people won't remember. You can't have... So what is that? Yeah, they have to be non-destructive, right? So you can't have things that override other things so that if you have words that are similar or they don't override each other, you shouldn't have more than two syllables in words because then it starts getting hard to recognize it or hard to. So there's all these guidelines around how you design your speech or your voice commands. And that's the hard part. So I want to show you the easy part, which is really, really quite simple. And I was surprised because this is... I've done a fair bit of Katana integration. Has anyone here done Windows apps? Is there any Windows developers? There's a few. Hello! Have you used Katana? Has anyone here used Katana on an Android, on an iOS, on a Windows phone, whatever it might be? Oh, come on. It's like the good version of Zerry. Come on. No, I know what you're saying because it's a thing we're not used to, it's an unnatural thing to go around and go, hey, Katana, tell me a joke. It gets boring quick. By the way, Katana's told over half a billion jokes now. Someone does it. Oh, no, no. Stop, stop, sorry. Told you she was going to wake up. All right, I can show you, but first I need to quit the current app. Is that okay? No! All right. I'm not even sure what she was going to quit. Wow. Calm down. So let's have a look at speech. And it is really simple. You have a keyword recognizer. Actually, I'll go down the bottom here first. Oh, that's an error. So you have a keyword recognizer and then you add keywords to essentially an array. And the keywords are text strings. So we're back to magic strings again. But they are just text strings, right? And that means you can do whatever you want. So we have two here. We have reset world and we have drop sphere. So they're different enough, right? They're short, they're different. All right, fair enough. And then with these keywords, we attach them in here to our keyword recognizer. And then whenever it recognizes a keyword, it fires this phrase recognized. And we get all this for free. It's all built for us. It's all part of the SDK. And then we have our event that says on phrase recognized. And we invoke whatever that lambda function that we gave it up here is. So there's other ways you can do it as well, obviously. But that's the gist of it. And by the way, all this code here, I've blatantly stolen from the Microsoft tutorial because I want you to do it. I want you to go out and do this. If you have any interest in holographic development, it is that simple. And again, follow the Twitter handle and you'll get where all the links are. And so what this does is that reset world sends an on reset event and drop sphere sends an on drop event. This is a broadcast message. So that's actually for every single thing, not just what you're looking at, whereas the drop sphere is only for what you're focused on, right? So that's a slight difference here. I'll go back to our app here. Let's see if I can find. That's the problem when you're, oh, there we go. So there's my other ball here. I'll need to get my, there we go. Did you remember that I dropped it to the floor, that right one? Now it's back again. It's because I said the keyword and it was still listening. So if I drop this, whee, and then I go and I do the same with the other one. Actually, no. I'm not sure if I could write. We just made a voice command. So I can say drop sphere, eee, and it falls down, right? That's how well the emulator works because my pronunciation is okay, but I'm certainly not British, right? It's not super clear. I picked up the hang on, eee, eee, right. The Australian part of the things, right? And then the other thing we could do obviously is that we could say reset world, eee, and they come back. And they have so much fun with this. Like you can keep making up, you know, keywords and do things and make them dance or whatever you want to do, right? It's actually implementing the keyword part of it is not hard, which is my point. Everyone here can do it. You can. It's really not hard. So let's have a look then at the last of the four pillars or building blocks that you need to know. So we've gone through gays, gestures, and now speech. So the last one, bleh, is sound, right? Sound is very important because as I said, you might have people put holograms here and everywhere. And you need to know, or you need to let your users know where these are if they can't find them. So having sound, especially spatial sound like this, what HoloLens does, is crucial. And to be honest, I haven't mastered this. It's really hard to get sounds that are appropriate. You don't have to worry about where the spatial sound bit of it because the SDK does all that for you. But this thing, the SDK does what's called the head-related transfer function, which is very clever, I'm sure. But all that means is that it mimics what your ears would normally hear. So a sound, the speakers above your ears on the HoloLens will put out sound that is slightly adjusted to each based on where the direction of the sound is. So if I heard something over there, it would reach this ear later than that ear. And that's the simulation of, I guess, what HoloLens does to make you turn your head and say, oh, it's over there. Very simple concept, but really, really hard to implement. But we don't have to, which is good. So I'll just see if I can get this demo to work as well. So does that make sense? Those four things is what you need to know, right? Those four things is everything you need to know to get started. I'm not saying it's everything you need to know ever. But it is to build relatively simple apps for HoloLens and get you started, that's all you need to know. So, sphere sounds. So these are assets in Unity as well. So, it's a bit small anyway. Down here, oops, sorry. Here we have an ambient sound, so that's a soundbite. So that's part of our project, right? It's an asset we have in this project. I don't want to go through how you build and all that because we already did that yesterday. Building and deploying your HoloLens app is a little bit tedious because you've got to go into Unity, export it to Visual Studio, open it to Visual Studio, start up, run it on the emulator, wait for it, and so on. Rene was a bit clever about it. He had some scripts that would alleviate some of that pain and make it a bit easier, but it still is a bit of a long, long haul. So, let's go back to Visual Studio. So we have, again, on start, we set up our audio source, and down here we have two clips, impact and rolling. So we have when something hits, and then we have obviously when the balls are rolling. So, naming. Remember, Jess's talk last night? The hardest thing is naming in computer science, right? Invalidating cache and naming, and it's so true. So try and make something, name something that makes sense. So on collision enters, that's when we hit something with the object. We then go and we calculate if we should play the audio clip. So that's the relative velocity. Does it roll fast enough? And then we have the uncollision stay. So that's when it stops, and then we still try to figure out, well, has it really stopped? Those are still going, type of thing. And then based on that, we play a video clip, sorry, video audio clip or not. And on collision exits, we just stop the audio source. So again, relatively simple code. The main thing again is to try and find audio clips that make sense. Now, this. Okay, now these didn't actually play audio, which I thought they would have. Let's try here again. And now it's frozen on me. All right. I'm pressing buttons. Yay. All right. Trust me, there is audio on it. I had to have one demo fail, of course. But I'll just keep going because the audio is, you click on something, it falls down, it makes a noise. These noises are sort of a bit crude. But what I was going to show you was that when it rolls down, you turn and I turn the emulator, the sound changes as well. So I haven't done anything to do that. So I'm going to go to say, when you turn your head left, make sure if the ball is rolling on the right, blah, blah, blah. There's none of that. It just comes out of the box, right? Okay. So please, any questions on any of this? Because that's kind of like, that's your super fast introduction to HoloLens coding. If, yeah. How is it to do things like WebViews or something like that? So how is it to build things like WebViews? Can I reword that and say, how is it to build 2D apps? Is that essentially what you're asking? Yeah. So 2D apps work really well because you can essentially pin them on the wall, right? And you, anywhere can become your screen. You can have them suspended in mid-air as well. And as you walk around them and see the backside, it's mirrored. Right? It's kind of cool. But anyway, it's not very cool when you need to work on it. So you can absolutely make 2D apps. And Outlook, so Mail, so the Windows Mail app, just called Mail, now works on HoloLens. There's a HoloLens version of it. And there was one more thing I can't remember. I think it was the Outlook or something. You guys might know. But there are Microsoft is building their 2D apps to work on HoloLens. Absolutely. And it's easy because they're all UWP. They're all Windows, Universal Windows platform apps. So you can take all your code almost and just get it and deploy it for HoloLens and have it in the HoloLens store as well. So yeah, you can absolutely do 2D apps. And I think there'll be lots and lots and lots of those because they're much easier to build. And why wouldn't you have, if you have a 3D modeling tool, you might need your Outlook there. It makes sense, right? So what's next? This is very early days. None of us are going to have a HoloLens, yet we're all here. We're all trying to find out what it is involved in building one. I hope it's not going to be like this because that would just be really awkward. But yeah, there are some social challenges because the truth is when you put one of these on, you can't help but look a little bit like a dick. I don't think it's a good look. And because you see everything else that no one else sees, you have this big grin on your face and you're starting to go, yeah, it's not a good look. So there's some social challenges for sure, but it's very early days, right? No one has one. If we see one, it's because, well, we geek, so we go, ooh, it's cool. We don't kind of think through the things that, you know, a mom will go, really? Actually, you go, really? So the challenges we have is to build software that's going to make sense and not be completely unachievable like this. The mass rover HoloLens experience. I couldn't build that. I don't have the mass data. Like, that just seems really far-fetched. But we need that to be inspired to build things that are maybe not as complex. Right? I think it's been more like some of these sort of toolbench type apps. So something where you go and you say, I can put my toolbox here and I can build things in real space. Actually, I move it over on this table because it's in the way. So you still have the HoloLens experience that I talked about earlier. Make sure that what you're building actually makes sense as a mixed reality app and not just a virtual reality app. So that's, I think this is what's going to happen a lot. As developers get their hands on this, I don't think it's going to be as professional as what Rene was building, the HoloFlight type app, because that's hard. That is a really impressive piece of software. So that's my theory, I guess. But the real cool thing is that now is your turn, right? Because all of these tools are free. Everything I used here was free. Visual Studio Community Edition, Unity for HoloLens, there's a dedicated HoloLens SDK for Unity. The emulator, all of these tools are free. And I would recommend though to find someone that knows 3D modeling, because I have no clue. I am not a 3D designer. I can't draw anything. So I'm relying on the Unity shop. So lots of 3D models you can actually buy or free ones from the shop. And there's many other ways that you can achieve or acquire 3D models as well. And basically let you imagine GoNuts, your imagination GoNuts. There's no limits almost, right? Because it's all virtual. It's a 3D world that you can build, but you can make it interact with your real world. And then we were speaking about devices before, and they are speeding up. So now Wave 3 has arrived. Waves are how they're inviting you as a developer to spend $3,000 on buying HoloLens. And it's tricky because I was speaking to various people of, can I just have one here and someone can hold it and just show it. And the kind of sentiment is well, we'd love to, but if it breaks, I can't get another one for one. There's no warranty on it, and I can't insure it. So it's very sort of, if you have one and you get access to one, you're still a bit privileged, right? And that's unfortunate, but that's why we have all the tooling. So, but they are shipping. Wave 3, I'm on Wave 4, as I said before, so hopefully I'll get invitations, get another mortgage. It's a bit like that. But I want to leave you with something that's hopefully will inspire you a bit, because we can't all be super fantastic 3D designers and modelers. I mean, most of us have a job as well, right? We need to actually make a living, and this would be something extra. So one of the guys that did acquire a HoloLens dev kit, thought, okay, what can I do with you? I'm sorry, I didn't catch that. She's very temperamental today. So a developer that did manage to acquire a developer kit built this. I just want to show you this. I think this is very cool. What I think I'm going to do, I think I'm going to just simply put a sphere in the middle of the bush, and then just go around it and see what areas are outside the sphere, and just trim them. And I don't know, let's see what happens. I'm going to try it out and show you guys whenever I get it done. So this is what it looks like after trimming. I think I did a pretty decent job. Let me show you without the sphere, which by the way is low polygon, so it's not quite spherical. So it's not that great, and I could keep going if I wanted to, but that's good enough. So this is what it looks like. I think it's pretty good. I just needed to grow a little bit more so I can trim it a little better, but I'm pretty happy with it. And that's how HoloLens can help you do gardening, right? How awesome is that? And it's simple. The implementation is really, really easy, great sphere, that's it, and you move around a bit maybe. But this guy had an idea, right? I don't know if I can get my mom to where I want to do that, that's probably the challenge. But it doesn't have to be a master over landing experience that you're building. It doesn't have to be a HoloFlight type experience, like you can do stuff that's really powerful without even having 3D modeling skills, just creating a sphere. But then the other side of things is that obviously we need more inspiration. We can't all build gardening apps, so that's not enough to get us going perhaps. So this is Microsoft Research. It's called HoloPortation, and I'll probably just let the video speak for itself, because it's pretty impressive. Maybe we'll play. Hi, today we're going to show you an exciting new technology that could fundamentally change the way that people will communicate in the future. Imagine being able to virtually teleport from one space to another in real time. Hey Sergio, how does it feel like to be HoloPorted? It feels great to be HoloPorted. So Sergio is to wear his HoloLens device, and I'm going to wear mine. We can see each other in full 3D in real time. We can interact and communicate as if we're co-present. Sergio, can you walk around my space? Can you walk around this chair? So we're doing everything to give the impression that Sergio and I are present in the same space. Sergio, let's just do a high five. That's great. Thank you Sergio. We call this technology HoloPortation. That's freaking awesome. Wow, obviously this is not for us necessarily because there's about a squillion krona worth of equipment around them. And they have an internet connection that is certainly not in Australia to manage all this data. So there are limitations for us, but that should deal with this to give you inspiration on what is possible. Now, before I just go on the last slide, any other questions? Yeah. So there is a lot of things like on the table on the walls, and you saw these nice libraries. Is there something which tells you where the table is? Is like where the supply is? So the question is that we saw how we can place things on physical objects, but is there something that tells you where the physical objects are? Why do you need to know that? Well, if I want to make an app of which I expand on my table. Oh, right. So if you want to have your app, just like we saw with the Minecraft example, he obviously looked at the table because he was very focused and went, great world. And that's, so the HoloLens will see that there's a surface and then it's up to your app. Well, your raycast to see, well, and what I'm looking at is that adequate to what my command is, I guess. So you need, there's a bit of logic there, obviously, to make sure that that surface is either big enough or adequate. But you can have surfaces on an angle as well and things will fall off. Right. If you give them, you can anchor them as well and there's all sorts of stuff you can do. But you shouldn't need, your app shouldn't need to know where physical elements are as such because it should work anywhere, right? It shouldn't matter what the room is. What your app should do is be able to say, oh, I'm interacting with something in that room. Is it possible to do whatever the user's asking? I think that's probably a better way of looking at it. That make sense? Yeah. Anyone else? Yes, sir. I had pitch, what was it? Pitch, roll, and heave. See, I don't know enough. So the question is, there was, was this in Unity? Long time ago. Okay, but there was somewhere I'd said pitch, roll, and heave, I think. And I don't know. I haven't actually seen that. Sorry, I don't know. I'm just like you guys, I'm learning this stuff. But I'll look it up. I want to find out now. Not right now. But anyone else? Wow, almost Norwegian. Before I leave you, I just want to mention PubConf. Tomorrow night at pubconf.io is all the details. I'm doing a five minute talks with lots and lots and lots of animated gifs. It'll be amazing. There's about 12, 14 speakers or something doing five minute talks. The idea is that slides automatically rotate every 15 seconds. Which would be very interesting. Bit like slide karaoke, but almost not. Thank you very much guys. Appreciate your time.
 Good morning. How cool was that press play to untape, what was it called? Press play  on tape last night? Holy crap. It made me feel really old because I knew all the games.  Thanks for coming. I just want to sort of start with a little bit of disclaimer. Did  anyone yesterday go to Renee's talk? Just to show hands please. It's a few. Alright,  so there was a HoloLens talk yesterday and Renee actually has a HoloLens and I don't.  See, no. No, it's good for you because I'm guessing you don't have a HoloLens either.  Does anyone have a HoloLens? Jimmy, hands down. No, and that's the whole idea, right?  There are really none of them around. There are, I think, so Jimmy has one. I'm pointing  it out. No, Jimmy has access to one and it's only because he gave up his first born, right?  And really hard to get. It's almost impossible. So therefore, I'm here to tell you how you  can develop for HoloLens anyway. And that's why it's cool. So I don't need a HoloLens.  So if you thought I should have a HoloLens, you can give me one. No? Okay. So why am I  here? My name is Lars and I am actually Danish, but I have lived in the land of fruit and  nuts. No, not California. Australia for a very long time, which is kind of why my accent  is like this. So I'm drink the Kool-Aid. I talk a lot about Microsoft. Windows platform,  MVP categories where HoloLens belongs for the moment. So that's kind of cool because  I fit in there. I do some PluralSight courses, which have been a bit of a mix. Kids' Cause,  some Windows phone stuff. Yes, I am the guy that uses Windows phone in Australia. And  I've been Troy's test subject on a couple as well. So if you don't know what PluralSight  is or if you would like to try, have some free passes. So feel free to grab one after  the talk. I'll write a bit about different things here. Also about HoloLens, although  I haven't written much because it's so new. And I just want to mention this. Did you know  that NDC is going to Sydney? Yeah? So yes, says the Australian. So if you want to have  another go and a trip to Australia, because we have really, really nice beaches. We have  beautiful fauna and we welcome everybody equally. We really like you to come and obviously  someone else likes would like that too. This is generally the perception of Australia.  It is not true. We are quite a civilized bunch as long as you provide us with beer. Now,  I have this experience from last year at NDC and I had a Norwegian audience. That's a thing.  Did you know that? It's like we use in the world. It's like, did you get Norwegian audience?  Oh no. Please don't be like this. Please don't be these guys. The best way to learn is to  do this. Ask questions, be interactive. I don't mind. I thrive on questions. And throughout  the talk, if you follow this hashtag, I'm going to tweet whatever is relevant in the  talk, hopefully at the same time as I talk about it. So that's just a bit of context  if you miss out on a thing or a link or a video or whatever it might be.  So enough about me. Let's talk a bit about what we do every day. So the current state  of software is that we develop things for a screen. Is that right? How many here are  software developers? Okay, everybody else? Anyone here is a manager or something? No?  Good. Out. No. This is for everyone. But the thing is we all live in this software world  and we more so than everybody else. We have a duty to provide good software to users.  The problem is though that we are limited by what we can display it on. So if you're  a mobile developer, you develop for a flat screen. It's in someone's hand. Granted, you  have lots of accelerometers and GPS and all sorts of sensors that you can use, but you  are limited to that screen. It's the same if you're a desktop software developer. You  can only develop for whatever people can show on their screen. And it goes on. If you do  games, does anyone here do game development, by the way? Do we have any game developers?  It's like one in a really, really long arm. He made Donkey Kong. Because that's an advantage  in holographic programming. No question, especially because of the 3D aspect. But with games, you  still, you sit down. Unless you are using a Kinect or a Wii, which I haven't seen anyone  do without injury, then you sit down, right? And you limited to the screen again. And the  same collaboration. And this is a really cool point about HoloLens, collaboration. But you  normally just have, I don't know if you sit in point at a screen. That seems a bit odd.  But there are very, very physical limitations to what you can do. Yeah, come on in, guys.  If you sit up the front, I'll point now. I won't point you out. But the gist of all  this is that when you've been doing it for 20 plus years, which is why I knew all the  games yesterday, oh, God. It kind of, you kind of become a little bit of a boring duck.  It all becomes a little bit the same, right? And you just sort of go, another one. And  you go, oh, this one, this thing, I can use GPS. Yes. And it's sort of you try to start  focusing on the little things that make it worthwhile, right? So I'm in the same boat.  I do mainly web development. I'd love to do more Windows apps, but no one gives a shit.  Unfortunately. So that's sort of the current state. And I guess I'm here to try and tell  you that it can be really, really different. And you can start right now, even though we  don't have these beautiful devices. So let's just, I know, so just to show hands again,  because we like like double of book before, if anyone were at Renee's talk yesterday,  please just show it. Yeah, there's a few there. So, and that's okay. I'm going to try and  do a different take on it. There will be some mobile app, but I don't have a HoloLens. So  I have to do it differently, right? Okay. Just so you know what you're in for. So if  you don't like that out, no, you can say. The first thing is virtual reality. So we have  this concept of virtual reality, which has been around for God knows how long I was,  I think it was eight or nine when I did virtual reality for the first time. And it was in,  I don't know, in this contraption thing, and you're holding a, like a trigger that would  make you move. And then you're supposed to shoot the other guy. And it was all eight  bit and it was all, it was kind of like the demo yesterday. It was all squares and spheres.  And that was it, right? So it's been around for a long time. But it is a complete virtual  world. So everything you see is replaced. Everything you see in this world is made by  someone else. It has very limited physical movement, obviously, because if I had an  Oculus Rift on now, I would walk off the stage, right? So you can't really move around because  you can't see where you're going. But it's great for prototyping. If you're trying to  show someone what a new building is going to be like, there's nothing better than giving  them on a virtual reality and they can look inside the building, right? So there are really,  really good use cases for it. Next thing is augmented reality. So you probably have a  smartphone. Yes, anyone doesn't have a smartphone? Because I want to see it. Now, it's so prevalent,  right? We all have these devices in our pockets, which are mini computers. Augmented is kind  of like a tiny bit of what HoloLens can do, I guess is the best way of explaining it.  It's usually triggered by some sort of object. And your phone will pick it up with a camera  and it'll show something, it'll augment the reality. Usually what I've seen is brochure  here. So you have, I want to buy a boat. I point my smartphone app at this boat and  the boat comes alive on the screen, right? It comes 3D. Very cool. It's really powerful.  And then because HoloLens didn't really fit into any of these sort of digital realities,  as I like to call them, they had to come up with their own. So it's called mixed reality.  And I'm sure this may change in the future as people find out more about what it can  do. But right now the term is mixed reality. And these are all digital realities, just  different ways that we are interacting with a digital reality. So the HoloLens, the whole  idea is you have seamless integration of real and digital. If you went to Renée's talk,  you saw this. You saw that it'll interact with whatever physical devices you have next  to you. The whole idea is to have natural interactions, right? So whatever you do normally,  it's still getting there. I wouldn't say it's still normal to do this all the time. But  we're getting there. It's an interaction that you can do. It's a physical movement that  you could do in real life. And then the whole idea is you can use any room, any surface,  anywhere. So normally when you look at virtual reality, and Troy kind of touched on this  in his keynote, you get this feeling, right? Like this is usually the reaction people get.  It's like, holy shit! Because it replaces everything. And you don't have your in-area  balance necessarily. You don't know what's coming. You can't escape it other than just  scream, right? And it does replace your entire physical reality to something that is much  more real, I guess. I love that one. So that's what Sonic looks like in real life. But HoloLens  to me is why we're here. And it is just absolutely pure magic. I tried it on. I have used one.  And it does blow your mind. Even though you've read everything about it, and you've seen  all the videos, it's just that whole change of what your world is, is quite impressive.  One important thing that I have come across though, the more I read about this, the more  I use the tooling, the more I try and come up with ideas that are really going to blow  people's mind, as we always say. We need to understand what is the difference between  HoloLens app and a virtual reality or augmented reality app? And it's an important difference  because I don't want to take any of Renee's thunder away because the flight, the Holo  was a Holo flight yesterday, was absolutely impressive, right? But to me it wasn't a HoloLens  app. And I might be a bastard for saying that, but it just wasn't. Because it wasn't interacting  with the real world. The planes, if they could crash maybe into the wall or whatever, it might  be, but that would just be morbid. It wasn't a proper HoloLens app. It was impressive and  it was really, really polished and well done. No question about it. But something like another  test or not test, but demo app that Microsoft has is this game. And in this game, which  I forget what was called, what was it called? Fragments, which is fragments of memory for  this boy and trying to solve a mystery. What happens to the boy? And you look around the  room and things get replaced depending on what room you're in. And people will come  into your reality, they'll sit down on your couch and talk to you. Like that to me is  a holographic app. It's HoloLens app. It changes what your physical world looks like. Important  distinction at least to me. Anyway, okay. Let's have a look at some of the hardware  specs. I love this picture because all we've seen is on the right. We've all we've seen  is that HoloLens, right? And it looks pretty good. Looks a little bit like you're from  X-Men, but it looks pretty good. But it came from the other one. It came from that prototype.  And that's the earliest prototype I could find any pictures of. And the thing he's  got on his head, that black bit is actually a Kinect. So he's got a Kinect on his face  along with all the other gadgets. So just bear in mind, we've come a really long way.  I'm not sure how long it's taken. I think it's about five years so far, five, six years  project. So it's not simple. So the specs for this. They are important as much as we  like building software. I'm not a hardware guy, really, as long as my laptop works and  it doesn't hold up. I'm pretty good. But it's important to remember what is inside this  thing. So that's why I say hardware and then I'm going to say Windows 10. It's not hardware.  But it's important to know that Windows 10 runs on this device. And as Renee touched  on yesterday as well, UWP is how you develop apps for this. So you use the universal Windows  platform. And you can use, if you're a masochist, C++. I'm not man enough to use C++. So I  use Cshare. Just because it works everywhere and it does a lot of the heavy lifting for  me. But there are scenarios where you have to use C++ just because of the performance.  So I'll just walk over here because I like you guys too. It's an Intel processor, so  it's 32 bits. And people are, oh, it's not 64. I don't think it matters. Often 64 just  means that it takes a little bit longer to process because the memory space is bigger  and you need a more powerful processor. There's lots of drawbacks as well. So I'm not too  worried about that. This is cool though. HPU. Anyone heard of HPU before? No? A holographic  processing unit. Yeah, now we're talking X-Ment, right? How cool is that? But it is the thing  that makes holograms real. It is the thing that calculates all these millions of polygons  that fast. And I don't know, there hasn't been any specs officially announced about  it, but rumor says it's about a terabyte a second of data that it processes. So I, you  know, it's a pretty serious thing. Two gigs of RAM, which doesn't seem like a lot. I thought  it would be like with today's RAM, we're like 200 or something. But two gigs of RAM, 64  gigabytes of storage is where you install your apps. So it still is a fully self-contained  computing device, right? Has four mics. So the mics above your head, essentially, if  you saw on the device, and it's really good. They're really, really good. I'm very impressed  with the speech recognition in Katana. It's Katana using it, but in the HoloLens. And  then four speakers, which sits around you, but I'll get back to that later. And then  Bluetooth, Wi-Fi as well. But the one thing that is the most important thing, take away  all this googly-gar here, is the fact that this thing is untethered. Like, there is no  wires off it. It is a completely self-contained unit. And that to me is, it blows everything  away. Because you can walk, you can put a hologram here on the wall. So that's in your  living room. And that might be, you know, Netflix streaming or something. And you say  to it, Netflix, stop. And it'll stop. And then you walk in your kitchen, you make a  cup of coffee, you come back, and it's exactly where you left it. Like, you don't have to  remove the whole, and you can take it off and on again if you want. But things are exactly  where you left them, if you're a good developer. Because it's not that easy. There are lots  and lots of abstractions and tools to let you do it. But it can be a little bit, let's  say daunting to read all the documentations on all the various things that make a hologram  stick. But I won't go into that, because it's boring. This is normally what you see. Have  anyone here followed HoloLens sort of marketing blurb and tweets and Facebook stuff and seen  the videos and all that sort of stuff? Yeah? Yeah, a few nods? Please don't be Norwegian.  Come on. So you've seen this. And this is normally what you get. You say, oh, this guy's  staying there. And look, he's got the ocean and it must be Australia because the shark's  about to eat someone. And this is actually not how it's going to look. Because the feel  of you and this thing is very, very narrow. It's more like this. So it is like a box in  front of you, right? And that's why we're talking about sound later, because sound becomes  really important. You can't see much. And that's why every single demo you see, they  kind of look like this, because they can't see anything, right? They're trying to look,  even the whole hologram can't fit inside that field of view sometimes. And it takes a little  while to get used to, maybe an hour or two. But it is a limitation for sure. It is something  that you need to be aware of, because people can't necessarily see everything they do.  And the other thing is you put a hologram over there, and then you saw Renee yesterday,  he just fired off 100 of these meatballs. If you had put 100 things different, how are  you going to remember where they are? Like you have no idea, right? So those that as  well, you can't see necessarily, you forget where you put things. All right. Just to blow  your mind a little bit, let's have a look at this video. It's from E3, if I can get  over here. Which is a gaming conference. Oops, I'm not putting sound on. Sound please.  Hello. Is there sound on? I'm not sending any sound. Oh no. All right. I'm going to  do that again. There are more guards. Still nothing. Oh, that's really disappointing.  That worked about five minutes ago. Hang on. No. It says it's plugged in. I got a notification.  Nothing. All right. Bear with me guys. Because it's important. I do need sound later. So  let me just get this sorted. No. Yeah, that's what I did. That one. Thank you. See, I'm  not a hardware guy. Thank you. Okay. Right, Clay. I really don't know what I'm doing here.  Yay. Uh oh. Problem. All right. Let's try again. I didn't enable the other one. I'm  not going to do it again. Really? All right. That's disabled. Okay. Touching. Welcome to  my talk on hardware configuration 101. All right. Let's keep going. So this is from E3  and this was one of the earlier videos or demos that Microsoft did on stage.  It wasognin e3a, since  on your Minecraft world.  It's awesome to play with the controller,  but could we show them something new, Sax?  Sure.  Let's take our experience off the wall  and then put it on the table over here.  Create World.  I see Lydia way up there.  As I run around and play, Sax can easily navigate  and manipulate the world using his voice and his hands.  He can walk around the hologram, pan around  for different viewpoints, and even look inside.  How cool is that?  Holy crap.  And that's why they use Minecraft, right?  Because we all know it, and we all know it as a thing  on a screen.  And then suddenly they make it 3D in real life.  So this was an early example.  Things have moved on since then.  But I still like this because it really  showcases what is possible in a way that we can relate to.  So let's have a look now.  Is there any questions, by the way?  I'm going to keep asking.  OK, building blocks of how do you build a HoloLens app?  Because there are elements that are really important.  There's four in particular that you need to pay attention to.  So we've got Gaze.  Gaze is what you're looking at.  Gaze is essentially how you select things  as with a mouse, with a mouse.  So it's your mouse cursor.  And it's not your eyes.  So you don't have to program for people looking.  It's the center of the HoloLens as a ray.  So it's usually called Raycast.  And that's because the object that you use in programming,  as we'll see in a second, is called Raycast.  And the Gaze is your mouse pointer for whatever object  you want to interact with.  So it's important.  It's important that you know what people are wanting  to interact with.  So let's have a look.  Just quickly.  So I've got, here's something I prepared earlier.  Can you all see that?  All right?  Yeah?  I know it's a little bit tricky in this room with the middle.  I mean, I feel like I'm in a tennis match, right?  I'm doing this all the time.  But you guys just try and see if you can see the screen, OK?  So all HoloLens objects are built in Unity.  Actually, we can start in Unity just for good measure.  So you set up a scene, which is essentially your app.  So an app has at least one scene.  And I have some 3D objects here.  So this is my cursor here.  So if I double click on that, you get lots of information  about it.  And if you don't know Unity, don't be scared.  I had, like six months ago, I had no idea what it is meant.  No clue.  It is not that scary.  Most of these things, you never have to touch,  or have to touch once.  But you set everything up in Unity.  In this case, we have this cursor, which is just a 3D  object of a ring.  And that ring, we want that to be where people are looking,  right?  We want that to be the indication that you're looking at  something and where is this placed.  So the way you do this is you create scripts.  So down here, I've got folders.  And I have some scripts in these folders.  And they are just C-sharp scripts.  If you notice, C-sharp, it really is as simple as that.  We create a script by going through all the far new  script, blah, blah, blah, blah.  And then we drag and drop that script onto the cursor.  In this case, it's the world cursor, right?  I just want to show you the code, because otherwise we  can spend really long time just going through all this.  Every single script that wants that you get attached to  objects have a start and an update.  Start is basically the way you set things up that runs once  on initialization of the app.  An update runs every frame.  So there's 60 frames a second on HoloLens, without  exception.  Everything has 60 frames a second, because that's the way  you make people not throw up.  So it's really important that you have really, really good  frame rate.  And there's lots of tooling to help you do that.  But all this does is that we have a mesh renderer.  Actually, that's the mesh for, yep.  And in here, we have the transform.  So that's where the camera, or HoloLens, is facing, the  position.  So that's your head position, whether you do this.  And then where forward.  So that's where you're looking.  What's the raycast, right?  And then you use this physics.raycast, which is a  boolean.  Give it the position of where you're looking.  And if that's true, then you can use this hit info, which  is a raycast hit.  So that's what am I looking at.  This turns false if you're not looking at anything, or if the  little ring doesn't hit anything, right?  Which is why we have enabled false.  Pretty simple.  If we do hit something, we enabled the cursor, the  mesh renderer.  And then we set the hit info.  So the hit info of the raycast and the point of that is  what we set the transform position to be.  So that's where we're looking.  And then we can rotate it.  So the position of the mesh, so this is complicated.  The position of the mesh compared to where we're  looking, right?  So if we're looking, as we've seen in a minute, at a ball,  we'll put that on the ball.  And then we have the rotation here, which we're saying,  well, let's make it hug the surface.  So it looks a bit more real.  So you don't just have this flat ring that sort of sits not  in real life.  So the way that looks is, if I just go in here, we'll just  debug it without debugging.  So this is the emulator, right?  And that's a really, really cool tool.  It's actually incredibly well done for a first-gen device.  The emulator does everything the HoloLens does, more or  less, obviously without the physical experience of it.  But it gives you 3D space.  It gives you all of the, I'll just do this.  Here we go.  Reactivate it.  Yeah.  Has all the tooling goes with it so we can see where  everything is.  Let's move it a little bit more.  So we have all the coordinates of everything that you would  be looking at.  So this is your head coordinates.  Why is this not starting?  Come on.  So that's the start menu.  I'll just press Start.  And I'm just going to go over here.  And I'm going to select my origami.  So the tap is what I was doing.  That's just the Enter key.  So it's as simple as that.  You just have Enter, and you have Start.  And that's pretty much it.  So here's our little app.  This is our, and I'm using the arrow keys and the gamer  keys, WASD.  And I'm going to move around this.  And as you can see, when I go over the surface here, hey,  let's just zoom in a bit.  I'll move in.  That's our ring, right?  So we're hugging the surface as we go around.  And that's it.  That's all you do.  Well, obviously, you can have this cursor be anything.  You can be your company logo.  It can be your finger.  It could be whatever you want it to be, right?  So that's gaze.  It is really as simple as that.  OK.  The next thing, which is probably a little bit more  interesting, is, what's that?  Here we go.  Gestures, right?  So we want to be able to interact with these elements.  I love that one.  We want to be able to interact with the objects that we're  looking at, and that's where we use gestures.  The most common one of the, hello?  Oh, you're still here.  Now, the most common one of these is the tap.  And there's what's called a gesture field of view, or  gesture frame.  And that's essentially where the whole lens is looking for  your finger.  So you can have actions based on whether it can see your  finger or not.  But essentially, it's all this, this.  You just do this all the time.  There's a whole lot of this.  And that's mouse clicks.  And it is meant for you to do this.  So if you move your head, it can't see your finger, right?  So you start becoming a robot because you've got to sort of  move your hand with everything.  It's a bit mechanical, but it works really well.  It does pick it up really, really well.  OK.  So that's the tap gesture.  It also comes as tap and hold, or tap and hold and move in  3D space so you can move things around and manipulate them.  And then you have the bloom, which is just that.  That's just start menu.  That will always work.  You can always get the start menu up.  And again, see, there's again more marketing shots.  No field of view displayed at all.  So let's just try and make a few gestures, shall we?  OK.  So the way that you do gestures is you have a gaze  gesture manager.  So it's built in the SDK.  So all of these tooling, by the way, are free.  And if you follow the last Twitter handle, I will eventually  tweet a link to where you can get all the tooling and how to  set it up.  And I wrote an article yesterday on the whole introduction  to building HoloLens apps.  So we have a game as a gesture recognizer, which we  actually recognize the gestures.  And we then, on start, which is our initialization, we  attach these events, event handlers.  So tapped event comes for free.  We're going to do that a lot.  And all that does is that if it detects a tapped event, then  we say, well, if we're actually looking at something, then  we send it an on select action.  And then we just start capturing gestures.  And that shall start.  And then every frame, we do a raycast again.  And we set the focused object.  So every frame, we get the raycast.  And we look at, OK, are you looking at something?  No.  Yes.  No.  Bull, bull, bull, bull, bull, bull.  Hold on.  60 times a second.  Which means that when we actually look at something, we  tap, it knows whether we focused on something or not.  And that is really as simple as that.  And then we cancel gestures and start capturing them.  Bit of optimization there.  So we get our right frame rate.  And all that does, so again, we up here.  I'll just click on my emulator.  So I can move around here.  So when I, oh sorry, I forgot to show you  what actually happens.  So here, on select, we click on the right frame rate.  On select, that's that method here.  So that's attached to the object itself.  The thing that I can select, the thing that I tap on,  we'll get this.  So if you tap on something that doesn't have the on select,  well, nothing happens.  It's just an event.  Does this look familiar?  C-sharp developers?  This is something you could do, right?  Well, you can read this.  It's not rocket science.  It really is quite straightforward.  Because initially, when I thought a HoloLens app,  oh, God, it's going to be tricky because there's maths  and there's 3D space in there.  But no, it is initially very simple like this.  As Rene said, you do get into situations  sometimes that are not.  All this does is it says, if you don't have a rigid body,  that essentially just means, do I have physics?  So does the 3D object have physics?  If not, you give it physics.  So what happens is that when you tap on it,  so I'm just going to hit the Enter key here, falls down.  Because I'm in the room.  Actually, I'll show the room.  And they're suspended in mid-air because they don't have physics.  They're like magic, right?  One thing I just want to show you now,  we're looking at the emulator.  So there's all these things here.  There's all the body was that moved.  If I tilt the head, so if I move here,  so Q and E will tilt the head.  So you see my head can roll.  Because you could do this when you have HoloLens on.  So you get that information as well.  You can select what room you're in.  So their room's preloaded.  If you have a HoloLens, you can record your own room  and put it into the emulator.  Makes it even more useful.  But all this is that has this device portal, which is a website.  So this is now connected to the emulator.  And if you had a physical device,  you would get the same information just  for the physical device.  And the really cool part of this is this 3D view.  So this is my plane.  So I just used the mouse key here to go in and look.  I mean, obviously, this doesn't look that impressive,  because it's just a bunch of squares.  If I update my surface here, it builds the room for me  that I'm in.  So I can fully completely see where this room is.  Just a little bit like that.  Move it over here.  And then if I open the emulator, as I move around,  see it moves over there as well.  So obviously, I would have this on a different screen.  I normally work with three screens,  just because there are so many tools  that you want to see at the same time.  So you get a fully, I don't want to say immersive,  but a full experience of what you would experience  in a HoloLens.  And there's many more tools in this,  especially around performance, which  is a complete talk on its own, because it's  a reasonably complex topic if you want to have lots of holograms,  especially at the same time, even if they need  to interact as well.  So still no questions.  Is this?  Yes?  Yes?  I have a question concerning, I think this is the coolest thing  that's happened in the last couple of years.  So do you know anything about the plans of rolling out  consumer versions?  Do I know anything about rolling out consumer versions?  Is anyone here from Microsoft?  No, I don't know a thing.  I have no idea.  No, I actually don't know.  There's been nothing I've heard at all.  Right now it's dev kits.  And the one I tried is very much a dev kit.  It's like a prototype, essentially.  They're quite fragile.  They do work fully as you see in the videos.  There is no magic there in terms of Photoshopping.  But it is, the field of view and the way that it feels  is still very much a prototype.  But consumer versions, I don't know.  I can guess.  Two years, three years?  I don't know.  I think it's probably about that far.  But I hope it won't be $3,000 for one.  But no, I don't.  I don't know when the consumer version would be released.  Yeah?  You said field of view.  Do you think there is any plan of expanding the field of view?  So is there any plans of expanding the field of view?  I really hope so.  That's the only I can say.  There was a job at someone posted that was announced  from Microsoft of an Optics specialist or something  for the something devices unit or something.  I mean, hopefully that's a good thing.  But I would have thought so.  I think it has to do with processing power.  So once they get their hardware better,  they might be able to do more.  But I actually don't know.  Again, I really hope so.  Because it would make it a lot more immersive, for sure.  OK.  Let's go back and look at the thing that I really  hope is going to work.  Bit temperamental sometimes.  So voice is quite important because you can't keep clicking  on things, right?  You can't keep doing this because it gets boring  and you get tired.  So you can talk to the device and it'll understand  everything you say.  I don't know what I'm saying.  I'm just saying it's not a big deal.  I mean, but in English, it almost doesn't matter what  accent you have.  I think if you have really like strong voice,  you can't really hear it.  It almost doesn't matter what accent you have.  I think if you have really strong Scottish eyes or  something, it might be a problem.  But it actually does work really well.  OK.  I'll get rid of that guy for you.  Sorry.  There's a command which is select.  So you can always look at something and say select.  If it is selectable, something will happen.  And that says you can't override that.  It's a reserved word.  Select will always work.  And of course, Katana works, right?  Yeah, Katana.  Because it's going to bleep everything over here.  But you can say, you know, Katana.  And she'll come up and she'll start working on your device  as well.  So yeah, it's internet connected.  And then there's custom commands, which is by far the most  useful thing.  And there's lots and lots of theory around this that I won't  go into as such.  But just bear in mind that speech design is by far the  hardest part of this.  Getting someone to do something when you say something is not  hard.  But having a whole catalog of commands that you want your  users to use is by far the hardest.  Because you can't have words that are too similar.  You can't have sentences that are too long because people  won't remember.  You can't have...  So what is that?  Yeah, they have to be non-destructive, right?  So you can't have things that override other things so that  if you have words that are similar or they don't override  each other, you shouldn't have more than two syllables in  words because then it starts getting hard to recognize it  or hard to.  So there's all these guidelines around how you design your  speech or your voice commands.  And that's the hard part.  So I want to show you the easy part, which is really, really  quite simple.  And I was surprised because this is...  I've done a fair bit of Katana integration.  Has anyone here done Windows apps?  Is there any Windows developers?  There's a few.  Hello!  Have you used Katana?  Has anyone here used Katana on an Android, on an iOS, on a  Windows phone, whatever it might be?  Oh, come on.  It's like the good version of Zerry.  Come on.  No, I know what you're saying because it's a thing we're not  used to, it's an unnatural thing to go around and go, hey,  Katana, tell me a joke.  It gets boring quick.  By the way, Katana's told over half a billion jokes now.  Someone does it.  Oh, no, no.  Stop, stop, sorry.  Told you she was going to wake up.  All right, I can show you, but first I need to quit the  current app.  Is that okay?  No!  All right.  I'm not even sure what she was going to quit.  Wow.  Calm down.  So let's have a look at speech.  And it is really simple.  You have a keyword recognizer.  Actually, I'll go down the bottom here first.  Oh, that's an error.  So you have a keyword recognizer and then you add keywords to  essentially an array.  And the keywords are text strings.  So we're back to magic strings again.  But they are just text strings, right?  And that means you can do whatever you want.  So we have two here.  We have reset world and we have drop sphere.  So they're different enough, right?  They're short, they're different.  All right, fair enough.  And then with these keywords, we attach them in here to our  keyword recognizer.  And then whenever it recognizes a keyword, it fires this  phrase recognized.  And we get all this for free.  It's all built for us.  It's all part of the SDK.  And then we have our event that says on phrase recognized.  And we invoke whatever that lambda function that we gave it  up here is.  So there's other ways you can do it as well, obviously.  But that's the gist of it.  And by the way, all this code here, I've blatantly stolen from  the Microsoft tutorial because I want you to do it.  I want you to go out and do this.  If you have any interest in holographic development, it is  that simple.  And again, follow the Twitter handle and you'll get where  all the links are.  And so what this does is that reset world sends an on reset  event and drop sphere sends an on drop event.  This is a broadcast message.  So that's actually for every single thing, not just what  you're looking at, whereas the drop sphere is only for what  you're focused on, right?  So that's a slight difference here.  I'll go back to our app here.  Let's see if I can find.  That's the problem when you're, oh, there we go.  So there's my other ball here.  I'll need to get my, there we go.  Did you remember that I dropped it to the floor, that right  one?  Now it's back again.  It's because I said the keyword and it was still listening.  So if I drop this, whee, and then I go and I do the same  with the other one.  Actually, no.  I'm not sure if I could write.  We just made a voice command.  So I can say drop sphere, eee, and it falls down, right?  That's how well the emulator works because my pronunciation  is okay, but I'm certainly not British, right?  It's not super clear.  I picked up the hang on, eee, eee, right.  The Australian part of the things, right?  And then the other thing we could do obviously is that we  could say reset world, eee, and they come back.  And they have so much fun with this.  Like you can keep making up, you know, keywords and do things  and make them dance or whatever you want to do, right?  It's actually implementing the keyword part of it is not hard,  which is my point.  Everyone here can do it.  You can.  It's really not hard.  So let's have a look then at the last of the four pillars  or building blocks that you need to know.  So we've gone through gays, gestures, and now speech.  So the last one, bleh, is sound, right?  Sound is very important because as I said, you might have people  put holograms here and everywhere.  And you need to know, or you need to let your users know  where these are if they can't find them.  So having sound, especially spatial sound like this,  what HoloLens does, is crucial.  And to be honest, I haven't mastered this.  It's really hard to get sounds that are appropriate.  You don't have to worry about where the spatial sound bit of it  because the SDK does all that for you.  But this thing, the SDK does what's called the head-related  transfer function, which is very clever, I'm sure.  But all that means is that it mimics what your ears would normally hear.  So a sound, the speakers above your ears on the HoloLens  will put out sound that is slightly adjusted to each based on  where the direction of the sound is.  So if I heard something over there, it would reach this ear later than that ear.  And that's the simulation of, I guess, what HoloLens does  to make you turn your head and say, oh, it's over there.  Very simple concept, but really, really hard to implement.  But we don't have to, which is good.  So I'll just see if I can get this demo to work as well.  So does that make sense?  Those four things is what you need to know, right?  Those four things is everything you need to know to get started.  I'm not saying it's everything you need to know ever.  But it is to build relatively simple apps for HoloLens  and get you started, that's all you need to know.  So, sphere sounds.  So these are assets in Unity as well.  So, it's a bit small anyway.  Down here, oops, sorry.  Here we have an ambient sound, so that's a soundbite.  So that's part of our project, right?  It's an asset we have in this project.  I don't want to go through how you build and all that  because we already did that yesterday.  Building and deploying your HoloLens app is a little bit tedious  because you've got to go into Unity,  export it to Visual Studio, open it to Visual Studio,  start up, run it on the emulator, wait for it, and so on.  Rene was a bit clever about it.  He had some scripts that would alleviate some of that pain  and make it a bit easier, but it still is a bit of a long, long haul.  So, let's go back to Visual Studio.  So we have, again, on start,  we set up our audio source,  and down here we have two clips, impact and rolling.  So we have when something hits,  and then we have obviously when the balls are rolling.  So, naming.  Remember, Jess's talk last night?  The hardest thing is naming in computer science, right?  Invalidating cache and naming, and it's so true.  So try and make something, name something that makes sense.  So on collision enters, that's when we hit something with the object.  We then go and we calculate if we should play the audio clip.  So that's the relative velocity.  Does it roll fast enough?  And then we have the uncollision stay.  So that's when it stops,  and then we still try to figure out,  well, has it really stopped?  Those are still going, type of thing.  And then based on that, we play a video clip,  sorry, video audio clip or not.  And on collision exits, we just stop the audio source.  So again, relatively simple code.  The main thing again is to try and find audio clips that make sense.  Now, this.  Okay, now these didn't actually play audio,  which I thought they would have.  Let's try here again.  And now it's frozen on me.  All right.  I'm pressing buttons.  Yay.  All right.  Trust me, there is audio on it.  I had to have one demo fail, of course.  But I'll just keep going because the audio is,  you click on something, it falls down, it makes a noise.  These noises are sort of a bit crude.  But what I was going to show you was that when it rolls down,  you turn and I turn the emulator, the sound changes as well.  So I haven't done anything to do that.  So I'm going to go to say, when you turn your head left,  make sure if the ball is rolling on the right, blah, blah, blah.  There's none of that.  It just comes out of the box, right?  Okay.  So please, any questions on any of this?  Because that's kind of like, that's your super fast introduction  to HoloLens coding.  If, yeah.  How is it to do things like WebViews or something like that?  So how is it to build things like WebViews?  Can I reword that and say, how is it to build 2D apps?  Is that essentially what you're asking?  Yeah.  So 2D apps work really well because you can essentially  pin them on the wall, right?  And you, anywhere can become your screen.  You can have them suspended in mid-air as well.  And as you walk around them and see the backside, it's mirrored.  Right?  It's kind of cool.  But anyway, it's not very cool when you need to work on it.  So you can absolutely make 2D apps.  And Outlook, so Mail, so the Windows Mail app, just called Mail,  now works on HoloLens.  There's a HoloLens version of it.  And there was one more thing I can't remember.  I think it was the Outlook or something.  You guys might know.  But there are Microsoft is building their 2D apps to work on HoloLens.  Absolutely.  And it's easy because they're all UWP.  They're all Windows, Universal Windows platform apps.  So you can take all your code almost and just get it  and deploy it for HoloLens and have it in the HoloLens store as well.  So yeah, you can absolutely do 2D apps.  And I think there'll be lots and lots and lots of those  because they're much easier to build.  And why wouldn't you have, if you have a 3D modeling tool,  you might need your Outlook there.  It makes sense, right?  So what's next?  This is very early days.  None of us are going to have a HoloLens, yet we're all here.  We're all trying to find out what it is involved in building one.  I hope it's not going to be like this  because that would just be really awkward.  But yeah, there are some social challenges  because the truth is when you put one of these on,  you can't help but look a little bit like a dick.  I don't think it's a good look.  And because you see everything else that no one else sees,  you have this big grin on your face and you're starting to go,  yeah, it's not a good look.  So there's some social challenges for sure,  but it's very early days, right?  No one has one.  If we see one, it's because, well, we geek, so we go,  ooh, it's cool.  We don't kind of think through the things that,  you know, a mom will go, really?  Actually, you go, really?  So the challenges we have is to build software  that's going to make sense and not be completely unachievable  like this.  The mass rover HoloLens experience.  I couldn't build that.  I don't have the mass data.  Like, that just seems really far-fetched.  But we need that to be inspired to build things  that are maybe not as complex.  Right?  I think it's been more like some of these sort of toolbench type apps.  So something where you go and you say,  I can put my toolbox here and I can build things in real space.  Actually, I move it over on this table because it's in the way.  So you still have the HoloLens experience that I talked about earlier.  Make sure that what you're building actually makes sense  as a mixed reality app and not just a virtual reality app.  So that's, I think this is what's going to happen a lot.  As developers get their hands on this,  I don't think it's going to be as professional as what Rene was building,  the HoloFlight type app, because that's hard.  That is a really impressive piece of software.  So that's my theory, I guess.  But the real cool thing is that now is your turn, right?  Because all of these tools are free.  Everything I used here was free.  Visual Studio Community Edition, Unity for HoloLens,  there's a dedicated HoloLens SDK for Unity.  The emulator, all of these tools are free.  And I would recommend though to find someone that knows 3D modeling,  because I have no clue.  I am not a 3D designer.  I can't draw anything.  So I'm relying on the Unity shop.  So lots of 3D models you can actually buy or free ones from the shop.  And there's many other ways that you can achieve or acquire 3D models as well.  And basically let you imagine GoNuts, your imagination GoNuts.  There's no limits almost, right?  Because it's all virtual.  It's a 3D world that you can build, but you can make it interact with your real world.  And then we were speaking about devices before, and they are speeding up.  So now Wave 3 has arrived.  Waves are how they're inviting you as a developer to spend $3,000 on buying HoloLens.  And it's tricky because I was speaking to various people of,  can I just have one here and someone can hold it and just show it.  And the kind of sentiment is well, we'd love to, but if it breaks, I can't get another one for one.  There's no warranty on it, and I can't insure it.  So it's very sort of, if you have one and you get access to one,  you're still a bit privileged, right?  And that's unfortunate, but that's why we have all the tooling.  So, but they are shipping.  Wave 3, I'm on Wave 4, as I said before, so hopefully I'll get invitations, get another mortgage.  It's a bit like that.  But I want to leave you with something that's hopefully will inspire you a bit,  because we can't all be super fantastic 3D designers and modelers.  I mean, most of us have a job as well, right?  We need to actually make a living, and this would be something extra.  So one of the guys that did acquire a HoloLens dev kit,  thought, okay, what can I do with you?  I'm sorry, I didn't catch that.  She's very temperamental today.  So a developer that did manage to acquire a developer kit built this.  I just want to show you this.  I think this is very cool.  What I think I'm going to do, I think I'm going to just simply put a sphere in the middle of the bush,  and then just go around it and see what areas are outside the sphere, and just trim them.  And I don't know, let's see what happens.  I'm going to try it out and show you guys whenever I get it done.  So this is what it looks like after trimming.  I think I did a pretty decent job.  Let me show you without the sphere, which by the way is low polygon, so it's not quite spherical.  So it's not that great, and I could keep going if I wanted to, but that's good enough.  So this is what it looks like.  I think it's pretty good.  I just needed to grow a little bit more so I can trim it a little better, but I'm pretty happy with it.  And that's how HoloLens can help you do gardening, right?  How awesome is that?  And it's simple.  The implementation is really, really easy, great sphere, that's it, and you move around a bit maybe.  But this guy had an idea, right?  I don't know if I can get my mom to where I want to do that, that's probably the challenge.  But it doesn't have to be a master over landing experience that you're building.  It doesn't have to be a HoloFlight type experience, like you can do stuff that's really powerful  without even having 3D modeling skills, just creating a sphere.  But then the other side of things is that obviously we need more inspiration.  We can't all build gardening apps, so that's not enough to get us going perhaps.  So this is Microsoft Research.  It's called HoloPortation, and I'll probably just let the video speak for itself, because it's pretty impressive.  Maybe we'll play.  Hi, today we're going to show you an exciting new technology that could fundamentally change the way that people will communicate in the future.  Imagine being able to virtually teleport from one space to another in real time.  Hey Sergio, how does it feel like to be HoloPorted?  It feels great to be HoloPorted.  So Sergio is to wear his HoloLens device, and I'm going to wear mine.  We can see each other in full 3D in real time.  We can interact and communicate as if we're co-present.  Sergio, can you walk around my space?  Can you walk around this chair?  So we're doing everything to give the impression that Sergio and I are present in the same space.  Sergio, let's just do a high five.  That's great. Thank you Sergio.  We call this technology HoloPortation.  That's freaking awesome.  Wow, obviously this is not for us necessarily because there's about a squillion krona worth of equipment around them.  And they have an internet connection that is certainly not in Australia to manage all this data.  So there are limitations for us, but that should deal with this to give you inspiration on what is possible.  Now, before I just go on the last slide, any other questions?  Yeah.  So there is a lot of things like on the table on the walls, and you saw these nice libraries.  Is there something which tells you where the table is?  Is like where the supply is?  So the question is that we saw how we can place things on physical objects,  but is there something that tells you where the physical objects are?  Why do you need to know that?  Well, if I want to make an app of which I expand on my table.  Oh, right. So if you want to have your app, just like we saw with the Minecraft example, he obviously looked at the table  because he was very focused and went, great world.  And that's, so the HoloLens will see that there's a surface and then it's up to your app.  Well, your raycast to see, well, and what I'm looking at is that adequate to what my command is, I guess.  So you need, there's a bit of logic there, obviously, to make sure that that surface is either big enough or adequate.  But you can have surfaces on an angle as well and things will fall off.  Right. If you give them, you can anchor them as well and there's all sorts of stuff you can do.  But you shouldn't need, your app shouldn't need to know where physical elements are as such because it should work anywhere, right?  It shouldn't matter what the room is.  What your app should do is be able to say, oh, I'm interacting with something in that room.  Is it possible to do whatever the user's asking?  I think that's probably a better way of looking at it.  That make sense? Yeah. Anyone else?  Yes, sir.  I had pitch, what was it?  Pitch, roll, and heave.  See, I don't know enough.  So the question is, there was, was this in Unity?  Long time ago.  Okay, but there was somewhere I'd said pitch, roll, and heave, I think.  And I don't know.  I haven't actually seen that.  Sorry, I don't know. I'm just like you guys, I'm learning this stuff.  But I'll look it up. I want to find out now. Not right now.  But anyone else?  Wow, almost Norwegian.  Before I leave you, I just want to mention PubConf.  Tomorrow night at pubconf.io is all the details.  I'm doing a five minute talks with lots and lots and lots of animated gifs.  It'll be amazing.  There's about 12, 14 speakers or something doing five minute talks.  The idea is that slides automatically rotate every 15 seconds.  Which would be very interesting.  Bit like slide karaoke, but almost not.  Thank you very much guys. Appreciate your time.
Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_0_59.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_60_2915.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_2916_3503.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_3504_3767.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_3768_4319.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_4320_7871.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_7872_8831.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_8832_13439.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_13440_13487.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_13488_13607.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_13608_14687.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_14688_16775.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_16776_22259.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_22260_25775.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_25776_29363.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_29364_29435.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_29436_30071.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_30072_30899.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_30900_31511.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_31512_32027.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_32028_32255.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_32256_32759.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_32760_40163.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_40164_40247.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_40248_40487.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_40488_40547.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_40548_40799.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_40800_41711.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_41712_41951.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_41952_42491.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_42492_44483.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_44484_44759.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_44760_47387.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_47388_49343.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_49344_51863.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_51864_51947.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_51948_52115.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52116_52163.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52164_52211.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52212_52271.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52272_52343.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52344_52427.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52428_52499.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52500_52583.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52584_52679.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52680_52739.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52740_52871.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_52872_53711.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_53712_57359.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_57360_57431.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_57432_58523.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_58524_61955.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_61956_63899.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_63900_64571.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_64572_66491.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_66492_66935.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_66936_69131.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_69132_72743.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_72744_73271.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_73272_74303.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_74304_75623.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_75624_77171.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_77172_77651.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_77652_77711.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_77712_77963.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_77964_78023.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_78024_78071.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_78072_78275.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_78276_78539.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_78540_78815.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_78816_78935.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_78936_79187.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_79188_79259.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_79260_79307.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_79308_79499.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_79500_80351.png Holographic_Programming_Exploring_the_HoloLens_Lars_Klint_80352_86480.png