High end augmented reality using Javascript
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Benetou (Utopiah), Fabien
VR is so 2018. Now is time for AR... or is it? In this talk we will explore practically what is feasible today and how. We'll rely on Exokit for the new Magic Leap ML1 after briefly exploring what is AR, what it's good for then some guiding principles. No AR expertise required, just basic JavaScript frontend of backend will do.
 So for our next speaker, sometimes you have to fill a dev room, you don't really know what to do, and then there is one guy who's like, oh, I like to talk. So I will present myself. The talk is from me. We'll still be super strict about the timing. So I'll probably, I also have a demo that might work and probably like it worked yesterday, but this morning kind of. So I'll just ask you to cross your fingers. So anyway, we'll try. So 25 minutes. High and augmented reality using JavaScript. That's the JavaScript dev room. So high and augmented reality means, in my opinion, the best commercially available device. You can take photo of this slide, but the very last slide of the presentation is a big ask your code that allows you to take all the slides, all the links, everything. So what are we going to see? So what's AR? What are the limitations of AR when it's native? What's AR in the browser? Specifications, because it's so exciting topic. Hardware, like an experiment life that might kind of work, but we'll see. I have video backup just in case. So am I. I'm Fabien. What do I do? I work for the European parliament. I can't really say what I do there. So that's just me smiling at a conference. I also work for UNICEF. UNICEF, who knows UNICEF? Who know, keep your hands up if you know, if UNICEF has an innovation fund. Okay. More than, more than zero, so I'm pretty happy. For others who don't know, I'll also put a link at the end. But yes, you can also do tech for good. Also I'm a Mozilla tech speaker and volunteer, so I present some of my VOR network there at Mozilla all hands last year. So to get everybody excited. It's not like we're not debate via any max because that's okay too hot, but what makes code beautiful? So is that beautiful code? I wish you can see in the back. Is that beautiful code? Is that beautiful code? That's beautiful. So we have different. So the idea is that it's kind of an excuse for me to show you after the shittiest code you've ever seen, but to argue a little bit why. So shitty, in my opinion, that's beautiful code in term of exploration. So that's a presentation from Brett Victor on how you can have a problem to solve. And if you have a way to have a vision of what's happening in the background so that you don't have to process it yourself, but let the computer do it for you, then you're going somewhere. So that's beautiful. If you're an artist or if you have to explore, let's say, a new API. That's beautiful code if you're doing the demo scene. So if you have to do a 4K demo, if you have to be really compact, then that's the kind of code you want. And that's beautiful code if you're linear-store-velled and if you think maintenance and clarity is intact. So that's up to you. But yes, be expect the shittiest code you've ever seen. What's augmented reality? So who has tried the HoloLens? Who has tried a little app on your phone to see how to paint your nails or how to change shoes or whatnot? Who has coded anything for AR? OK, quite a few. Who has coded anything for AR on the web? So somebody knows it's possible. So I think AR is really cool. That's why I work in AR. But I think also it's a lot of hype. And I think at the very beginning, when you want to have AR, you're pretty much ready to do anything, which means including downloading and then installing an app that lasts about out of five minutes and then use the app for about one minute. So that's not really worth it. Like you spend more time installing the app than running it. So I think if you're one of those big guys, nobody actually cares about your app. No offense, but it's a lot of numbers. So yeah, just don't make an app for two minutes experience if it takes you longer to download it. And most of AR now is just bite-sized experience. Just a little taste, just a little something, but you're not going to spend three hours on it. So AR, the definition, it's pretty simple. It's augmented reality. So that T-pad doesn't actually exist. It has a URL. That's it. That's the one key thing to bring home. Just for history, Kohler, it started in Georgia Tech. I mean, of course, if you do history in Tech, it always go back earlier, but in my opinion, that's where it really started with a Blur Mac entire in our app. Specs, spec, specs. The promise, of course, of WebAR, it's different on every machine possible. So it means that all the major browser vendor or the major hardware need to have it running. So the idea is to have a debate, discussion on what is your specific device that is the most exciting, but still get together on what makes a proper API. So basically, what you need for AR, lack for virtual reality is the position of the headset in space. Let's say I start here. I'm at 0, 0, 0. I step forward. I'm at 0, 1. No, 0, 0, 1. Well, it depends on your code in the system, but basically that you need to keep track of your position in space. Same for the controllers. So that's, let's say, the basics. So that's an emulator, plug-in for Chrome, if I remember correctly, that emulates the XR API. So you can virtually have a virtual headset in your browser. You can move it around. So yeah, the specs are changing all the time. And hardware supports and grows. So basically, hardware support goes from the HoloLens at the bottom to the Google Tango to your normal iPad to any kind of random phone with a camera as long as you have a marker on it. So this kind of device is about $3,000. The HoloLens is about the same price, so it's pretty expensive. I don't have it at home just for fun. I don't think it's worth it yet. But still, when you have an expensive device like this, then you have a depth camera, then you have a bunch of extra sensors, and all the processing is done on the device. So it is, in my opinion, not worth it yet for an individual, but as a professional, it's the best way to explore. And then there is a new contender. MagicLib just made this device public for creators in July, I think, last year. So the concept of really how it's working. So you have motion tracking, so you're able to see in space, thanks to either the depth camera or the RGB camera, and you do a fusion of those sensors. So you're able to see really when you're from one point to the next. So you have different techniques for it. And out of this, both your movement and scan of the room or wherever you're at, you gain room knowledge. So for example, as you probably have tried before, where are the flat surfaces? That's usually where you will put objects on. Or the largest flat surface that is vertical, or so you can keep on building on those abstractions to have, like, detecting some more complex objects. One of those key difficulties is relocalization. So for example, if I use my AR device here, and then I go away, when I come back, if I have elements in space, then I need to know that they are still there, and they are back when I'm back. So that's actually a pretty tricky problem. I was trying to think about a way to make you experience it, but it's a bit tricky. It's a bit like you close your eyes, and then a friend brings you somewhere else, and then you open your eyes, and you're like, OK, where am I actually? So you need to find similar markers or point of interest that allow you to compare where you were before. It's a hard problem. It's a hard problem. In terms of browser and implementation of the specifications, if you just use the normal RGB camera, then you use a marker, for example. It works on pretty much anything. If you use more experimental hardware, well, you'll have to go to other lenses for now. Yeah, the different browsers, you have Oregon, like I said, from Georgia Tech, Chromium, so from Google, with WebAR, you have the Modilla WebExterviewer that you can use on your iPhone already now, and a bunch of different libraries. I'm not going to explore those just for the magic clip. That's a cool video. I don't have time to show it to you, but it's like one vision of AR that, in my opinion, is quite this topic, where you have so much information and a lot about manipulation. So I think that's why I'm excited to present it to you today at FirstDem. I think AR is pretty exciting. It's pretty cool, but then in the end, it's what you can do with it. So it means what you guys can build with it. So that's also the new guy. So it's pretty straightforward architecture. So I can just keep that. You will see it after. It will make more sense. Don't worry about it. License, MIT. I'll do a little example. So that's, oh yeah, we'll see. The network allows. So that's the result. Okay, no, we don't want to do this. So that's a point cloud, all the little dots you see, thanks to the depth camera. And then you can put element in space. So those, for example, are my notes that I just put in space. And then I can move around, interact with them, make them smaller or bigger. And then thanks to the different marker, it makes in space. If I go somewhere else and come back, it will localize again and put the notes back in the right position. So the construction of her, so that's the moment. Let's see how it's, if you have questions that might also be the time, but let's try and just add all if it works. I'm not religious, but if I was, it would be the moment I would pray. Okay, let's try. I can see you guys just in case you're wondering. You will see. Hopefully everything works. You'll see yourself too, maybe. So I need to make sure it's connected. It's connected. So I need to remember now which it was. And I'll explain a bit more. Maybe not. Yes, yes, yes. Okay. So I hope I have the same IP address. Otherwise I need to around a bit more. So I'll explain a bit. So you see what I'm saying in the headset from the browser. So on the headset, I think I can actually, let me see if I can move. That's a bit abstract, but I'll explain. So what do you think is the blue box? Do you see a blue box? Yes. What do you think it is? That's my eye. So I can see I get the position of where each of my eyes looking. And then what is the gray area? Okay, easier one. What's the red area? It's a flat surface. And then the gray area is basically, maybe if I look at you guys, what is the gray area? It's a 3D mesh that's being constructed of the room. Okay. Oh, that's quick. Nice. So one of the biggest difficulties to be honest, that's why I insisted to make this one demo work is that it's really hard to convey. It's like a personal individual experience. It's a bit out there still. So to have the video running so that you can see it, well, that wasn't working yesterday. So that was a good surprise. So a little bit more example, that was the backup, but I guess we can just keep that. Let's see. Yeah. So that was another simple example with a little 3D scene added on top. So how did it actually work? Getting entire cables. So some of the basic functions, like I said before, or actually like you guys said before, the blue code is basically eye tracking. So you have your position in space on a 2D plane or further, you're looking at. Some of the basic functions are to get the mesh of the room, to get the red planes, then hand tracking or eye tracking. So eye tracking also allows you to do more fun stuff. So that was yesterday at the JavaScript stand. And that was yesterday afternoon. So everything is super fresh. My first proper component for Exocit, which is totally useless, but so that was a bit actually earlier this morning. So I start the code that I made and then it loads the Exocit, which is kind of a browser, but not, I'll explain a bit in a while. So that loading moment is always a bit of suspense. So what's happening? So I have a 3D scene, super basic, just the primitives. And then it blinks. We change color rather when I blink. So what's the point of this? I'll explain a little bit more at the end. Basically, that's another way to interact and something you cannot do basically with any device. So how is it done more precisely? So I warned you before about the ugly code. I think it's now, yes, it's now. It's ugly, but it's working because I just work on prototypes and proof of concept. So I don't have to be code that's beautiful and attainable. Kind of works. And I put console logs everywhere. And just because I have to, I can put two in a row just because I don't know what's happening. I don't know those APIs. So that's the kind of exploration I do. So basically, can you actually see this? So who is familiar with A-frame? Is that the same guy who tried before? So A-frame is basically how to do 3D on the web, either for VR, for AR, or just for 3D. It's using web components. So it's basically some kind of HTML, super basic. So for example, to have a box, you have a box. To have a sphere, you have a sphere. To have, I mean, you get it by now. So what I do, once I have those entities, I have a little component. So I register this component, which I call Blink. Then I get the entity where it is. I store the very magic function. So that's the magic specific API. Then I can see, actually, I do have data on eye tracking. And then every text, which is about few milliseconds, if I do have data on my eye and it is actually blinking, I change the color. So it's actually, if it was presented a little bit better. It's really, really clear, really simple. And yeah, it's just straight from the browser. I'm using Glitch. I don't know if you use Glitch, but it's a way to host on my page for free and then you can run it everywhere. Yeah, I think I'm pretty good on time. So now, that actually works. The trickiest part, let's say, which is not about technology, which is a lot more about, okay, you got all those new sensors and we saw about IoT before, but then it has consequences. I'm pretty excited by this kind of devices, but that's a house of a colleague. And then after that, he used his HoloLens. Then you have basically, I don't know if you can, yeah, it's a bit tricky to see, but here you have like a semi-transparent plane. Here you have a little bit of a plane. So you're outside of his house, but thanks to the model built by the device, you still have this kind of information. So if you start, I don't know if you heard about the AugmentalityDAT cloud or the AR cloud, that's very practical because before I go to see my friend's place, I can already see where it is. But if my friend is not my friend anymore, because I've done some silly presentation, I say, that's enough. I need to stop seeing you. Well, I still have access, eventually, to this data. The worst, if we're not even friends, but I mean, you know about security, you know that it's not perfect. Well, if anybody else, like an adversarial, can have access to the information, that's a lot of problems. So those kind of devices are so exciting because you can do so much with it. But that comes again with a lot of constraints on what actually information you can share and how to do it. So I become an yes. Post blog post on the Modzilla blog, and then how to do it more precisely. Well, you need a bit of information, like actually how does the device work. So you need to have workshops or presentation or help users to actually understand what are the consequences of their action. And well, you know the guy in the t-shirt, right? So to give a little bit of perspective, so that was Mark, in case you don't. A little bit of perspective, I was earlier in Brussels this week at the CPDP conference on data protection and democracy. How actually, maybe that's one of the most important non-technical part of the talk is a great book that started an article, maybe some of you have read it, on the age of surveillance capitalism. So I'm excited by this, that's cool, I can do a lot of stuff. But then it's also sold by companies, it's also some of the stack is open, some is not, and those companies have an agenda. Does it fit precisely with your lifestyle, if you need, maybe not. And if it doesn't, well maybe if it happens once again with Facebook, with Google, with others, well if that pattern is there, then maybe it's not by luck. So I really recommend this kind of book or she has also a shorter article on the topic, but there is a logic behind it. Which of course lead me to you. Somebody asked me yesterday, I was looking all dorky next to the canteen trying to finish the presentation, thinking our shit's never going to work. And then somebody came, looked a little bit at me like, no, does he see me on that? And then that person asked, okay, so I showed him a bit what it was. And it's like, but what is it for? So I have a bunch of ideas. I don't know if they're good or bad, but it doesn't really matter. So AR is for what you want it to be. That's also why it's FuzzDem, that's also why it's open source and free software is because you have the capacity to make it happen. You have the capacity to actually implement. So it means, I don't know what it is for. I have a bunch of ideas. I think I'll show you one little, but the point is it's for you to implement. So ExoKit is on MIT license, AFRAM is on Matilla license, and those are just examples still. So this little video is from a friend who was at the MIT Magically Packathon two or three weeks ago, basically. And they did an interface, a binary interface, where you can navigate in space using just a yes, no option. If you cannot see, I'll zoom actually a bit more. So it's a person in the wheelchair because he or she cannot move anything except his eyes. So that's the kind of usage you can think of. You can use augmented reality for eye tracking and then for somebody who would not be able to use VR and AR for these kind of experiences. Also like I said, I work for UNICEF and the Innovation Fund. So that's the kind of application usage you can use it for. So a little bit out there that you can use to explore. Two references and some of the doc and the slides. I don't know if you can. Is it high quality enough to scan it? You'll see anyway. It's on the website. So I have three minutes for questions. Or you get everything and then use the need for questions. How much does the device cost? Too much. 3,000. Like I said, to play at home, maybe not if you need to work with it. It's OK. Again, so for fun, it's not worth it. For exploration, to work on it's worth it. And then if you do any kind of software project, I don't know, you charge minimum 10,000 for a project that makes any kind of sense. So 3,000 for 10 minimum 1,000 euros. It's still not negligible, but it's not that much really in my opinion. Yeah. You said you work for UNICEF and the Innovation Fund. What kind of application do you see for the presentation? So I see a bunch of applications, but the thing, the beauty of the Innovation Fund is we don't, we tried actually before to say, hey, you have a problem. So I was in Ghana in Nigeria last week, no, two weeks ago. And we tried before to say, hey, you in Ghana, your problem is this, your solution is that. That didn't work. So now what we do, we search entrepreneurs for profit companies so that they're well after and they're knowledge desperate in the local ecosystem to actually implement those solutions. Because we don't know. So we have ideas, we have technology, we focus on whatever you, the IoT, IoT blockchain, digitalization, machine learning, AI, VR, AR and on the web. So we have technology, we identify some places with problems, but how, what is the technical solution that's precisely not up to us. We evaluate the different startups that apply to the fund, but after that it's really up to them. 30 seconds. Last question. Get everything. I'll be outside then, and if you have more questions because you're a little bit shy, don't worry, I'm there for like 25 minutes. That's it. Thank you. I hope you enjoyed it.
 So for our next speaker, sometimes you have to fill a dev room, you don't really know  what to do, and then there is one guy who's like, oh, I like to talk.  So I will present myself.  The talk is from me.  We'll still be super strict about the timing.  So I'll probably, I also have a demo that might work and probably like it worked yesterday,  but this morning kind of.  So I'll just ask you to cross your fingers.  So anyway, we'll try.  So 25 minutes.  High and augmented reality using JavaScript.  That's the JavaScript dev room.  So high and augmented reality means, in my opinion, the best commercially available  device.  You can take photo of this slide, but the very last slide of the presentation is a big  ask your code that allows you to take all the slides, all the links, everything.  So what are we going to see?  So what's AR?  What are the limitations of AR when it's native?  What's AR in the browser?  Specifications, because it's so exciting topic.  Hardware, like an experiment life that might kind of work, but we'll see.  I have video backup just in case.  So am I.  I'm Fabien.  What do I do?  I work for the European parliament.  I can't really say what I do there.  So that's just me smiling at a conference.  I also work for UNICEF.  UNICEF, who knows UNICEF?  Who know, keep your hands up if you know, if UNICEF has an innovation fund.  Okay.  More than, more than zero, so I'm pretty happy.  For others who don't know, I'll also put a link at the end.  But yes, you can also do tech for good.  Also I'm a Mozilla tech speaker and volunteer, so I present some of my VOR network there  at Mozilla all hands last year.  So to get everybody excited.  It's not like we're not debate via any max because that's okay too hot, but what makes  code beautiful?  So is that beautiful code?  I wish you can see in the back.  Is that beautiful code?  Is that beautiful code?  That's beautiful.  So we have different.  So the idea is that it's kind of an excuse for me to show you after the shittiest code  you've ever seen, but to argue a little bit why.  So shitty, in my opinion, that's beautiful code in term of exploration.  So that's a presentation from Brett Victor on how you can have a problem to solve.  And if you have a way to have a vision of what's happening in the background so that  you don't have to process it yourself, but let the computer do it for you, then you're  going somewhere.  So that's beautiful.  If you're an artist or if you have to explore, let's say, a new API.  That's beautiful code if you're doing the demo scene.  So if you have to do a 4K demo, if you have to be really compact, then that's the kind  of code you want.  And that's beautiful code if you're linear-store-velled and if you think maintenance and clarity  is intact.  So that's up to you.  But yes, be expect the shittiest code you've ever seen.  What's augmented reality?  So who has tried the HoloLens?  Who has tried a little app on your phone to see how to paint your nails or how to change  shoes or whatnot?  Who has coded anything for AR?  OK, quite a few.  Who has coded anything for AR on the web?  So somebody knows it's possible.  So I think AR is really cool.  That's why I work in AR.  But I think also it's a lot of hype.  And I think at the very beginning, when you want to have AR, you're pretty much ready  to do anything, which means including downloading and then installing an app that lasts about  out of five minutes and then use the app for about one minute.  So that's not really worth it.  Like you spend more time installing the app than running it.  So I think if you're one of those big guys, nobody actually cares about your app.  No offense, but it's a lot of numbers.  So yeah, just don't make an app for two minutes experience if it takes you longer to download  it.  And most of AR now is just bite-sized experience.  Just a little taste, just a little something, but you're not going to spend three hours  on it.  So AR, the definition, it's pretty simple.  It's augmented reality.  So that T-pad doesn't actually exist.  It has a URL.  That's it.  That's the one key thing to bring home.  Just for history, Kohler, it started in Georgia Tech.  I mean, of course, if you do history in Tech, it always go back earlier, but in my opinion,  that's where it really started with a Blur Mac entire in our app.  Specs, spec, specs.  The promise, of course, of WebAR, it's different on every machine possible.  So it means that all the major browser vendor or the major hardware need to have it running.  So the idea is to have a debate, discussion on what is your specific device that is the  most exciting, but still get together on what makes a proper API.  So basically, what you need for AR, lack for virtual reality is the position of the headset  in space.  Let's say I start here.  I'm at 0, 0, 0.  I step forward.  I'm at 0, 1.  No, 0, 0, 1.  Well, it depends on your code in the system, but basically that you need to keep track  of your position in space.  Same for the controllers.  So that's, let's say, the basics.  So that's an emulator, plug-in for Chrome, if I remember correctly, that emulates the  XR API.  So you can virtually have a virtual headset in your browser.  You can move it around.  So yeah, the specs are changing all the time.  And hardware supports and grows.  So basically, hardware support goes from the HoloLens at the bottom to the Google Tango  to your normal iPad to any kind of random phone with a camera as long as you have a marker  on it.  So this kind of device is about $3,000.  The HoloLens is about the same price, so it's pretty expensive.  I don't have it at home just for fun.  I don't think it's worth it yet.  But still, when you have an expensive device like this, then you have a depth camera, then  you have a bunch of extra sensors, and all the processing is done on the device.  So it is, in my opinion, not worth it yet for an individual, but as a professional, it's  the best way to explore.  And then there is a new contender.  MagicLib just made this device public for creators in July, I think, last year.  So the concept of really how it's working.  So you have motion tracking, so you're able to see in space, thanks to either the depth  camera or the RGB camera, and you do a fusion of those sensors.  So you're able to see really when you're from one point to the next.  So you have different techniques for it.  And out of this, both your movement and scan of the room or wherever you're at, you gain  room knowledge.  So for example, as you probably have tried before, where are the flat surfaces?  That's usually where you will put objects on.  Or the largest flat surface that is vertical, or so you can keep on building on those abstractions  to have, like, detecting some more complex objects.  One of those key difficulties is relocalization.  So for example, if I use my AR device here, and then I go away, when I come back, if I  have elements in space, then I need to know that they are still there, and they are back  when I'm back.  So that's actually a pretty tricky problem.  I was trying to think about a way to make you experience it, but it's a bit tricky.  It's a bit like you close your eyes, and then a friend brings you somewhere else, and then  you open your eyes, and you're like, OK, where am I actually?  So you need to find similar markers or point of interest that allow you to compare where  you were before.  It's a hard problem.  It's a hard problem.  In terms of browser and implementation of the specifications, if you just use the normal  RGB camera, then you use a marker, for example.  It works on pretty much anything.  If you use more experimental hardware, well, you'll have to go to other lenses for now.  Yeah, the different browsers, you have Oregon, like I said, from Georgia Tech, Chromium,  so from Google, with WebAR, you have the Modilla WebExterviewer that you can use on your iPhone  already now, and a bunch of different libraries.  I'm not going to explore those just for the magic clip.  That's a cool video.  I don't have time to show it to you, but it's like one vision of AR that, in my opinion,  is quite this topic, where you have so much information and a lot about manipulation.  So I think that's why I'm excited to present it to you today at FirstDem.  I think AR is pretty exciting.  It's pretty cool, but then in the end, it's what you can do with it.  So it means what you guys can build with it.  So that's also the new guy.  So it's pretty straightforward architecture.  So I can just keep that.  You will see it after.  It will make more sense.  Don't worry about it.  License, MIT.  I'll do a little example.  So that's, oh yeah, we'll see.  The network allows.  So that's the result.  Okay, no, we don't want to do this.  So that's a point cloud, all the little dots you see, thanks to the depth camera.  And then you can put element in space.  So those, for example, are my notes that I just put in space.  And then I can move around, interact with them, make them smaller or bigger.  And then thanks to the different marker, it makes in space.  If I go somewhere else and come back, it will localize again and put the notes back in the  right position.  So the construction of her, so that's the moment.  Let's see how it's, if you have questions that might also be the time, but let's try  and just add all if it works.  I'm not religious, but if I was, it would be the moment I would pray.  Okay, let's try.  I can see you guys just in case you're wondering.  You will see.  Hopefully everything works.  You'll see yourself too, maybe.  So I need to make sure it's connected.  It's connected.  So I need to remember now which it was.  And I'll explain a bit more.  Maybe not.  Yes, yes, yes.  Okay.  So I hope I have the same IP address.  Otherwise I need to around a bit more.  So I'll explain a bit.  So you see what I'm saying in the headset from the browser.  So on the headset, I think I can actually, let me see if I can move.  That's a bit abstract, but I'll explain.  So what do you think is the blue box?  Do you see a blue box?  Yes.  What do you think it is?  That's my eye.  So I can see I get the position of where each of my eyes looking.  And then what is the gray area?  Okay, easier one.  What's the red area?  It's a flat surface.  And then the gray area is basically, maybe if I look at you guys, what is the gray area?  It's a 3D mesh that's being constructed of the room.  Okay.  Oh, that's quick.  Nice.  So one of the biggest difficulties to be honest, that's why I insisted to make this one demo  work is that it's really hard to convey.  It's like a personal individual experience.  It's a bit out there still.  So to have the video running so that you can see it, well, that wasn't working yesterday.  So that was a good surprise.  So a little bit more example, that was the backup, but I guess we can just keep that.  Let's see.  Yeah.  So that was another simple example with a little 3D scene added on top.  So how did it actually work?  Getting entire cables.  So some of the basic functions, like I said before, or actually like you guys said before,  the blue code is basically eye tracking.  So you have your position in space on a 2D plane or further, you're looking at.  Some of the basic functions are to get the mesh of the room, to get the red planes, then  hand tracking or eye tracking.  So eye tracking also allows you to do more fun stuff.  So that was yesterday at the JavaScript stand.  And that was yesterday afternoon.  So everything is super fresh.  My first proper component for Exocit, which is totally useless, but so that was a bit  actually earlier this morning.  So I start the code that I made and then it loads the Exocit, which is kind of a browser,  but not, I'll explain a bit in a while.  So that loading moment is always a bit of suspense.  So what's happening?  So I have a 3D scene, super basic, just the primitives.  And then it blinks.  We change color rather when I blink.  So what's the point of this?  I'll explain a little bit more at the end.  Basically, that's another way to interact and something you cannot do basically with  any device.  So how is it done more precisely?  So I warned you before about the ugly code.  I think it's now, yes, it's now.  It's ugly, but it's working because I just work on prototypes and proof of concept.  So I don't have to be code that's beautiful and attainable.  Kind of works.  And I put console logs everywhere.  And just because I have to, I can put two in a row just because I don't know what's happening.  I don't know those APIs.  So that's the kind of exploration I do.  So basically, can you actually see this?  So who is familiar with A-frame?  Is that the same guy who tried before?  So A-frame is basically how to do 3D on the web, either for VR, for AR, or just for 3D.  It's using web components.  So it's basically some kind of HTML, super basic.  So for example, to have a box, you have a box.  To have a sphere, you have a sphere.  To have, I mean, you get it by now.  So what I do, once I have those entities, I have a little component.  So I register this component, which I call Blink.  Then I get the entity where it is.  I store the very magic function.  So that's the magic specific API.  Then I can see, actually, I do have data on eye tracking.  And then every text, which is about few milliseconds, if I do have data on my eye and it is actually  blinking, I change the color.  So it's actually, if it was presented a little bit better.  It's really, really clear, really simple.  And yeah, it's just straight from the browser.  I'm using Glitch.  I don't know if you use Glitch, but it's a way to host on my page for free and then  you can run it everywhere.  Yeah, I think I'm pretty good on time.  So now, that actually works.  The trickiest part, let's say, which is not about technology, which is a lot more about,  okay, you got all those new sensors and we saw about IoT before, but then it has consequences.  I'm pretty excited by this kind of devices, but that's a house of a colleague.  And then after that, he used his HoloLens.  Then you have basically, I don't know if you can, yeah, it's a bit tricky to see, but here  you have like a semi-transparent plane.  Here you have a little bit of a plane.  So you're outside of his house, but thanks to the model built by the device, you still  have this kind of information.  So if you start, I don't know if you heard about the AugmentalityDAT cloud or the AR  cloud, that's very practical because before I go to see my friend's place, I can already  see where it is.  But if my friend is not my friend anymore, because I've done some silly presentation,  I say, that's enough.  I need to stop seeing you.  Well, I still have access, eventually, to this data.  The worst, if we're not even friends, but I mean, you know about security, you know that  it's not perfect.  Well, if anybody else, like an adversarial, can have access to the information, that's  a lot of problems.  So those kind of devices are so exciting because you can do so much with it.  But that comes again with a lot of constraints on what actually information you can share  and how to do it.  So I become an yes.  Post blog post on the Modzilla blog, and then how to do it more precisely.  Well, you need a bit of information, like actually how does the device work.  So you need to have workshops or presentation or help users to actually understand what  are the consequences of their action.  And well, you know the guy in the t-shirt, right?  So to give a little bit of perspective, so that was Mark, in case you don't.  A little bit of perspective, I was earlier in Brussels this week at the CPDP conference  on data protection and democracy.  How actually, maybe that's one of the most important non-technical part of the talk is  a great book that started an article, maybe some of you have read it, on the age of surveillance  capitalism.  So I'm excited by this, that's cool, I can do a lot of stuff.  But then it's also sold by companies, it's also some of the stack is open, some is not,  and those companies have an agenda.  Does it fit precisely with your lifestyle, if you need, maybe not.  And if it doesn't, well maybe if it happens once again with Facebook, with Google, with  others, well if that pattern is there, then maybe it's not by luck.  So I really recommend this kind of book or she has also a shorter article on the topic,  but there is a logic behind it.  Which of course lead me to you.  Somebody asked me yesterday, I was looking all dorky next to the canteen trying to finish  the presentation, thinking our shit's never going to work.  And then somebody came, looked a little bit at me like, no, does he see me on that?  And then that person asked, okay, so I showed him a bit what it was.  And it's like, but what is it for?  So I have a bunch of ideas.  I don't know if they're good or bad, but it doesn't really matter.  So AR is for what you want it to be.  That's also why it's FuzzDem, that's also why it's open source and free software is because  you have the capacity to make it happen.  You have the capacity to actually implement.  So it means, I don't know what it is for.  I have a bunch of ideas.  I think I'll show you one little, but the point is it's for you to implement.  So ExoKit is on MIT license, AFRAM is on Matilla license, and those are just examples still.  So this little video is from a friend who was at the MIT Magically Packathon two or  three weeks ago, basically.  And they did an interface, a binary interface, where you can navigate in space using just  a yes, no option.  If you cannot see, I'll zoom actually a bit more.  So it's a person in the wheelchair because he or she cannot move anything except his  eyes.  So that's the kind of usage you can think of.  You can use augmented reality for eye tracking and then for somebody who would not be able  to use VR and AR for these kind of experiences.  Also like I said, I work for UNICEF and the Innovation Fund.  So that's the kind of application usage you can use it for.  So a little bit out there that you can use to explore.  Two references and some of the doc and the slides.  I don't know if you can.  Is it high quality enough to scan it?  You'll see anyway.  It's on the website.  So I have three minutes for questions.  Or you get everything and then use the need for questions.  How much does the device cost?  Too much.  3,000.  Like I said, to play at home, maybe not if you need to work with it.  It's OK.  Again, so for fun, it's not worth it.  For exploration, to work on it's worth it.  And then if you do any kind of software project, I don't know, you charge minimum 10,000 for  a project that makes any kind of sense.  So 3,000 for 10 minimum 1,000 euros.  It's still not negligible, but it's not that much really in my opinion.  Yeah.  You said you work for UNICEF and the Innovation Fund.  What kind of application do you see for the presentation?  So I see a bunch of applications, but the thing, the beauty of the Innovation Fund is  we don't, we tried actually before to say, hey, you have a problem.  So I was in Ghana in Nigeria last week, no, two weeks ago.  And we tried before to say, hey, you in Ghana, your problem is this, your solution is that.  That didn't work.  So now what we do, we search entrepreneurs for profit companies so that they're well  after and they're knowledge desperate in the local ecosystem to actually implement those  solutions.  Because we don't know.  So we have ideas, we have technology, we focus on whatever you, the IoT, IoT blockchain,  digitalization, machine learning, AI, VR, AR and on the web.  So we have technology, we identify some places with problems, but how, what is the technical  solution that's precisely not up to us.  We evaluate the different startups that apply to the fund, but after that it's really up  to them.  30 seconds.  Last question.  Get everything.  I'll be outside then, and if you have more questions because you're a little bit shy,  don't worry, I'm there for like 25 minutes.  That's it.  Thank you.  I hope you enjoyed it.
High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__0_167.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__168_287.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__288_539.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__540_719.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__720_851.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__852_971.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__972_1799.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__1800_2795.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__2796_6911.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__6912_7259.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__7260_8723.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__8724_9143.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__9144_10523.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__10524_13823.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__13824_14387.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__14388_14555.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__14556_14723.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__14724_14771.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__14772_16211.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__16212_17315.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__17316_17531.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__17532_17675.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__17676_18011.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__18012_18503.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__18504_18551.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__18552_18635.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__18636_18683.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__18684_18791.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__18792_20579.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__20580_21131.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__21132_21503.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__21504_22055.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__22056_22319.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__22320_31763.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__31764_36143.png High_end_augmented_reality_using_Javascript_Using_markers_with_a_webcam_is_cool_but_what_about_a_Ho__36144_36256.png