Teaching machines new tricks
CC Attribution 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Drost-Fromm, Isabel
According to the Gartner Hype Cycle Machine Learning is currently at the peak of being hyped. Scanning current press publications we can find anything from Elon Musk warning about AI being the biggest existential threat to humanity, scientists fooling machine learning models with seemingly tiny modifications to street signs, machine learning enhancing smart phone pictures, as well as introductory material trying to explain what machine learning is about. According to Wikipedia "Machine learning is the subfield of computer science that, according to Arthur Samuel, gives "computers the ability to learn without being explicitly programmed." This keynote will detail what it takes to build a successful machine learning pipeline. We will explore some examples of how machine learning has evolved over the last twenty years and close with highlighting some of the implications that new machine learning based systems have.
 Hello and welcome to the keynote of the second day of the Frostcon, of the 12th Frostcon. It is really a pleasure for me to introduce Isabel Trostrom. She is a member of Apache Foundation, co-founder of the Apache Mahut project, serial incubator. So to say Mahut is a platform for environment for quickly creating scalable machine learning application and this is also what the talk is about, teaching machine your tricks, so whether bullet or route to evil, please welcome Isabel. Thank you for this warm welcome. So today we are going to talk a little bit about machine learning and its implications. Why is this even relevant? If you take a look at the gardener hype cycle and see where AI and machine learnings are floating, that's pretty much at the top. So there's a lot of hype involved, but it's also like a technology that's more and more pervasive. What we are going to look at is what it looks like to build a machine learning application so that we are going from magical fairy dust to actual understanding of what it looks like on a very, very high level. We are going to see a few of the successful applications as well as their historic background and where we are coming from. And I will conclude with a few implications. As a warning, I will probably not use up the whole time slot, so if you have any questions we will probably have a lot of time for a Q&A session at the end. So we had a little bit about an introduction of why I'm giving us this talk about machine learning. I entered this field as a researcher back in 2003 before I realized that writing real software is more interesting than publishing papers, which is when I went into industry. In industry I realized that was like 10 years ago that the libraries that we had back then, the Delta's machine learning, were either licensed under something strange like give me a call if you want to use it, so no real license actually, or they were active and visible for as long as they had research funding and afterwards that, or they couldn't scale to large amounts of data. And that's how together with a few people at the Apache Software Foundation, namely Grand Inglesol and others, we founded Apache Mahood all around a proven, scalable, open source license, was the goal of building a community that survives individual contributors not having time or interest anymore. And that's how Mahood came to be. Apart from that, I come from Berlin. If you know someone from Berlin, they don't like traveling outside. They will more likely invite you inside. So if you need an excuse to make your employer pay for your train or flight ticket to come to Berlin in summer, go to Berlin Buswords if you're interested in anything scalable, like data analysis, search, machine learning, by now also virtualization systems like Kubernetes. That's early June. Trust me, Berlin is lovely in summer. If you're not into big data and you still want to convince your employer to travel to Berlin, I'm currently trying to make a conference on everything free and open source backstage fly. It's named fospectstage.de. It's going to be two days after Berlin Buswords next year or November 20. November 20 is going to be like a kickoff workshop in this year. All things open source governance, licensing, community building, Apache way, et cetera. Why do I get time to do this? I'm currently working as open source strategist at Europeace AG. You may not know them, but raise your hand if you'll build a house or need financing for some kind of construction work in your house. Because then you probably have spoken in Germany to someone who gave you a mortgage offering, and if they were comparing different offerings, they were probably using our system in order to figure out what is best for you. The processes that we use internally to develop software are pretty much converting towards independent teams, towards making decisions close to the team where they are needed, towards making decisions without having huge hierarchies and escalation paths. This is fairly aligned with how at least the Apache Software Foundation works. The Apache Software Foundation is also a duocracy, so people who want to do something, those people that takes a decision who want to do something and who actually submit a patch. There's quite some alignment, and that's why Europeace is supporting making this conference on open source behind the scenes fly. Okay, you know a little bit about me. I'm famous for doing meetups and for taking this microphone and handing it through the audience. I will save you from that today. What I want to do with you is a quick show of hands. How many of you know what this formula is all about? Okay, pretty much more than half. Good. Someone heard about MXNet. I would love to see at least one Amazonian hand up. So it's a deep learning framework currently under incubation at the Apache Software Foundation, which is heavily sponsored by Amazon with development time. Someone knows this logo? Tens of low? A few more? Okay. About a third, maybe? Quarter? Anyone seen these logos before? Spark, Mahoud, before I mentioned it. Okay. Got a mix back. Anyone taken a machine learning course before? Okay. Hopefully you won't feel bored because the base theorems that we saw earlier is pretty much the only equations that I have in my slides. Sorry. Any geeks in the room? Okay. Anyone else who's scared of AI? Or at least who knows the movie? Okay. Why do I make this quick show of hands? Recently, I came across a study one of my colleagues sent it to me, actually, Tainer. What customers really think about AI? The 72% believe that they understand what artificial intelligence is all about. However, when asked about whether they actually used any artificial intelligence technology, only 34% answered with yes. If you then take a look at the devices that these people who have been asked were using and applications and services that they are using, it would truly more be something like 84%. So chances are high that at least if you're using some form of mobile phone, you did interact with some sort of machine learning based system before in your life. What does the press say? According to Elon Musk, artificial intelligence is our biggest existential threat. According to the MIT, maybe it could solve the world's biggest problems. If I look at the spectrum, to me, this looks like people are thinking we are talking about magical fairy dust. You take it, you sprinkle it over your project, and suddenly everything works like a charm. Or does it? And see what Wikipedia has to say about it. It's not particularly helpful. If you look at machine learning, what we read is something like, we want to have a program that makes data-driven predictions or decisions through building a model from sample inputs. Machine learning is employed in a range of computing tasks where designing and programming explicit algorithms with good performance is difficult or infeasible. It's kind of more usable. If I take a look at the machine learning book by Tom Mitchell back from 1997, machine learning is a study of computer algorithms that improve automatically through experience. Okay, sounds like magic, doesn't it? Actually it's a little bit more like mass. This is how your machine learning algorithm sees your data. This is by now fairly old-fashioned. It's an SVM model. It's a very simple SVM model, and said it only finds a linear separation. So what do we have to do to our data? As I said, for instance, in this case, a classifier can do something with it. Sometimes our language consists of two words, one of them being high-performance computing, the other word being sunny weather. All texts that contain only sunny weather, it's a green dot over here, but not high-performance computing. Probably weather forecasts, maybe. What about the red dot over here? It talks about high-performance computing and sunny weather. It's probably some research publications on how to create weather forecasts, maybe. The plus in the lower right, the blue one, only high-performance computing, probably something computer science related. So what our algorithm will do is try to see a lot of examples. We tell them which categories the example belongs to, and then it will find just the lines that separate those pluses that belong to the class from those that don't belong to it, create like a weight vector that tells it what the line should look like, plus maybe a transformation from the origin where it should lie, and then we've got our separation. All easy. In reality, of course, we don't have only two words in our language, otherwise life would be boring. So imagine doing this not in 2D, but in a more high-dimensional space, like a couple thousand, couple million dimensions. But the only things that we are doing is just drawing lines in space, just creating hyperplanes. That's everything. If you bring deep learning into the mix, you're no longer drawing hyperplanes, what you're drawing is some kind of like nonlinear separation, but essentially the concept remains the same. Okay. In order to come up with this hyperplane, what do we need to do? We want to learn from data. So where do we get this data from? Where does it live? Does it live in a database? Do we have it somewhere in a CSV format? Is it maybe in some kind of proprietary format that we first have to read and convert? Is it maybe just available on a few sheets of paper on your employees desk? What if it's not even recorded at all? In order to train a machine learning model, you will spend a lot of time figuring out where in your company the data that you need to build this model lives and to get this to a format that makes sense. So no shiny model training, model tuning, parameter tuning, just figuring out where the data lives and talking to a lot of teams and people. Great. We've got the data. We are done, right? Not quite yet. What we have now is a huge pile of documents. Could be images, could be voice, could be audio, could be music, could be something like text. But we don't have mathematical vectors yet. So what we need beforehand is like a first transformation step. We need you to come up with some kind of feature generation, feature selection. Let me tell you a story of a team that tried to build a information extraction for identifying information relevant in job advertisements. What you need for that is like a location plus at least a job title. Back in the days, these advertisements were published not only in HTML but also in PDF. So they ran these PDF documents through a text converter and then created features in order to identify what was the job title. Now guess what the most important feature was? Is this line of text located at the very bottom of the document? Why on earth? Because the PDF to text generator would take all the text that was marked as title and put it at the very bottom. So the very bottom line often was the job titles that people were looking for. There was another team building a spam classifier. The data scientists spent weeks and weeks optimizing the model, coming up with different feature transformations, with different normalizations, but he couldn't quite get it above the threshold where he could publish it and deploy it. So what this person ended up doing was going out to the ops people, giving some spreadsheet where he had all of the features sorted by how much influence they had on the decisions that the machine learning model would take. The ops people would then go ahead and say, hey, there's a feature missing over here. We know it. That's what it looks like in production. It took a couple cycles, maybe a week, maybe two. Suddenly there was a huge performance boost. Things could go out to production and everything was great. So what this means is that for feature generation, you will need to spend a lot of time talking to your teams, talking to your ops teams, talking to people who know the business in order to figure out what the best features are. When converting data, you will run into all sorts of lovely issues. How many of you have dealt with timestamp conversion? Same here. You will have to deal with typos in your data either because that's just the human error, because someone couldn't spell the whatever feature is correctly or on purpose. At least for me personally, I've registered a couple dozen times in the past with just fake identity information in order to retrieve like a PDF document that was published. So there's going to be a lot of noisy data out there. Or to speak from a location provider, there's hotel owners who purposefully give you the wrong latitude and longitude for their hotel. Why would they do that? Well you'll probably be more likely to book it if it's closer to the ocean. Suddenly it's moved. Oftentimes the data that you will need isn't captured. You will have to go through additional loops in order to deal with privacy protection. But there's also the issue of, well, nobody actually thought that this piece of data would be interesting. Maybe there's not enough data to train off. Maybe the records that you would like to put into one bucket are not linkable together. So overall, you will spend a lot of time just managing data, just getting pipelines together. All sorts of boring works that you will have to do before you come to some kind of fancy machine learning. If you want to know more about this, there's two papers out there, one by Google and one by Amazon. The one on the lower right was published like a couple weeks ago at VLDB. Okay now we've got, we've found our data. We've pre-processed. We've trained our model. We are done, right? Well, if you're a software engineer, you know that you need to test and identify how good the quality of your solution is, so you need to evaluate quality. What's the quality metrics that you want to use? If you look at literature, you will find dozens of metrics that you can use only for classification. So you will have to deal, you will have to identify whether it's accuracy, whether it's precision recall, whether it's F measure, or whether it's something like AUC or what have you. On top of that, think about the spam classification example. That's the most, that's the best known one. Your algorithm, putting something in your inbox you didn't want to see is probably not quite bad. It's annoying at worst. But your spam classifier, putting something in your spam folder that you would have wanted to read, not quite so good. So you want to put weighting on your errors, depending on your business use case. It's all fancy and shiny if you use deep learning models, if you use support vector machines like if you use these black boxes. But as soon as you talk to your manager, you will probably want to be able to explain why a certain decision was made. If you've ever built a search engine, you probably remember that your manager comes into your office and tells you, hey, why is this search result not at the very top when I search for this query? Well, clearly there's a few ranking factors, et cetera, involved that were optimized. So often what happens is that in many cases, something that is easy to explain is something that wins. Also you want to make what happens visually available. So just having a number that tells you yes or no is helpful to some extent, but you probably also want to visualize what your algorithm does. And finally, you need to deploy what you did to production. Sounds easy. You just take what you trained and put it on your server. But if you talk to your average data scientist, they will use and they will want an environment which lets them experiment very easily. And what's easy to experiment with is often something that cannot be deployed easily. So you will have to figure out how to make this transition. Now to make things worse, you will probably have a loop where your quality evaluation informs which additional data you should have explored to begin with. Plus you will probably have an additional loop. Once you've put something to production, user behavior will probably change so you will have to retrain your model. Just to tell you one story about a email provider that was building a spam classifier. If they build a model today, it was usually effective for roughly three months and after that it was useless. Already ten years ago, what happened was that at some point their disks were filling up. Why was this happening? Because spammers had figured out that the spam classifier didn't look at images so they were hiding the spam message with an image. So suddenly you have a different classification problem there. So to summarize, you will always have 80-20 rules that you already know from engineering. You will spend 20% of your time that goes into selecting and tuning your model. Plus you will spend 80% of your time in integration and feature preparation and deployment of your pipeline. You probably want this to be a continuous process where you observe your usage, you measure how good you are, you decide which feature to implement next and then you actually act and make those changes in order to re-enter the cycle. So the title you make is the cycle. The faster you make these iterations, the faster you can be than your competitors. What you want to do is to automate as much as possible. What you want to do as well is to be able to fail fast and cheaply. You want to be able to learn from others. That means that you want to build API services that make it possible that people are being tricked into supplying you more data. You probably want to make something like annotations a game. Back in research time what we did was to pay students to annotate documents. You can still do something today if you go to something like Amazon Mechanical Tour, go to other click workers. What happens there is you can train these people but still these annotations will be noisy so even there is a cost will go up. You can put out rewards for better data. Think about being local scouts on Google Maps. Okay, let's look at the history of a few applications. Anyone know where the image might be coming from? Yeah. So that's actually handwritten character recognition tasks, standard data mining tasks, standard machine learning tasks published back in 1995. Fast forward like 10 years and scientists were trying to learn to complete sentences. Imagine like a help desk scenario. You've got emails coming in and it's often the same questions that are being asked so you can try to train a model off of that data in order to help these agents to provide better and faster answers. Fast forward in other 10 years and customers swearing at their mobile phones because autocorrect doesn't do what they want. More often than not like swipe to type actually does work. Plus you've got tiny little devices that you can talk to and they will answer your questions hopefully correctly. So what we are looking at is 20 years going from raw character recognition to speech recognition and interaction. Again like roughly mid-90s what we had was something like little robots trying to play soccer. Make a random guess. When was the first time the car crossed the desert autonomously? Anyone have a guess? Five years ago? 2003. 2003? It was pretty close. It was 2005. There was a car driving through Berlin. If you come from Berlin there is this street which goes from the Siegelssoil which has a large roundabout to Brandenburger Gate. When was it? Actual real traffic. Capital of Germany. Guess? Two years ago. Two years ago? 2011. 2011. You can tell. 2011. You can still watch the video on YouTube. It's fairly impressive. So going from there we now got an advertisement for actual autonomous cars and it actually reached public discourse. So we had 20 years going from indoor roadblocks to autonomous cars let loose on real world streets at least in the US. How did we get there? What we needed was collaboration. If you think back at the soccer games at Robocup what was the basis there was that they had multi-disciplinary teams and they were sharing the code and the data sets after the competition like after the yearly competition was over. The winning team was supposed to share the code. So that others could be kick started and could work off of that. It could be far start. Essentially what this means also if you think back to your corporate environment is it's a lone data scientist coming up with a magical idea that's a lie. Think about the data scientist trying to build a spam classifier who was only boosted after talking to ops people. You will need to have a production understanding in order to understand your data. You will need to have a production understanding in order to figure out if like a drop in data is something meaningful or if it's just an outage. It being an outage is very interesting to your business but probably not to your prediction model. You will need to have an understanding of data privacy because as soon as you link data sets together and as soon as you start to acquire more data you will need to understand if you are allowed to do that and you will need to understand what kind of warning you have to give to your users. You will need to make a decision what is better for you if you want to ship pre-made models or if you want to build services and with these services gain more insight into additional data. What also helped was competition. Back in the old days what we had were user data sets that were fairly limited and what you learn on these probably don't transfer one-on-one to production settings. It helps to have something like Kaggle where you have got a wide range of data sets and have tasks that you can deal with. It's still not ideal but you want to have some tasks where you can compete on pre-agreed metrics and you want to have teams work on these tasks together. What also helps is to focus on your customers, focus on the problems that have business value. What is the most valuable problem that you can solve right now? For that you need more than just a mathematical understanding and the theoretical understanding you need a business understanding. Looking at the implications. Data intensive applications, machine learning based applications are being deployed in a wide range of settings. One of them being pages and websites where ordinary people share their thoughts and become what before was only possible for a press, namely become someone who shares the message with the wider public plus share a message that's not only verbal but which is written and which is permanent. We are going beyond traditional media. We are increasing this publication process, publication cycle and speed. We are increasing it in terms of reach. The false statements suddenly make it very much faster to the end user. We are dealing with amplification effects. As engineers what we typically say is that these are just algorithms. But there is someone training them, there is someone deploying them, there is someone tuning them. Plus there is someone selecting the data that is being fed into these algorithms in order to make decisions. We have seen changes in communications that have had impact on politics before. Sometimes they were first exploited before they were regulated or before they were used for good. Where do we see influence? It's all fun and games until it has real world impact. Looking at the ecosystem today, if you look at these algorithms and if you look at these models, they can impact real human beings life. They can have an impact on elections. Whatever algorithm selects to be shown top or to be shown as a trend or to be shown to many users can impact how these people think and act. What about training algorithms in order to predict if someone should receive an insurance contract or someone should receive a mortgage? What if there is implicit racism or implicit bias from prior decisions? Of course this will manifest in the models themselves. There is this rule of garbage in, garbage out. Whatever you see in the data that you train with, you will see in the model afterwards. You've got suddenly a question whether machine driven bias is any worse than human bias. Was human bias worse? Or is there something that we can do about the bias inherent in these models? Something we somehow deal with our input data and get the bias out of it? What about models deployed among judges or among police officers in order to drive their decisions? What about bias in the data sets over there? See autonomous vehicles are being tested on real-world traffic situations. What I mentioned earlier when we had the cycle of development, what I told you was that it's good to go through this cycle very fast in order to improve fast. What I told you was that it's good to have a fail fast scenario. We are here in Germany. In a fail fast scenario, in a car that goes on a motorway. On the other hand, machine learning can lead to more automation and a different landscape when it comes to jobs and employment. We've had that in the past as well. This is just one poem that's famous in Germany. Usually first it becomes worse before regulation helps and makes the situation better. What we have to think about is do we want machines to entirely replace certain jobs? How will these machines change jobs? What will the transition period look like? What implications will we have on society? As an engineer and I'm an engineer myself, we believe that we are taking pure technical decisions. After all, it's just a computer. It's just a program that we write. These pure technical decisions are starting to have real life consequences on fellow human beings. Even worse, there's real life consequences on human beings if you imagine these same advances being rolled out in military, impacting how warfare is being made. What impact will it have if face recognition enters military if other machine learning models are being trained in order to be used in the military? What type of regulation do we want any regulations there and if so, which type? There's one open letter there signed by several AI and robotics researchers as well as practitioners. Urging the world to regulate AI being deployed in a warfare setting. How many of you still believe that AI or machine learning really is magical pixie fairy dust? I hopefully helped the spell with magic. At the end of the day, machine learning is just a tool. It's a little bit of algorithm, it's a little bit of data that you use in order to train your models. It can be used for good and evil as so many things. At the end of the day, what it's being used for is our decision. And for that, I'm open for questions. Any questions, comments, thoughts? Yeah? Sorry? Whether or really think it's our decision. I do think to some extent it is our decision. After all, we can go and vote. And we can talk to people who can vote. Yeah? You go to your politician and you talk to them about what regulation should look like. And they will then have to enforce it. I don't think I'm that confident. As a single-dee, if I can vote, I need something to vote on. This will not come in this election. It will not come in the next four years. And by then, it's all decided anyway. So I don't see where we can vote. You can rebel, you can act against it. Maybe I did what I think the vote means. The other option would be to get organized like CFSF Egypt in order to further the cause of free software and free software licenses and become a lobbyist in that topic. And draw people together who think like you. But then it's not just a decision. Then it gets a lot more than that. Sure. Yeah? What are some practical methods and techniques to get rid of bias and forcing data to speak better? How do you answer it? So techniques for getting rid of bias in your data. First of all, you have to analyze your data set. You have to figure out where this data is coming from and then maybe start to stratify it. Like when I was working at a maps provider, what we did was like a very simple example. We had a lot of traffic, say, in Germany. So our quality metrics were always dominated by the good German performance. What we started was to look at individual metrics, like how well are we was in France or how well are we was in China. If you want to expand to China and then look at these metrics individually and start to optimize for these. Yeah. Yeah. Yeah. That would be one option. Yeah. Sorry, I don't have an answer for that. I usually start, so what's easier is if you have access to the data, you can start looking at the decisions they make and try to figure out if there's like a pattern. There's a very famous video right now, viral in the social media space where you see someone with a black hand trying to get soap in a soap dispenser and it doesn't work. As soon as they put something white over the hand, it suddenly works. Then apparently there's some kind of bias in there. Yeah. So the question was rather agencies employing machine learning are real of this bias. I'm pretty sure that those who train the models are real of this bias. You can't go through a machine learning training without being taught that there is bias in data that you have to deal with. I would doubt that the police officer on the street still understands the model so deeply that he is aware of it. Other than people telling him and training him or her. Yeah. Definitely But this is also the part where people come toücke on their жi or for repression of is the most of the kind of beautiful view on that, and the higher, the kind of the cooperative view that you just said. People tend to be afraid of change. And people tend to be afraid of what they don't understand. And I believe that so far, the ways this works isn't so well known amongst the general public. Like if I show you the model up there, which is just the line going through space, it's fairly simple, actually. But what we tend to show people is what it can do in the end user application. And this sounds magical. They don't see, like it's a bit of the tip of the iceberg problem. They see what works, but they don't see how much works they had to go into this model in order to make this one use case work. And this tip of the iceberg looks so magical that it almost looks fairly simple in order to build something similar. They don't see the 20 years of research going into it beforehand. That's my personal take. Also, the idea of turning it around is more cooperative to go ahead with AI and models. Have a public discussion. Have more better education. Yeah? I think the quote from in a mask at the beginning, I'm sure it's a tool, but I think it's motivated by the idea of a singularity. That what's your take on singularity? Will it come? Will it not come? I can't predict the future. I can't predict the future. But given what you've seen in this talk about how it works today, what's your take? When will it come? I think there is a point that there might be a point where machines get so autonomous that all of a sudden there, the ability to replicate and innovate will exponentially outgrow the same ability of mankind. If the proposition that this will happen is true, then we will have to know. So I don't know if the proposition is true. And I would like to assess it. Is there really the data that that moment will come? So the issue is that as engineers, we love talking about theoretical issues. Right now with AI, we have real issues out there. We've got cars driving on machine learning models set on where there is no law in place how to deal with them. There are machine learning models deployed out there that are capable of influencing our political decisions right now as we talk. I believe we should rather deal with these problems first before thinking about something theoretical. Totally agree. We will take a premature discussion. So we're not yet at that point. In my personal opinion, we're not there yet. We've got way more problems out there to deal with before we go into this discussion. I totally agree. I did not look into that much detail into this theory about singularity. It's complicated. So thank you very much. It's a lovely problem. And it's nice to think about it. I believe right now we've got other issues to talk about. Yeah, thank you. Yeah? I think some of these thoughts are even too hypothetical because we're disabled ourselves, even with the technology today. We have issues with driving cars autonomously in an environment that is built for human riders. And if we support our autonomic cars by technique in the streets, we can even reduce the distance from the cars and narrow the streets and all of this stuff. So that we're able ourselves to use our highways, driving our cars ourselves. And I think that's the main problem, not the technique, not the AI, but disabling ourselves to live in our environment independently of machines. That is a more critical part of my story. So what I would like to do in this discussion is to focus on the real use cases we already have out there and to figure out whether we want to do something there and if so, what do we want to do there? Instead of arguing if there is going to be in the distant future something happening, look at what's out there right now and see what we have to do if we do have to do something. Yeah? So recently, Google has given the best goal players in the world in the soft. So it was quite amazing for me to see that kind of machine and professional in the world. And I wonder, Google has a company, a big corporation, has this power now that can use this. And it's quite scary that it can predict so many things with that. So I wonder your thoughts on the preparation that has this technology and making use it for something and how can you kind of keep the balance? So we are back to where you need to get involved in politics in order to talk about regulation. Because Google isn't the only one who has this data and who has this power over you. There is a ones making it accessible. There's other companies who have the same data and who have the same models. There's state institutions that have the same data and build the same models. So it's not only about Google, but it's about the topics that you have to discuss at a wider circle. Yeah? So thanks for a very interesting talk. My question is kind of a problem to the questioner right now. And there was very little talk about general general direction that you mentioned yesterday. So first of all, I am aware of it. And I wanted to get a say from the user side and from the company side. And from the user side, it's good to be protected by hardware and tech. Sometimes we don't know what to expect kind of a separate effect. So having the ability to use this power system to see how dangerous potentially in personal information could be coming. And from the business side, will European companies be shaded in the Gulf with companies that don't have the restrictiveness of general strategies? Of course, all of them that will be. What's on your mind? So about data protection being forced on European companies versus this is an advantage or an disadvantage. Let me turn the question around. Maybe it's even an advantage to European companies being subject to this law. Because people can be certain that their data is being used for purposes that they didn't agree on. From a company perspective, who wants to roll out machine learning, of course, they also always want more data in order to build better models and in order to build better services. But from a customer's perspective, it's like two sides of a coin. You want better services that are being based on your data. So you will have to share it somehow. But on the other hand, you want to have control over your data. So maybe it's even an advantage to be able to tell your customers, hey, I'm subject to this law. It's being enforced right here. Your data is safe with us. Not just because we say so, but because the system is not just because we say so, but because the state enforces us to do so. Yeah? I was just regarding Apache. What is your views regarding deep learning and stuff? What is the roadmap for creating scale for deep learning? The question was about the roadmap of Apache Mahoud. Roadmaps go to the mailing list. It's an all volunteer project. Right now, for deep learning, most people that are active in Mahoud are also active in MXNet. So there are ideas flowing back and forth. But other than that, I would ask you to go to the dev list, ask there so that others can benefit as well. I won't make predictions on behalf of other people there. I don't see any further questions. If I may take the opportunity again, I think this shows that this development about the deep learning and AI takes the responsibilities of the engineers who are doing this and the ethical implications that the work has a bit further. I mean, I think this is what happened with IT through the last 50 years. I mean, it was just like number crunching in a not number crunching. It was just like like like Lonon Gehalso. It was just something like corporate data that were not very, and now IT developed more and more sophisticated systems. And now there it's going into the public realm where it goes about personal preferences, decisions about personality. So the societies get more involved from the outcome what happens. So this is very important work, raises very important questions. And I would like to thank Isabel a lot again. Thank you very much.
 Hello and welcome to the keynote of the second day of the Frostcon, of the 12th Frostcon.  It is really a pleasure for me to introduce Isabel Trostrom.  She is a member of Apache Foundation, co-founder of the Apache Mahut project, serial incubator.  So to say Mahut is a platform for environment for quickly creating scalable machine learning  application and this is also what the talk is about, teaching machine your tricks, so  whether bullet or route to evil, please welcome Isabel.  Thank you for this warm welcome.  So today we are going to talk a little bit about machine learning and its implications.  Why is this even relevant?  If you take a look at the gardener hype cycle and see where AI and machine learnings are  floating, that's pretty much at the top.  So there's a lot of hype involved, but it's also like a technology that's more and more  pervasive.  What we are going to look at is what it looks like to build a machine learning application  so that we are going from magical fairy dust to actual understanding of what it looks like  on a very, very high level.  We are going to see a few of the successful applications as well as their historic background  and where we are coming from.  And I will conclude with a few implications.  As a warning, I will probably not use up the whole time slot, so if you have any questions  we will probably have a lot of time for a Q&A session at the end.  So we had a little bit about an introduction of why I'm giving us this talk about machine  learning.  I entered this field as a researcher back in 2003 before I realized that writing real  software is more interesting than publishing papers, which is when I went into industry.  In industry I realized that was like 10 years ago that the libraries that we had back then,  the Delta's machine learning, were either licensed under something strange like give  me a call if you want to use it, so no real license actually, or they were active and visible  for as long as they had research funding and afterwards that, or they couldn't scale to  large amounts of data.  And that's how together with a few people at the Apache Software Foundation, namely Grand  Inglesol and others, we founded Apache Mahood all around a proven, scalable, open source  license, was the goal of building a community that survives individual contributors not  having time or interest anymore.  And that's how Mahood came to be.  Apart from that, I come from Berlin.  If you know someone from Berlin, they don't like traveling outside.  They will more likely invite you inside.  So if you need an excuse to make your employer pay for your train or flight ticket to come  to Berlin in summer, go to Berlin Buswords if you're interested in anything scalable,  like data analysis, search, machine learning, by now also virtualization systems like Kubernetes.  That's early June.  Trust me, Berlin is lovely in summer.  If you're not into big data and you still want to convince your employer to travel to  Berlin, I'm currently trying to make a conference on everything free and open source backstage  fly.  It's named fospectstage.de.  It's going to be two days after Berlin Buswords next year or November 20.  November 20 is going to be like a kickoff workshop in this year.  All things open source governance, licensing, community building, Apache way, et cetera.  Why do I get time to do this?  I'm currently working as open source strategist at Europeace AG.  You may not know them, but raise your hand if you'll build a house or need financing  for some kind of construction work in your house.  Because then you probably have spoken in Germany to someone who gave you a mortgage offering,  and if they were comparing different offerings, they were probably using our system in order  to figure out what is best for you.  The processes that we use internally to develop software are pretty much converting towards  independent teams, towards making decisions close to the team where they are needed, towards  making decisions without having huge hierarchies and escalation paths.  This is fairly aligned with how at least the Apache Software Foundation works.  The Apache Software Foundation is also a duocracy, so people who want to do something, those  people that takes a decision who want to do something and who actually submit a patch.  There's quite some alignment, and that's why Europeace is supporting making this conference  on open source behind the scenes fly.  Okay, you know a little bit about me.  I'm famous for doing meetups and for taking this microphone and handing it through the  audience.  I will save you from that today.  What I want to do with you is a quick show of hands.  How many of you know what this formula is all about?  Okay, pretty much more than half.  Good.  Someone heard about MXNet.  I would love to see at least one Amazonian hand up.  So it's a deep learning framework currently under incubation at the Apache Software Foundation,  which is heavily sponsored by Amazon with development time.  Someone knows this logo?  Tens of low?  A few more?  Okay.  About a third, maybe?  Quarter?  Anyone seen these logos before?  Spark, Mahoud, before I mentioned it.  Okay.  Got a mix back.  Anyone taken a machine learning course before?  Okay.  Hopefully you won't feel bored because the base theorems that we saw earlier is pretty  much the only equations that I have in my slides.  Sorry.  Any geeks in the room?  Okay.  Anyone else who's scared of AI?  Or at least who knows the movie?  Okay.  Why do I make this quick show of hands?  Recently, I came across a study one of my colleagues sent it to me, actually, Tainer.  What customers really think about AI?  The 72% believe that they understand what artificial intelligence is all about.  However, when asked about whether they actually used any artificial intelligence technology,  only 34% answered with yes.  If you then take a look at the devices that these people who have been asked were using  and applications and services that they are using, it would truly more be something like  84%.  So chances are high that at least if you're using some form of mobile phone, you did interact  with some sort of machine learning based system before in your life.  What does the press say?  According to Elon Musk, artificial intelligence is our biggest existential threat.  According to the MIT, maybe it could solve the world's biggest problems.  If I look at the spectrum, to me, this looks like people are thinking we are talking about  magical fairy dust.  You take it, you sprinkle it over your project, and suddenly everything works like a charm.  Or does it?  And see what Wikipedia has to say about it.  It's not particularly helpful.  If you look at machine learning, what we read is something like, we want to have a program  that makes data-driven predictions or decisions through building a model from sample inputs.  Machine learning is employed in a range of computing tasks where designing and programming  explicit algorithms with good performance is difficult or infeasible.  It's kind of more usable.  If I take a look at the machine learning book by Tom Mitchell back from 1997, machine learning  is a study of computer algorithms that improve automatically through experience.  Okay, sounds like magic, doesn't it?  Actually it's a little bit more like mass.  This is how your machine learning algorithm sees your data.  This is by now fairly old-fashioned.  It's an SVM model.  It's a very simple SVM model, and said it only finds a linear separation.  So what do we have to do to our data?  As I said, for instance, in this case, a classifier can do something with it.  Sometimes our language consists of two words, one of them being high-performance computing,  the other word being sunny weather.  All texts that contain only sunny weather, it's a green dot over here, but not high-performance  computing.  Probably weather forecasts, maybe.  What about the red dot over here?  It talks about high-performance computing and sunny weather.  It's probably some research publications on how to create weather forecasts, maybe.  The plus in the lower right, the blue one, only high-performance computing, probably  something computer science related.  So what our algorithm will do is try to see a lot of examples.  We tell them which categories the example belongs to, and then it will find just the  lines that separate those pluses that belong to the class from those that don't belong  to it, create like a weight vector that tells it what the line should look like, plus maybe  a transformation from the origin where it should lie, and then we've got our separation.  All easy.  In reality, of course, we don't have only two words in our language, otherwise life  would be boring.  So imagine doing this not in 2D, but in a more high-dimensional space, like a couple  thousand, couple million dimensions.  But the only things that we are doing is just drawing lines in space, just creating hyperplanes.  That's everything.  If you bring deep learning into the mix, you're no longer drawing hyperplanes, what you're  drawing is some kind of like nonlinear separation, but essentially the concept remains the same.  Okay.  In order to come up with this hyperplane, what do we need to do?  We want to learn from data.  So where do we get this data from?  Where does it live?  Does it live in a database?  Do we have it somewhere in a CSV format?  Is it maybe in some kind of proprietary format that we first have to read and convert?  Is it maybe just available on a few sheets of paper on your employees desk?  What if it's not even recorded at all?  In order to train a machine learning model, you will spend a lot of time figuring out where  in your company the data that you need to build this model lives and to get this to  a format that makes sense.  So no shiny model training, model tuning, parameter tuning, just figuring out where the  data lives and talking to a lot of teams and people.  Great.  We've got the data.  We are done, right?  Not quite yet.  What we have now is a huge pile of documents.  Could be images, could be voice, could be audio, could be music, could be something  like text.  But we don't have mathematical vectors yet.  So what we need beforehand is like a first transformation step.  We need you to come up with some kind of feature generation, feature selection.  Let me tell you a story of a team that tried to build a information extraction for identifying  information relevant in job advertisements.  What you need for that is like a location plus at least a job title.  Back in the days, these advertisements were published not only in HTML but also in PDF.  So they ran these PDF documents through a text converter and then created features in  order to identify what was the job title.  Now guess what the most important feature was?  Is this line of text located at the very bottom of the document?  Why on earth?  Because the PDF to text generator would take all the text that was marked as title and  put it at the very bottom.  So the very bottom line often was the job titles that people were looking for.  There was another team building a spam classifier.  The data scientists spent weeks and weeks optimizing the model, coming up with different  feature transformations, with different normalizations, but he couldn't quite get it above the threshold  where he could publish it and deploy it.  So what this person ended up doing was going out to the ops people, giving some spreadsheet  where he had all of the features sorted by how much influence they had on the decisions  that the machine learning model would take.  The ops people would then go ahead and say, hey, there's a feature missing over here.  We know it.  That's what it looks like in production.  It took a couple cycles, maybe a week, maybe two.  Suddenly there was a huge performance boost.  Things could go out to production and everything was great.  So what this means is that for feature generation, you will need to spend a lot of time talking  to your teams, talking to your ops teams, talking to people who know the business in  order to figure out what the best features are.  When converting data, you will run into all sorts of lovely issues.  How many of you have dealt with timestamp conversion?  Same here.  You will have to deal with typos in your data either because that's just the human error,  because someone couldn't spell the whatever feature is correctly or on purpose.  At least for me personally, I've registered a couple dozen times in the past with just  fake identity information in order to retrieve like a PDF document that was published.  So there's going to be a lot of noisy data out there.  Or to speak from a location provider, there's hotel owners who purposefully give you the  wrong latitude and longitude for their hotel.  Why would they do that?  Well you'll probably be more likely to book it if it's closer to the ocean.  Suddenly it's moved.  Oftentimes the data that you will need isn't captured.  You will have to go through additional loops in order to deal with privacy protection.  But there's also the issue of, well, nobody actually thought that this piece of data would  be interesting.  Maybe there's not enough data to train off.  Maybe the records that you would like to put into one bucket are not linkable together.  So overall, you will spend a lot of time just managing data, just getting pipelines together.  All sorts of boring works that you will have to do before you come to some kind of fancy  machine learning.  If you want to know more about this, there's two papers out there, one by Google and one  by Amazon.  The one on the lower right was published like a couple weeks ago at VLDB.  Okay now we've got, we've found our data.  We've pre-processed.  We've trained our model.  We are done, right?  Well, if you're a software engineer, you know that you need to test and identify how good  the quality of your solution is, so you need to evaluate quality.  What's the quality metrics that you want to use?  If you look at literature, you will find dozens of metrics that you can use only for classification.  So you will have to deal, you will have to identify whether it's accuracy, whether it's  precision recall, whether it's F measure, or whether it's something like AUC or what  have you.  On top of that, think about the spam classification example.  That's the most, that's the best known one.  Your algorithm, putting something in your inbox you didn't want to see is probably not quite  bad.  It's annoying at worst.  But your spam classifier, putting something in your spam folder that you would have wanted  to read, not quite so good.  So you want to put weighting on your errors, depending on your business use case.  It's all fancy and shiny if you use deep learning models, if you use support vector  machines like if you use these black boxes.  But as soon as you talk to your manager, you will probably want to be able to explain why  a certain decision was made.  If you've ever built a search engine, you probably remember that your manager comes  into your office and tells you, hey, why is this search result not at the very top when  I search for this query?  Well, clearly there's a few ranking factors, et cetera, involved that were optimized.  So often what happens is that in many cases, something that is easy to explain is something  that wins.  Also you want to make what happens visually available.  So just having a number that tells you yes or no is helpful to some extent, but you probably  also want to visualize what your algorithm does.  And finally, you need to deploy what you did to production.  Sounds easy.  You just take what you trained and put it on your server.  But if you talk to your average data scientist, they will use and they will want an environment  which lets them experiment very easily.  And what's easy to experiment with is often something that cannot be deployed easily.  So you will have to figure out how to make this transition.  Now to make things worse, you will probably have a loop where your quality evaluation  informs which additional data you should have explored to begin with.  Plus you will probably have an additional loop.  Once you've put something to production, user behavior will probably change so you will  have to retrain your model.  Just to tell you one story about a email provider that was building a spam classifier.  If they build a model today, it was usually effective for roughly three months and after  that it was useless.  Already ten years ago, what happened was that at some point their disks were filling up.  Why was this happening?  Because spammers had figured out that the spam classifier didn't look at images so they  were hiding the spam message with an image.  So suddenly you have a different classification problem there.  So to summarize, you will always have 80-20 rules that you already know from engineering.  You will spend 20% of your time that goes into selecting and tuning your model.  Plus you will spend 80% of your time in integration and feature preparation and deployment of  your pipeline.  You probably want this to be a continuous process where you observe your usage, you  measure how good you are, you decide which feature to implement next and then you actually  act and make those changes in order to re-enter the cycle.  So the title you make is the cycle.  The faster you make these iterations, the faster you can be than your competitors.  What you want to do is to automate as much as possible.  What you want to do as well is to be able to fail fast and cheaply.  You want to be able to learn from others.  That means that you want to build API services that make it possible that people are being  tricked into supplying you more data.  You probably want to make something like annotations a game.  Back in research time what we did was to pay students to annotate documents.  You can still do something today if you go to something like Amazon Mechanical Tour,  go to other click workers.  What happens there is you can train these people but still these annotations will be  noisy so even there is a cost will go up.  You can put out rewards for better data.  Think about being local scouts on Google Maps.  Okay, let's look at the history of a few applications.  Anyone know where the image might be coming from?  Yeah.  So that's actually handwritten character recognition tasks, standard data mining tasks, standard  machine learning tasks published back in 1995.  Fast forward like 10 years and scientists were trying to learn to complete sentences.  Imagine like a help desk scenario.  You've got emails coming in and it's often the same questions that are being asked so  you can try to train a model off of that data in order to help these agents to provide better  and faster answers.  Fast forward in other 10 years and customers swearing at their mobile phones because autocorrect  doesn't do what they want.  More often than not like swipe to type actually does work.  Plus you've got tiny little devices that you can talk to and they will answer your questions  hopefully correctly.  So what we are looking at is 20 years going from raw character recognition to speech recognition  and interaction.  Again like roughly mid-90s what we had was something like little robots trying to play  soccer.  Make a random guess.  When was the first time the car crossed the desert autonomously?  Anyone have a guess?  Five years ago?  2003.  2003?  It was pretty close.  It was 2005.  There was a car driving through Berlin.  If you come from Berlin there is this street which goes from the Siegelssoil which has  a large roundabout to Brandenburger Gate.  When was it?  Actual real traffic.  Capital of Germany.  Guess?  Two years ago.  Two years ago?  2011.  2011.  You can tell.  2011.  You can still watch the video on YouTube.  It's fairly impressive.  So going from there we now got an advertisement for actual autonomous cars and it actually  reached public discourse.  So we had 20 years going from indoor roadblocks to autonomous cars let loose on real world  streets at least in the US.  How did we get there?  What we needed was collaboration.  If you think back at the soccer games at Robocup what was the basis there was that they had  multi-disciplinary teams and they were sharing the code and the data sets after the competition  like after the yearly competition was over.  The winning team was supposed to share the code.  So that others could be kick started and could work off of that.  It could be far start.  Essentially what this means also if you think back to your corporate environment is it's  a lone data scientist coming up with a magical idea that's a lie.  Think about the data scientist trying to build a spam classifier who was only boosted after  talking to ops people.  You will need to have a production understanding in order to understand your data.  You will need to have a production understanding in order to figure out if like a drop in data  is something meaningful or if it's just an outage.  It being an outage is very interesting to your business but probably not to your prediction  model.  You will need to have an understanding of data privacy because as soon as you link data  sets together and as soon as you start to acquire more data you will need to understand  if you are allowed to do that and you will need to understand what kind of warning you  have to give to your users.  You will need to make a decision what is better for you if you want to ship pre-made models  or if you want to build services and with these services gain more insight into additional  data.  What also helped was competition.  Back in the old days what we had were user data sets that were fairly limited and what  you learn on these probably don't transfer one-on-one to production settings.  It helps to have something like Kaggle where you have got a wide range of data sets and  have tasks that you can deal with.  It's still not ideal but you want to have some tasks where you can compete on pre-agreed  metrics and you want to have teams work on these tasks together.  What also helps is to focus on your customers, focus on the problems that have business  value.  What is the most valuable problem that you can solve right now?  For that you need more than just a mathematical understanding and the theoretical understanding  you need a business understanding.  Looking at the implications.  Data intensive applications, machine learning based applications are being deployed in a  wide range of settings.  One of them being pages and websites where ordinary people share their thoughts and become  what before was only possible for a press, namely become someone who shares the message  with the wider public plus share a message that's not only verbal but which is written  and which is permanent.  We are going beyond traditional media.  We are increasing this publication process, publication cycle and speed.  We are increasing it in terms of reach.  The false statements suddenly make it very much faster to the end user.  We are dealing with amplification effects.  As engineers what we typically say is that these are just algorithms.  But there is someone training them, there is someone deploying them, there is someone  tuning them.  Plus there is someone selecting the data that is being fed into these algorithms in order  to make decisions.  We have seen changes in communications that have had impact on politics before.  Sometimes they were first exploited before they were regulated or before they were used  for good.  Where do we see influence?  It's all fun and games until it has real world impact.  Looking at the ecosystem today, if you look at these algorithms and if you look at these  models, they can impact real human beings life.  They can have an impact on elections.  Whatever algorithm selects to be shown top or to be shown as a trend or to be shown to  many users can impact how these people think and act.  What about training algorithms in order to predict if someone should receive an insurance  contract or someone should receive a mortgage?  What if there is implicit racism or implicit bias from prior decisions?  Of course this will manifest in the models themselves.  There is this rule of garbage in, garbage out.  Whatever you see in the data that you train with, you will see in the model afterwards.  You've got suddenly a question whether machine driven bias is any worse than human bias.  Was human bias worse?  Or is there something that we can do about the bias inherent in these models?  Something we somehow deal with our input data and get the bias out of it?  What about models deployed among judges or among police officers in order to drive their  decisions?  What about bias in the data sets over there?  See autonomous vehicles are being tested on real-world traffic situations.  What I mentioned earlier when we had the cycle of development, what I told you was that it's  good to go through this cycle very fast in order to improve fast.  What I told you was that it's good to have a fail fast scenario.  We are here in Germany.  In a fail fast scenario, in a car that goes on a motorway.  On the other hand, machine learning can lead to more automation and a different landscape  when it comes to jobs and employment.  We've had that in the past as well.  This is just one poem that's famous in Germany.  Usually first it becomes worse before regulation helps and makes the situation better.  What we have to think about is do we want machines to entirely replace certain jobs?  How will these machines change jobs?  What will the transition period look like?  What implications will we have on society?  As an engineer and I'm an engineer myself, we believe that we are taking pure technical  decisions.  After all, it's just a computer.  It's just a program that we write.  These pure technical decisions are starting to have real life consequences on fellow  human beings.  Even worse, there's real life consequences on human beings if you imagine these same  advances being rolled out in military, impacting how warfare is being made.  What impact will it have if face recognition enters military if other machine learning  models are being trained in order to be used in the military?  What type of regulation do we want any regulations there and if so, which type?  There's one open letter there signed by several AI and robotics researchers as well as practitioners.  Urging the world to regulate AI being deployed in a warfare setting.  How many of you still believe that AI or machine learning really is magical pixie fairy dust?  I hopefully helped the spell with magic.  At the end of the day, machine learning is just a tool.  It's a little bit of algorithm, it's a little bit of data that you use in order to train  your models.  It can be used for good and evil as so many things.  At the end of the day, what it's being used for is our decision.  And for that, I'm open for questions.  Any questions, comments, thoughts?  Yeah?  Sorry?  Whether or really think it's our decision.  I do think to some extent it is our decision.  After all, we can go and vote.  And we can talk to people who can vote.  Yeah?  You go to your politician and you talk to them about what regulation should look like.  And they will then have to enforce it.  I don't think I'm that confident.  As a single-dee, if I can vote, I need something to vote on.  This will not come in this election.  It will not come in the next four years.  And by then, it's all decided anyway.  So I don't see where we can vote.  You can rebel, you can act against it.  Maybe I did what I think the vote means.  The other option would be to get organized like CFSF Egypt in order to further the cause  of free software and free software licenses and become a lobbyist in that topic.  And draw people together who think like you.  But then it's not just a decision.  Then it gets a lot more than that.  Sure.  Yeah?  What are some practical methods and techniques to get rid of bias and forcing data to speak  better?  How do you answer it?  So techniques for getting rid of bias in your data.  First of all, you have to analyze your data set.  You have to figure out where this data is coming from and then maybe start to stratify  it.  Like when I was working at a maps provider, what we did was like a very simple example.  We had a lot of traffic, say, in Germany.  So our quality metrics were always dominated by the good German performance.  What we started was to look at individual metrics, like how well are we was in France  or how well are we was in China.  If you want to expand to China and then look at these metrics individually and start to  optimize for these.  Yeah.  Yeah.  Yeah.  That would be one option.  Yeah.  Sorry, I don't have an answer for that.  I usually start, so what's easier is if you have access to the data, you can start looking  at the decisions they make and try to figure out if there's like a pattern.  There's a very famous video right now, viral in the social media space where you see someone  with a black hand trying to get soap in a soap dispenser and it doesn't work.  As soon as they put something white over the hand, it suddenly works.  Then apparently there's some kind of bias in there.  Yeah.  So the question was rather agencies employing machine learning are real of this bias.  I'm pretty sure that those who train the models are real of this bias.  You can't go through a machine learning training without being taught that there is bias in  data that you have to deal with.  I would doubt that the police officer on the street still understands the model so deeply  that he is aware of it.  Other than people telling him and training him or her.  Yeah.  Definitely  But this is also the part where people come toücke on their жi or for repression of  is the most of the kind of beautiful view on that,  and the higher, the kind of the cooperative view  that you just said.  People tend to be afraid of change.  And people tend to be afraid of what they don't understand.  And I believe that so far, the ways this works  isn't so well known amongst the general public.  Like if I show you the model up there, which is just  the line going through space, it's fairly simple, actually.  But what we tend to show people is what it can do in the end  user application.  And this sounds magical.  They don't see, like it's a bit of the tip of the iceberg  problem.  They see what works, but they don't see how much  works they had to go into this model in order  to make this one use case work.  And this tip of the iceberg looks so magical  that it almost looks fairly simple in order  to build something similar.  They don't see the 20 years of research going  into it beforehand.  That's my personal take.  Also, the idea of turning it around  is more cooperative to go ahead with AI and models.  Have a public discussion.  Have more better education.  Yeah?  I think the quote from in a mask at the beginning,  I'm sure it's a tool, but I think it's  motivated by the idea of a singularity.  That what's your take on singularity?  Will it come?  Will it not come?  I can't predict the future.  I can't predict the future.  But given what you've seen in this talk about how it works  today, what's your take?  When will it come?  I think there is a point that there  might be a point where machines get so autonomous  that all of a sudden there, the ability  to replicate and innovate will exponentially outgrow  the same ability of mankind.  If the proposition that this will happen is true,  then we will have to know.  So I don't know if the proposition is true.  And I would like to assess it.  Is there really the data that that moment will come?  So the issue is that as engineers,  we love talking about theoretical issues.  Right now with AI, we have real issues out there.  We've got cars driving on machine learning models  set on where there is no law in place how to deal with them.  There are machine learning models deployed out there  that are capable of influencing our political decisions  right now as we talk.  I believe we should rather deal with these problems first  before thinking about something theoretical.  Totally agree.  We will take a premature discussion.  So we're not yet at that point.  In my personal opinion, we're not there yet.  We've got way more problems out there  to deal with before we go into this discussion.  I totally agree.  I did not look into that much detail  into this theory about singularity.  It's complicated.  So thank you very much.  It's a lovely problem.  And it's nice to think about it.  I believe right now we've got other issues to talk about.  Yeah, thank you.  Yeah?  I think some of these thoughts are even too hypothetical  because we're disabled ourselves,  even with the technology today.  We have issues with driving cars autonomously  in an environment that is built for human riders.  And if we support our autonomic cars by technique in the streets,  we can even reduce the distance from the cars  and narrow the streets and all of this stuff.  So that we're able ourselves to use our highways,  driving our cars ourselves.  And I think that's the main problem, not the technique,  not the AI, but disabling ourselves  to live in our environment independently of machines.  That is a more critical part of my story.  So what I would like to do in this discussion  is to focus on the real use cases we already have out there  and to figure out whether we want to do something there  and if so, what do we want to do there?  Instead of arguing if there is going to be in the distant future  something happening, look at what's out there right now  and see what we have to do if we do have to do something.  Yeah?  So recently, Google has given the best goal players  in the world in the soft.  So it was quite amazing for me to see that kind of machine  and professional in the world.  And I wonder, Google has a company,  a big corporation, has this power now that can use this.  And it's quite scary that it can predict so many things  with that.  So I wonder your thoughts on the preparation that  has this technology and making use it for something  and how can you kind of keep the balance?  So we are back to where you need to get involved in politics  in order to talk about regulation.  Because Google isn't the only one who has this data  and who has this power over you.  There is a ones making it accessible.  There's other companies who have the same data  and who have the same models.  There's state institutions that have the same data  and build the same models.  So it's not only about Google, but it's  about the topics that you have to discuss at a wider circle.  Yeah?  So thanks for a very interesting talk.  My question is kind of a problem to the questioner right now.  And there was very little talk about general general direction  that you mentioned yesterday.  So first of all, I am aware of it.  And I wanted to get a say from the user side  and from the company side.  And from the user side, it's good to be protected by hardware  and tech.  Sometimes we don't know what to expect  kind of a separate effect.  So having the ability to use this power system  to see how dangerous potentially  in personal information could be coming.  And from the business side, will European companies  be shaded in the Gulf with companies  that don't have the restrictiveness  of general strategies?  Of course, all of them that will be.  What's on your mind?  So about data protection being forced on European companies  versus this is an advantage or an disadvantage.  Let me turn the question around.  Maybe it's even an advantage to European companies  being subject to this law.  Because people can be certain that their data is being used  for purposes that they didn't agree on.  From a company perspective, who wants to roll out machine  learning, of course, they also always want more data  in order to build better models and in order  to build better services.  But from a customer's perspective,  it's like two sides of a coin.  You want better services that are being based on your data.  So you will have to share it somehow.  But on the other hand, you want to have control over your data.  So maybe it's even an advantage to be  able to tell your customers, hey, I'm  subject to this law.  It's being enforced right here.  Your data is safe with us.  Not just because we say so, but because the system is  not just because we say so, but because the state enforces us  to do so.  Yeah?  I was just regarding Apache.  What is your views regarding deep learning and stuff?  What is the roadmap for creating scale for deep learning?  The question was about the roadmap of Apache Mahoud.  Roadmaps go to the mailing list.  It's an all volunteer project.  Right now, for deep learning, most people  that are active in Mahoud are also active in MXNet.  So there are ideas flowing back and forth.  But other than that, I would ask you  to go to the dev list, ask there so that others can benefit  as well.  I won't make predictions on behalf of other people there.  I don't see any further questions.  If I may take the opportunity again,  I think this shows that this development about the deep  learning and AI takes the responsibilities  of the engineers who are doing this  and the ethical implications that the work has a bit further.  I mean, I think this is what happened with IT  through the last 50 years.  I mean, it was just like number crunching in a not number  crunching.  It was just like like like Lonon Gehalso.  It was just something like corporate data  that were not very, and now IT developed more and more  sophisticated systems.  And now there it's going into the public realm  where it goes about personal preferences,  decisions about personality.  So the societies get more involved from the outcome  what happens.  So this is very important work, raises very important questions.  And I would like to thank Isabel a lot again.  Thank you very much.
froscon2017-2083-teaching_machines_new_tricks_0_155.png froscon2017-2083-teaching_machines_new_tricks_156_1895.png froscon2017-2083-teaching_machines_new_tricks_1896_7835.png froscon2017-2083-teaching_machines_new_tricks_7836_7883.png froscon2017-2083-teaching_machines_new_tricks_7884_8579.png froscon2017-2083-teaching_machines_new_tricks_8580_9383.png froscon2017-2083-teaching_machines_new_tricks_9384_9659.png froscon2017-2083-teaching_machines_new_tricks_9660_10139.png froscon2017-2083-teaching_machines_new_tricks_10140_14603.png froscon2017-2083-teaching_machines_new_tricks_14604_19835.png froscon2017-2083-teaching_machines_new_tricks_19836_21107.png froscon2017-2083-teaching_machines_new_tricks_21108_21455.png froscon2017-2083-teaching_machines_new_tricks_21456_21719.png froscon2017-2083-teaching_machines_new_tricks_21720_21827.png froscon2017-2083-teaching_machines_new_tricks_21828_21899.png froscon2017-2083-teaching_machines_new_tricks_21900_22463.png froscon2017-2083-teaching_machines_new_tricks_22464_22691.png froscon2017-2083-teaching_machines_new_tricks_22692_27635.png froscon2017-2083-teaching_machines_new_tricks_27636_28475.png froscon2017-2083-teaching_machines_new_tricks_28476_32279.png froscon2017-2083-teaching_machines_new_tricks_32280_38591.png froscon2017-2083-teaching_machines_new_tricks_38592_39191.png froscon2017-2083-teaching_machines_new_tricks_39192_52427.png froscon2017-2083-teaching_machines_new_tricks_52428_52487.png froscon2017-2083-teaching_machines_new_tricks_52488_53195.png froscon2017-2083-teaching_machines_new_tricks_53196_54527.png froscon2017-2083-teaching_machines_new_tricks_54528_58751.png froscon2017-2083-teaching_machines_new_tricks_58752_59327.png froscon2017-2083-teaching_machines_new_tricks_59328_65519.png froscon2017-2083-teaching_machines_new_tricks_65520_76559.png froscon2017-2083-teaching_machines_new_tricks_76560_78059.png froscon2017-2083-teaching_machines_new_tricks_78060_78299.png froscon2017-2083-teaching_machines_new_tricks_78300_80211.png