The Responsibility Challenge for Data
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Jagadish, H. V.
Bonchi, Francesco
Eliassi-Rad, Tina
Getoor, Lise
Gummadi, Krishna
Stoyanovich, Julia
As data science and artificial intelligence become ubiquitous, they have an increasing impact on society. While many of these impacts are beneficial, others may not be. So understanding and managing these impacts is required of every responsible data scientist. Nevertheless, most human decision-makers use algorithms for efficiency purposes and not to make a better (i.e., fairer) decisions. Even the task of risk assessment in the criminal justice system enables efficiency instead of (and often at the expense of) fairness. So we need to frame the problem with fairness, and other societal impacts, as primary objectives. In this context, most attention has been paid to the machine learning of a model for a task, such as recognition, prediction, or classification. However, issues arise in all parts of the data eco-system, from data acquisition to data presentation. For example, the majority of the population is not white and male, yet this demographic is over-represented in the training data. It is challenging for a data scientist to satisfactorily discharge this broad responsibility.
 If you have large impacts, whether it's loans, jobs, or staying out of jail, you've got to think about what those impacts are and how they work. And that's, I think, the key takeaway from the morning talk. What we want to talk about in this panel is what does that mean operationally for us as a field? And just in terms of setting the stage, I also want to point out that when somebody tries to wake us up, it's important to talk about all the things that could go wrong and do go wrong if you're not careful. But that doesn't mean that things are necessarily bad. The reason that we have this much interest in big data or data science or AI or FUBAR, as it's going to be called when the next buzzword comes along, is that these things have great power to help us. And so we want to get that benefit to the extent we can. And it's just that it's not a clean, unalloyed benefit as tech boosters would say. The other thing to point out is humans have all kinds of biases. And if you have a judge who has been, who's biased in making decisions or discriminating against some category of people, these decisions, each decisions will get explained away because the numbers of decisions are small enough. With algorithms, you can actually test them. And so one can hope to do a lot better because one has algorithms. So that's a positive framing of the work that we're trying to do. So the question is, how do we exercise this responsibility? And as we talk about this, we've got a really distinguished panel of people working not just in core databases, but also in a number of neighboring fields, giving us a breadth of perspective that I think is going to serve us in good stead. So I've got the names listed here in alphabetical order. So we have in the middle there Francesco Bonchi from the ISI Foundation in Turin. To his left is Tina Eliassirad who's at the North Eastern University and leads the Just AI project. I love the name of the project. To her left is Krishna Gumadi from the Max Planck Institute in Germany. And rightmost is Julia Stojanovic from NYU. And you all heard Lisa's talk this morning. What we're going to do is do a five minute position statement each for everybody other than Lisa who gave an 80 minute position statement already. And then we'll open it up for questions and discussion. So with that, I guess I'd like to get started. So Krishna. Thanks Jack. So rather than make a position statement, I would like you to view this as a couple of lessons that I learned while working on this topic of ethics and algorithm design over the last couple of years. So it took me, so I'm just going to talk about two lessons. Now to get on to the first lesson, one of the things that surprised me in tracking with philosophers who do ethics research is how much I was, I had a fetish for utilitarianism. Now and how I didn't recognize the limits of utilitarianism. Now to explain this, let me just walk you through a thought experiment that I would encourage you guys to participate in that is often used to illustrate the limits of utilitarianism. This is called the trolley problem. Suppose you have an out of control trolley that is running down the tracks and you are the person in charge of that lever over there. Now you have the power to pull the lever and divert the trolley from going down straight and killing five people but instead move to a side track. The problem is on the side track you have one person. Now how many of you would pull the lever with a quick show of hands? Okay that's far less than what I expected but it's interesting because you are choosing to not save five lives. One way to think about this if you're a utilitarian would be you're choosing to lose five lives instead of one. Now let's take another version of this trolley problem where you have again an out of control trolley aimed at five people on the tracks but now you're the person who can push a fat person on the bridge in front of the trolley to stop it. Now how many of you would actually do this? Again a quick raise of hands. Now notice that the number of hands actually went down. Now let's look at the final, a different version of this where you're a doctor at a hospital and you have a patient who has come to check in and you could actually harvest their organs to save five lives, lives of five other patients. How many of you would actually kill or harvest the organs of this patient to save five lives? Okay, there are three or four of you and that's extremely worrisome. So we would do this. Now the point that I wanted to make, the reason why these trolley problems are interesting because if you looked at it from a purely utilitarian perspective all three scenarios are the same. It's a trade off between five lives and one life. However, there is something about the specific scenarios that made a big difference. Now I don't have the time to go through why it is that we might have felt differently but to just highlight what utilitarianism is, one where we decide the righteousness or the fairness of a decision system purely based on the outcomes where we define some utility function at the level of individual outcomes and then we look at, we pick the decision system that maximizes some of the utilities of outcomes for each individual in a population. Now that is what utilitarianism is about. Now if you look at many algorithmic designs and particularly machine learning, it is overrun with utilitarian ethics. So for instance, as Lisa pointed out in her talk, when you're trying to learn the optimal boundaries we often try to define and optimize the loss or accuracy function where we try to minimize expected loss over all examples in the training data. Now this is a classic utilitarian ethics. And this actually explains the root cause of discrimination. I'm sorry for the mess up of the slides. I think we did this over a Mac laptop. So utilitarianism leads to minimizing the overall prediction errors but this doesn't address discrimination because discrimination is about utility at the level of groups of individuals, not at the level of single individuals. So to just quickly illustrate this, suppose you have three different decision systems, system one, system two, system three. What I'm showing in the table are how accurate these decision systems are for men and women and overall. Now if you were a utilitarian, you would probably pick system one because that's the one that maximizes the overall accuracy. If you are a person who cares extremely about equality of group error rate, you might actually pick the system three. Whereas in practice, many people might actually pick system two because you use a different normative criteria, one which shows that actually system two is giving better accuracy for both men, women and overall compared to system three. So what I've just pointed out is a different normative criteria than just discrimination that actually kicks in when we choose which decision systems to follow. So the first take away or the first lesson is don't rely exclusively on utilitarianism to design and evaluate algorithm designs. Now this of course raises the question, how should algorithm designers select the right set of normative criteria? So utilitarianism is one criteria. Now there are other types of criteria like discrimination is a normative criteria. How should one select the right set of them? Now I actually think, or this is my position maybe, that the search for right normative criteria is not the job of algorithm designers. Our job is to operationalize a given normative criteria. Now what I mean by operationalization is how to make these criteria formally measurable in terms of empirical observations. So to take a quick example, if you think about discrimination, discrimination has been studied in moral philosophy, political sciences, economics for over 100 years now. And there is actually a approximative normative definition that is here, which is discrimination is about wrongfully imposing a relative disadvantage on persons based on their membership in some salient social group. Now typically if you look at ethics, and this is a definition that I picked from Stanford Encyclopedia of Philosophy, the definitions are usually in English. And the challenge for us as algorithm designers is how do we operationalize these fuzzy notions like relative disadvantage, wrongful imposition based on salient social group. Now over the past several years, if you looked at the literature on algorithmic discrimination, people have operationalized or formally defined various different ways to measure these fuzzy notions. And this has led to a plethora of definitions of discrimination. Now one might wonder, aren't these one too many interpretations? Yes, but given the different contexts in which we want to use these systems, they are actually quite needed. And so I think we should stop worrying about there being 20 or 30. In fact, it's surprising that there are only 20 or 30 and not hundreds given that fairness is a topic that has occupied ethicists for like hundreds of years, if not thousand. And of course, when we are operationalizing, designers should maintain informed neutrality between different reasonable ethical positions. And what I mean by that is we shouldn't look at each of these ethical positions as necessarily competing with each other as opposed to being applicable in different contexts. And designers should not attempt to resolve which position is better. And finally, once you define the formal measures, there is a lot of interesting work that comes in terms of designing algorithms that's for the formal criteria. Lisa mentioned some impossibility results of designing some algorithm that would satisfy multiple criteria. Then there are all sorts of algorithms, designs that try to achieve optimal performance based on lexicographic orderings or Pareto optimal tradeoff between the different criteria. So the take away for the lesson two for me is as algorithm designers, we should study the connections between the designs and consequences while maintaining informed neutrality between several different reasonable ethical positions as opposed to thinking that we could actually determine what is the right ethical position. Thank you. Track café and it's not but Lee's selection. I don't know how I got killed that way. So I'll be brief. What is my main point here is that where we stand with this fairness of quotability, transparency research, I believe the topic is still in its infancy and it reminds me a lot about where the privacy preserving that analysis and management area was at the beginning of the 2000s. There are many similarities. It's a very hot topic with an exploding interest. There is an explosion of number of papers that are coming out on this topic, but those are in my opinion mostly simple ad hoc solutions. I would say like a good 90% are on the classification task. Maybe even a very large fraction of these are on the binary classification task. And so overall I feel that there is a lack, I mean still a lack because the areas just boom of some deep robust theoretical foundation. So the two areas have many similar challenges. There is this proliferation of different notions and definitions that also Krishna's was mentioning before. It is the problem of different regulations which was a problem also for the privacy preserving that management research, the lack of clear benchmarks. And the need to interact with all the players in the play field that are the legislators, the policy makers and the need for multidisciplinary approaches. So interacting for instance with social scientists and philosophers. And both privacy preservation and these fairness accountability transparency, they technically they usually have to deal with this trade off between receiving some desired privacy or fairness or transparency goal while not losing too much utility in our data and our algorithms and so on. So there are also interplays between the two notions because privacy preservation is about preventing sensitive information from being leaked and when we talk about fairness we also talk about sensitive information, we want to prevent it from being abused. This notion of sensitive information can overlap in some attributes or can be distinct in others. And there is not only positive synergy, there are also potential conflict between the two notions because for instance having a sanitized database for privacy issue might even hide the presence of discriminatory patterns. So if you take a database which is the output of some algorithm and you want to see if there is some form of gender bias, if the column gender has been removed for instance for imposing key anonymity, then good luck with that. So I also believe that an holistic approach for this reason, an holistic approach taking privacy and fairness accountability transparency all at once would be very desirable and probably is also achievable. So going forward I believe that after this initial phase of excitement and increasing of awareness in the society and among the research community, the FAT area should produce some theoretically foundational definition and result like for instance a nice example is differential privacy for the field of privacy preserving data management. And in this I think this community can contribute a lot as it was the case for privacy in the early 2000s, just think about all this notion of key anonymity, diversity, closeness, privacy preserving data mining, privacy preserving all up, apocalyptic databases that were all presented or introduced by this community. So we need you, get on board, the field is exciting, there are many low hanging fruits, many interesting problems to be solved and I would say it's not too late, we just started so get on board. Thank you. We are not here at all. Hi, everyone. So actually my first slide is this. So the message that my colleagues and I have been trying to impart on this community is that we all should be working on problems in data ethics, in responsible data science, in another name for this field that Francesco used, this fairness, accountability, and transparency or FAT. This is a name that we are inheriting from another community, FAT ML. So since 2015 or so, we started telling all of you that data ethics is a very important area of research. And I think that at today's, at this particular sigmod, we can declare victory. The job is done, right? I mean, we have, we just heard a wonderful keynote on responsible data science. The best paper award is going to a paper that is on this topic. There is going to be a wonderful tutorial by Suresh and Pods, so please go to that. And there are several other papers at this conference that are on these issues of data ethics, broadly speaking. So in one sense, I'm happy. I think, you know, the job is done to some extent. But what I want to tell you today is a completely opposite message, right? Just to be controversial, because you've been hearing me say the same thing, so now I'm going to tell you something completely different. And that is that we should not, the database community should not work on data ethics. Now, what do I mean by that specifically? What I mean is that we should not be working on data ethics with only our technical head on. That a lot of work has been done in communities like machine learning and like data mining, where many, many fairness definitions were proposed and operationalized, and lots of papers have been published, but this work has not had an impact on the real world to the extent to which it should have had an impact, given how much time was spent by really, really smart people thinking about these problems. And an even scarier thought here is that, in fact, many of the results that came out of the technical community in this field have been harmful to society, because they have helped decision makers at all levels whitewash results of structural discrimination in our society. And this is something that we must work very hard to avoid. So I will give you an example of this. Before I do, I want to read a quote by a colleague and a friend, Solon Barokas, who taught me all I knew at the time in 2016 about this area, and it's very insightful. He says, I worry very much about the co-opting of the work on fairness and machine learning, about how it may no longer be serving the progressive goals that it had at its inception, how it can be deployed to whitewash the problematic and unjust practices that continue to plug some of our most prominent companies and institutions. So my message to you today is that, as we work on these topics, we really must be aware of the impacts that our work has on society, and in particular, that our work in this area has on society. Whom are we helping? Right? Whose interests are we protecting? And an example that I want to give here is, and this is my exhibit A, because I'm talking about the criminal justice system, is the following. So the state of New Jersey recently enacted a comprehensive reform of the criminal justice system, and one of the goals that they had was to make it so that people who are accused of a crime are not compelled to stay in jail pending trial, but rather are able to post bail and then go home and continue working and continue with their daily lives and then return to court. So we wanted to be able to release people on bail much more so than we were doing in the past. New Jersey has this goal, right? This is sort of a vaguely stated goal. And the reform that they enacted is made up of several interventions. All of these were deployed at once, and one of the interventions that they deployed was a framework called PSA, and this framework is in a similar vein as the compass system that Lisa described in her keynote earlier today, where it essentially produces a risk score, a number, that the judge then takes under advisement when making their decisions, bail decisions in this case. And so one thing that's distinctive about New Jersey is that if a judge's decision differs from the recommendation that the PSA framework gives him or her, then the judge needs to document their disagreement in writing. So think about that for a second, right? A judge who is not a data scientist gets a number from a machine that has been presumably validated and thoughtfully produced, and they now need to actually state a disagreement if they want to release somebody despite the recommendation of the system. Raise your hand if you think that a judge would release somebody under these conditions despite the recommendation of a system to keep somebody in jail pending trial. None of us think that the judge would take that risk, right? So a thing that disturbs me here the most, perhaps, is this quote that you've been staring at while I've been talking, and this is from a New Jersey pretrial justice manual that was produced by some of the good guys, the ACLU among them, and that advises defenders on how to now defend, it advises lawyers on how to defend those accused in this environment in which PSA is deployed. So the manual says, switching from a system based solely on instinct and experience to one in which judges have access to scientific, objective risk assessment tools, could further the criminal justice system's central goals of increasing public safety, reducing crime, and making the most effective, fair, and efficient use of public resources. Why am I showing you this quote? Well, it's because whoever produced this system actually didn't do their job of explaining to the people using the system that the system is not objective, that it involves trade-offs, that the data that it uses and the society in which it is deployed is one in which we have results of historical injustice and discrimination, right? So this particular system does not pass by any measure, the scientific and objective descriptors that we have in this statement. So this is very misleading, and it's essentially objectivizing some of these decisions, and it's legitimizing them and scaling them up that these systems embed. And we have, of course, here an example of the compass system that we already looked at that tells us that, in fact, mathematically and societally, it is impossible for us to be fair in the sense of how the false positive and false negative errors are distributed in a society that is itself unfair. And this result, of course, also applies to PSA and to any other system of the sort that is deployed in our society today. So what do I want to say with this picture that Krishna already showed? It's that the real trolley car problems that we're facing as a society and as technologists are not those where we have an autonomous vehicle that's going to run somebody over one person or five people, and then we have to choose how many. The car should stop. Rather, the trolley car problems that we need to worry about are those where we're thinking about how we're trading off different fairness and utility notions in the use of these predictive analytics in society. So I'll stop here because perhaps I'm over time, but I'll just kind of point you to this potential solution. So what do we need to do? How should we, in fact, work on these problems? Well, we need to think about law and social sciences and talk to people in these fields because guess what? It's much easier for us to read a law paper. Believe me, I've done it. Then it is for a lawyer to read our papers because their papers are well-written and in English, right? And ours are formulas and less well-written. So I'm going to just leave this here for a second for you to try and decipher. Solutionism, something that we're proposing with technology fixing the problems of society, like reductionism, is the deadly sin that we need to carefully think about and avoid as best we can. So I'll stop here. So we had a technical glitch because of which Tina Slides didn't make it to the prediction laptop. And so we're chained up. I'm very good at projecting. Don't worry about it. So while we were doing this, when Judy Thompson, who's a philosopher at MIT, came up with a problem back in 1976, she never thought that computer scientists were going to use it this way. So be careful when you come up with something. It can be used in ways you do not wish for it to be used, as was mentioned by Lisa earlier. And while this is being set up, so I started working in this area because Daniel Allen, who's the director of the Ethics Center at Harvard, asked me to come and give a public lecture on this topic. And I thought, oh, this is great for a broader impact for an NSF proposal, right? Computer scientist goes and talks about computer science to a bunch of ethicists. So I don't need my slides. So then that led to a project called Just Machine Learning. And if you think about machine learning, I really like the definition that Tom Mitchell has in his 1997 textbook, which is called Machine Learning. He defined the well-closed learning problem as a triple. You have a task, you have experience, and you have performance measures. And when machine learning goes wrong, usually one or more of these things have gone wrong. One of the things in this particular area that people have focused a lot about is that experience, that data. That for example, the data is predominantly, so the data predominantly includes white males, even though the majority of the population is not white male, right? And that's why, for example, Alexa works better for white male voices than for female voices. And so, thank you. And so this is his definition of the well-closed learning problem, right? So a computer program is said to learn on a particular task, given an experience, if over time it improves on that task given the performance measures. So people have focused a lot on the experience and the performance measures, but not so much on the task. In fact, the tasks that people have worked on predominantly has been risk assessment and ranking. And here are just two reasons about why people have worked on them. One is computer scientists know a lot about them. We know a lot about classification. Risk assessment basically boils down to classification, which is sorting, right? Good, bad. High risk, low risk, right? And ranking. We have done a lot of work on ranking. And also the human decision makers like these tasks because it's easier for them, right? Tina's bad, Lisa's good. Perfect. Tina goes to jail, Lisa goes free, right? And so there has been a lot of issues with these tasks that have been well-documented and celebrated like impossibility results that were mentioned earlier. But if you think about it, these tasks are just too abstract, they're too under-described to have any kind of ethical consideration in terms of what you should do. And one of the things which is horrible is that they provide no uncertainty values, right? It's not like the judge, you go to the judge and say, yes, Tina's risk of recidivism is high, but the algorithm is only 65% sure. You don't tell that to the judge, right? You don't provide any kind of context to the judge. It's not like, well, Tina is similar to Lisa in these ways and not similar to Lisa in these other ways. And in particular, these tasks cannot incorporate anything about how the human decision maker makes a decision, right? They're just sorting. Everything is just not good enough when you want to have some ethical consideration on your task. And this slide, Lisa showed where a judge is trying to cover his behind. He's saying the algorithm says, Tina's bad, I'm not going to let Tina go out. Because if Tina commits a crime, then I'm going to show up on the front page of New York Times. And so he's trying to cover his behind. And he is throwing out the plea deal that experts have come up with, which leads to what are the incentives of the human decision maker using your algorithms, using your system? Is the judge using it just for efficiency purposes? I have this pile of cases to go through, so I use Lisa's algorithm to sort through them as quickly as possible. Is it profits in terms of health care system, right? So you need to be very cognizant of what are the incentives of the human decision maker in using your algorithm. In almost all the work that's out there, this is not discussed. And so we should think about other kinds of tasks, right? And as was mentioned earlier on, if you think about normative ethics, you can break it down into three main groups. One is the ontology, so be like Kant and follow your duty. Two is consequentialism, which utilitarian falls under it. And three is virtue ethics. So if an algorithm is going to make a decision on me, I will like that algorithm to have been trained to be a virtuous agent, right? So what I would like to do is learn algorithms that fall under virtue ethics. And so, and in fact, if you think about professional fields like medicine or law, there is this notion of apprenticeship. Now, if you think about here, we actually have a task. It's called apprenticeship learning or imitation learning. It's just a hard task. You can't have zillions of papers coming out, like it's money that you're printing on, right? And so it's a hard task. It is a task that's been studied in machine learning. And it has not been applied here, because what I need to do is I need to learn how a moral agent, how Sonia Sotomayor makes a decision. And I need to know context, and I need to know many, many things. And there's a nice tutorial at ICML 2018 on this in terms of learning from a demonstrator or learning from demonstration to learn a policy about how to make a decision. As opposed to I get 30 years of data, throw it into this classification algorithm, which sorts people out and bins for me. And again, I'm not the first person to come up here and say virtue ethics, but computer scientists usually don't like virtue ethics, because it doesn't have these nice rules, right? I'm going to follow duty. I care about what the consequences are. And then the other thing is that perhaps there are certain tasks that we as a society should say that machine learning or any kind of automation should not be used. Or the bar for using them should be a lot higher, like dangerousness, right? In terms of pre-child disposition in America, the judge is trying to decide two things, flight risk and dangerousness. And flight risk maybe is easier for an algorithm to decide, right? Did I take Tina's passport away? How long has Tina lived here? Does Tina have a job in this place? Right? What are Tina's connection to the community versus dangerousness, which is just horrible, right? Tina is not white, so Tina's dangerous, right? So there should be a much higher bar. And in fact, one of the things that has happened is that bail is being used for crime control and wasn't designed for it, as my colleague, David Rovirez, at Northeastern talks about it. And lastly, even if you do come up with a perfect task, one of the questions is, did the algorithm learn for the task, right? And so Lisa was mentioning about the deep network and classifying wolves. This has been going on for a long time. Back in the 80s, there's a classic example of the army wanting to classify tanks that were camouflage or not. And it looks like what they learned was basically whether there's a shadow or not. And then recently, there was some awful science that came out of Stanford that said that, oh, we can, from a picture, tell whether you're gay or not. And it looks, it seems, what they learned was some pictures have better lighting. But imagine being gay in a country that you would be murdered because you're gay. I wouldn't want to have that algorithm, right? And have all of those souls on my head. So the takeaway message is, so far, what has been done is we've been trying to pack all the normativity versus description. So being normative is what ought to happen, what should happen. Being descriptive is what happens. Murder happens, but murder should not happen. And they're trying to build all of this normativity into these fairness measures as part of performance measure. And that is wrong. It's just not enough there. What you need to do is think about the task, think about the experience, and think about the performance measures, and try to build normativity, what ought to happen in all of them. There is a task out there, I'm not saying it's a panacea, but it's apprenticeship learning. And again, put yourself in front of a judge and think about, OK, if a judge is going to use an algorithm on me, would I want the algorithm to behave like Sonia Sotomayor? Like Ruth Bader Ginsburg, right? Or would I just like it to learn from 30 years of data that have lots of prejudice and it's just starting sorting? So on that note, I'll finish. Thank you. So this is the time for you to queue up questions. And while you're getting your questions ready, I want to start the panel off with one question to try to bring together what has been said thus far. How should a DB researcher approach this problem? We've talked about there is a problem, and we heard that in the keynote this morning. And what you got in terms of the presentations were some of the challenges in trying to address this problem and some of the difficulties with how some people might be thinking about this problem in general. And one thing there is that this is not a purely, that's pretty obvious, is that this isn't a purely technical thing. And so if the questions to address go beyond databases, and in fact, they go beyond computer science, then there are at least two ways in which you could conduct research. One is to say, I'm going to work on the problems that I understand and I know how to solve. And I realize that this is only a piece of the puzzle, but I'm going to work on the piece of the puzzle that I'm good at. And other people will work on the other pieces. The other way to think about this is to say that the only way to address this is to formulate a disciplinary team, and we've got to collaborate with experts in other fields, and that's the only way we'll be able to address this problem. And so what I'd like to do is for the panelists quickly to say either A or B, and if they want to perhaps a couple of sentences to justify their choice. Okay, so I'd begin with Krishna and just go down. It's mostly A with some information, with some aspects of B in terms of figuring out how to understand what the problem is about, but it's mostly A. It should be B. I mean, again, what you ought to do, you should think about what you ought to do. What would make your mother proud is B. Think about it. It's B. I think both. I mean, if... That's not an option. No, I mean, I'm fine with Krishna doing A and Tina doing B. I think as researchers we have different aspirations, different approaches at the research, and all together A and B can help progress the field, I think. Neither. So I think, first off, you have to be introspective and think about yourself. How are you going to make impact? And there are ways of making impact. I personally really enjoy going across the boundaries, but I've been a party to a number of conversations where there's a lot of eye-rolling. The technologist rolling their eyes, the humanist rolling their eyes, and being able to be comfortable with that. But then you have to think about what works for you. You can't do... Yeah. So I'm going to say B as well, although I've, of course, done A starting out and then moved slowly into B, as many of us have. So the time has sort of passed where you could just say fairness plus X equals paper, right? And we've all done that work. But the train has left the station. This is something that we missed in this community. So now we actually have to be thoughtful. And I think an important aspect of B is that we get to educate the others about technology. And people, even those who practice technology law, for example, can use a lot of our help, and it is our professional responsibility to educate others about what we know. And interdisciplinary teams can give us that opportunity. And we can also learn from them very importantly, in particular, to be conservative about the claims that we make about the applicability of the technology that we develop, right? I mean, we can't just go out there and say, my fairness measure, best thing after sliced bread. Everybody should just use this, right? It doesn't work. That is irresponsible to do. OK. So you've got some thoughts on this question. Are there thoughts that people in the audience have or questions for the panelists on what they've said so far? I think there's a question in the back there. Hi. So thanks for these insights. It was really nice to see all these different opinions. This question here is about what I should be doing in the future, but I think many of us might be worrying what I'm doing or what we're doing right now. So I'm working on problem X. How do I recognize that I should also worry about fairness or ethics? So if I'm developing the next best B-tree, maybe it's not quite as relevant versus I'm evaluating surveillance data. So is there like a recipe or is it just a gray area? Is there any way for me to tell, should I be talking to lawyers and philosophers or am I OK with the current research I'm doing? Can I? OK. So the ACM recently issued an update to its code of ethics and professional conduct. And we as a community have actually participated in that revision. I was volunteered by Don Cosman here to be on that task force. And what I encourage all of you to look at that document, it's quite thoughtful. It's an improvement on the previous version. And what it tells us is that ethics is not a recipe really, right? There isn't an algorithm out there that you can just run and it will tell you, you know, worry about this and this is a way to fix it. It gives you a set of questions to ponder and you need to essentially figure out how you're going to trade these off, right? It's neither consequentialism nor the ontology that is right. So you have to think about these issues on your own. It's very important to do this. And look at the code. So if I may, Kate Crawford, who is at MSR New York, said that no computer scientist can just walk around and say I'm an engineer anymore, right? Because of our success, right? Our tools are being used. And so you should, you are a moral agent. Think about it yourself, right? Are there issues in terms of them using your tools, your algorithms that you would not be okay with? And if that is the case, then you should seek out a philosopher, an ethicist, a lawyer, and discuss it with them. I think especially about law, there's a big thing in terms of liability, right? If Tina's algorithm is used and somebody dies because of it, who's liable? Am I liable? Is the person who used my algorithm liable, right? So there's a lot of aspects in terms of law that need to be sorted out, which hasn't sort out itself yet. And we should be part of that discussion in terms of liability of our tools, algorithms, et cetera. So there's sort of a, hopefully, a conflict that I want to point out between the families that maybe you can resolve. So Francesco said earlier that remarking on the similarities between privacy and fairness that we should, one thing we might want to do is look for the equivalent of differential privacy for fairness, which I will call the one true definition. And I think Julia and others are arguing, don't take your fairness definition and go and wave it around like a sword and understand the context of what you're doing. To me, as a researcher who wants to think about this, instinctively, I would like to gravitate towards Francesco's point of view. I would like to think that there is a definition I can find that if I find it, or if someone finds it, that we can all look at it. And that is the way we will think about this. And of course, from yesterday's talk, there was a lot of success with that. But also, I feel like that's not quite right. And I wonder if the discussion on the panel, if you could sort of thrash out if there's actually a disagreement there between folks there on that. I believe we should not simply come out with our definitions and then convince the rest of the world. But all together with the policy makers and the legislators, we can have a virtue cycle. So the laws give us constraints, objective function. And once we do progress with our research, we can get back to them, try to explain what we achieved. And this can go in cycles. It's not just one shot and we are done. So I think along those lines, one of the things which is interesting is, can you reconstruct the intent of a policy? So if you talk to policy makers, every policy has an intent. And as Francesca was saying, the laws are constraints and then the policies are how those laws are interpreted. And then you have police and other people who are implementing actually in the environment. And so one thing that could happen is that I'm seeing the droppings of how that policy is being implemented. And can I reconstruct the intent of the policy? And if the intent of the policy is not the same as what the policy maker is saying, then perhaps we can change the policy. But I think this notion that you can code up ethics in a set of consistent logical rules is just wrong. You can't because to be ethical, you need to have empathy. You need to be able to walk your way through conflicts. And that's very difficult. To have one formal definition of that, just think about it. May I add, so I think that a critical shortcoming of the differential privacy technology and the reason why we should not be trying to mimic differential privacy as a kind of a model for a solution in ethics is that it is very difficult, possibly hopeless, to explain how differential privacy works, what the epsilon parameter means, what the tradeoff is between privacy and utility to a member of the public. And I would love to be proven wrong by members of the differential privacy community. So I think that this technology and its use in society will live or die by whether we are able to provide explanations that are meaningful, not about how it works, but what effects it has to members of the public. So please take that challenge. And if you are able to explain this technology to us, then perhaps we can look for a common technical framework for other aspects of ethics. But at the moment, I'm not convinced. So I think the last two comments in a way I think may have said what I wanted to say. I think these things have the roots and value systems. And whether it's people that are making the decisions, they're still not perfect because they're rooted in the value system that those people bring in to fairness or ethics or any of these things. And I think that we're trying to go a step further and say AI ethics or how would algorithms view fairness and so on, which is even more complicated because now you have to figure out what value system these algorithms and this is what some of the comments that were made. So I mean, I don't think that even if you brought two lawyers that would reach the same decision because there's a lot of culture and value systems and so on and so forth. So I mean, I think I just, I'm reading this here also, but I don't think we can simplify this. This is going to be something that needs to be not only be, but really much more complicated than even be. And the answers are not going to be with computer scientists or acknowledges. The answers are going to be, you know, because philosophers and ethicists and so on have looked at this for hundreds and hundreds of years. And there's a lot of depth into it that, you know, but I think the last two, you made the same remark. So I think this is, oh, sorry. Go ahead. So addressing that point and the previous point, I think one of the places where we can take responsibility is more of education about the limitations of our systems. So we're talking about a particular formalization of an optimization problem, a formalization that leaves out important pieces, and that's always going to have limitations. And to act like it doesn't, I think, and you can see in the comments that are shared that non-technologists don't understand this. I've had conversations with folks from the STS community where they're kind of explaining to me, you know, well, your algorithms, you know, leaving this out. And I'm like, of course it's leaving it out. You don't think I actually believe that this is true. I think we have to figure out how to not leave people with this impression. And that is a place where kind of doing comparative modeling, being able to look at more of the context and then think really about what the task is. And how do you aid the human decision maker in this process? What are their goals as important as well? In fact, I think this is where there could be a lot of innovation from the computer science community is what the right task is. In fact, this is why I think we should do virtue ethics going back to your point. Because again, if somebody is making a decision about me, I would like them to be a good person, right, that makes a decision about me. So I want my algorithms, my tools to have been learned on virtues and vices. Now, there are a lot of people who do not like virtue ethics. For example, they say, I don't care that you're doing the right thing. I want to know why you're doing the right thing. You need to explain it to me. So that's meta-effects. But I think at least for us right now, if our algorithms are going to be used to make human decisions, it's best if they learn to behave like a good human. Now, there's a question of who is a good human and how you decide and all that kind of stuff. And that's why we need interdisciplinary teams. This actually is a very good segue to my next question for the panel, which is, how should we evaluate work in this area? You know, if you do systems research, sort of the standard thing is you build system you measure performance. Maybe you're not after performance, you're after functionality or usability or whatever. Again, you build system and you evaluate. Now if your innovation is in building a more responsible system, how are you going to evaluate it? But one answer is it's not enough just to build the system. You can't do a measurement there. So you've got to deploy the system and evaluate success in the field. If you don't do that, then as was pointed out in the keynote this morning, you're introducing all kinds of assumptions about what your test set is. And so in effect, you're measuring success only with respect to some mathematical criterion of choice. And you know, that has some value, but that's a very limited test. So if one says one is going to measure success in the field, you have an even bigger problem besides actually getting your system deployed. And that is not everyone agrees on what success is. As Ahmed's question was saying, what's right is often a value-driven thing. And so, you know, depending on who is evaluating it, their choice might be different. And if you're going to look at these subjective assessments, then perhaps what it all boils down to is popular acclaim. So like the panelists to comment on, if you're working on data responsibility, how should that work be evaluated? We'll start with Julia. Thank you. So I think maybe this is my cue to talk a little bit about the policy work that I've been involved in. So in my mind, you know, success here may not be in translating the algorithms and the techniques that we develop into commercial systems, perhaps sometime down the road. But right now, there is a tremendous void in having technologists participate in defining how we regulate in this space. And regulate, we must. We cannot let technology, you know, run unchecked and make decisions that we don't understand and enact policies that we didn't even intend to encode just by accident. They made it into code. So in Europe, there is the GDPR. But there are also other frameworks that people are thinking about at the national levels and where there is always hunger for participation from us, from, you know, just explaining what an algorithm is and isn't, what you can and cannot learn from data to really helping hands on come up with recommendations. So in New York City, there is a law that we passed, and this is the first such law in the U.S. that is compelling city agencies, government agencies, to start thinking about how they can become transparent and accountable in their use of algorithms and data. And they designate this ADS, automated decision systems. And so I am on a task force that is working on a set of recommendations for city agencies, and this is a very difficult set of recommendations to come up with. So that an agency can respond to an individual who comes and requests an explanation why a decision was made that affects them. So for example, I come to the Department of Education and I ask, why was my son given a spot in this school and not in this other school? How can we explain that? Technically, how can we explain it? How can we make it interpretable to a person? And how can we measure that in fact what we are showing them they understand and then they can act on? Another task here is that agencies need to show that they are not biased in the sense of not discriminating against members of legally protected groups. So working in efforts of this sort I think is very important for us. It certainly is an output that we as a community can help produce. And all it takes is for me, I went to a city hall when there was a hearing on this law. I read the draft, it made no sense to me, and I testified. And then they call you back and actually want your input. So please get engaged. It's very interesting and it's very rewarding. So one of the places where I think this work emerges or the place that I have seen it emerge is from the area of data science for social good. So many of you may be aware of efforts in this space or maybe efforts in your own universities and so on. So I think of Raeid Ghani's program at University of Chicago, a bunch of things came out of that. And the interesting thing, so they start off saying, okay, data science for social good. And I think some of the deepest work on ethics and data science has come out of that community because they've realized how challenging it is to define social good and so on. So kind of watching that process of seeing them try and evaluate their work and see the challenges that came up is a place that can give us some inspiration. I think this is a very hard question, especially for short-term impact. I think it's quite hard to judge. Of course, longer term, we might be able to say that result was really game changer, was really important for the other to build on it. Of course, we need to always keep an eye or two on the real world and try to show that in a specific application domain, a specific context, there was something very bad going on and practically we can improve this. But again, I don't have a strong convincing answer because I believe it's really important and tough question to answer right now. I have a strong convincing answer. Would you want your system to be used to make a decision on you, conditioned on the fact that you are not a white straight male? If your answer is yes, success. If your answer is no, go think about it. That's the answer. I guess I have a bit of mixed thoughts on this. On one hand, I think that it's absolutely important that we talk to researchers from different areas. In fact, one of the things that I experienced in these interdisciplinary collaborations with legal scholars, economists, and philosophers who I was lucky to collaborate with is that I started realizing how deep these notions are and how much literature already exists on these topics. There are lots of interesting notions, fairness notions that these social scientists have studied. I see a huge opportunity under research agenda where one could try to formalize, try to think about if you were to follow this specific notion, how would you incorporate it or how would you introduce it into an algorithm design? I see that as very technical work that one could involve themselves in. For instance, I would highly encourage you to just go and look at the discussion about anti-discrimination in Stanford Encyclopedia of Philosophy. They have the definition in English, and then they have a long discussion of how in different contexts you might want to think about applying this law in different ways. A lot of papers that we wrote were essentially inspired by this discussion where we said, well, what if we tried to operationalize this value judgment? What I fear at this point is that I fear that because there's context matters, it would be a mistake to rush and try to, at that point, go and influence policy in the real world because the real world is a lot more complex. In fact, if there was anything where I was going for option A before, in fact, here is a quick backstory about Compass. The people who designed and deployed Compass wanted to reform the criminal justice system. These were actually not people from hardcore computer scientists who were excited about applying machine learning techniques, but it is just that they observed that human decisions, the judges' decisions in the US were extremely biased, and they thought nudging these judges with objective decision making can actually improve the system. These were people who actually ended up deploying something in the real world without actually understanding fully about it. I get the enthusiasm or the desire to influence policy, but at the same time, I think there is a, and I think people should be doing different sorts of things, and I think we should be doing things where we have expertise on. I think if you are a bit more of a technical person, I do think there is a lot of interesting technical work that can be done. If you are a much more of an interdisciplinary person who can engage with the policy makers, I think you could do that as well. But I think we should not think that, I think we should not miss the primary purpose of what I think is of academic research, which is actually be able to isolate our results a little bit from the real world and actually study the various different consequences of different forms of different ideas. Knowing fully well that the damage wouldn't be immediate because these things are not going to be immediately implemented in the real world. And I think it would be a mistake if everybody tried to do this. I think some people like Julia and Tina, who have a bit more of experience with this, with interacting with policy makers might be able to put it off, but not necessarily everyone. We have a disagreement. Yes, but we do have a disagreement and unfortunately, just as the panel is heating up, we are out of time. Much awaits. So thank you all for coming.
 If you have large impacts, whether it's loans, jobs, or staying out of jail, you've got to  think about what those impacts are and how they work.  And that's, I think, the key takeaway from the morning talk.  What we want to talk about in this panel is what does that mean operationally for us as  a field?  And just in terms of setting the stage, I also want to point out that when somebody tries  to wake us up, it's important to talk about all the things that could go wrong and do  go wrong if you're not careful.  But that doesn't mean that things are necessarily bad.  The reason that we have this much interest in big data or data science or AI or FUBAR,  as it's going to be called when the next buzzword comes along, is that these things have great  power to help us.  And so we want to get that benefit to the extent we can.  And it's just that it's not a clean, unalloyed benefit as tech boosters would say.  The other thing to point out is humans have all kinds of biases.  And if you have a judge who has been, who's biased in making decisions or discriminating  against some category of people, these decisions, each decisions will get explained away because  the numbers of decisions are small enough.  With algorithms, you can actually test them.  And so one can hope to do a lot better because one has algorithms.  So that's a positive framing of the work that we're trying to do.  So the question is, how do we exercise this responsibility?  And as we talk about this, we've got a really distinguished panel of people working not  just in core databases, but also in a number of neighboring fields, giving us a breadth  of perspective that I think is going to serve us in good stead.  So I've got the names listed here in alphabetical order.  So we have in the middle there Francesco Bonchi from the ISI Foundation in Turin.  To his left is Tina Eliassirad who's at the North Eastern University and leads the Just  AI project.  I love the name of the project.  To her left is Krishna Gumadi from the Max Planck Institute in Germany.  And rightmost is Julia Stojanovic from NYU.  And you all heard Lisa's talk this morning.  What we're going to do is do a five minute position statement each for everybody other  than Lisa who gave an 80 minute position statement already.  And then we'll open it up for questions and discussion.  So with that, I guess I'd like to get started.  So Krishna.  Thanks Jack.  So rather than make a position statement, I would like you to view this as a couple of  lessons that I learned while working on this topic of ethics and algorithm design over the  last couple of years.  So it took me, so I'm just going to talk about two lessons.  Now to get on to the first lesson, one of the things that surprised me in tracking with  philosophers who do ethics research is how much I was, I had a fetish for utilitarianism.  Now and how I didn't recognize the limits of utilitarianism.  Now to explain this, let me just walk you through a thought experiment that I would encourage  you guys to participate in that is often used to illustrate the limits of utilitarianism.  This is called the trolley problem.  Suppose you have an out of control trolley that is running down the tracks and you are  the person in charge of that lever over there.  Now you have the power to pull the lever and divert the trolley from going down straight  and killing five people but instead move to a side track.  The problem is on the side track you have one person.  Now how many of you would pull the lever with a quick show of hands?  Okay that's far less than what I expected but it's interesting because you are choosing  to not save five lives.  One way to think about this if you're a utilitarian would be you're choosing to lose five lives  instead of one.  Now let's take another version of this trolley problem where you have again an out of control  trolley aimed at five people on the tracks but now you're the person who can push a fat  person on the bridge in front of the trolley to stop it.  Now how many of you would actually do this?  Again a quick raise of hands.  Now notice that the number of hands actually went down.  Now let's look at the final, a different version of this where you're a doctor at a hospital  and you have a patient who has come to check in and you could actually harvest their organs  to save five lives, lives of five other patients.  How many of you would actually kill or harvest the organs of this patient to save five lives?  Okay, there are three or four of you and that's extremely worrisome.  So we would do this.  Now the point that I wanted to make, the reason why these trolley problems are interesting  because if you looked at it from a purely utilitarian perspective all three scenarios  are the same.  It's a trade off between five lives and one life.  However, there is something about the specific scenarios that made a big difference.  Now I don't have the time to go through why it is that we might have felt differently  but to just highlight what utilitarianism is, one where we decide the righteousness  or the fairness of a decision system purely based on the outcomes where we define some  utility function at the level of individual outcomes and then we look at, we pick the  decision system that maximizes some of the utilities of outcomes for each individual  in a population.  Now that is what utilitarianism is about.  Now if you look at many algorithmic designs and particularly machine learning, it is overrun  with utilitarian ethics.  So for instance, as Lisa pointed out in her talk, when you're trying to learn the optimal  boundaries we often try to define and optimize the loss or accuracy function where we try  to minimize expected loss over all examples in the training data.  Now this is a classic utilitarian ethics.  And this actually explains the root cause of discrimination.  I'm sorry for the mess up of the slides.  I think we did this over a Mac laptop.  So utilitarianism leads to minimizing the overall prediction errors but this doesn't address  discrimination because discrimination is about utility at the level of groups of individuals,  not at the level of single individuals.  So to just quickly illustrate this, suppose you have three different decision systems,  system one, system two, system three.  What I'm showing in the table are how accurate these decision systems are for men and women  and overall.  Now if you were a utilitarian, you would probably pick system one because that's the one that  maximizes the overall accuracy.  If you are a person who cares extremely about equality of group error rate, you might actually  pick the system three.  Whereas in practice, many people might actually pick system two because you use a different  normative criteria, one which shows that actually system two is giving better accuracy for both  men, women and overall compared to system three.  So what I've just pointed out is a different normative criteria than just discrimination  that actually kicks in when we choose which decision systems to follow.  So the first take away or the first lesson is don't rely exclusively on utilitarianism  to design and evaluate algorithm designs.  Now this of course raises the question, how should algorithm designers select the right  set of normative criteria?  So utilitarianism is one criteria.  Now there are other types of criteria like discrimination is a normative criteria.  How should one select the right set of them?  Now I actually think, or this is my position maybe, that the search for right normative  criteria is not the job of algorithm designers.  Our job is to operationalize a given normative criteria.  Now what I mean by operationalization is how to make these criteria formally measurable  in terms of empirical observations.  So to take a quick example, if you think about discrimination, discrimination has been studied  in moral philosophy, political sciences, economics for over 100 years now.  And there is actually a approximative normative definition that is here, which is discrimination  is about wrongfully imposing a relative disadvantage on persons based on their membership in some  salient social group.  Now typically if you look at ethics, and this is a definition that I picked from Stanford  Encyclopedia of Philosophy, the definitions are usually in English.  And the challenge for us as algorithm designers is how do we operationalize these fuzzy notions  like relative disadvantage, wrongful imposition based on salient social group.  Now over the past several years, if you looked at the literature on algorithmic discrimination,  people have operationalized or formally defined various different ways to measure these fuzzy  notions.  And this has led to a plethora of definitions of discrimination.  Now one might wonder, aren't these one too many interpretations?  Yes, but given the different contexts in which we want to use these systems, they are actually  quite needed.  And so I think we should stop worrying about there being 20 or 30.  In fact, it's surprising that there are only 20 or 30 and not hundreds given that fairness  is a topic that has occupied ethicists for like hundreds of years, if not thousand.  And of course, when we are operationalizing, designers should maintain informed neutrality  between different reasonable ethical positions.  And what I mean by that is we shouldn't look at each of these ethical positions as necessarily  competing with each other as opposed to being applicable in different contexts.  And designers should not attempt to resolve which position is better.  And finally, once you define the formal measures, there is a lot of interesting work that comes  in terms of designing algorithms that's for the formal criteria.  Lisa mentioned some impossibility results of designing some algorithm that would satisfy  multiple criteria.  Then there are all sorts of algorithms, designs that try to achieve optimal performance based  on lexicographic orderings or Pareto optimal tradeoff between the different criteria.  So the take away for the lesson two for me is as algorithm designers, we should study  the connections between the designs and consequences while maintaining informed neutrality between  several different reasonable ethical positions as opposed to thinking that we could actually  determine what is the right ethical position.  Thank you.  Track café and  it's not but Lee's selection.  I don't know how I got killed that way.  So I'll be brief.  What is my main point here is that where we stand with this fairness of quotability, transparency  research, I believe the topic is still in its infancy and it reminds me a lot about where  the privacy preserving that analysis and management area was at the beginning of the 2000s.  There are many similarities.  It's a very hot topic with an exploding interest.  There is an explosion of number of papers that are coming out on this topic, but those  are in my opinion mostly simple ad hoc solutions.  I would say like a good 90% are on the classification task.  Maybe even a very large fraction of these are on the binary classification task.  And so overall I feel that there is a lack, I mean still a lack because the areas just  boom of some deep robust theoretical foundation.  So the two areas have many similar challenges.  There is this proliferation of different notions and definitions that also Krishna's was mentioning  before.  It is the problem of different regulations which was a problem also for the privacy preserving  that management research, the lack of clear benchmarks.  And the need to interact with all the players in the play field that are the legislators,  the policy makers and the need for multidisciplinary approaches.  So interacting for instance with social scientists and philosophers.  And both privacy preservation and these fairness accountability transparency, they technically  they usually have to deal with this trade off between receiving some desired privacy or  fairness or transparency goal while not losing too much utility in our data and our algorithms  and so on.  So there are also interplays between the two notions because privacy preservation is about  preventing sensitive information from being leaked and when we talk about fairness we also  talk about sensitive information, we want to prevent it from being abused.  This notion of sensitive information can overlap in some attributes or can be distinct in others.  And there is not only positive synergy, there are also potential conflict between the two  notions because for instance having a sanitized database for privacy issue might even hide  the presence of discriminatory patterns.  So if you take a database which is the output of some algorithm and you want to see if there  is some form of gender bias, if the column gender has been removed for instance for imposing  key anonymity, then good luck with that.  So I also believe that an holistic approach for this reason, an holistic approach taking  privacy and fairness accountability transparency all at once would be very desirable and probably  is also achievable.  So going forward I believe that after this initial phase of excitement and increasing  of awareness in the society and among the research community, the FAT area should produce  some theoretically foundational definition and result like for instance a nice example  is differential privacy for the field of privacy preserving data management.  And in this I think this community can contribute a lot as it was the case for privacy in the  early 2000s, just think about all this notion of key anonymity, diversity, closeness, privacy  preserving data mining, privacy preserving all up, apocalyptic databases that were all  presented or introduced by this community.  So we need you, get on board, the field is exciting, there are many low hanging fruits,  many interesting problems to be solved and I would say it's not too late, we just started  so get on board.  Thank you.  We are not here at all.  Hi, everyone.  So actually my first slide is this.  So the message that my colleagues and I have been trying to impart on this community is  that we all should be working on problems in data ethics, in responsible data science,  in another name for this field that Francesco used, this fairness, accountability, and transparency  or FAT.  This is a name that we are inheriting from another community, FAT ML.  So since 2015 or so, we started telling all of you that data ethics is a very important  area of research.  And I think that at today's, at this particular sigmod, we can declare victory.  The job is done, right?  I mean, we have, we just heard a wonderful keynote on responsible data science.  The best paper award is going to a paper that is on this topic.  There is going to be a wonderful tutorial by Suresh and Pods, so please go to that.  And there are several other papers at this conference that are on these issues of data  ethics, broadly speaking.  So in one sense, I'm happy.  I think, you know, the job is done to some extent.  But what I want to tell you today is a completely opposite message, right?  Just to be controversial, because you've been hearing me say the same thing, so now  I'm going to tell you something completely different.  And that is that we should not, the database community should not work on data ethics.  Now, what do I mean by that specifically?  What I mean is that we should not be working on data ethics with only our technical head  on.  That a lot of work has been done in communities like machine learning and like data mining,  where many, many fairness definitions were proposed and operationalized, and lots of  papers have been published, but this work has not had an impact on the real world to  the extent to which it should have had an impact, given how much time was spent by really,  really smart people thinking about these problems.  And an even scarier thought here is that, in fact, many of the results that came out  of the technical community in this field have been harmful to society, because they have  helped decision makers at all levels whitewash results of structural discrimination in our  society.  And this is something that we must work very hard to avoid.  So I will give you an example of this.  Before I do, I want to read a quote by a colleague and a friend, Solon Barokas, who taught me  all I knew at the time in 2016 about this area, and it's very insightful.  He says, I worry very much about the co-opting of the work on fairness and machine learning,  about how it may no longer be serving the progressive goals that it had at its inception,  how it can be deployed to whitewash the problematic and unjust practices that continue to plug  some of our most prominent companies and institutions.  So my message to you today is that, as we work on these topics, we really must be aware  of the impacts that our work has on society, and in particular, that our work in this area  has on society.  Whom are we helping?  Right?  Whose interests are we protecting?  And an example that I want to give here is, and this is my exhibit A, because I'm talking  about the criminal justice system, is the following.  So the state of New Jersey recently enacted a comprehensive reform of the criminal justice  system, and one of the goals that they had was to make it so that people who are accused  of a crime are not compelled to stay in jail pending trial, but rather are able to post  bail and then go home and continue working and continue with their daily lives and then  return to court.  So we wanted to be able to release people on bail much more so than we were doing in  the past.  New Jersey has this goal, right?  This is sort of a vaguely stated goal.  And the reform that they enacted is made up of several interventions.  All of these were deployed at once, and one of the interventions that they deployed was  a framework called PSA, and this framework is in a similar vein as the compass system  that Lisa described in her keynote earlier today, where it essentially produces a risk  score, a number, that the judge then takes under advisement when making their decisions,  bail decisions in this case.  And so one thing that's distinctive about New Jersey is that if a judge's decision  differs from the recommendation that the PSA framework gives him or her, then the judge  needs to document their disagreement in writing.  So think about that for a second, right?  A judge who is not a data scientist gets a number from a machine that has been presumably  validated and thoughtfully produced, and they now need to actually state a disagreement  if they want to release somebody despite the recommendation of the system.  Raise your hand if you think that a judge would release somebody under these conditions  despite the recommendation of a system to keep somebody in jail pending trial.  None of us think that the judge would take that risk, right?  So a thing that disturbs me here the most, perhaps, is this quote that you've been staring  at while I've been talking, and this is from a New Jersey pretrial justice manual that  was produced by some of the good guys, the ACLU among them, and that advises defenders  on how to now defend, it advises lawyers on how to defend those accused in this environment  in which PSA is deployed.  So the manual says, switching from a system based solely on instinct and experience to  one in which judges have access to scientific, objective risk assessment tools, could further  the criminal justice system's central goals of increasing public safety, reducing crime,  and making the most effective, fair, and efficient use of public resources.  Why am I showing you this quote?  Well, it's because whoever produced this system actually didn't do their job of explaining  to the people using the system that the system is not objective, that it involves trade-offs,  that the data that it uses and the society in which it is deployed is one in which we  have results of historical injustice and discrimination, right?  So this particular system does not pass by any measure, the scientific and objective  descriptors that we have in this statement.  So this is very misleading, and it's essentially objectivizing some of these decisions, and  it's legitimizing them and scaling them up that these systems embed.  And we have, of course, here an example of the compass system that we already looked  at that tells us that, in fact, mathematically and societally, it is impossible for us to  be fair in the sense of how the false positive and false negative errors are distributed  in a society that is itself unfair.  And this result, of course, also applies to PSA and to any other system of the sort that  is deployed in our society today.  So what do I want to say with this picture that Krishna already showed?  It's that the real trolley car problems that we're facing as a society and as technologists  are not those where we have an autonomous vehicle that's going to run somebody over  one person or five people, and then we have to choose how many.  The car should stop.  Rather, the trolley car problems that we need to worry about are those where we're thinking  about how we're trading off different fairness and utility notions in the use of these predictive  analytics in society.  So I'll stop here because perhaps I'm over time, but I'll just kind of point you to this  potential solution.  So what do we need to do?  How should we, in fact, work on these problems?  Well, we need to think about law and social sciences and talk to people in these fields  because guess what?  It's much easier for us to read a law paper.  Believe me, I've done it.  Then it is for a lawyer to read our papers because their papers are well-written and  in English, right?  And ours are formulas and less well-written.  So I'm going to just leave this here for a second for you to try and decipher.  Solutionism, something that we're proposing with technology fixing the problems of society,  like reductionism, is the deadly sin that we need to carefully think about and avoid  as best we can.  So I'll stop here.  So we had a technical glitch because of which Tina Slides didn't make it to the prediction  laptop.  And so we're chained up.  I'm very good at projecting.  Don't worry about it.  So while we were doing this, when Judy Thompson, who's a philosopher at MIT, came up with a  problem back in 1976, she never thought that computer scientists were going to use it this  way.  So be careful when you come up with something.  It can be used in ways you do not wish for it to be used, as was mentioned by Lisa earlier.  And while this is being set up, so I started working in this area because Daniel Allen,  who's the director of the Ethics Center at Harvard, asked me to come and give a public  lecture on this topic.  And I thought, oh, this is great for a broader impact for an NSF proposal, right?  Computer scientist goes and talks about computer science to a bunch of ethicists.  So I don't need my slides.  So then that led to a project called Just Machine Learning.  And if you think about machine learning, I really like the definition that Tom Mitchell  has in his 1997 textbook, which is called Machine Learning.  He defined the well-closed learning problem as a triple.  You have a task, you have experience, and you have performance measures.  And when machine learning goes wrong, usually one or more of these things have gone wrong.  One of the things in this particular area that people have focused a lot about is that  experience, that data.  That for example, the data is predominantly, so the data predominantly includes white males,  even though the majority of the population is not white male, right?  And that's why, for example, Alexa works better for white male voices than for female voices.  And so, thank you.  And so this is his definition of the well-closed learning problem, right?  So a computer program is said to learn on a particular task, given an experience, if  over time it improves on that task given the performance measures.  So people have focused a lot on the experience and the performance measures, but not so much  on the task.  In fact, the tasks that people have worked on predominantly has been risk assessment  and ranking.  And here are just two reasons about why people have worked on them.  One is computer scientists know a lot about them.  We know a lot about classification.  Risk assessment basically boils down to classification, which is sorting, right?  Good, bad.  High risk, low risk, right?  And ranking.  We have done a lot of work on ranking.  And also the human decision makers like these tasks because it's easier for them, right?  Tina's bad, Lisa's good.  Perfect.  Tina goes to jail, Lisa goes free, right?  And so there has been a lot of issues with these tasks that have been well-documented  and celebrated like impossibility results that were mentioned earlier.  But if you think about it, these tasks are just too abstract, they're too under-described  to have any kind of ethical consideration in terms of what you should do.  And one of the things which is horrible is that they provide no uncertainty values, right?  It's not like the judge, you go to the judge and say, yes, Tina's risk of recidivism is  high, but the algorithm is only 65% sure.  You don't tell that to the judge, right?  You don't provide any kind of context to the judge.  It's not like, well, Tina is similar to Lisa in these ways and not similar to Lisa in these  other ways.  And in particular, these tasks cannot incorporate anything about how the human decision maker  makes a decision, right?  They're just sorting.  Everything is just not good enough when you want to have some ethical consideration on  your task.  And this slide, Lisa showed where a judge is trying to cover his behind.  He's saying the algorithm says, Tina's bad, I'm not going to let Tina go out.  Because if Tina commits a crime, then I'm going to show up on the front page of New York  Times.  And so he's trying to cover his behind.  And he is throwing out the plea deal that experts have come up with, which leads to what are  the incentives of the human decision maker using your algorithms, using your system?  Is the judge using it just for efficiency purposes?  I have this pile of cases to go through, so I use Lisa's algorithm to sort through them  as quickly as possible.  Is it profits in terms of health care system, right?  So you need to be very cognizant of what are the incentives of the human decision maker  in using your algorithm.  In almost all the work that's out there, this is not discussed.  And so we should think about other kinds of tasks, right?  And as was mentioned earlier on, if you think about normative ethics, you can break it down  into three main groups.  One is the ontology, so be like Kant and follow your duty.  Two is consequentialism, which utilitarian falls under it.  And three is virtue ethics.  So if an algorithm is going to make a decision on me, I will like that algorithm to have  been trained to be a virtuous agent, right?  So what I would like to do is learn algorithms that fall under virtue ethics.  And so, and in fact, if you think about professional fields like medicine or law, there is this  notion of apprenticeship.  Now, if you think about here, we actually have a task.  It's called apprenticeship learning or imitation learning.  It's just a hard task.  You can't have zillions of papers coming out, like it's money that you're printing on,  right?  And so it's a hard task.  It is a task that's been studied in machine learning.  And it has not been applied here, because what I need to do is I need to learn how a  moral agent, how Sonia Sotomayor makes a decision.  And I need to know context, and I need to know many, many things.  And there's a nice tutorial at ICML 2018 on this in terms of learning from a demonstrator  or learning from demonstration to learn a policy about how to make a decision.  As opposed to I get 30 years of data, throw it into this classification algorithm, which  sorts people out and bins for me.  And again, I'm not the first person to come up here and say virtue ethics, but computer  scientists usually don't like virtue ethics, because it doesn't have these nice rules,  right?  I'm going to follow duty.  I care about what the consequences are.  And then the other thing is that perhaps there are certain tasks that we as a society should  say that machine learning or any kind of automation should not be used.  Or the bar for using them should be a lot higher, like dangerousness, right?  In terms of pre-child disposition in America, the judge is trying to decide two things,  flight risk and dangerousness.  And flight risk maybe is easier for an algorithm to decide, right?  Did I take Tina's passport away?  How long has Tina lived here?  Does Tina have a job in this place?  Right?  What are Tina's connection to the community versus dangerousness, which is just horrible,  right?  Tina is not white, so Tina's dangerous, right?  So there should be a much higher bar.  And in fact, one of the things that has happened is that bail is being used for crime control  and wasn't designed for it, as my colleague, David Rovirez, at Northeastern talks about  it.  And lastly, even if you do come up with a perfect task, one of the questions is, did  the algorithm learn for the task, right?  And so Lisa was mentioning about the deep network and classifying wolves.  This has been going on for a long time.  Back in the 80s, there's a classic example of the army wanting to classify tanks that  were camouflage or not.  And it looks like what they learned was basically whether there's a shadow or not.  And then recently, there was some awful science that came out of Stanford that said that, oh,  we can, from a picture, tell whether you're gay or not.  And it looks, it seems, what they learned was some pictures have better lighting.  But imagine being gay in a country that you would be murdered because you're gay.  I wouldn't want to have that algorithm, right?  And have all of those souls on my head.  So the takeaway message is, so far, what has been done is we've been trying to pack all  the normativity versus description.  So being normative is what ought to happen, what should happen.  Being descriptive is what happens.  Murder happens, but murder should not happen.  And they're trying to build all of this normativity into these fairness measures as part of performance  measure.  And that is wrong.  It's just not enough there.  What you need to do is think about the task, think about the experience, and think about  the performance measures, and try to build normativity, what ought to happen in all of  them.  There is a task out there, I'm not saying it's a panacea, but it's apprenticeship learning.  And again, put yourself in front of a judge and think about, OK, if a judge is going to  use an algorithm on me, would I want the algorithm to behave like Sonia Sotomayor?  Like Ruth Bader Ginsburg, right?  Or would I just like it to learn from 30 years of data that have lots of prejudice and it's  just starting sorting?  So on that note, I'll finish.  Thank you.  So this is the time for you to queue up questions.  And while you're getting your questions ready, I want to start the panel off with one question  to try to bring together what has been said thus far.  How should a DB researcher approach this problem?  We've talked about there is a problem, and we heard that in the keynote this morning.  And what you got in terms of the presentations were some of the challenges in trying to address  this problem and some of the difficulties with how some people might be thinking about  this problem in general.  And one thing there is that this is not a purely, that's pretty obvious, is that this  isn't a purely technical thing.  And so if the questions to address go beyond databases, and in fact, they go beyond computer  science, then there are at least two ways in which you could conduct research.  One is to say, I'm going to work on the problems that I understand and I know how to solve.  And I realize that this is only a piece of the puzzle, but I'm going to work on the piece  of the puzzle that I'm good at.  And other people will work on the other pieces.  The other way to think about this is to say that the only way to address this is to formulate  a disciplinary team, and we've got to collaborate with experts in other fields, and that's the  only way we'll be able to address this problem.  And so what I'd like to do is for the panelists quickly to say either A or B, and if they  want to perhaps a couple of sentences to justify their choice.  Okay, so I'd begin with Krishna and just go down.  It's mostly A with some information, with some aspects of B in terms of figuring out  how to understand what the problem is about, but it's mostly A.  It should be B.  I mean, again, what you ought to do, you should think about what you ought to do.  What would make your mother proud is B.  Think about it.  It's B.  I think both.  I mean, if...  That's not an option.  No, I mean, I'm fine with Krishna doing A and Tina doing B.  I think as researchers we have different aspirations, different approaches at the research, and all  together A and B can help progress the field, I think.  Neither.  So I think, first off, you have to be introspective and think about yourself.  How are you going to make impact?  And there are ways of making impact.  I personally really enjoy going across the boundaries, but I've been a party to a number  of conversations where there's a lot of eye-rolling.  The technologist rolling their eyes, the humanist rolling their eyes, and being able  to be comfortable with that.  But then you have to think about what works for you.  You can't do...  Yeah.  So I'm going to say B as well, although I've, of course, done A starting out and then moved  slowly into B, as many of us have.  So the time has sort of passed where you could just say fairness plus X equals paper, right?  And we've all done that work.  But the train has left the station.  This is something that we missed in this community.  So now we actually have to be thoughtful.  And I think an important aspect of B is that we get to educate the others about technology.  And people, even those who practice technology law, for example, can use a lot of our help,  and it is our professional responsibility to educate others about what we know.  And interdisciplinary teams can give us that opportunity.  And we can also learn from them very importantly, in particular, to be conservative about the  claims that we make about the applicability of the technology that we develop, right?  I mean, we can't just go out there and say, my fairness measure, best thing after sliced  bread.  Everybody should just use this, right?  It doesn't work.  That is irresponsible to do.  OK.  So you've got some thoughts on this question.  Are there thoughts that people in the audience have or questions for the panelists on what  they've said so far?  I think there's a question in the back there.  Hi.  So thanks for these insights.  It was really nice to see all these different opinions.  This question here is about what I should be doing in the future, but I think many of  us might be worrying what I'm doing or what we're doing right now.  So I'm working on problem X. How do I recognize that I should also worry about fairness or  ethics?  So if I'm developing the next best B-tree, maybe it's not quite as relevant versus I'm  evaluating surveillance data.  So is there like a recipe or is it just a gray area?  Is there any way for me to tell, should I be talking to lawyers and philosophers or  am I OK with the current research I'm doing?  Can I?  OK.  So the ACM recently issued an update to its code of ethics and professional conduct.  And we as a community have actually participated in that revision.  I was volunteered by Don Cosman here to be on that task force.  And what I encourage all of you to look at that document, it's quite thoughtful.  It's an improvement on the previous version.  And what it tells us is that ethics is not a recipe really, right?  There isn't an algorithm out there that you can just run and it will tell you, you know,  worry about this and this is a way to fix it.  It gives you a set of questions to ponder and you need to essentially figure out how  you're going to trade these off, right?  It's neither consequentialism nor the ontology that is right.  So you have to think about these issues on your own.  It's very important to do this.  And look at the code.  So if I may, Kate Crawford, who is at MSR New York, said that no computer scientist  can just walk around and say I'm an engineer anymore, right?  Because of our success, right?  Our tools are being used.  And so you should, you are a moral agent.  Think about it yourself, right?  Are there issues in terms of them using your tools, your algorithms that you would not be  okay with?  And if that is the case, then you should seek out a philosopher, an ethicist, a lawyer,  and discuss it with them.  I think especially about law, there's a big thing in terms of liability, right?  If Tina's algorithm is used and somebody dies because of it, who's liable?  Am I liable?  Is the person who used my algorithm liable, right?  So there's a lot of aspects in terms of law that need to be sorted out, which hasn't  sort out itself yet.  And we should be part of that discussion in terms of liability of our tools, algorithms,  et cetera.  So there's sort of a, hopefully, a conflict that I want to point out between the families  that maybe you can resolve.  So Francesco said earlier that remarking on the similarities between privacy and fairness  that we should, one thing we might want to do is look for the equivalent of differential  privacy for fairness, which I will call the one true definition.  And I think Julia and others are arguing, don't take your fairness definition and go  and wave it around like a sword and understand the context of what you're doing.  To me, as a researcher who wants to think about this, instinctively, I would like to  gravitate towards Francesco's point of view.  I would like to think that there is a definition I can find that if I find it, or if someone  finds it, that we can all look at it.  And that is the way we will think about this.  And of course, from yesterday's talk, there was a lot of success with that.  But also, I feel like that's not quite right.  And I wonder if the discussion on the panel, if you could sort of thrash out if there's  actually a disagreement there between folks there on that.  I believe we should not simply come out with our definitions and then convince the rest  of the world.  But all together with the policy makers and the legislators, we can have a virtue cycle.  So the laws give us constraints, objective function.  And once we do progress with our research, we can get back to them, try to explain what  we achieved.  And this can go in cycles.  It's not just one shot and we are done.  So I think along those lines, one of the things which is interesting is, can you reconstruct  the intent of a policy?  So if you talk to policy makers, every policy has an intent.  And as Francesca was saying, the laws are constraints and then the policies are how  those laws are interpreted.  And then you have police and other people who are implementing actually in the environment.  And so one thing that could happen is that I'm seeing the droppings of how that policy  is being implemented.  And can I reconstruct the intent of the policy?  And if the intent of the policy is not the same as what the policy maker is saying, then  perhaps we can change the policy.  But I think this notion that you can code up ethics in a set of consistent logical rules  is just wrong.  You can't because to be ethical, you need to have empathy.  You need to be able to walk your way through conflicts.  And that's very difficult.  To have one formal definition of that, just think about it.  May I add, so I think that a critical shortcoming of the differential privacy technology and  the reason why we should not be trying to mimic differential privacy as a kind of a  model for a solution in ethics is that it is very difficult, possibly hopeless, to explain  how differential privacy works, what the epsilon parameter means, what the tradeoff is between  privacy and utility to a member of the public.  And I would love to be proven wrong by members of the differential privacy community.  So I think that this technology and its use in society will live or die by whether we  are able to provide explanations that are meaningful, not about how it works, but what  effects it has to members of the public.  So please take that challenge.  And if you are able to explain this technology to us, then perhaps we can look for a common  technical framework for other aspects of ethics.  But at the moment, I'm not convinced.  So I think the last two comments in a way I think may have said what I wanted to say.  I think these things have the roots and value systems.  And whether it's people that are making the decisions, they're still not perfect because  they're rooted in the value system that those people bring in to fairness or ethics or any  of these things.  And I think that we're trying to go a step further and say AI ethics or how would algorithms  view fairness and so on, which is even more complicated because now you have to figure  out what value system these algorithms and this is what some of the comments that were  made.  So I mean, I don't think that even if you brought two lawyers that would reach the same  decision because there's a lot of culture and value systems and so on and so forth.  So I mean, I think I just, I'm reading this here also, but I don't think we can simplify  this.  This is going to be something that needs to be not only be, but really much more complicated  than even be.  And the answers are not going to be with computer scientists or acknowledges.  The answers are going to be, you know, because philosophers and ethicists and so on have looked  at this for hundreds and hundreds of years.  And there's a lot of depth into it that, you know, but I think the last two, you made  the same remark.  So I think this is, oh, sorry.  Go ahead.  So addressing that point and the previous point, I think one of the places where we  can take responsibility is more of education about the limitations of our systems.  So we're talking about a particular formalization of an optimization problem, a formalization  that leaves out important pieces, and that's always going to have limitations.  And to act like it doesn't, I think, and you can see in the comments that are shared that  non-technologists don't understand this.  I've had conversations with folks from the STS community where they're kind of explaining  to me, you know, well, your algorithms, you know, leaving this out.  And I'm like, of course it's leaving it out.  You don't think I actually believe that this is true.  I think we have to figure out how to not leave people with this impression.  And that is a place where kind of doing comparative modeling, being able to look at more of the  context and then think really about what the task is.  And how do you aid the human decision maker in this process?  What are their goals as important as well?  In fact, I think this is where there could be a lot of innovation from the computer  science community is what the right task is.  In fact, this is why I think we should do virtue ethics going back to your point.  Because again, if somebody is making a decision about me, I would like them to be a good person,  right, that makes a decision about me.  So I want my algorithms, my tools to have been learned on virtues and vices.  Now, there are a lot of people who do not like virtue ethics.  For example, they say, I don't care that you're doing the right thing.  I want to know why you're doing the right thing.  You need to explain it to me.  So that's meta-effects.  But I think at least for us right now, if our algorithms are going to be used to make  human decisions, it's best if they learn to behave like a good human.  Now, there's a question of who is a good human and how you decide and all that kind of stuff.  And that's why we need interdisciplinary teams.  This actually is a very good segue to my next question for the panel, which is, how  should we evaluate work in this area?  You know, if you do systems research, sort of the standard thing is you build system  you measure performance.  Maybe you're not after performance, you're after functionality or usability or whatever.  Again, you build system and you evaluate.  Now if your innovation is in building a more responsible system, how are you going to evaluate  it?  But one answer is it's not enough just to build the system.  You can't do a measurement there.  So you've got to deploy the system and evaluate success in the field.  If you don't do that, then as was pointed out in the keynote this morning, you're introducing  all kinds of assumptions about what your test set is.  And so in effect, you're measuring success only with respect to some mathematical criterion  of choice.  And you know, that has some value, but that's a very limited test.  So if one says one is going to measure success in the field, you have an even bigger problem  besides actually getting your system deployed.  And that is not everyone agrees on what success is.  As Ahmed's question was saying, what's right is often a value-driven thing.  And so, you know, depending on who is evaluating it, their choice might be different.  And if you're going to look at these subjective assessments, then perhaps what it all boils  down to is popular acclaim.  So like the panelists to comment on, if you're working on data responsibility, how should  that work be evaluated?  We'll start with Julia.  Thank you.  So I think maybe this is my cue to talk a little bit about the policy work that I've been involved  in.  So in my mind, you know, success here may not be in translating the algorithms and the  techniques that we develop into commercial systems, perhaps sometime down the road.  But right now, there is a tremendous void in having technologists participate in defining  how we regulate in this space.  And regulate, we must.  We cannot let technology, you know, run unchecked and make decisions that we don't understand  and enact policies that we didn't even intend to encode just by accident.  They made it into code.  So in Europe, there is the GDPR.  But there are also other frameworks that people are thinking about at the national levels  and where there is always hunger for participation from us, from, you know, just explaining what  an algorithm is and isn't, what you can and cannot learn from data to really helping hands  on come up with recommendations.  So in New York City, there is a law that we passed, and this is the first such law in  the U.S. that is compelling city agencies, government agencies, to start thinking about  how they can become transparent and accountable in their use of algorithms and data.  And they designate this ADS, automated decision systems.  And so I am on a task force that is working on a set of recommendations for city agencies,  and this is a very difficult set of recommendations to come up with.  So that an agency can respond to an individual who comes and requests an explanation why  a decision was made that affects them.  So for example, I come to the Department of Education and I ask, why was my son given  a spot in this school and not in this other school?  How can we explain that?  Technically, how can we explain it?  How can we make it interpretable to a person?  And how can we measure that in fact what we are showing them they understand and then  they can act on?  Another task here is that agencies need to show that they are not biased in the sense  of not discriminating against members of legally protected groups.  So working in efforts of this sort I think is very important for us.  It certainly is an output that we as a community can help produce.  And all it takes is for me, I went to a city hall when there was a hearing on this law.  I read the draft, it made no sense to me, and I testified.  And then they call you back and actually want your input.  So please get engaged.  It's very interesting and it's very rewarding.  So one of the places where I think this work emerges or the place that I have seen it emerge  is from the area of data science for social good.  So many of you may be aware of efforts in this space or maybe efforts in your own universities  and so on.  So I think of Raeid Ghani's program at University of Chicago, a bunch of things came out of  that.  And the interesting thing, so they start off saying, okay, data science for social good.  And I think some of the deepest work on ethics and data science has come out of that community  because they've realized how challenging it is to define social good and so on.  So kind of watching that process of seeing them try and evaluate their work and see the  challenges that came up is a place that can give us some inspiration.  I think this is a very hard question, especially for short-term impact.  I think it's quite hard to judge.  Of course, longer term, we might be able to say that result was really game changer,  was really important for the other to build on it.  Of course, we need to always keep an eye or two on the real world and try to show that  in a specific application domain, a specific context, there was something very bad going  on and practically we can improve this.  But again, I don't have a strong convincing answer because I believe it's really important  and tough question to answer right now.  I have a strong convincing answer.  Would you want your system to be used to make a decision on you, conditioned on the fact  that you are not a white straight male?  If your answer is yes, success.  If your answer is no, go think about it.  That's the answer.  I guess I have a bit of mixed thoughts on this.  On one hand, I think that it's absolutely important that we talk to researchers from  different areas.  In fact, one of the things that I experienced in these interdisciplinary collaborations  with legal scholars, economists, and philosophers who I was lucky to collaborate with is that  I started realizing how deep these notions are and how much literature already exists  on these topics.  There are lots of interesting notions, fairness notions that these social scientists have  studied.  I see a huge opportunity under research agenda where one could try to formalize, try to think  about if you were to follow this specific notion, how would you incorporate it or how  would you introduce it into an algorithm design?  I see that as very technical work that one could involve themselves in.  For instance, I would highly encourage you to just go and look at the discussion about  anti-discrimination in Stanford Encyclopedia of Philosophy.  They have the definition in English, and then they have a long discussion of how in different  contexts you might want to think about applying this law in different ways.  A lot of papers that we wrote were essentially inspired by this discussion where we said,  well, what if we tried to operationalize this value judgment?  What I fear at this point is that I fear that because there's context matters, it would  be a mistake to rush and try to, at that point, go and influence policy in the real world  because the real world is a lot more complex.  In fact, if there was anything where I was going for option A before, in fact, here is  a quick backstory about Compass.  The people who designed and deployed Compass wanted to reform the criminal justice system.  These were actually not people from hardcore computer scientists who were excited about  applying machine learning techniques, but it is just that they observed that human decisions,  the judges' decisions in the US were extremely biased, and they thought nudging these judges  with objective decision making can actually improve the system.  These were people who actually ended up deploying something in the real world without actually  understanding fully about it.  I get the enthusiasm or the desire to influence policy, but at the same time, I think there  is a, and I think people should be doing different sorts of things, and I think we should be  doing things where we have expertise on.  I think if you are a bit more of a technical person, I do think there is a lot of interesting  technical work that can be done.  If you are a much more of an interdisciplinary person who can engage with the policy makers,  I think you could do that as well.  But I think we should not think that, I think we should not miss the primary purpose of  what I think is of academic research, which is actually be able to isolate our results  a little bit from the real world and actually study the various different consequences of  different forms of different ideas.  Knowing fully well that the damage wouldn't be immediate because these things are not  going to be immediately implemented in the real world.  And I think it would be a mistake if everybody tried to do this.  I think some people like Julia and Tina, who have a bit more of experience with this, with  interacting with policy makers might be able to put it off, but not necessarily everyone.  We have a disagreement.  Yes, but we do have a disagreement and unfortunately, just as the panel is heating up, we are out  of time.  Much awaits.  So thank you all for coming.
Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_0_143.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_144_299.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_300_563.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_564_3119.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_3120_7715.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_7716_14843.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_14844_28187.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_28188_28259.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_28260_29039.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_29040_29243.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_29244_33155.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_33156_39935.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_39936_41075.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_41076_41447.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_41448_41567.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_41568_42791.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_42792_43283.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_43284_43595.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_43596_43667.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_43668_44075.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_44076_44651.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_44652_44759.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_44760_44903.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_44904_45155.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_45156_46139.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_46140_46283.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_46284_46907.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_46908_47375.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_47376_47819.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_47820_52583.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_52584_53543.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_53544_53927.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_53928_56891.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_56892_58631.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_58632_58811.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_58812_59219.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_59220_59723.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_59724_60563.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_60564_62435.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_62436_65471.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_65472_66371.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_66372_67655.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_67656_68339.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_68340_69083.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_69084_69959.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_69960_70739.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_70740_72443.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_72444_73487.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_73488_73847.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_73848_74255.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_74256_74519.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_74520_74807.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_74808_75431.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_75432_77207.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_77208_78983.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_78984_79331.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_79332_81491.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_81492_81575.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_81576_81887.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_81888_81959.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_81960_82739.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_82740_83231.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_83232_83435.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_83436_84047.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_84048_84551.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_84552_85883.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_85884_86087.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_86088_86291.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_86292_86435.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_86436_86567.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_86568_87107.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_87108_87275.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_87276_87323.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_87324_90359.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_90360_90503.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_90504_90659.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_90660_91235.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_91236_92603.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_92604_93059.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_93060_94295.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_94296_94847.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_94848_94955.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_94956_95135.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_95136_95579.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_95580_97391.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_97392_97451.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_97452_97571.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_97572_98231.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_98232_100619.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_100620_100859.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_100860_101135.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_101136_101219.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_101220_101519.png Jagadish_Bonchi_EliassiRad_Getoor_Gummadi_Stoyanovich_101520_101737.png