[ WP ] Your future self driving car has to choose between saving you or saving some strangers ( from r/futurology )

*ring-ring* 
 
 [ > > > obstacleObject ( classification, distance ) 
 
 classification= human ( s ) 
 
 distance= 800m 
 
 run eSTOP 
 
 > DangerException: Action unsuccessful 
 
 run emergencyReRoute 
 > Error: Unable to safely reroute 
 
 obstacleObject ( classification, distance ) 
 
 classification= human ( s ) 
 
 distance= 799.5m 
 
 run eSTOP; 
 
 > DangerException: Action unsuccessful 
 
 run emergencyReRoute; 
 
 > Error: Unable to safely reroute; 
 
 run eSTOP; 
 
 > DangerException: Action unsuccessful 
 
 run emergencyReRoute; 
 > Error: Unable to safely reroute; 
 
 obstacleObject ( classification, distance ) 
 
 classification= human ( s ) 
 
 distance= 799.2m 
... ] 
 
 *ring-ring* 
 
 `` It goes on like that for the next few hundred meters,'' Hank said sheepishly, moving the monitor away from the desk with an almost embarrassed gaze downward. He held the monitor to his chest and addressed the man behind the desk. `` I... I think we know what happens next.'' 
 
 The man's gaze remained fixed to where the monitor was moments before. He placed his elbows on his desk and clasped his hands together, as if in prayer. His eyes slowly closed, and opened again. He opened his mouth, just as slowly, and then closed it again. Finally, he stood, every ounce of authority in his station radiating off his dark suit bombarding the timid programmer in front of him. 
 `` So explain this.'' He said, with a softness in his voice that betrayed his gaze. `` Explain how this happened. Explain how your system failed.'' 
 
 `` Well I would n't say it *failed*...'' came a wheezy, more confident voice from over Hank's shoulders. 
 
 *ring-ring* 
 
 `` Tim!'' Hank warned. 
 
 ``... sir.'' Tim added. 
 
 The man behind the desk ignored him, and fixed his grey, piercing eyes on Hank's. The programmer's eyes sank from behind his thick glasses as he searched for words that did n't sound as stupid as the ones in his head did. 
 
 *ring-ring* 
 
 
 `` You have to understand... it was a very, *very* rare event -'' 
 
 `` Rare?'' Interrupted the man with a feigned patience in his voice. `` You set out to create a system for self-driving vehicles, and you did n't account the the event that it would need to avoid someone walking across the street?'' 
 
 *ring-ring* 
 
 `` But it's... it was more complicated than that.'' Hank explained. `` It's programmed to reroute or swerve when it can, or stop when it was out of options. But the road conditions -'' 
 
 `` The black ice.'' Tim aided. 
 
 `` - the black ice,'' Hank continued. `` The camera's do n't see the ice, but the pressure sensors on the tires feed information to the processor. The friction calculations it ran showed that swerving would result in the death of the driver, and braking would be futile, it would n't be able to reduce it's velocity in time. Any other action would have resulted in the death of the two bystanders. The program tried millions of times a second to find another option,'' he continued, indicating the monitor. `` But... as we know...'' 
 
 *ring-ring* 
 
 `` But that does n't explain how it **failed**. I need answer-'' 
 
 `` It did n't *fail*!'' Said Tim again, this time with frustration in his voice. 
 
 The man's patience broke as he placed his closed fists on the table. `` Then explain why PR has been up my ass all morning, blowing up my god damn phone -'' 
 
 *ring- ri-* 
 
 The man raised his fist and quickly dropped it back down on the cell phone on the table, silencing it. He continued, in a softer tone and through gritted teeth, `` - all fucking morning, asking for a statement. What am I supposed to tell them? The system *succeeded*? Killing those people was all part of the plan? It **crashed** for god's sake-'' 
 
 `` Tim's right.'' Hank said. `` It did n't crash. Well, I mean the *system* did n't crash. The built-in artificial intelligence did exactly what it was supposed to.'' 
 
 `` And what's that Hank?'' 
 
 `` It was faced with an impossible situation. None of the built in methods were helping so it had to make it's own.'' 
 
 Hank turned the monitor around to show the man the last line of code on the screen. 
 
 [... public void 4gc780cko ( *parameters* ) { 
 
 Action 4gc780cko = new Action ( reroute.terminate ); 
 
 execute 4gc780cko; 
 
 } 
... ] 
 
 `` It created a method to terminate it's action sequence. It figured the only way to win an un-winnable situation was to not attempt to win at all. To quit.'' 
 
 `` And it did n't reason that by quitting he would in fact kill both the pedestrians and the driver?'' 
 
 `` It probably did. And it probably knew that one way or another, he could n't save both -'' 
 
 `` *She* could n't save both,'' Tim interrupted. `` We, er... we decided C.O.D.A was a girl.'' 
 
 `` Sorry. *She* could n't save both. So it stopped being about saving the humans, and about proving itself. It became about not failing.'' 
 
 The man sunk back into his chair, defeated, and stared into emptiness. He broke a laugh under his breath. `` Did you ever think a machine could be selfish?'' 
 
 `` Sir,'' responded Tim. `` I think that is the most human thing a machine *can* do.'' 
 
 *EDIT: Some name misspellings and other minor errors. 

