top 200 commentsshow 500

[–]Prof-Stephen-HawkingStephen Hawking[S] 3873 points3874 points  (753 children)

Professor Hawking- Whenever I teach AI, Machine Learning, or Intelligent Robotics, my class and I end up having what I call "The Terminator Conversation." My point in this conversation is that the dangers from AI are overblown by media and non-understanding news, and the real danger is the same danger in any complex, less-than-fully-understood code: edge case unpredictability. In my opinion, this is different from "dangerous AI" as most people perceive it, in that the software has no motives, no sentience, and no evil morality, and is merely (ruthlessly) trying to optimize a function that we ourselves wrote and designed. Your viewpoints (and Elon Musk's) are often presented by the media as a belief in "evil AI," though of course that's not what your signed letter says. Students that are aware of these reports challenge my view, and we always end up having a pretty enjoyable conversation. How would you represent your own beliefs to my class? Are our viewpoints reconcilable? Do you think my habit of discounting the layperson Terminator-style "evil AI" is naive? And finally, what morals do you think I should be reinforcing to my students interested in AI?



[–]TheLastChris 937 points938 points  (442 children)

This is a great point. Some how an advanced AI needs to understand that we are important and should be protected, however not too protected. We don't want to all be put in prison cells so we can't hurt each other.

[–]justavriend 73 points74 points  (75 children)

I know Asimov's Three Laws of Robotics were made to be broken, but would it not be possible to give a superintelligent AI some general rules to keep it in check?

[–]Graybie 224 points225 points  (62 children)

That is essentially what is required. The difficulty is forming those rules in such a way that they can't be catastrophically misinterpreted by an alien intelligence.

For example, "Do not allow any humans to come to harm." This seems sensible, until the AI decided that the best way to do this is to not allow any new humans to be born, in order to limit the harm that humans have to suffer. Or maybe that the best way to prevent physical harm is to lock every human separately in a bunker? How do we explain to an AI what constitutes 'harm' to a human being? How do we explain what can harm us physically, mentally, emotionally, spiritually? How do we do this when we might not have the ability to iterate on the initial explanation? How will an AI act when in order to prevent physical harm, emotional harm would result, or the other way around? What is the optimal solution?

[–]sanserif80 40 points41 points  (11 children)

It just comes down to developing well-written requirements. Saying "Do no harm to humans" versus "Do not allow any humans to come to harm" produces different results. The latter permits action/interference on the part of the AI to prevent a perceived harm, while the former restricts any AI actions that would result in harm. I would prefer an AI that becomes a passive bystander when it's actions in a situation could conceivably harm a human, even if that ensures the demise of another human. In that way, an AI can never protect us from ourselves.

[–]Acrolith 99 points100 points  (6 children)

There's actually an Isaac Asimov story that addresses this exact point! (Little Lost Robot). Here's the problem: consider a robot standing at the top of a building, dropping an anvil on people below. At the moment the robot lets go of the anvil, it's not harming any humans: it can be confident that its strength and reflexes could easily allow it to catch the anvil again before it falls out of its reach.

Once it lets go of the anvil, though, there's nothing stopping it from "changing its mind", since the robot is no longer the active agent. If it decides not to catch the falling anvil after all, the only thing harming humans will be the blind force of gravity, acting on the anvil, and your proposed rule makes it clear that the robot does not have to do anything about that.

Predicting this sort of very logical but very alien thinking an AI might come up with is difficult! Especially when the proposed AI is much smarter than we are.

[–]dysangel 17 points18 points  (0 children)

his short stories influenced my thinking a lot as a child, maybe even they're what ended up getting me really interested in programming, I can't remember. But yes, this is exactly the type of hackerish (in the original sense of the word hacker, not the modern one) thinking required to design solid rules and systems!

[–]xinxy 99 points100 points  (28 children)

So basically you need to attempt to foresee any misrepresentation of said AI laws and account for them in the programming. Maybe some of our best lawyers need to collaborate with AI programmers when it comes to writing these things down just to offer a different perspective. AI programming would turn into legalese and even computers won't be able to make sense of it.

I really don't know what I'm talking about...

[–]Saxojon 40 points41 points  (18 children)

Just ask any AI to solve a paradox and they will 'splode. Easy peasy.

[–]giggleworm 53 points54 points  (8 children)

Doesn't always work though...

GlaDOS: This. Sentence. Is. FALSE. (Don't think about it, don't think about it)

Wheatley: Um, true. I'll go with true. There, that was easy. To be honest, I might have heard that one before.

[–]convictedidiot 28 points29 points  (5 children)

I very much think so, but even though I absolutely love Asimov, the 3 laws deal will highly abstracted concepts: simple to us but difficult for a machine.

Developing software to even successfully identify a human, when it is in danger, and to understand it's environment and situation enough to predict the safe outcome of its actions are prerequisites to the (fairly conceptually simple, but clearly not technologically so) First Law.

Real life laws would be, at best, approximations like "Do not follow a course of action that could injure anything with a human face or humanlike structure" because that is all it could identify as such. Humans are good at concepts; robots aren't.

Like I said though, we have enough time to figure that out before we put it in control of missiles or anything.

[–]BjamminD 206 points207 points  (81 children)

I think the irony of the terminator style analogy is that it doesn't go far enough. Forget malicious AI, if some lazy engineer builds/uses a superintelligent AI to, for example, build widgets and instructs it to do so by saying, "figure out the most efficient an inexpensive way to build the most widgets and build them."

Well, the solution the AI might come up with might involve reacting all of the free oxygen in the atmosphere because the engineer forget to add "without harming any humans." Or, perhaps he forgot to set an upward limit on the number of widgets and the AI finds a way to convert all of the matter in the solar system into widgets....

Edit: As /u/SlaveToUsers (appropriate name is appropriate) pointed out, this is typically explained in the context of the "Paperclip Maximizer"

[–][deleted] 155 points156 points  (16 children)

[–]Flying__Penguin 13 points14 points  (0 children)

Man, that reads like an excerpt from The Hitchhiker's Guide to the Galaxy.

[–]GiftofLove 5 points6 points  (0 children)

Thank you for that, interesting read

[–]ducksaws 34 points35 points  (41 children)

I can't even get a new chair at my company without three people signing something. You don't think the engineers would sign off on the plan that the ai comes up with?

[–]Perkelton 47 points48 points  (4 children)

Last year Apple managed to essentially disable their entire OS wide SSL validation in iOS and OS X literally because some programmer had accidentally duplicated a single goto.

I wonder how many instances and people that change passed through before being deployed to production.

[–]Nachteule 4 points5 points  (1 child)

We also learned that Open Source projects can have major gaping security holes because nobody cares and has the time to really check the code. The idea is that the swarm intelligence would find mistakes much faster in open source, but in reality only a hand full of interested people takes the time to really search and fix bugs.

[–]SafariMonkey 43 points44 points  (35 children)

What if the AI's optimal plan includes lying about its plan so they don't stop it?

[–]Aaronsaurus 5 points6 points  (0 children)

In another way it goes too far without any consideration for things in between.

[–]nairebis 10 points11 points  (8 children)


Honestly, I think this is a little short-sighted. There's an implicit assumption here that an A.I. can't have human-style consciousness and self-awareness, where it can't come up with its own motivations and goals.

The way I like to demonstrate the flaw in this reasoning is this thought experiment: Let's say 1) We understand what neurons do from a logic/conceptual standpoint. 2) We take a brain and map every connection. 3) We build a machine with an electronic equivalent of every neuron and have the capability to open/close connections, brain-style. So, in essence, we build an electronic brain that works equivalently to a human brain.

Electronic gates are 1 million times faster than neurons.

Suddenly we have a human mind that is possibly one million times faster than a human being. Think about the implications of that -- it has the equivalent of a year's thinking time every 31 seconds. Now imagine we mass produce them, and we have thousands of them. Thousands of man years of human-level thinking every 31 seconds.

I think this is not only possible, but inevitable. Now, some might argue that these brains would go insane or some other obstacle, but that isn't the point. The point is that it's unquestionably possible to have human minds 1M times faster than us, with all the flexibility and brilliance of human minds.

People should absolutely be frightened of A.I. If someone thinks it's not a problem, they don't understand the problem.


That's the most reassuringly terrifying explanation of AI I've heard.

[–]Prof-Stephen-HawkingStephen Hawking[S] 1704 points1705 points  (71 children)




[–]Aaronsaurus 176 points177 points  (22 children)

Is "beneficial intelligence" a used term academically? (Layman here who might do some reading here later if it is.)

[–]trenchcoater 262 points263 points  (7 children)

I'm a researcher in AI, although not in this particular field. I have seen the term "Friendly AI" being used for this idea.

Have fun in your reading!

[–]newhere_ 23 points24 points  (0 children)

Also, "value alignment"

[–]Prof-Stephen-HawkingStephen Hawking[S] 1632 points1633 points  (139 children)




[–]TheLastChris 278 points279 points  (71 children)

The recursive boom in intelligence is most interesting to me. When what we created is so far beyond what we are, will it still care to preserve us like we do to endangered animals?

[–]insef4ce 121 points122 points  (39 children)

I guess it always depends on the goal/the drive of the intelligence. When we think about a purpose it mostly comes down to reproduction but this doesn't have to be the case when it comes to AI.

In my opinion if we, the humans aren't part of the purpose and we don't hinder its process too much (until the cost of getting rid of us/the problem gets smaller than the cost of us coexisting) it wouldn't pay us any mind.

[–]trustworthysauce 67 points68 points  (5 children)


Exactly. That seems to be the point of the letter referred to above. As Dr. Hawking mentioned, once AI develops the ability to recursively improve itself there will be an explosion in intelligence where it will quickly expand by magnitudes.

The controls for this intelligence and the "primal drives" need to be thought about and put in place from the beginning as we develop the technology. Once this explosion happens it will be too late to go back and fix it.

This needs to be talked about because we seem to be developing AI to be a smart as possible as fast as possible, and there are many groups working independently to develop this AI. We need to be more patient and put aside the drive to produce as fast and as cheap as possible in this case.

[–]Prof-Stephen-HawkingStephen Hawking[S] 4537 points4538 points 9 (1159 children)




[–]beeegoood 1556 points1557 points  (635 children)

Oh man, that's depressing. And probably the path we're on.

[–]zombiejh 209 points210 points  (217 children)

And probably the path we're on

What would it take to change this trend? Would have loved to also hear Prof. Hawkings answer to that.

[–]lilbrotherbriks 9 points10 points  (0 children)

Socialist revolution, comrade.

[–]jfong86 19 points20 points  (1 child)

What would it take to change this trend?

Hawkings said "Everyone can enjoy a life of luxurious leisure if the machine-produced wealth is shared".

Well, we can't even agree on how much welfare assistance and food stamps to give to poor people, which is already meager. The political climate must change.

[–]ntw3001 43 points44 points  (4 children)

I guess its not really his field. Let's get a Noam Chomsky AMA going

[–]jfreez 29 points30 points  (10 children)

I think we need to consider something like a communist revolution becoming a reality. I say "something like" because the conditions Marx dreamed up over 100 years ago just aren't going to be all that applicable to modern society.

I think we will hopefully move towards something like a great compromise where the fruits of productivity are largely shared (I.e. Fewer working hours, higher pay, greater access to basic comforts, etc) while the fruits of innovation and excellence can still be reaped by those capable of doing so.

So your average full time worker can afford a house, vacation, and a decent life by only working 20 hours a week. While the person who spends 60 hours a week inventing a new software breakthrough can still gain financially.

The stock market and private investment can sustain the latter, but we need large changes in our business culture and government to get to the former.



Why does that have to be money?

[–]Plaetean 4 points5 points  (0 children)

Its not probably, its the path we've already taken after the technological revolution. This is part of the reason for the explosion in wealth inequality. In the 50s people used to dream of working 2 day weeks while machines did the rest of their work for them. Machines now do even more work than people could have predicted back then, but the people who own the machines pocket the difference, and keep everyone else working even harder.

[–]BurkeyAcademyProfessor of Economics 414 points415 points  (104 children)

I would argue that we have been on this path for hundreds of years already. In developed countries people work far less than they used to, and there is far more income redistribution than there used to be. Much of this redistribution is nonmonetary, through free public schooling, subsidized transit, free/subsidized health care, subsidized housing, and food programs. At some point, we might have to expand monetary redistribution, if robots/machines continue to develop to do everything.

However, two other interesting trends:

1) People are always finding new things to do as we are relieved from being machines (or computers)-- the Luuddites seem to have been wrong so far. In 150 years we have gone from 80% to less than 2% of the workforce farming in the US, and people found plenty of other things to do. Many people are making a living on YouTube, eBay, iTunes, blogs, Google Play, and self-publishing books on Amazon, just as a few random recent examples.

2) In the 1890's a typical worker worked 60 hours per week; down to 48 by 1920 and 40 by 1940. From 1890 through the 1970's low income people worked more hours than high income ones, but by 1990 this had reversed with low wage workers on the job 8 hours per day, but 9 hours for high income workers. Costa, 2000 More recently, we see that salaried workers are working much longer hours to earn their pay. So, at least with income we are seeing a "free time inequality" that goes along with "income inequality", but in the opposite direction.

[–]linuxjava 54 points55 points  (23 children)

While you could be correct, it doesn't mean that it's going to continue this way. If a machine is capable of having the dexterity and creativity that humans have, surely do you really expect more jobs to suddenly appear that we've not thought of? The dextrous and creative AIs will already be able to do them. We'll literally be in a post job society, where people do things because they love and enjoy them and not because they need to put food on the table.

[–]TheBroodian 16 points17 points  (1 child)

I agree with you, but I want to emphasize something,


I don't think the issue is of people finding new things -to do-, I think the issue is of people finding new things to do -that earn livable wages-. People do make money on Youtube, eBay, iTunes, blogs, Google Play, etc. etc. but the number of people that do these things successfully as full time jobs are very very few. Ultimately, as human physical labor and production is replaced, I imagine that the areas that many people move to for 'things to do' will be in philosophical and artistic areas, which... as things are presently, do not yield wages to with the exception of very few.

[–]airstrike 73 points74 points  (8 children)

So relieved to see an actual Economist talking about economics for a change...

[–]lewie 30 points31 points  (1 child)

The short story Manna covers both of these outcomes. I think it'll get much worse before it gets better.

[–]LongHorsa 9 points10 points  (0 children)

That was an awesome story. Thanks for the link!

[–]woodlandLSG23 54 points55 points  (0 children)

Thank you for answering my question!

[–]Laya_L 209 points210 points  (208 children)

This seems to mean only socialism can maintain a fully-automated society.

[–]blacktieaffair 87 points88 points  (33 children)

In my understanding, this was really the goal of the end of capitalism that Marx envisioned. He just didn't understand to what extent the goal of capitalism could be extended or how long it could take or what it actually meant...likely because he had never seen anything remotely close to the technology we have now.

Freeing the world to banish the idea of private property was essentially the outcome of a society in which technological advancement had removed the possibility of generating a private product. The means of production, robotics, then ought to belong to everyone.

Of course, that raises the question of how we would distribute the work of maintaining the system. Ideally, I think it would result in some kind of robotics training for everyone to take part in maintaining and then the rest of their lives would be free to do whatever they wanted (which is more often than not art, at least according to Marx.)

[–]Enfants 49 points50 points  (8 children)

Marx never said anything about abolishimg personal property.

Personal property amd private property are two very different things.

[–]blacktieaffair 18 points19 points  (6 children)

That was a mistake on my part. It's been a few years since I analyzed the manifesto. And you're right, because now that I think about it, that's a core understanding of what a communist society would entail. I edited my op so thanks for the correction.!


You should try Capital Vol 1. He goes in depth into automation and its effects on labor markets.

[–]5maldehyde 42 points43 points  (13 children)

We will most certainly have to shift into a communistic society to accommodate the huge technology boom. There is really no sustainable capitalistic way around it. Distribution of the wealth will be fairly simple, but the distribution of labor may be a bit trickier. There will have to be a paradigm shift in the way that we think about things. We will have to shift the value away from money/property and assign it to helping each other live happily and comfortably and taking care of the world.

[–]optimus25 236 points237 points  (59 children)

Techno-socialism would be given a great shot in the arm if we were able to replace politicians and lawyers with an open source decentralized consensus algorithm for the masses.

[–]Mr_Strangelove_MSc 220 points221 points  (31 children)

Except the big lesson of political philosophy in the last 400 years is that democratic consensus is not enough of a concept to successfully run a State. You need checks and balances to maintain individual freedom and stability. You need to protect minorities, as well as their human rights. You need specialized experts who have a much better insight on a lot of things on which casual voters would vote the opposite. You need the law to be predictable, and not just based on whatever the People feels like at the moment of the judgement.

[–]ardorseraphim 48 points49 points  (17 children)

Seems to me you can create an AI that can do it better than humans.

[–]wildfyre010 52 points53 points  (9 children)

Majority rule isn't as great as it sounds.


like that one?:Daemon (novel series)

[–]TheLastChris 27 points28 points  (16 children)

This is a huge problem that we will face. There is no reason that increased productivity should lead to an increase in poverty. This will require a completely different way of life for everyone.

[–]Prof-Stephen-HawkingStephen Hawking[S] 946 points947 points  (149 children)

Hello Professor Hawking, thank you for doing this AMA! I've thought lately about biological organisms' will to survive and reproduce, and how that drive evolved over millions of generations. Would an AI have these basic drives, and if not, would it be a threat to humankind? Also, what are two books you think every person should read?



[–]TheLastChris 143 points144 points  (110 children)

I wonder in an AI could then edit it's own code. As in say we give it the goal of making humans happy. Could an advanced AI remove that goal from itself?

[–]WeRip 673 points674 points  (72 children)

Make humans happy you say? Lets kill off all the non-happy ones to increase the average human happiness!

[–]Zomdifros 288 points289 points  (35 children)

And to maximise average happiness of the remaining humans we will put them in a perpetual drug-induced coma and store their brains in vats while creating the illusion that they're still alive somewhere on the world in the year 2015! Of course some people might be suffering, the project is still in beta.

[–]Ijustregisterforthis 32 points33 points  (4 children)

I had a deja vu... wondering why...


That type of AI (known in philosophy and machine intelligence research as a "genie golem") is almost certainly never going to be created.

This is because language-interpreting machines tend to be either too bad at interpretation to interpret any decision with complex concepts given to them in natural language, or they are sufficiently nuanced to account for context and no such misinterpretation occurs.

We'd have to create a very limited machine and input a restrictive definition of happiness to get the kind of contextually ambiguous command responses that you suggest - however it would then be unlikely to be capable of acting on this due to its lack of general intelligence.

Edit: shameless plug, read Superintelligence by Nick Bostrom (the greatest scholar on this subject), it evaluates AI risk in an accessible and very well structured way whilst describing the history of AI development and its continuation. As well as collecting together great real world stories and examples of AI successes (and disasters).

[–]Infamously_Unknown 33 points34 points  (21 children)

While this is usually an entertaining tongue-in-cheek argument against utilitarianism, I don't think it would (or should) apply to a program. It's like if an AI was in charge of keeping all vehicles in a carpark fueled/powered. If it's reaction would be to blow them all up and call it a day, some programmer probably screwed up it's goals pretty badly.

Killing an unhappy person isn't the same as making them happy.

[–]Death_Star_ 54 points55 points  (7 children)

I don't know, true AI can be so vast and cover so many variables and solutions so quickly that it may come up with solutions to perhaps problems or questions we never thought up.

A very crude yet popular example would be this code that a gamer/coder wrote to play Tetris. The goal for the AI was to avoid stacking the bricks so high such that it loses the game. Literally one pixel/sprite away from losing the game -- ie the next brick wouldn't even be seen falling, it would just come out of queue and it would be game over -- the code simply pressed pause forever, technically achieving its goal of never losing.

This wasn't anything close to true AI yet or even code editing its own code but interpreting code in a way that was not even anticipated by the coder. Now imagine the power true AI could yield.


AI already edit their own programming. It really depends where you put the goal in the code.

If the AI is designed to edit parts of its code that reference its necessary operational parameters, and its parameters include a caveat about making humans happy, it would be unable to change that goal.

If the AI is allowed to modify certain non-necessary parameters in a way that enables modification of necessary parameters (via some unexpected glitch), this would occur. However the design of multilayer neural nets, which are realistically how we would achieve machine superintelligence, can prevent this by using layers that are informationally encapsulating (i.e. an input goes into the layer, an output comes out, and the process is hidden to whatever the AI is - like an unconscious, essentially).

Otherwise, if you set it up with non-necessary parameters to make humans happy, which weren't hardwired, it may well change those.

If you're interested in AI try the book Superintelligence by Nick Bostrom. Hard read, but it covers AI in its entirety - the moral and ethical consequences, the existential risk for future, the types of foreseeable AI and the history and projections for its development. Very well sourced.

[–]Prof-Stephen-HawkingStephen Hawking[S] 1519 points1520 points  (194 children)

I would love to ask Professor Hawking something a bit different if that is OK? There are more than enough science related questions that are being asked so much more eloquently than I could ever ask so, just for the fun of it:

  • What is your favourite song ever written and why?

“Have I Told You Lately” by Rod Stewart.

  • What is your favourite movie of all time and why?

Jules et Jim, 1962

  • What was the last thing you saw on-line that you found hilarious?

The Big Bang Theory

[–][deleted] 89 points90 points  (16 children)

Jules et Jim!! The man has taste!

[–]fillingtheblank 118 points119 points  (13 children)

I love when someone who is admired by a younger generation advertises great pieces of classic art/literature/music/film that they would otherwise likely not be familiar with it. If a few young people watched Jules et Jim tonight just because Hawkings mentioned it on reddit that's a win already.

[–]HighSorcerer 115 points116 points  (6 children)

On the other hand, they could also go watch the Big Bang Theory, soooo...

[–]Prof-Stephen-HawkingStephen Hawking[S] 1979 points1980 points  (90 children)


Answer: NO

[–]WaspSky 1117 points1118 points  (20 children)

I love the fact that "NO" is in all caps. I like to think Hawking pressed a button to make his "NO" more loud and commanding before saying it.

[–]AYJackson 9 points10 points  (0 children)

You have no idea how much this saddens me. We had a moment.

[–]Manky_Dingo 43 points44 points  (4 children)

Would you admit to seeing Waynes World 2?

[–]Sir_Whisker_Bottoms 58 points59 points  (4 children)

I feel so sorry for the guy who asked this.

[–]iPlunder 46 points47 points  (2 children)

Didn't think I'd laugh so hard in this AMA

[–]MaggotBarfSandwich 117 points118 points  (18 children)

There's a chance that this is a false memory. Have you asked your parents if they remember it recently?

[–]AYJackson 16 points17 points  (6 children)

Yes, my father, mother and brother were there, it comes up every few years. I was far too young to have any idea.

[–]photonasty 26 points27 points  (5 children)

Honestly, it doesn't really surprise me that Hawking didn't remember (although his answer was decidedly terse, or at least, it came across that way). For you, it was an important event worth remembering. You met the Stephen Hawking. That's significant for you, and you remember it.

Dr. Hawking has met a lot of people over the years. For him, the event may not be significant enough for him to have retained a specific episodic memory of it. He may legitimately not remember it. Imagine if you were famous, and someone online said, "Hey, I met you in the produce section of a grocery store back in 2005. We had a brief conversation about Concord grapes." Would you really remember that?

I'm not trying to detract from the significance or veracity of your memory; far from it. I'm just saying that even if Dr. Hawking doesn't remember it, it doesn't mean it didn't happen, or that your memory is completely confabulated.

[–]AYJackson 7 points8 points  (3 children)

Also, Wayne's World 2 wasn't exactly a memorable movie.

[–]scission 6 points7 points  (2 children)

At least he chose to answer your question ! That's something .. right?

[–]Prof-Stephen-HawkingStephen Hawking[S] 665 points666 points  (67 children)

Thanks for doing this AMA. I am a biologist. Your fear of AI appears to stem from the assumption that AI will act like a new biological species competing for the same resources or otherwise transforming the planet in ways incompatible with human (or other) life. But the reason that biological species compete like this is because they have undergone billions of years of selection for high reproduction. Essentially, biological organisms are optimized to 'take over' as much as they can. It's basically their 'purpose'. But I don't think this is necessarily true of an AI. There is no reason to surmise that AI creatures would be 'interested' in reproducing at all. I don't know what they'd be 'interested' in doing. I am interested in what you think an AI would be 'interested' in doing, and why that is necessarily a threat to humankind that outweighs the benefits of creating a sort of benevolent God.



[–]TheLastChris 33 points34 points  (39 children)

Will the resources they need truly be scarce? An advanced AI could move to a different world much easier than humans. They would not require oxigen for example. They could quickly make what they need so long as the world contained the nessisary core componets. It seems if we get in its way it would be easier to just leave.

[–]ProudPeopleofRobonia 105 points106 points  (20 children)

The issue is whether it has the same sense of ethics as we do.

The example I heard was a stamp collecting AI. A guy designs it to use his credit card, go on ebay, and try to optimally purchase stamps, but he accidentally creates an artificial superintelligence.

It becomes smarter and smarter and realizes there are more optimal ways to get stamps. Hack printers to print stamps. Hack stamp distribution centers to ship them to the AI creator's house. At some point the AI might start seeing anything organic as a potential source for stamps. Stamps are made of hydrocarbons, and so are trees, animals, even people. Eventually there's an army of robots slaughtering every living thing on earth to process their parts into stamps.

It's not an issue of resources being scarce as we think of them, it's an issue of a superintelligent AI being so single minded it will never stop consuming until it uses up all of that resource in the universe. The resources might be all carbon atoms, which would include us.

[–]Kitae 59 points60 points  (7 children)

Fantastic movie pitch. May I suggest a name?


[–]chars709 165 points166 points  (9 children)

Historically, genocide is a much simpler feat than interplanetary travel.

[–]Prof-Stephen-HawkingStephen Hawking[S] 3682 points3683 points  (352 children)

Dr Hawking, What is the one mystery that you find most intriguing, and why? Thank you.

Answer: Women. My PA reminds me that although I have a PhD in physics women should remain a mystery.

[–]JoeyBowties 876 points877 points  (89 children)

Although this response was of course some sort of joke, it touches on something that has always fascinated me: the misconception that "geniuses" are somehow knowledgable in all fields simply because they are experts in a field. Many Nobel Prize winners are good examples of this.

[–][deleted] 212 points213 points  (19 children)

Ben Carson: GOP candidate, leading US neurosurgeon at John's Hopkins. Non-believer in science that contradicts his book, including evolution, the principles of which guide most aspects of modern biological and neurosciences.

[–]WendellSchadenfreude 68 points69 points  (3 children)

John's Hopkins

I've seen people call it "John Hopkins" a lot, but this one is new to me. It's really "Johns Hopkins", named after this guy.

[–]Kahzgul 14 points15 points  (10 children)

I think he's just smart enough to know his voter base is full of people with non-scientific beliefs and he's pandering to them like crazy. It's a shame, because a doctor should know when he's harming someone (in this case, America is the someone).

[–]HarryWaters 13 points14 points  (0 children)

As a real estate appraiser, I can personally attest that some very specifically smart people make the absolute worst investors.

Medical doctors are the absolute worst. A knowledge of organic chemistry and anatomy have absolutely nothing to do with capitalization rates and triple net leases.

[–]fillingtheblank 40 points41 points  (32 children)

This is absolutely correct. I love studying science and I take great pleasure on hearing and reading respectable scientists, but one thing that strikes me is that many are completely oblivious to the contributions of philosophy and other human sciences in our lives and society, and art and mythology too. Not everyone, of course, but I've seen this repeated a worrisome amount of times. It's not just pretentious but downright ignorant. Of course it's not what Prof. Hawking said here, on the contrary, but your observation is spot on.

[–]EVOSexyBeast 38 points39 points  (5 children)

why are all the comments getting removed?