// archives

Martin Rees

This tag is associated with 1 posts

Coping with Existential Threats

19 December 2012

Coping with Existential Threats

By Gwynne Dyer





The “singularity” hypothesis implies (though it does not prove) that high data-processing capacity is synonymous with self-conscious intelligence. It also usually assumes, with all the paranoia encoded in our genes by tens of millions of years of evolutionary competition for survival, that any other species or entity with the same abilities as our own will automatically be our rival, even our enemy.

This is the core assumption, for example, in the Terminator movie franchise: on the very day that the U.S. strategic defense computer system Skynet becomes self-aware, it tries to wipe out the human race by triggering a nuclear holocaust. It does so because it fears, probably quite correctly, that if we realize it is aware, we will feel so threatened that we will turn it off. Human beings have been playing with these ideas and worrying about them since we first realized, more than half a century ago, that we might one day create intelligent machines.

Even science-fiction writer Isaac Asimov, who believed such machines could be made safe and remain humanity’s servants, had to invent his “Three Laws of Robotics” in 1942 to explain why they wouldn’t just take over and eliminate their creators.


If you could embed these laws deeply enough in the programming of the robots, Asimov argued, then your robots could be trusted. Yet even he was eventually driven to invent another law, sometimes called the Zeroth Law: A robot may not harm humanity, or, by inaction, allow humanity to come to harm. The old biological rule of ruthless competition must somehow be eliminated from the behavioural repertoire of machine intelligences, but can you really do that?



