Intelligence explosion

From Lesswrongwiki
Revision as of 05:42, 9 November 2009 by Vladimir Nesov (talk | contribs) (separated core claim from the example, integrated an excerpt from "three schools" article.)
Jump to: navigation, search


Intelligence explosion is the idea of a positive feedback loop in which an intelligence is making itself smarter, thus getting better at making itself even smarter. A strong version of this idea suggests that once the positive feedback starts to play a role, it will lead to a dramatic leap in capability very quickly. Depending on the mechanism underlying the feedback loop, the transition may take years or hours (intelligence explosion doesn't necessarily mean recursively self-improving AIs, other options include brain-computer interfaces or even genetic engineering). At some point technological progress drops into the characteristic timescale of transistors (or super-transistors) rather than human neurons, and the ascent rapidly surges upward and creates superintelligence (minds orders of magnitude more powerful than human) before it hits physical limits.

To pick just one possibility for illustrative purposes, an AI undergoing a hard takeoff might invent molecular nanotechnology, use the internet to gain physical manipulators, deploy the nanotech, use it to expand its computational capabilities and reach singleton status within a matter of weeks. Recursive self-improvement would be a genuinely new phenomenon on Earth. Humans study, and human societies accumulate new technologies and ways of doing things, but we don't directly redesign our brains. A cleanly-designed AI could redesign itself, and reap the benefits of recursive self-improvement.

Blog posts

Posts by Eliezer Yudkowsky:

See also