long blog post
https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html
How (and Why) SpaceX Will Colonize Mars — Wait But Why
2015-08-16
Tim Urban
<div><div><p><i>This is Part 3 of a four-part series on Elon Musk’s companies. For an explanation of why this series is happening and how Musk is involved, </i><i><a href="https://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-man.html">start with Part 1</a>.</i></p>
<p><strong>Pre-Post Note: </strong>I started working on this post ten weeks ago. When I started, I never intended for it to become such an ordeal. But like the <a href="https://waitbutwhy.com/2015/06/how-tesla-will-change-your-life.html">Tesla post</a>, I decided as I researched that this was A) a supremely important topic that will only become more important in the years to come, and B) something most people don’t know nearly enough about. My weeks of research and discussions with Musk and others built me an in-depth, tree-trunk understanding of what’s happening in what I’m calling The Story of Humans and Space—one that has totally reframed my mental picture of the future (yet again). And as I planned out what to include in the post, I wanted to make sure every Wait But Why reader ended up with the same foundation moving forward—because with everything that’s coming, we’re gonna need it. So like the Tesla post, this post became a full situation. Even the <a href="https://waitbutwhy.com/2015/07/the-spacex-post-progress-meter.html">progress updates</a> leading up to its publication became a full situation.</p>
<p>Thanks for your patience. I know you’d prefer this not to be a site that updates every two months, and I would too. The Tesla and SpaceX posts were special cases, and you can expect a return to more normal-length WBW posts now that they’re done.</p>
<p>About the post itself: There are three main parts. Part 1 provides the context and background, Part 2 explores the “Why” part of colonizing Mars, and Part 3 digs into the “How.” To make reading this post as accessible as possible, it’s broken into five pages, each about the length of a normal WBW post, and you can jump to any part of the post easily by clicking the links in the Table of Contents below. We’re also trying two new things, both coming in the next couple days:</p>
<p><strong>1) PDF and ebook options: </strong>We made a fancy PDF of this post for printing and offline viewing (see a preview <a href="https://www.waitbutwhy.com/wp-content/uploads/2015/11/How-and-Why-SpaceX-Will-Colonize-Mars-preview.pdf">here</a>), and an ebook containing the whole four-part Elon Musk series:</p>
<p><a href="https://gum.co/wbw-musk"></a><a href="http://amzn.to/1RkVeEm"></a></p>
<p><strong>2) <a href="https://waitbutwhy.com/2015/09/spacex-audio-stuff.html">An audio version</a></strong><strong>. </strong>You can find an unabridged audio version of the post, read by me, as well as a discussion about the post between Andrew and me <a href="https://waitbutwhy.com/2015/09/spacex-audio-stuff.html">here</a>.</p>
<p>___________</p>
<p>Contents </p>
<p><strong><a href="#part1">Part 1: The Story of Humans and Space</a></strong></p>
<p><strong><a href="https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html/2#part2">Part 2: Musk’s Mission</a></strong></p>
<p><strong><a href="https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html/3#part3">Part 3: How to Colonize Mars</a></strong><strong><br />
</strong>→ <a href="https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html/3#phase1">Phase 1: Figure out how to put things into space</a><br />
<strong>→ </strong><a href="https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html/4#phase2">Phase 2: Revolutionize the cost of space travel</a><br />
<strong>→ </strong><a href="https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html/5#phase3">Phase 3: Colonize Mars</a></p>
<p><strong><a href="https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html/5#future">A SpaceX Future</a></strong></p>
<hr />
<p><strong><br />
2365 AD, Ganymede</strong></p>
<p><em>One more day until departure.</em> It was so surreal to picture actually <em>being </em>there that she still didn’t really believe it would happen. All those things she had always heard about—buildings that were constructed hundreds of years before the first human set foot on Ganymede; animals the size of a house; oceans the size of her whole world; tropical beaches; the famous blue sky; the giant sun that’s so close it can burn your skin; and the weirdest part—<em>no Jupiter hovering overhead</em>. Having seen it all in so many movies, she felt like she was going to visit a legendary movie set. It was too much to think about all at once. For now, she just had to focus on making sure she had everything she needed and saying goodbye to everyone—it would be a long time before she would see them again…</p>
<p>___________</p>
<p><strong><a></a>Part 1: The Story of Humans and Space</strong></p>
<p>About six million years ago, a very important female great ape had two children. One of her children would go on to become the common ancestor of all chimpanzees. The other would give birth to a line that would one day include the entire human race. While the descendants of her first child would end up being pretty normal and monkey-ish, as time passed, strange things began to happen with the lineage of the other.<a href="#footnote-1-3902">1</a><a href="#footnote2-1-3902">1</a> <strong><sup>← click these</sup></strong></p>
<p>We’re not quite sure why, but over the next six million years, our ancestral line started to do something no creatures on Earth had ever done before—they woke up.</p>
<p>It happened slowly and gradually through the thousands of generations the same way your brain slowly comes to in the first few seconds after you rouse from sleep. But as the clarity increased, our ancestors started to look around and, for the very first time, <em>wonder</em>.</p>
<p>Emerging from a 3.6-billion-year dream, life on Earth had its first questions.</p>
<p><em>What is this big room we’re in, and who put us here? What is that bright yellow circle on the ceiling and where does it go every night? Where does the ocean end and what happens when you get there? Where are all the dead people now that they’re not here anymore? </em></p>
<p>We had discovered our species’ great mystery novel—<em>Where Are We?</em>—and we wanted to learn how to read it.</p>
<p>As the light of human consciousness grew brighter and brighter, we began to arrive at answers that seemed to make sense. Maybe we were on top of a floating disk, and maybe that disk was on top of a huge turtle. Maybe the pinpricks of light above us at night are a glimpse into what lies beyond this big room—and maybe that’s where we go when we die. Maybe if we can find the place where the ceiling meets the floor, we can poke our heads through and see all the super fun stuff on the other side.<a href="#footnote2-2-3902">2</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Flammarion1.jpg"></a></p>
<p>Around 10,000 years ago, isolated tribes of humans began to merge together and form the first cities. In larger communities, people were able to talk to each other about this mystery novel we had found, comparing notes across tribes and through the generations. As the techniques for learning became more sophisticated and the clues piled up, new discoveries surfaced.</p>
<p>The world was apparently a ball, not a disk. Which meant that the ceiling was actually a larger sphere surrounding us. The sizes of the other objects floating out there in the sphere with us, and the distances between them, were vaster than we had ever imagined. And then, something upsetting:</p>
<p>The sun wasn’t revolving around us. We were revolving around the sun.</p>
<p>This was a <em>super </em>unwarm, unfuzzy discovery. Why the <em>hell</em> weren’t we in the center of things? What did that mean?</p>
<p><em>Where are we?</em></p>
<p>The sphere was already unpleasantly big—if we weren’t in the center of it, were we just on a random ball inside of it, kind of for no apparent reason? Could this really be what was happening?</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Tiny-Earth.png"></a></p>
<p>Scary.</p>
<p>Then things got worse.</p>
<p>It seemed that the pinpricks of light on the edge of the sphere weren’t what we thought they were—they were other suns like ours. And they were out there floating just like our sun—which means <em>we weren’t inside of a sphere at all</em>. Not only was our planet not the center of things, even our <em>sun </em>was just a random dude out there, in the middle of nowhere, surrounded by nothingness.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Tiny-Sun.png"></a></p>
<p><em>Scary.</em></p>
<p>Our sun turned out to be a little piece of something much bigger. A beautiful, vast cloud of billions of suns. The everything of everything.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Milky-Way.jpg"></a></p>
<p>At least we had that. Until we realized that it wasn’t everything, it was this:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Milky-Way-small.jpg"></a></p>
<p>Darkness.</p>
<p>The better our tools and understanding became, the more we could zoom out, and the more we zoomed out, the more things sucked. We were deciphering the pages of <em>Where Are We?</em> at our own peril, and we had deciphered our way right into the knowledge that we’re unbelievably alone, living on a lonely island inside a lonely island inside a lonely island, buried in layers of isolation, with no one to talk to.</p>
<p>That’s our situation.</p>
<p>In the most recent 1% of our species’ short existence, we have become the first life on Earth to know about the Situation—and we’ve been having a collective existential crisis ever since.</p>
<p>You really can’t blame us. Imagine not realizing that the universe is a thing and <em>then realizing the universe is a thing</em>. It’s a <em>lot </em>to take in.</p>
<p>Most of us handle it by living in a pleasant delusion, pretending that the only place we live is in an endless land of colors and warmth. We’re like this guy, who’s doing <a href="https://www.youtube.com/watch?v=6bWyhj7siEY&amp;feature=youtu.be&amp;t=13s">everything he possibly can</a> to ignore the Situation:<a href="#footnote2-3-3902">3</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/zipgif1.gif"></a></p>
<p>And our best friend for this activity? The clear blue sky. The blue sky seems like it was <em>invented </em>to help humans pretend the Situation doesn’t exist, serving as the perfect whimsical backdrop to shield us from reality.</p>
<p>Then nighttime happens, and there’s the Situation, staring us right in the face.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Stars.jpg"></a></p>
<p><i>Oh yeah…</i></p>
<p>This la-di-da → oh yeah… → la-di-da → oh yeah… merry-go-round of psychosis was, for most of recent history, the extent of our relationship with the Situation.</p>
<p>But in the last 60 years, that relationship has vaulted to a whole new level. During World War II, missile technology leapt forward,<a href="#footnote-2-3902">2</a> and for the first time, a new, mind-blowing concept was possible—</p>
<p><em>Space travel.</em></p>
<p>For thousands of years, The Story of Humans and Space had been the story of staring out and wondering. The possibility of people leaving our Earth island and <em>venturing out</em> <em>into space</em> burst open the human spirit of adventure.</p>
<p>I imagine a similar feeling in the people of the 15th century, during the Age of Discovery, when we were working our way through the <a href="https://waitbutwhy.com/wp-content/uploads/2020/07/Cantino_planisphere_1502@0.5x-scaled.jpg">world map chapter</a> of <em>Where Are We?</em> and the notion of cross-ocean voyages dazzled people’s imaginations. If you asked a child in 1495 what they wanted to be when they grew up, “an ocean explorer” would probably have been a common response.</p>
<p>In 1970, if you asked a child the same question, the answer would be, “an astronaut”—i.e. <em>a Situation explorer</em>.</p>
<p>WWII advanced the possibility of human space travel, but it was in late 1957, when the Soviets launched the first man-made object into orbit, the adorable <a href="http://www.bisbos.com/images_spacecraft/sputnik_1_800.jpg">Sputnik 1</a>, that space travel became <em>the </em>defining quest of the world’s great powers.</p>
<p>At the time, the Cold War was in full throttle, and the US and Soviets had their measuring sticks out for an internationally-televised penis-measuring contest. With the successful launch of Sputnik, the Soviet penis bolted out by a few centimeters, horrifying the Americans.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Contest1.png"></a></p>
<p>To the Soviets, putting a satellite into space before the US was proof that Soviet technology was superior to American technology, which in turn was put forward as proof, for all the world to see, that communism was a system superior to capitalism.</p>
<p>Eight months later, NASA was born.</p>
<p>The Space Race had begun, and NASA’s first order of business would be to get a man into space, and then a man into full orbit, preferably both before the Soviets. The US was not to be shown up again.</p>
<p>In 1959, NASA launched Project Mercury to carry out the mission. They were on the verge of success when in April of 1961, the Soviets launched Yuri Gagarin into a full orbit around the Earth, making the first human in space <em>and</em> in orbit a Soviet.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Contest2.png"></a></p>
<p>It was time for drastic measures. John F. Kennedy’s advisors <a href="http://www.hq.nasa.gov/office/pao/History/Apollomon/apollo2.pdf">told him</a> that the Soviets had too big a lead for the US to beat them at any near-term achievements—but that the prospect of a manned <em>moon</em> landing was far enough in the future that the US had a fighting chance to get <em>there</em> first. So Kennedy gave his famous “we choose to go to the moon, not because it is easy, but because it is hahhd” <a href="https://www.youtube.com/watch?v=g25G1M4EXrQ&amp;feature=youtu.be&amp;t=1m32s">speech</a>, and directed an outrageous amount of funding at the mission ($20 billion, or $205 billion in today’s dollars).</p>
<p>The result was Project Apollo. Apollo’s mission was to land an American on the moon—and to do it <em>first</em>. The Soviets answered with Soyuz, their own moon program, and the race was on.</p>
<p>As the early phases of Apollo started coming together, Project Mercury finally hit its stride. Just a month after Yuri Gagarin became the first man in space, American astronaut Alan Shepard became the second man in space, completing a <a href="https://upload.wikimedia.org/wikipedia/commons/3/36/Freedom_7_Diagram.jpg">little arc</a> that didn’t put him in full orbit but allowed him to give space a high-five at the top of the arc. A few months later, in February of 1962, John Glenn became the first American to orbit the Earth.</p>
<p>The next seven years saw 22 US and Soviet manned launches as the superpowers honed their skills and technology. By late 1968, the furiously-sprinting US had more total launches under their belt (17) than the Soviets (10), and together, the two nations had mastered what we call Low Earth Orbit (LEO).</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/LEO.jpg"></a></p>
<p>But LEO hadn’t really excited anyone since the early ’60s. Both powers had their sights firmly set on the moon. The Apollo program was making quick leaps, and in December of 1968, the US became the first nation to soar outside of LEO. Apollo 8 made it all the way to the moon’s orbit and circled around 10 times before returning home safely. The crew, which included James Lovell (who a few months later played the role of Tom Hanks on the Apollo 13 mission), shattered the human altitude record and became the first people to see the moon up close, the first to see the “dark” side of the moon, and the first to see the Earth as a whole planet, snapping this iconic photo:<a href="#footnote2-4-3902">4</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/06/1280px-NASA-Apollo8-Dec24-Earthrise.jpg"></a></p>
<p>Upon return, the crew became America’s most celebrated heroes—which I hope they enjoyed for eight months. Three Apollo missions later, in July of 1969, Apollo 11 made Americans Neil Armstrong<a href="#footnote-3-3902">3</a> and Buzz Aldrin the first humans on the moon, and Armstrong took this famous photo of Aldrin looking all puffy:<a href="#footnote2-5-3902">5</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/06/Aldrin_Apollo_11.jpg"></a></p>
<p>It’s hard to fully emphasize what a big deal this was. Ever since life on Earth began 3.6 billion years ago, no earthly creature had set foot on any celestial body other than the Earth. Suddenly, there are Armstrong and Aldrin, bouncing around <em>another </em>sphere, looking up in the sky where the moon is supposed to be and seeing the Earth instead. Insane.</p>
<p>Project Apollo proved to be a smashing success. Not only did Apollo get a man on the moon before the Soviets, the program sent 10 more men to the moon over the next 3.5 years on five other Apollo missions. There were six successful moon trips in seven tries, with the famous exception being Apollo 13, which was safely aborted after an explosion in the oxygen tank.<a href="#footnote-4-3902">4</a>
</p><p>The Soviet Soyuz program kept running into technical problems, and it never ended up putting someone on the moon.</p>
<p>The final Apollo moonwalk took place in late 1972. In only one decade, we had conquered nearby space, and progress was accelerating. If at that time you had asked any American, or any other human, what the coming decades of space travel would bring, they’d have made big, bold predictions. Many more people on the moon, a permanent moon base, people on Mars, and beyond.</p>
<p>So you can only imagine how surprised they’d be if you told them in 1972, after just watching 12 humans walk on the moon, that 43 years later, in the impossibly futuristic-sounding year 2015, the number of people to set foot on the moon <em>would still be 12</em>. Or that after leaving Low Earth Orbit in the dust years earlier and using it now as our pre-moon trip parking lot, 2015 would roll around and LEO would be the farthest out humans would ever go.</p>
<p>1972 people would be blown away by our smart phones and our internet, but they’d be just as shocked that we gave up on pushing our boundaries in space.</p>
<p>So what happened? After such a wildly exciting decade of human space adventure, why did we just <em>stop?</em></p>
<p>Well, like we found in the <a href="https://waitbutwhy.com/2015/06/how-tesla-will-change-your-life.html">Tesla post</a>, “Why did we stop?” is the wrong question. Instead, we should ask:</p>
<p><em>Why were we ever adventurous about sending humans into space in the first place?</em></p>
<p>Space travel is unbelievably expensive. National budgets are incredibly tight. The fact is, it’s kind of surprising that a nation <em>ever</em> ponied up a sizable chunk of its budget for the sake of adventure and inspiration and pushing our boundaries.</p>
<p>And that’s actually because no nation <em>did </em>blow their budget for the sake of adventure and inspiration and pushing our boundaries—two nations blew their budgets because of a <em>penis</em>–<em>length</em> contest. In the face of international embarrassment at a time when everyone was trying to figure out whose economic system was better, the US government agreed to drop the usual rules for a few years to pour whatever resources were necessary on the problem to make sure they won that argument—</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Early-budget.png"></a></p>
<p>And once they won it, the contest was over and so were the special rules. And the US went back to spending money like a normal person.<a href="#footnote2-6-3902">6</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Full-budget.png"></a></p>
<p>Instead of continuing to push the limits at all costs, the US and the Soviets got a grip, put their pants back on, shook hands, and started working together like adults on far more practical projects, like setting up a joint space station in LEO.</p>
<p>In the four decades since then, the Story of Humans and Space has again become confined to Earth, where we find ourselves with two primary reasons to interact with space (Note: the next whole chunk of the post is a slight diversion for an overview on satellites, space probes, and space telescopes. If that doesn’t excite you, I won’t be hurt if you skip down to the International Space Station section):</p>
<p><strong>1) Support for Earth Industries</strong></p>
<p>The first and primary reason humans have interacted with space since the Apollo program isn’t about human interest in space. It’s about using space for practical purposes in support of industries on Earth—mostly in the form of satellites. The bulk of today’s rocket launches into space are simply putting things into LEO whose purpose is to look back down at Earth, not to the great expanses in the other direction.</p>
<p>Here’s a little satellite overview:</p>
<div>
<p><strong>Satellites Blue Box</strong></p>
<p>We don’t think about them that often, but above us are hundreds of flying robots that play a large part in our lives on Earth. In 1957, lonely Sputnik circled the Earth by itself, but today, the worlds of communication, weather forecasting, television, navigation, and aerial photography all rely heavily on satellites, as do many national militaries and government intelligence agencies.</p>
<p>The total market for satellite manufacturing, the launches that carry them to space, and related equipment and services has ballooned from $60 billion in 2004 to over $200 billion in 2015. Satellite industry revenue today makes up only 4% of the global telecommunications industry but accounts for over 60% of space industry revenue.<a href="#footnote2-7-3902">7</a>
</p><p>Here’s how the world’s satellites break down by role (in 2013):<a href="#footnote2-8-3902">8</a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/SatFunction1.png"></a></p>
<p>Of the 1,265 active satellites in orbit at the beginning of 2015, the US owns by far <a href="http://www.statista.com/statistics/264472/number-of-satellites-in-orbit-by-operating-country/">the largest number</a> at 528—over 40% of the total—but over 50 countries own at least one orbiting satellite.</p>
<p>As for <em>where </em>all of these satellites are, most of them fall into two distinct “layers” of space:</p>
<p>About two-thirds of active satellites are in Low Earth Orbit. LEO starts up at 99 miles (160 km) above the Earth, the lowest altitude at which an object can orbit without atmospheric drag messing things up. The top of LEO is 1,240 miles (2,000 km) up. Typically, the lowest satellites are at around 220 miles (350 km) up or higher.</p>
<p>Most of the rest (about one-third) of the satellites are much farther out, in a place called geostationary orbit (GEO). It’s right at 22,236 miles (35,786 km) above the Earth, and it’s called geostationary because something orbiting in it rotates at the exact speed that the Earth turns, making its position in the sky stationary relative to a point on the Earth. It’ll seem to be motionless to an observer on the ground.<a href="#footnote2-9-3902">9</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/Geostationaryjava3D.gif"></a></p>
<p>GEO is ideal for something like a TV satellite because a dish on the Earth can aim at the same fixed spot all the time.</p>
<p>A small percentage of other satellites are in medium Earth orbit (MEO), which is everything in between LEO and GEO. One notable resident of MEO is the GPS system that most Americans, and people from many other countries, use every day. I never realized that the entire GPS system, a US Department of Defense project that went live in 1995, only uses 32 satellites total. And until 2012, the number was only 24—six orbits, each with four satellites. But you can see in the GIF below that even with 24, a given point on the Earth can be seen by at least six of the satellites at any given time, and usually it’s nine or higher (in the GIF, the blue dot on the Earth is a hypothetical person on the ground, and whichever satellites can see him at a given time are blue, with the green lines showing their line of sight to the person):<a href="#footnote2-10-3902">10</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/ConstellationGPS.gif"></a></p>
<p>This is why your phone’s map can still show your location even when you’re somewhere with no cellular service—because it has nothing to do with cellular service. The system is also set up to be redundant—only four satellites need to simultaneously see you in order for the system to pinpoint your location. GPS satellites have an orbital period of about 12 hours, making two full rotations of the Earth each day.<a href="#footnote-5-3902">5</a>
</p><p>You can see satellite locations using <a href="http://www.gearthblog.com/satellites">Google Earth</a> (here’s a <a href="https://www.youtube.com/watch?v=ydbbd-4oEds">cool video</a> of Google Earth showing the satellites).</p>
<div>
<p><b>Space Debris Bluer Box</b></p>
<p>There’s a big problem happening in the world of satellites. In addition to the 1,265 active satellites up in orbit, there are thousands more inactive satellites, as well as a bunch of spent rockets from previous missions. And once in a while, one of them explodes, or two of them collide, creating a ton of tiny fragments called space debris. The number of objects in space has risen quickly over recent decades, as a GIF<a href="#footnote-6-3902">6</a> made by the ESA shows (with exaggerated-sized objects relative to the Earth’s size):<a href="#footnote2-11-3902">11</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/2599e0ad79e09729f1ef751b422dccfb.gif"></a></p>
<p>The majority of satellites and debris are bunched around the Earth in LEO, and the outer ring of objects is what’s located in GEO.</p>
<p>Earth space agencies track about 17,000 objects in space, only 7% of which are active satellites. Here’s a map showing every known object in space today.<a href="#footnote2-12-3902">12</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/boZhtHC-11.jpg"></a></p>
<p>But the crazy thing is they only track the large objects, and that’s what we’re seeing in that image. Estimates for the number of smaller debris objects (1 – 10 cm) range from 150,000 to 500,000, and there are over a million total pieces of debris larger than 2 mm.<a href="#footnote2-13-3902">13</a>
</p><p>The issue is that at the incredible speeds at which space objects move (most LEO objects zip along at over 17,000 mph), a collision with even a tiny object can cause devastating damage to an active satellite or spacecraft. An object of only 1 cm at those speeds will cause the same damage in a collision as a small <em>hand grenade</em>.<a href="#footnote-7-3902">7</a><a href="#footnote2-14-3902">14</a>
</p><p>Over a third of all space debris originated from just two events: China’s 2007 anti-satellite test, when China shat on the world’s face by intentionally blowing up one of its own satellites, creating <em>3,000</em> new pieces of debris large enough to be trackable, and a 2009 collision between two satellites that exploded into 2,000 debris chunks.<a href="#footnote2-15-3902">15</a> Each collision increases the amount of debris, which in turn increases the likelihood of more collisions, and there’s danger of a domino effect situation, which scientists call the <a href="https://www.youtube.com/watch?v=xgGm5odlIh4">Kessler Syndrome</a>. A bunch of parties are proposing ways to mitigate the amount of debris in LEO—everything from <a href="http://www.popsci.com/technology/article/2013-04/scientists-are-building-space-harpoon-take-out-orbiting-junk">harpooning</a> the debris to <a href="http://www.space.com/29271-space-station-laser-cannon-orbital-debris.html">laser blasting</a> it to <a href="http://www.popsci.com/technology/article/2012-10/boeings-new-space-junk-scheme-clears-debris-cloud-ballistic-gas">intercepting</a> it with a cloud of gas.</p>
</div>
<p>Here’s a chart that sums up each nation’s “space footprint,” showing the quantity of active satellites, inactive satellites, and space debris caused by each country:<a href="#footnote2-16-3902">16</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/18moky6ntooezjpg-2.jpg"></a></p>
</div>
<p>There are a few other space activities in the “Support for Earth Industries” category of human/space interaction—like <a href="http://www.planetaryresources.com/">space mining</a>, <a href="http://www.celestis.com/">space burial</a>, and <a href="http://www.virgingalactic.com/">space tourism</a>—but at least for now, satellites account for almost the entire category.</p>
<p><strong>2) Looking and Learning</strong></p>
<p>The second reason humans have interacted with space in the last four decades proves that while we may have stopped sending people into The Situation, we never lost our hunger to learn about what’s out there. As society moved on from space and turned its attention elsewhere, astronomers have kept busy at work deciphering their way through page after page of the old mystery novel, <em>Where Are We?</em> <em> </em></p>
<p>Astronomers learn best with their eyes, and a side effect of the Space Race was the development of far better technology for seeing<em> </em>what’s out there. There are two high-tech ways modern astronomers see things:</p>
<p><strong>Looking and Learning Tool #1: Sending probes around the Solar System</strong></p>
<p>Basically, scientists fire a fancy robot toward some distant planet, moon, or asteroid, and the robot spends months or years flying through space, bored, until it finally arrives. Then, depending on the plan, it either just flies by the object, taking some pictures on the way, orbits the object to get more detailed information, or lands on the object for a full inspection. Everything it learns, it sends back to us, and one day, when its job is done, we either kill the probe by crashing it into the object or let it just fly out into deep space to be depressed.</p>
<p>I often use myself as a litmus test for what the public probably knows about or doesn’t know about. As I’ve mentioned before on this blog, I’ve been seriously dating astronomy ever since I was three years old—so if I don’t know something going on in the world of space, I assume that most people don’t. And when it comes to space probes, I’ve felt pretty disoriented. Are there 200 of them flying around out there? 50? 9? Why are they out there, who sent them, and what are they doing? All I’d know is that sometimes there would be a random story about some probe sending back stunning pictures—I’d open the cnn.com gallery, click through them, be thrilled for a second, send the link to the three friends of mine who are also dating astronomy, and then try to close the page but instead see some trashy CNN clickbait headline on the side of the page, click that, and ruin my life for the next three hateful hours. That’s my relationship with humanity’s space probes.</p>
<p>But in researching this post, I quickly realized there’s not that much to know, and it doesn’t take too big an effort to get fully oriented. Here are what I consider the eight key space robots to know about right now:<a href="#footnote2-17-3902">17</a>
</p><p><strong>1) <a href="https://www.nasa.gov/mission_pages/newhorizons/main/index.html">New Horizons</a> </strong>(Pluto, NASA)</p>
<p><a href="http://www.henrywurst.com/higher-resolutions-images-work-best-for-print-hello-pluto/"></a></p>
<p>New Horizons goes first because its big moment just happened. Launched in 2006 on a decade-long trip to Pluto (sped up on its way by a Jupiter fly-by in 2007 that gravity-zinged it to a much faster speed), New Horizons <em>finally</em> reached Pluto on July 14th, 2015. It didn’t land on Pluto, but it flew very near to it and showed us Pluto for the first time:<a href="#footnote-8-3902">8</a><a href="#footnote2-18-3902">18</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Nh-pluto-in-true-color_2x_JPEG.jpg"></a></p>
<p>Next, New Horizons will be on its way further outwards into the Kuiper belt to send back images of comets and dwarf planets. You can track New Horizons’ location <a href="http://pluto.jhuapl.edu/Mission/Where-is-New-Horizons/index.php">here</a>.</p>
<p>Awkwardly, Pluto was still a planet when New Horizons launched, and everyone spent the years following Pluto’s demotion avoiding making eye contact with the New Horizons team. While I agree with the common sentiment that it’s sad that Pluto’s sad about its demotion,<a href="#footnote-9-3902">9</a> the truth is, Pluto should probably appreciate that it got away with 76 illegitimate years as a planet celebrity, pulling in a ton of Kuiper belt ass in the process, given that fellow Kuiper belt dwarf planet Eris spent that whole time living its life in total obscurity, only discovered in 2005.</p>
<p><strong>2) <a href="http://www.nasa.gov/mission_pages/msl/index.html">Curiosity</a></strong> (Mars, NASA)</p>
<p><a href="http://www.gizmag.com/curiosity-2nd-year/33248/"></a></p>
<p>Curiosity is a now-famous rover. A car-sized lovable lander robot dropped down on Mars’s surface in 2012, Curiosity is studying a bunch of things inside a large crater, with its primary objective being to figure out if there’s ever been life on Mars. The last two Mars rovers, Opportunity and Spirit, landed in 2004 with a planned mission of 90 days. Both lasted way past their expiry date, and <a href="http://www.geek.com/wp-content/uploads/2013/06/Opportunity-590x330.jpg">Opportunity</a> is <em>still </em>active. Such a good boy.</p>
<p>There are a bunch of other probes orbiting around Mars as well, but Curiosity is the main event there.</p>
<p>In my research, I came across <a href="https://www.youtube.com/watch?v=XRCIzZHpFtY">this video</a> from an IMAX movie about getting the rover Spirit from Earth to the surface of Mars and thought it was the coolest video ever. Until I found <a href="https://www.youtube.com/watch?v=BudlaGh1A0o">this video</a> about getting Curiosity on Mars, which was even cooler.</p>
<p><strong>3) <a href="http://www.nasa.gov/mission_pages/juno/main/index.html">Juno</a> </strong>(Jupiter, NASA)</p>
<p><a href="http://www.jpl.nasa.gov/news/news.php?feature=3501"></a></p>
<p>Juno left Earth in 2011, made a big loop and came back to Earth in 2013 to get a gravity zing (during which it captured a <a href="https://www.youtube.com/watch?v=_CzBlSXgzqI">cool video</a> of the moon circling the Earth), and is now on its way to Jupiter, where it’ll arrive in July of 2016.<a href="#footnote2-19-3902">19</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/06/Junos_interplanetary_trajectory.jpg"></a></p>
<p>Once it arrives, Juno will orbit Jupiter, taking pictures and using sensors to try to figure out what’s going on in there underneath all the <a href="https://waitbutwhy.com/wp-content/uploads/2020/07/j4mWuqcBmfDJukZUciLgUk-1024-80.jpg">succulent-looking</a> cloud tops. It’ll die by falling into Jupiter, hopefully snapping and relaying some quick photos of what it looks like inside Jupiter’s atmosphere before burning up so that someone can make a virtual reality video that lets you descend into Jupiter’s surface.</p>
<p><b>4) <a href="http://saturn.jpl.nasa.gov/spacecraft/overview/">Cassini</a> </b>(Saturn, NASA / European Space Agency / Italian Space Agency collaboration)</p>
<p><a href="http://www.esa.int/Our_Activities/Space_Science/Cassini-Huygens/Cassini_spacecraft"></a></p>
<p>Launched in 1997, Cassini set off towards Saturn, the only planet in the Solar System who decided it was okay to wear a tutu. Reaching Saturn in 2004, Cassini became the first probe in history to orbit the planet, sending back some jaw-dropping pictures, like this one:<a href="#footnote2-20-3902">20</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/06/saturn-2-1240x600.jpg"></a></p>
<p>And this one:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/06/21NASA2_SPAN-superJumbo.jpg"></a></p>
<p>And this close-up of the rings:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/06/saturn-fly-ring-1-1066x600.jpg"></a></p>
<p>And this absurdly cool picture of Saturn with the sun behind it:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/06/Saturn1-1371x600.jpg"></a></p>
<p>In 2005, Cassini dropped its attached lander, the upsettingly-named Huygens, down onto the largest of Saturn’s moons, Titan. Here’s a real image of the surface of Titan, taken by Huygens (it’s creepily fascinating seeing the actual surface of something as far away and mysterious as a Saturn moon):<a href="#footnote2-21-3902">21</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/06/Huygens_surface_color_sr.jpg"></a></p>
<p><strong>5 and 6) <a href="http://voyager.jpl.nasa.gov/">Voyager 1 and 2</a> </strong>(Jupiter, Saturn, Uranus, Neptune; NASA)</p>
<p><a href="http://voyager.jpl.nasa.gov/spacecraft/"></a></p>
<p>Launched in 1977, the two Voyager probes were the first probes to collect images of the four outer giants of the Solar System. Voyager 2 is still the only probe to visit Uranus and Neptune, taking these eerie photos of the two, respectively:<a href="#footnote2-22-3902">22</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/06/TheIceGiants.jpg"></a></p>
<p>The cool thing about the Voyagers is that even though their original missions are now long over, they’re still zooming outward. They’re both ridiculously far away now and going super fast. Voyager 1 is the faster of the two, going 38,000 mph (61,000 km/h)—so fast that it would cross the Atlantic Ocean in five minutes—and it’s the farthest man-made object from Earth, <a href="http://voyager.jpl.nasa.gov/where/">currently</a> 131 AU<a href="#footnote-10-3902">10</a> away from Earth. It was also the first man-made object to leave the Solar System. At this rate, Voyager 1 will reach Proxima Centauri, the closest star to us, in about 73,000 years.</p>
<p>Another cool thing about the Voyagers is that before they launched, a NASA committee, led by Carl Sagan, loaded them each up with a <a href="http://voyager.jpl.nasa.gov/spacecraft/goldenrec.html">time capsule</a>, full of symbols, sounds, and images of Earth (and symbol instructions about how to play and view the media), so the probes can one day tell aliens what our deal is. Probably a waste of everyone’s time, but who knows.</p>
<p><strong>7) <a href="http://rosetta.esa.int/">Rosetta</a> </strong>(comet, ESA)</p>
<p><a href="http://www.space.com/24292-rosetta-spacecraft.html"></a></p>
<p>Launched in 2004, Rosetta got a lot of attention last year when it reached comet 67P in August 2014 and successfully dropped its little lander, Philae, onto the comet a couple months later. Comet 67P turned out to kind of just be a big rock (2.7 mi/4.3 km long), but the <a href="http://www.nytimes.com/interactive/2015/03/20/science/space/rosetta-comet-photos.html?_r=0">images</a> taken by Rosetta were cool:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/06/150131-rosetta.jpg"></a></p>
<p><strong>8) <a href="http://dawn.jpl.nasa.gov/">Dawn</a></strong> (Vesta and Ceres, NASA)</p>
<p><a href="http://www.thisweekintomorrow.com/vol-2-no-10-dawn-approaches-two-balloons-and-a-cool-million-on-arxiv/"></a></p>
<p>Dawn can’t believe it made the cut on this list. The reason I included it is that I’m not sure people realize that there are huge, almost planet-size objects in the asteroid belt. The asteroid belt, a huge ring of millions of asteroids, including over 750,000 that are at least 1 km in diameter,<a href="#footnote2-23-3902">23</a> lies between the orbits of Mars and Jupiter (not to be confused with the much larger Kuiper belt that surrounds the outer Solar System). Among the many asteroids in the asteroid belt is <a href="http://www.popsci.com/sites/popsci.com/files/styles/large_1x_/public/images/2015/03/ceres-craters.jpg?itok=p_-m1zJ2">Ceres</a>, a dwarf planet 27% the diameter of the moon that makes up one-third of the asteroid belt’s total mass, and <a href="http://www.nasa.gov/sites/default/files/images/685735main_pia15678-43_full.jpg">Vesta</a>, the second largest object in the belt after Ceres and the brightest belt object in our night sky.<a href="#footnote-11-3902">11</a> I didn’t really know Ceres and Vesta were things. Anyway, Dawn, which was launched in 2007, spent nine months orbiting Vesta in 2011 before heading off to Ceres, where it arrived in March 2015 (making it the first probe to orbit two different bodies).</p>
<p>There’s another handful of probes out there as well. Like <a href="http://messenger.jhuapl.edu/">Messenger</a>, which orbited Mercury for seven years until intentionally crashing into it in April 2015; <a href="http://www.space.com/18945-akatsuki-venus-probe-second-chance-2015.html">Akatsuki</a>, a Japanese probe that was supposed to start orbiting Venus in 2010 but botched it, and will try again this year; a bunch of probes uneventfully circling the moon, including China’s Chang’e 3, which dropped the first <a href="http://www.spaceflight101.com/change-3-mission-updates.html">lander</a> on the moon since 1976; and a group of others taking measurements from the sun. Here’s an <a href="https://en.wikipedia.org/wiki/List_of_Solar_System_probes">exhaustive list</a> of all past and present probes, and an awesome National Geographic visualization that sums it all up (click the graphic for a larger view):<a href="#footnote2-24-3902">24</a>
</p><p><a href="http://cosmicdiary.org/fmarchis/files/2014/05/50-years-of-exploration.jpg"></a></p>
<p><strong><br />
Looking and Learning Tool #2: Telescopes</strong></p>
<p>Telescopes have been around since the early 17th century, and as they got more and more powerful over the next 400 years, they became humanity’s primary tool for turning the pages of <i>Where Are We?</i></p>
<p>But there came a point when ground telescopes ran into a limit on what they’d be able to see, no matter how advanced they became. You know when you look at a light through a glass of water and the light is all bendy and silly? That’s what’s happening when stars twinkle, except instead of water, we’re looking at them through the Earth’s atmosphere. The atmosphere doesn’t distort light as much as water does, but stars and galaxies are <em>tiny </em>pinpricks of light in our sky, so any level of blur is a big problem—it’s like being underwater in a swimming pool and looking upwards, trying to examine a bunch of birds flying in the sky above.</p>
<p>In the 1960s, humans gained the ability to put telescopes in <em>space, </em>where they’d show us the first crystal-clear view of the stars in history. In 1990, NASA launched the first truly badass space telescope, the Hubble.<a href="#footnote-12-3902">12</a><a href="#footnote2-25-3902">25</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/Hubble_01.jpg"></a></p>
<p>The 13-ton, school bus-length Hubble Space Telescope’s 7.9 foot (2.4 m) lens is accurate enough to shine a laser beam on a dime 200 miles away and powerful enough to see a pair of fireflies in Tokyo from your home in Boston (if the Earth were flat). And in its position in orbit 340 miles above Earth, where there’s no atmosphere or light pollution in the way, the Hubble is on what NASA calls “the ultimate mountaintop.”<a href="#footnote2-26-3902">26</a> All of this gives the Hubble an unprecedented view of the universe, allowing it to spend the last 25 years sending us the most astounding photographs of things I can’t really believe are real. Like this epic galaxy:<a href="#footnote2-27-3902">27</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/03-hats-off-890.jpg"></a></p>
<p>Or these two galaxies, which are in the slow process of merging:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/54916fb9-02f7-4b8c-98df-def86bad895d-2016x2040.jpeg"></a></p>
<p>Or the inconceivably huge Pillars of Creation (the left finger is so big, at four light years from top to bottom, that if you started at the knuckle and flew in an airplane upwards, it would take 4.5 <em>million</em> years to get to the fingertip):</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/c47a074d-e53f-4856-83da-b7b8a928c156-1489x2040.jpeg"></a></p>
<p>Or the time Hubble aimed its lens at a tiny, seemingly empty square of the sky (seen here next to the moon to show the size of the square):</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/HDF.jpg"></a></p>
<p>And found thousands of galaxies:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/690958main_p1237a1.jpg"></a></p>
<p>What Hubble and other space telescopes<a href="#footnote-13-3902">13</a> have shown us has revealed worlds of new information about where we are and how we got here, expanding our knowledge about everything from dark energy to the origin and age and size of the universe to the number of planets out there like ours that might have life on them.</p>
<p>For over 40 years now, those two objectives—supporting Earth industries and continuing to learn and discover—have been the extent of our relationship with space.</p>
<p>And because those two goals are both best accomplished by <em>machine </em>space travelers, the most recent chapter of The Story of Humans and Space has been all about space faring machines, with the human role taking place on or very near Earth, controlling things with joysticks.</p>
<p>The only reason any humans have gone to space since Apollo 17 returned to Earth in 1972 is that sometimes, the machines aren’t yet advanced enough to do a certain task, so we need to send a human up to do it instead. Of the roughly 550 people who have ever been in space, over 400 of them have gone there in the post-Space Race era. But since Apollo, the reasons have been practical—scientists and technicians going to space to do a job. That’s why each and every manned mission of the past four decades has kept within the thin blanket of space surrounding the Earth—Low Earth Orbit.</p>
<p><strong>The International Space Station</strong></p>
<p>Today, the purpose of almost every manned space mission is to take astronauts to and from the International Space Station (ISS). <a href="#footnote2-28-3902">28</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/AaHfbHo.jpg"></a></p>
<p>The ISS is an international collaboration among 16 countries, started in 1998 and constructed over the span of a decade. The space station orbits the Earth in the lowest strip of LEO at an altitude of between 205 and 255 miles (330–410 km<a href="#footnote-14-3902">14</a>), about the distance across Iceland—close enough to the ground that you can easily see it at night with your <a href="http://spotthestation.nasa.gov/">naked eye</a>.<a href="#footnote-15-3902">15</a> And it’s bigger than people realize, weighing as much as 320 cars and spanning the full length of an American football field:<a href="#footnote2-29-3902">29</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/693259main_jsc2012e219094_big-1.jpg"></a></p>
<div>
<p><strong>What the Hell Does Anyone Do in the ISS? Blue Box</strong></p>
<p>As I began working on this post, I realized I didn’t really know what the ISS was for or what anyone did while they were there. Every time I see a video of what goes on inside the space station, it’s just some adult floating around having playtime.</p>
<p>Conveniently, there’s such a thing as an ISS <em>conference</em>, and it happened to take place last month, in Boston. So I went. The conference was run by the Center for the Advancement of Science in Space (<a href="http://www.iss-casis.org/">CASIS</a>), which manages the US portion of the ISS. Here’s what I learned at the conference:</p>
<ul>
<li>The ISS is a science laboratory. It’s kind of like other labs, except with the party trick that it’s soaring through space, so it’s the one lab where you can test things in zero gravity (it’s not actually zero gravity—it’s <em>micro</em>gravity<em>—</em>something I’ll explain later in the post).</li>
</ul>
<ul>
<li>What most <a href="https://www.nasa.gov/mission_pages/station/research/news/casis_spacex4">ISS experiments</a> have in common is that they’re there for the gravity situation, but beyond that, they span a wide range of purposes—everything from learning about osteoporosis as astronauts’ bones atrophy (because they don’t have to fight against gravity), to testing how equipment holds up in space, to analyzing how fluids behave and interact without the influence of any other forces, to using the change in gravity to trick bacteria into revealing which genes make them immune to certain medicines.</li>
</ul>
<ul>
<li>Astronauts in the ISS have a tight and controlled schedule during the week. At all times, they’re either sleeping (8.5 hours), eating (1.5 hours for breakfast/dinner, 1 hour for lunch) exercising (mandatory 2.5 hours a day), or working on experiments (9 hours a day)—I took <a href="https://waitbutwhy.com/wp-content/uploads/2015/07/IMG_7334s.jpg">this photo</a> of the current schedule of the three astronauts on the ISS.<a href="#footnote-16-3902">16</a> Weekends are off, which could not possibly sound more fun—you get to spend the whole time floating around and looking out the window.</li>
</ul>
<ul>
<li>I’m not the only one who badly wants to play on the ISS—there’s a furiously competitive process to be selected by NASA to go. Thousands apply, 100 are picked for a final round interview and physical examination, and only one or two end up getting the nod. On rare occasion, a private company or individual can buy a spot on the station for a few days, but it costs around $60 million.</li>
</ul>
<p>If you want to get a better feel for what it’s like to live on the ISS, here’s a <a href="https://www.youtube.com/watch?v=doN4t5NKW-k">video tour</a> of the space station by a floaty astronaut.</p>
</div>
<p>So far, 216 people have gotten to play on the ISS, from 15 countries:<a href="#footnote2-30-3902">30</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/iss_visitors_country.png"></a></p>
<p><strong><br />
How Stuff Gets to Space<br />
</strong></p>
<p>We’ve gone over what’s in space, but how does all that stuff <em>get </em>to space? Have you ever asked yourself how something like the GPS satellite gets up there in the first place? The answer is that there are nine countries that have the ability to launch something into orbit: Russia, the US, France, Japan, China, India, Israel, Iran and, um, North Korea—along with one non-national entity, the European Space Agency (ESA). If a satellite goes up into space, it’s because someone paid one of those ten entities to bring it there atop a massive, expensive rocket (or because a country is putting one up there for its own uses).</p>
<p>As for launching <em>humans </em>into space, only three countries in history have done it—Russia, the US, and China (who is a fast-growing newcomer to the space industry). Since the 60s, Russia has used its Soyuz rockets to launch people into space, and the US, after wrapping up the Apollo program in 1972, regained the ability to put people in orbit in 1981 with the Space Shuttle program.<a href="#footnote2-31-3902">31</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/07/6938210-space-shuttle-photos.jpg"></a></p>
<p>Over the next 30 years, the US launched 135 Space Shuttles into LEO, with 133 successes. The two exceptions are fairly traumatizing parts of American history—<em><a href="https://www.youtube.com/watch?v=fSTrmJtHLFU">Challenger</a> </em>in 1986 and <i><a href="https://www.youtube.com/watch?v=aIJiW8d_c68">Columbia</a> </i>in 2003.</p>
<p>The Space Shuttle Program retired in 2011. Today, only two countries can launch a human into orbit—Russia and China. With no capability themselves, the US—the country that once triumphantly put a man on the moon while the world watched—now has to launch their astronauts on Russian rockets, at Russia’s whim.</p>
<p>___________</p>
<p>So what are we to make of The Story of Humans and Space? It’s a bit of an odd tale. In 1970, the story looked like this:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/graph1.png"></a></p>
<p>So the assumption about where the story was headed was this:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/graph2.png"></a></p>
<p>But now it’s 2015, and it turns out that this is what was happening:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/graph3.png"></a></p>
<p>When I look at what’s going on with humans and space today, I should think it’s incredible. Just 58 years after the Soviets put the first man-made object into orbit, we now have a swarm of high-tech equipment soaring around our planet, giving humans magical capabilities in vision and communication. There’s a team of flying robot messengers spread out through the Solar System, reporting back to us with their findings. There’s a <em>huge flying telescope </em>high above Earth, showing us exactly what the observable universe looks like. There’s a football field-sized <i>science lab </i>250 miles above our heads with <em>people </em>in it.</p>
<p>Everything I just said is <em>amazing</em>.</p>
<p>And if only The Story of Humans and Space looked like this—</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/graph4.png"></a></p>
<p>—I would be <em>marveling </em>at the things we’re currently doing out in The Situation.</p>
<p>But unfortunately, the 60s happened. So instead, it’s like this:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic1.png"></a><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic2.jpg"></a><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic3.jpg"></a><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic4.jpg"></a><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic5.png"></a><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic6.jpg"></a><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic7.png"></a><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic8.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic8.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic9.png"></a><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/magic10.png"></a></p>
<p>A good magic show follows a simple rule—make the act get better as it goes along. If you can’t continue to stay a step ahead of the increasingly-jaded crowd, they’ll quickly tune you out.</p>
<p>In some areas, the Humans and Space magic show has continued steadily upward. In our quest for knowledge and understanding, for example, we continue to outdo ourselves, learning significantly more about the universe every decade. The human spirit of <em>discovery </em>is alive and well, having thrived in space in the years since Apollo.</p>
<p>But as fascinated as we are by discovery—as much as we yearn to know all the secrets hidden in the pages of <em>Where Are We?—</em>when it comes to filling us with true excitement and inspiration and getting our <em>adrenaline </em>pumping, discovery doesn’t hold a candle to <em>adventure. </em>Probes and telescopes may fill us with wonder and light up our curiosity, but nothing gets us in our <em>animal core</em> like watching our species go <em>where no man has gone before. </em>And in that arena, the last four decades have left us feeling empty. After watching people land on the moon, following manned missions to and from the ISS is, as Ross Andersen <a href="http://aeon.co/magazine/technology/the-elon-musk-interview-on-mars/">said</a>, “about as thrilling as watching Columbus sail to Ibiza.”</p>
<p>And that’s why, in today’s world, The Story of Humans and Space has drifted off the front page of our consciousness. The topic that should drop all of us to our knees has become a geeky sideshow. Ask 10 well-educated people you know about what’s going on with Solar System probes or the ISS or NASA or SpaceX and most won’t be able to tell you very much. Some won’t even know that people ever go to space anymore. People don’t know because people don’t care. Because of the way it played out, The Story of Humans and Space feels like a disappointment. And looking at the world around us today, it’s intuitive to predict that future chapters of the space story will continue to putter along as they do today:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/08/graph52.png"></a></p>
<p>Many people don’t think this is a bad thing. “Why spend exorbitant amounts of money sending people to the far reaches of space when we have so many problems right here on Earth?” they ask. Massachusetts Congressman Barney Frank, who spent three decades playing a key role in US budget decision-making, calls ambitious manned space travel “at best a luxury that the country ought not to be indulging in” and “a complete and total waste of money” and “pure boondoggle.”<a href="#footnote2-32-3902">32</a> And the dramatic slashes to NASA’s budget since the Space Race ended suggest that Frank isn’t the only US politician to hold this view.</p>
<p>Upon first assessment, Frank is being perfectly rational—after all, in the face of concerns like healthcare, national security, education, and poverty, should we really make room for an “adventure budget”? And in that light, the graph projection above for The Future of Humans and Space seems all the more likely to continue on its current course.</p>
<p>I’ve spent the last couple months reading, talking, and thinking almost non-stop about what the coming chapters of this story will look like—and my assumptions about the future have now changed dramatically.</p>
<p>I think we’re all in for a big surprise.</p>
<p><a href="https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html/2">Part 2: Musk’s Mission →</a></p>
</div>||||I|||| * Home

 Menu
 * Homepage
 * about
 + wait but who
 + faq
 + contact
 * archive
 * minis
 * the shed
 * dinner table
 * store
 + store home
 + new releases
 + posters
 + phone cases
 + cards & wrapping paper
 + squishy things
 + men’s tees
 + women’s tees
 + coffee mugs
 + store support
 * support wbw
 * #7246 (no title)
 * Finn’s Cave
 * 

 How (and Why) SpaceX Will Colonize Mars

 August 16, 2015 By Tim Urban


 This is Part 3 of a four-part series on Elon Musk’s companies. For an explanation of why this series is happening and how Musk is involved, start with Part 1.

 Pre-Post Note: I started working on this post ten weeks ago. When I started, I never intended for it to become such an ordeal. But like the Tesla post, I decided as I researched that this was A) a supremely important topic that will only become more important in the years to come, and B) something most people don’t know nearly enough about. My weeks of research and discussions with Musk and others built me an in-depth, tree-trunk understanding of what’s happening in what I’m calling The Story of Humans and Space—one that has totally reframed my mental picture of the future (yet again). And as I planned out what to include in the post, I wanted to make sure every Wait But Why reader ended up with the same foundation moving forward—because with everything that’s coming, we’re gonna need it. So like the Tesla post, this post became a full situation. Even the progress updates leading up to its publication became a full situation.

 Thanks for your patience. I know you’d prefer this not to be a site that updates every two months, and I would too. The Tesla and SpaceX posts were special cases, and you can expect a return to more normal-length WBW posts now that they’re done.

 About the post itself: There are three main parts. Part 1 provides the context and background, Part 2 explores the “Why” part of colonizing Mars, and Part 3 digs into the “How.” To make reading this post as accessible as possible, it’s broken into five pages, each about the length of a normal WBW post, and you can jump to any part of the post easily by clicking the links in the Table of Contents below. We’re also trying two new things, both coming in the next couple days:

 1) PDF and ebook options: We made a fancy PDF of this post for printing and offline viewing (see a preview here), and an ebook containing the whole four-part Elon Musk series:

 2) An audio version. You can find an unabridged audio version of the post, read by me, as well as a discussion about the post between Andrew and me here.

 ___________

 Contents

 Part 1: The Story of Humans and Space

 Part 2: Musk’s Mission

 Part 3: How to Colonize Mars
 → Phase 1: Figure out how to put things into space
 → Phase 2: Revolutionize the cost of space travel
 → Phase 3: Colonize Mars

 A SpaceX Future


 2365 AD, Ganymede

 One more day until departure. It was so surreal to picture actually being there that she still didn’t really believe it would happen. All those things she had always heard about—buildings that were constructed hundreds of years before the first human set foot on Ganymede; animals the size of a house; oceans the size of her whole world; tropical beaches; the famous blue sky; the giant sun that’s so close it can burn your skin; and the weirdest part— no Jupiter hovering overhead . Having seen it all in so many movies, she felt like she was going to visit a legendary movie set. It was too much to think about all at once. For now, she just had to focus on making sure she had everything she needed and saying goodbye to everyone—it would be a long time before she would see them again…

 ___________

 Part 1: The Story of Humans and Space

 About six million years ago, a very important female great ape had two children. One of her children would go on to become the common ancestor of all chimpanzees. The other would give birth to a line that would one day include the entire human race. While the descendants of her first child would end up being pretty normal and monkey-ish, as time passed, strange things began to happen with the lineage of the other.11 ← click these

 We’re not quite sure why, but over the next six million years, our ancestral line started to do something no creatures on Earth had ever done before—they woke up.

 It happened slowly and gradually through the thousands of generations the same way your brain slowly comes to in the first few seconds after you rouse from sleep. But as the clarity increased, our ancestors started to look around and, for the very first time, wonder.

 Emerging from a 3.6-billion-year dream, life on Earth had its first questions.

 What is this big room we’re in, and who put us here? What is that bright yellow circle on the ceiling and where does it go every night? Where does the ocean end and what happens when you get there? Where are all the dead people now that they’re not here anymore?

 We had discovered our species’ great mystery novel—Where Are We?—and we wanted to learn how to read it.

 As the light of human consciousness grew brighter and brighter, we began to arrive at answers that seemed to make sense. Maybe we were on top of a floating disk, and maybe that disk was on top of a huge turtle. Maybe the pinpricks of light above us at night are a glimpse into what lies beyond this big room—and maybe that’s where we go when we die. Maybe if we can find the place where the ceiling meets the floor, we can poke our heads through and see all the super fun stuff on the other side.2

 Around 10,000 years ago, isolated tribes of humans began to merge together and form the first cities. In larger communities, people were able to talk to each other about this mystery novel we had found, comparing notes across tribes and through the generations. As the techniques for learning became more sophisticated and the clues piled up, new discoveries surfaced.

 The world was apparently a ball, not a disk. Which meant that the ceiling was actually a larger sphere surrounding us. The sizes of the other objects floating out there in the sphere with us, and the distances between them, were vaster than we had ever imagined. And then, something upsetting:

 The sun wasn’t revolving around us. We were revolving around the sun.

 This was a super unwarm, unfuzzy discovery. Why the hell weren’t we in the center of things? What did that mean?

 Where are we?

 The sphere was already unpleasantly big—if we weren’t in the center of it, were we just on a random ball inside of it, kind of for no apparent reason? Could this really be what was happening?

 Scary.

 Then things got worse.

 It seemed that the pinpricks of light on the edge of the sphere weren’t what we thought they were—they were other suns like ours. And they were out there floating just like our sun—which means we weren’t inside of a sphere at all. Not only was our planet not the center of things, even our sun was just a random dude out there, in the middle of nowhere, surrounded by nothingness.

 Scary.

 Our sun turned out to be a little piece of something much bigger. A beautiful, vast cloud of billions of suns. The everything of everything.

 At least we had that. Until we realized that it wasn’t everything, it was this:

 Darkness.

 The better our tools and understanding became, the more we could zoom out, and the more we zoomed out, the more things sucked. We were deciphering the pages of Where Are We? at our own peril, and we had deciphered our way right into the knowledge that we’re unbelievably alone, living on a lonely island inside a lonely island inside a lonely island, buried in layers of isolation, with no one to talk to.

 That’s our situation.

 In the most recent 1% of our species’ short existence, we have become the first life on Earth to know about the Situation—and we’ve been having a collective existential crisis ever since.

 You really can’t blame us. Imagine not realizing that the universe is a thing and then realizing the universe is a thing. It’s a lot to take in.

 Most of us handle it by living in a pleasant delusion, pretending that the only place we live is in an endless land of colors and warmth. We’re like this guy, who’s doing everything he possibly can to ignore the Situation:3

 And our best friend for this activity? The clear blue sky. The blue sky seems like it was invented to help humans pretend the Situation doesn’t exist, serving as the perfect whimsical backdrop to shield us from reality.

 Then nighttime happens, and there’s the Situation, staring us right in the face.

 Oh yeah…

 This la-di-da → oh yeah… → la-di-da → oh yeah… merry-go-round of psychosis was, for most of recent history, the extent of our relationship with the Situation.

 But in the last 60 years, that relationship has vaulted to a whole new level. During World War II, missile technology leapt forward,2 and for the first time, a new, mind-blowing concept was possible—

 Space travel.

 For thousands of years, The Story of Humans and Space had been the story of staring out and wondering. The possibility of people leaving our Earth island and venturing out into space burst open the human spirit of adventure.

 I imagine a similar feeling in the people of the 15th century, during the Age of Discovery, when we were working our way through the world map chapter of Where Are We? and the notion of cross-ocean voyages dazzled people’s imaginations. If you asked a child in 1495 what they wanted to be when they grew up, “an ocean explorer” would probably have been a common response.

 In 1970, if you asked a child the same question, the answer would be, “an astronaut”—i.e. a Situation explorer.

 WWII advanced the possibility of human space travel, but it was in late 1957, when the Soviets launched the first man-made object into orbit, the adorable Sputnik 1, that space travel became the defining quest of the world’s great powers.

 At the time, the Cold War was in full throttle, and the US and Soviets had their measuring sticks out for an internationally-televised penis-measuring contest. With the successful launch of Sputnik, the Soviet penis bolted out by a few centimeters, horrifying the Americans.

 To the Soviets, putting a satellite into space before the US was proof that Soviet technology was superior to American technology, which in turn was put forward as proof, for all the world to see, that communism was a system superior to capitalism.

 Eight months later, NASA was born.

 The Space Race had begun, and NASA’s first order of business would be to get a man into space, and then a man into full orbit, preferably both before the Soviets. The US was not to be shown up again.

 In 1959, NASA launched Project Mercury to carry out the mission. They were on the verge of success when in April of 1961, the Soviets launched Yuri Gagarin into a full orbit around the Earth, making the first human in space and in orbit a Soviet.

 It was time for drastic measures. John F. Kennedy’s advisors told him that the Soviets had too big a lead for the US to beat them at any near-term achievements—but that the prospect of a manned moon landing was far enough in the future that the US had a fighting chance to get there first. So Kennedy gave his famous “we choose to go to the moon, not because it is easy, but because it is hahhd” speech, and directed an outrageous amount of funding at the mission ($20 billion, or $205 billion in today’s dollars).

 The result was Project Apollo. Apollo’s mission was to land an American on the moon—and to do it first. The Soviets answered with Soyuz, their own moon program, and the race was on.

 As the early phases of Apollo started coming together, Project Mercury finally hit its stride. Just a month after Yuri Gagarin became the first man in space, American astronaut Alan Shepard became the second man in space, completing a little arc that didn’t put him in full orbit but allowed him to give space a high-five at the top of the arc. A few months later, in February of 1962, John Glenn became the first American to orbit the Earth.

 The next seven years saw 22 US and Soviet manned launches as the superpowers honed their skills and technology. By late 1968, the furiously-sprinting US had more total launches under their belt (17) than the Soviets (10), and together, the two nations had mastered what we call Low Earth Orbit (LEO).

 But LEO hadn’t really excited anyone since the early ’60s. Both powers had their sights firmly set on the moon. The Apollo program was making quick leaps, and in December of 1968, the US became the first nation to soar outside of LEO. Apollo 8 made it all the way to the moon’s orbit and circled around 10 times before returning home safely. The crew, which included James Lovell (who a few months later played the role of Tom Hanks on the Apollo 13 mission), shattered the human altitude record and became the first people to see the moon up close, the first to see the “dark” side of the moon, and the first to see the Earth as a whole planet, snapping this iconic photo:4

 Upon return, the crew became America’s most celebrated heroes—which I hope they enjoyed for eight months. Three Apollo missions later, in July of 1969, Apollo 11 made Americans Neil Armstrong3 and Buzz Aldrin the first humans on the moon, and Armstrong took this famous photo of Aldrin looking all puffy:5

 It’s hard to fully emphasize what a big deal this was. Ever since life on Earth began 3.6 billion years ago, no earthly creature had set foot on any celestial body other than the Earth. Suddenly, there are Armstrong and Aldrin, bouncing around another sphere, looking up in the sky where the moon is supposed to be and seeing the Earth instead. Insane.

 Project Apollo proved to be a smashing success. Not only did Apollo get a man on the moon before the Soviets, the program sent 10 more men to the moon over the next 3.5 years on five other Apollo missions. There were six successful moon trips in seven tries, with the famous exception being Apollo 13, which was safely aborted after an explosion in the oxygen tank.4

 The Soviet Soyuz program kept running into technical problems, and it never ended up putting someone on the moon.

 The final Apollo moonwalk took place in late 1972. In only one decade, we had conquered nearby space, and progress was accelerating. If at that time you had asked any American, or any other human, what the coming decades of space travel would bring, they’d have made big, bold predictions. Many more people on the moon, a permanent moon base, people on Mars, and beyond.

 So you can only imagine how surprised they’d be if you told them in 1972, after just watching 12 humans walk on the moon, that 43 years later, in the impossibly futuristic-sounding year 2015, the number of people to set foot on the moon would still be 12. Or that after leaving Low Earth Orbit in the dust years earlier and using it now as our pre-moon trip parking lot, 2015 would roll around and LEO would be the farthest out humans would ever go.

 1972 people would be blown away by our smart phones and our internet, but they’d be just as shocked that we gave up on pushing our boundaries in space.

 So what happened? After such a wildly exciting decade of human space adventure, why did we just stop?

 Well, like we found in the Tesla post, “Why did we stop?” is the wrong question. Instead, we should ask:

 Why were we ever adventurous about sending humans into space in the first place?

 Space travel is unbelievably expensive. National budgets are incredibly tight. The fact is, it’s kind of surprising that a nation ever ponied up a sizable chunk of its budget for the sake of adventure and inspiration and pushing our boundaries.

 And that’s actually because no nation did blow their budget for the sake of adventure and inspiration and pushing our boundaries—two nations blew their budgets because of a penis–length contest. In the face of international embarrassment at a time when everyone was trying to figure out whose economic system was better, the US government agreed to drop the usual rules for a few years to pour whatever resources were necessary on the problem to make sure they won that argument—

 And once they won it, the contest was over and so were the special rules. And the US went back to spending money like a normal person.6

 Instead of continuing to push the limits at all costs, the US and the Soviets got a grip, put their pants back on, shook hands, and started working together like adults on far more practical projects, like setting up a joint space station in LEO.

 In the four decades since then, the Story of Humans and Space has again become confined to Earth, where we find ourselves with two primary reasons to interact with space (Note: the next whole chunk of the post is a slight diversion for an overview on satellites, space probes, and space telescopes. If that doesn’t excite you, I won’t be hurt if you skip down to the International Space Station section):

 1) Support for Earth Industries

 The first and primary reason humans have interacted with space since the Apollo program isn’t about human interest in space. It’s about using space for practical purposes in support of industries on Earth—mostly in the form of satellites. The bulk of today’s rocket launches into space are simply putting things into LEO whose purpose is to look back down at Earth, not to the great expanses in the other direction.

 Here’s a little satellite overview:

 Satellites Blue Box

 We don’t think about them that often, but above us are hundreds of flying robots that play a large part in our lives on Earth. In 1957, lonely Sputnik circled the Earth by itself, but today, the worlds of communication, weather forecasting, television, navigation, and aerial photography all rely heavily on satellites, as do many national militaries and government intelligence agencies.

 The total market for satellite manufacturing, the launches that carry them to space, and related equipment and services has ballooned from $60 billion in 2004 to over $200 billion in 2015. Satellite industry revenue today makes up only 4% of the global telecommunications industry but accounts for over 60% of space industry revenue.7

 Here’s how the world’s satellites break down by role (in 2013): 8

 Of the 1,265 active satellites in orbit at the beginning of 2015, the US owns by far the largest number at 528—over 40% of the total—but over 50 countries own at least one orbiting satellite.

 As for where all of these satellites are, most of them fall into two distinct “layers” of space:

 About two-thirds of active satellites are in Low Earth Orbit. LEO starts up at 99 miles (160 km) above the Earth, the lowest altitude at which an object can orbit without atmospheric drag messing things up. The top of LEO is 1,240 miles (2,000 km) up. Typically, the lowest satellites are at around 220 miles (350 km) up or higher.

 Most of the rest (about one-third) of the satellites are much farther out, in a place called geostationary orbit (GEO). It’s right at 22,236 miles (35,786 km) above the Earth, and it’s called geostationary because something orbiting in it rotates at the exact speed that the Earth turns, making its position in the sky stationary relative to a point on the Earth. It’ll seem to be motionless to an observer on the ground.9

 GEO is ideal for something like a TV satellite because a dish on the Earth can aim at the same fixed spot all the time.

 A small percentage of other satellites are in medium Earth orbit (MEO), which is everything in between LEO and GEO. One notable resident of MEO is the GPS system that most Americans, and people from many other countries, use every day. I never realized that the entire GPS system, a US Department of Defense project that went live in 1995, only uses 32 satellites total. And until 2012, the number was only 24—six orbits, each with four satellites. But you can see in the GIF below that even with 24, a given point on the Earth can be seen by at least six of the satellites at any given time, and usually it’s nine or higher (in the GIF, the blue dot on the Earth is a hypothetical person on the ground, and whichever satellites can see him at a given time are blue, with the green lines showing their line of sight to the person):10

 This is why your phone’s map can still show your location even when you’re somewhere with no cellular service—because it has nothing to do with cellular service. The system is also set up to be redundant—only four satellites need to simultaneously see you in order for the system to pinpoint your location. GPS satellites have an orbital period of about 12 hours, making two full rotations of the Earth each day.5

 You can see satellite locations using Google Earth (here’s a cool video of Google Earth showing the satellites).

 Space Debris Bluer Box

 There’s a big problem happening in the world of satellites. In addition to the 1,265 active satellites up in orbit, there are thousands more inactive satellites, as well as a bunch of spent rockets from previous missions. And once in a while, one of them explodes, or two of them collide, creating a ton of tiny fragments called space debris. The number of objects in space has risen quickly over recent decades, as a GIF6 made by the ESA shows (with exaggerated-sized objects relative to the Earth’s size):11

 The majority of satellites and debris are bunched around the Earth in LEO, and the outer ring of objects is what’s located in GEO.

 Earth space agencies track about 17,000 objects in space, only 7% of which are active satellites. Here’s a map showing every known object in space today.12

 But the crazy thing is they only track the large objects, and that’s what we’re seeing in that image. Estimates for the number of smaller debris objects (1 – 10 cm) range from 150,000 to 500,000, and there are over a million total pieces of debris larger than 2 mm.13

 The issue is that at the incredible speeds at which space objects move (most LEO objects zip along at over 17,000 mph), a collision with even a tiny object can cause devastating damage to an active satellite or spacecraft. An object of only 1 cm at those speeds will cause the same damage in a collision as a small hand grenade.714

 Over a third of all space debris originated from just two events: China’s 2007 anti-satellite test, when China shat on the world’s face by intentionally blowing up one of its own satellites, creating 3,000 new pieces of debris large enough to be trackable, and a 2009 collision between two satellites that exploded into 2,000 debris chunks.15 Each collision increases the amount of debris, which in turn increases the likelihood of more collisions, and there’s danger of a domino effect situation, which scientists call the Kessler Syndrome. A bunch of parties are proposing ways to mitigate the amount of debris in LEO—everything from harpooning the debris to laser blasting it to intercepting it with a cloud of gas.

 Here’s a chart that sums up each nation’s “space footprint,” showing the quantity of active satellites, inactive satellites, and space debris caused by each country:16

 There are a few other space activities in the “Support for Earth Industries” category of human/space interaction—like space mining, space burial, and space tourism—but at least for now, satellites account for almost the entire category.

 2) Looking and Learning

 The second reason humans have interacted with space in the last four decades proves that while we may have stopped sending people into The Situation, we never lost our hunger to learn about what’s out there. As society moved on from space and turned its attention elsewhere, astronomers have kept busy at work deciphering their way through page after page of the old mystery novel, Where Are We?

 Astronomers learn best with their eyes, and a side effect of the Space Race was the development of far better technology for seeing what’s out there. There are two high-tech ways modern astronomers see things:

 Looking and Learning Tool #1: Sending probes around the Solar System

 Basically, scientists fire a fancy robot toward some distant planet, moon, or asteroid, and the robot spends months or years flying through space, bored, until it finally arrives. Then, depending on the plan, it either just flies by the object, taking some pictures on the way, orbits the object to get more detailed information, or lands on the object for a full inspection. Everything it learns, it sends back to us, and one day, when its job is done, we either kill the probe by crashing it into the object or let it just fly out into deep space to be depressed.

 I often use myself as a litmus test for what the public probably knows about or doesn’t know about. As I’ve mentioned before on this blog, I’ve been seriously dating astronomy ever since I was three years old—so if I don’t know something going on in the world of space, I assume that most people don’t. And when it comes to space probes, I’ve felt pretty disoriented. Are there 200 of them flying around out there? 50? 9? Why are they out there, who sent them, and what are they doing? All I’d know is that sometimes there would be a random story about some probe sending back stunning pictures—I’d open the cnn.com gallery, click through them, be thrilled for a second, send the link to the three friends of mine who are also dating astronomy, and then try to close the page but instead see some trashy CNN clickbait headline on the side of the page, click that, and ruin my life for the next three hateful hours. That’s my relationship with humanity’s space probes.

 But in researching this post, I quickly realized there’s not that much to know, and it doesn’t take too big an effort to get fully oriented. Here are what I consider the eight key space robots to know about right now:17

 1) New Horizons (Pluto, NASA)

 New Horizons goes first because its big moment just happened. Launched in 2006 on a decade-long trip to Pluto (sped up on its way by a Jupiter fly-by in 2007 that gravity-zinged it to a much faster speed), New Horizons finally reached Pluto on July 14th, 2015. It didn’t land on Pluto, but it flew very near to it and showed us Pluto for the first time:818

 Next, New Horizons will be on its way further outwards into the Kuiper belt to send back images of comets and dwarf planets. You can track New Horizons’ location here.

 Awkwardly, Pluto was still a planet when New Horizons launched, and everyone spent the years following Pluto’s demotion avoiding making eye contact with the New Horizons team. While I agree with the common sentiment that it’s sad that Pluto’s sad about its demotion,9 the truth is, Pluto should probably appreciate that it got away with 76 illegitimate years as a planet celebrity, pulling in a ton of Kuiper belt ass in the process, given that fellow Kuiper belt dwarf planet Eris spent that whole time living its life in total obscurity, only discovered in 2005.

 2) Curiosity (Mars, NASA)

 Curiosity is a now-famous rover. A car-sized lovable lander robot dropped down on Mars’s surface in 2012, Curiosity is studying a bunch of things inside a large crater, with its primary objective being to figure out if there’s ever been life on Mars. The last two Mars rovers, Opportunity and Spirit, landed in 2004 with a planned mission of 90 days. Both lasted way past their expiry date, and Opportunity is still active. Such a good boy.

 There are a bunch of other probes orbiting around Mars as well, but Curiosity is the main event there.

 In my research, I came across this video from an IMAX movie about getting the rover Spirit from Earth to the surface of Mars and thought it was the coolest video ever. Until I found this video about getting Curiosity on Mars, which was even cooler.

 3) Juno (Jupiter, NASA)

 Juno left Earth in 2011, made a big loop and came back to Earth in 2013 to get a gravity zing (during which it captured a cool video of the moon circling the Earth), and is now on its way to Jupiter, where it’ll arrive in July of 2016.19

 Once it arrives, Juno will orbit Jupiter, taking pictures and using sensors to try to figure out what’s going on in there underneath all the succulent-looking cloud tops. It’ll die by falling into Jupiter, hopefully snapping and relaying some quick photos of what it looks like inside Jupiter’s atmosphere before burning up so that someone can make a virtual reality video that lets you descend into Jupiter’s surface.

 4) Cassini (Saturn, NASA / European Space Agency / Italian Space Agency collaboration)

 Launched in 1997, Cassini set off towards Saturn, the only planet in the Solar System who decided it was okay to wear a tutu. Reaching Saturn in 2004, Cassini became the first probe in history to orbit the planet, sending back some jaw-dropping pictures, like this one:20

 And this one:

 And this close-up of the rings:

 And this absurdly cool picture of Saturn with the sun behind it:

 In 2005, Cassini dropped its attached lander, the upsettingly-named Huygens, down onto the largest of Saturn’s moons, Titan. Here’s a real image of the surface of Titan, taken by Huygens (it’s creepily fascinating seeing the actual surface of something as far away and mysterious as a Saturn moon):21

 5 and 6) Voyager 1 and 2 (Jupiter, Saturn, Uranus, Neptune; NASA)

 Launched in 1977, the two Voyager probes were the first probes to collect images of the four outer giants of the Solar System. Voyager 2 is still the only probe to visit Uranus and Neptune, taking these eerie photos of the two, respectively:22

 The cool thing about the Voyagers is that even though their original missions are now long over, they’re still zooming outward. They’re both ridiculously far away now and going super fast. Voyager 1 is the faster of the two, going 38,000 mph (61,000 km/h)—so fast that it would cross the Atlantic Ocean in five minutes—and it’s the farthest man-made object from Earth, currently 131 AU10 away from Earth. It was also the first man-made object to leave the Solar System. At this rate, Voyager 1 will reach Proxima Centauri, the closest star to us, in about 73,000 years.

 Another cool thing about the Voyagers is that before they launched, a NASA committee, led by Carl Sagan, loaded them each up with a time capsule, full of symbols, sounds, and images of Earth (and symbol instructions about how to play and view the media), so the probes can one day tell aliens what our deal is. Probably a waste of everyone’s time, but who knows.

 7) Rosetta (comet, ESA)

 Launched in 2004, Rosetta got a lot of attention last year when it reached comet 67P in August 2014 and successfully dropped its little lander, Philae, onto the comet a couple months later. Comet 67P turned out to kind of just be a big rock (2.7 mi/4.3 km long), but the images taken by Rosetta were cool:

 8) Dawn (Vesta and Ceres, NASA)

 Dawn can’t believe it made the cut on this list. The reason I included it is that I’m not sure people realize that there are huge, almost planet-size objects in the asteroid belt. The asteroid belt, a huge ring of millions of asteroids, including over 750,000 that are at least 1 km in diameter,23 lies between the orbits of Mars and Jupiter (not to be confused with the much larger Kuiper belt that surrounds the outer Solar System). Among the many asteroids in the asteroid belt is Ceres, a dwarf planet 27% the diameter of the moon that makes up one-third of the asteroid belt’s total mass, and Vesta, the second largest object in the belt after Ceres and the brightest belt object in our night sky.11 I didn’t really know Ceres and Vesta were things. Anyway, Dawn, which was launched in 2007, spent nine months orbiting Vesta in 2011 before heading off to Ceres, where it arrived in March 2015 (making it the first probe to orbit two different bodies).

 There’s another handful of probes out there as well. Like Messenger, which orbited Mercury for seven years until intentionally crashing into it in April 2015; Akatsuki, a Japanese probe that was supposed to start orbiting Venus in 2010 but botched it, and will try again this year; a bunch of probes uneventfully circling the moon, including China’s Chang’e 3, which dropped the first lander on the moon since 1976; and a group of others taking measurements from the sun. Here’s an exhaustive list of all past and present probes, and an awesome National Geographic visualization that sums it all up (click the graphic for a larger view):24


 Looking and Learning Tool #2: Telescopes

 Telescopes have been around since the early 17th century, and as they got more and more powerful over the next 400 years, they became humanity’s primary tool for turning the pages of Where Are We?

 But there came a point when ground telescopes ran into a limit on what they’d be able to see, no matter how advanced they became. You know when you look at a light through a glass of water and the light is all bendy and silly? That’s what’s happening when stars twinkle, except instead of water, we’re looking at them through the Earth’s atmosphere. The atmosphere doesn’t distort light as much as water does, but stars and galaxies are tiny pinpricks of light in our sky, so any level of blur is a big problem—it’s like being underwater in a swimming pool and looking upwards, trying to examine a bunch of birds flying in the sky above.

 In the 1960s, humans gained the ability to put telescopes in space, where they’d show us the first crystal-clear view of the stars in history. In 1990, NASA launched the first truly badass space telescope, the Hubble.1225

 The 13-ton, school bus-length Hubble Space Telescope’s 7.9 foot (2.4 m) lens is accurate enough to shine a laser beam on a dime 200 miles away and powerful enough to see a pair of fireflies in Tokyo from your home in Boston (if the Earth were flat). And in its position in orbit 340 miles above Earth, where there’s no atmosphere or light pollution in the way, the Hubble is on what NASA calls “the ultimate mountaintop.”26 All of this gives the Hubble an unprecedented view of the universe, allowing it to spend the last 25 years sending us the most astounding photographs of things I can’t really believe are real. Like this epic galaxy:27

 Or these two galaxies, which are in the slow process of merging:

 Or the inconceivably huge Pillars of Creation (the left finger is so big, at four light years from top to bottom, that if you started at the knuckle and flew in an airplane upwards, it would take 4.5 million years to get to the fingertip):

 Or the time Hubble aimed its lens at a tiny, seemingly empty square of the sky (seen here next to the moon to show the size of the square):

 And found thousands of galaxies:

 What Hubble and other space telescopes13 have shown us has revealed worlds of new information about where we are and how we got here, expanding our knowledge about everything from dark energy to the origin and age and size of the universe to the number of planets out there like ours that might have life on them.

 For over 40 years now, those two objectives—supporting Earth industries and continuing to learn and discover—have been the extent of our relationship with space.

 And because those two goals are both best accomplished by machine space travelers, the most recent chapter of The Story of Humans and Space has been all about space faring machines, with the human role taking place on or very near Earth, controlling things with joysticks.

 The only reason any humans have gone to space since Apollo 17 returned to Earth in 1972 is that sometimes, the machines aren’t yet advanced enough to do a certain task, so we need to send a human up to do it instead. Of the roughly 550 people who have ever been in space, over 400 of them have gone there in the post-Space Race era. But since Apollo, the reasons have been practical—scientists and technicians going to space to do a job. That’s why each and every manned mission of the past four decades has kept within the thin blanket of space surrounding the Earth—Low Earth Orbit.

 The International Space Station

 Today, the purpose of almost every manned space mission is to take astronauts to and from the International Space Station (ISS). 28

 The ISS is an international collaboration among 16 countries, started in 1998 and constructed over the span of a decade. The space station orbits the Earth in the lowest strip of LEO at an altitude of between 205 and 255 miles (330–410 km14), about the distance across Iceland—close enough to the ground that you can easily see it at night with your naked eye.15 And it’s bigger than people realize, weighing as much as 320 cars and spanning the full length of an American football field:29

 What the Hell Does Anyone Do in the ISS? Blue Box

 As I began working on this post, I realized I didn’t really know what the ISS was for or what anyone did while they were there. Every time I see a video of what goes on inside the space station, it’s just some adult floating around having playtime.

 Conveniently, there’s such a thing as an ISS conference, and it happened to take place last month, in Boston. So I went. The conference was run by the Center for the Advancement of Science in Space (CASIS), which manages the US portion of the ISS. Here’s what I learned at the conference:

 * The ISS is a science laboratory. It’s kind of like other labs, except with the party trick that it’s soaring through space, so it’s the one lab where you can test things in zero gravity (it’s not actually zero gravity—it’s microgravity—something I’ll explain later in the post).
 * What most ISS experiments have in common is that they’re there for the gravity situation, but beyond that, they span a wide range of purposes—everything from learning about osteoporosis as astronauts’ bones atrophy (because they don’t have to fight against gravity), to testing how equipment holds up in space, to analyzing how fluids behave and interact without the influence of any other forces, to using the change in gravity to trick bacteria into revealing which genes make them immune to certain medicines.
 * Astronauts in the ISS have a tight and controlled schedule during the week. At all times, they’re either sleeping (8.5 hours), eating (1.5 hours for breakfast/dinner, 1 hour for lunch) exercising (mandatory 2.5 hours a day), or working on experiments (9 hours a day)—I took this photo of the current schedule of the three astronauts on the ISS.16 Weekends are off, which could not possibly sound more fun—you get to spend the whole time floating around and looking out the window.
 * I’m not the only one who badly wants to play on the ISS—there’s a furiously competitive process to be selected by NASA to go. Thousands apply, 100 are picked for a final round interview and physical examination, and only one or two end up getting the nod. On rare occasion, a private company or individual can buy a spot on the station for a few days, but it costs around $60 million.

 If you want to get a better feel for what it’s like to live on the ISS, here’s a video tour of the space station by a floaty astronaut.

 So far, 216 people have gotten to play on the ISS, from 15 countries:30


 How Stuff Gets to Space

 We’ve gone over what’s in space, but how does all that stuff get to space? Have you ever asked yourself how something like the GPS satellite gets up there in the first place? The answer is that there are nine countries that have the ability to launch something into orbit: Russia, the US, France, Japan, China, India, Israel, Iran and, um, North Korea—along with one non-national entity, the European Space Agency (ESA). If a satellite goes up into space, it’s because someone paid one of those ten entities to bring it there atop a massive, expensive rocket (or because a country is putting one up there for its own uses).

 As for launching humans into space, only three countries in history have done it—Russia, the US, and China (who is a fast-growing newcomer to the space industry). Since the 60s, Russia has used its Soyuz rockets to launch people into space, and the US, after wrapping up the Apollo program in 1972, regained the ability to put people in orbit in 1981 with the Space Shuttle program.31

 Over the next 30 years, the US launched 135 Space Shuttles into LEO, with 133 successes. The two exceptions are fairly traumatizing parts of American history—Challenger in 1986 and Columbia in 2003.

 The Space Shuttle Program retired in 2011. Today, only two countries can launch a human into orbit—Russia and China. With no capability themselves, the US—the country that once triumphantly put a man on the moon while the world watched—now has to launch their astronauts on Russian rockets, at Russia’s whim.

 ___________

 So what are we to make of The Story of Humans and Space? It’s a bit of an odd tale. In 1970, the story looked like this:

 So the assumption about where the story was headed was this:

 But now it’s 2015, and it turns out that this is what was happening:

 When I look at what’s going on with humans and space today, I should think it’s incredible. Just 58 years after the Soviets put the first man-made object into orbit, we now have a swarm of high-tech equipment soaring around our planet, giving humans magical capabilities in vision and communication. There’s a team of flying robot messengers spread out through the Solar System, reporting back to us with their findings. There’s a huge flying telescope high above Earth, showing us exactly what the observable universe looks like. There’s a football field-sized science lab 250 miles above our heads with people in it.

 Everything I just said is amazing.

 And if only The Story of Humans and Space looked like this—

 —I would be marveling at the things we’re currently doing out in The Situation.

 But unfortunately, the 60s happened. So instead, it’s like this:

 A good magic show follows a simple rule—make the act get better as it goes along. If you can’t continue to stay a step ahead of the increasingly-jaded crowd, they’ll quickly tune you out.

 In some areas, the Humans and Space magic show has continued steadily upward. In our quest for knowledge and understanding, for example, we continue to outdo ourselves, learning significantly more about the universe every decade. The human spirit of discovery is alive and well, having thrived in space in the years since Apollo.

 But as fascinated as we are by discovery—as much as we yearn to know all the secrets hidden in the pages of Where Are We?—when it comes to filling us with true excitement and inspiration and getting our adrenaline pumping, discovery doesn’t hold a candle to adventure. Probes and telescopes may fill us with wonder and light up our curiosity, but nothing gets us in our animal core like watching our species go where no man has gone before. And in that arena, the last four decades have left us feeling empty. After watching people land on the moon, following manned missions to and from the ISS is, as Ross Andersen said, “about as thrilling as watching Columbus sail to Ibiza.”

 And that’s why, in today’s world, The Story of Humans and Space has drifted off the front page of our consciousness. The topic that should drop all of us to our knees has become a geeky sideshow. Ask 10 well-educated people you know about what’s going on with Solar System probes or the ISS or NASA or SpaceX and most won’t be able to tell you very much. Some won’t even know that people ever go to space anymore. People don’t know because people don’t care. Because of the way it played out, The Story of Humans and Space feels like a disappointment. And looking at the world around us today, it’s intuitive to predict that future chapters of the space story will continue to putter along as they do today:

 Many people don’t think this is a bad thing. “Why spend exorbitant amounts of money sending people to the far reaches of space when we have so many problems right here on Earth?” they ask. Massachusetts Congressman Barney Frank, who spent three decades playing a key role in US budget decision-making, calls ambitious manned space travel “at best a luxury that the country ought not to be indulging in” and “a complete and total waste of money” and “pure boondoggle.”32 And the dramatic slashes to NASA’s budget since the Space Race ended suggest that Frank isn’t the only US politician to hold this view.

 Upon first assessment, Frank is being perfectly rational—after all, in the face of concerns like healthcare, national security, education, and poverty, should we really make room for an “adventure budget”? And in that light, the graph projection above for The Future of Humans and Space seems all the more likely to continue on its current course.

 I’ve spent the last couple months reading, talking, and thinking almost non-stop about what the coming chapters of this story will look like—and my assumptions about the future have now changed dramatically.

 I think we’re all in for a big surprise.

 Part 2: Musk’s Mission →

 1. For those new to Wait But Why, blue circle footnotes (like this one) are good to click on—they’re for fun facts, extra thoughts, extraneous quotes from my conversations with Musk, and further explanation.↩

 2. It was actually the Germans who had the world’s early lead in rocket technology, but when they lost the war, the Americans, Soviets, and British pillaged Germany’s rocket engineers, with each successfully recruiting a number of them. The US was probably the biggest winner, snagging Wernher von Braun, who would ultimately lead them to their moon landing rocket, the Saturn V.↩

 3. Armstrong was selected to be the first man to walk on the moon, partially because he was known not to have an over-inflated ego. Gus Grissom might have been the front runner for the job, but in 1967, slated to command Apollo 1 on a mission to Low Earth Orbit, he and two other astronauts burned to death when they were trapped in a spacecraft as it caught fire during an on-the-ground test. Stressful.↩

 4. One inadvertent accomplishment during the Apollo 13 debacle was that the spacecraft at one point was farther away from Earth than any of the other Apollo missions, leaving the three Apollo 13 astronauts with the human high altitude record (248,665 miles / 400,187 km) that stands to this day.↩

 5. It’s technically two rotations every sidereal day, which is about 23 hours and 56 minutes, and correlates to the Earth’s rotation with respect to the stars instead of the sun. This annoys me because I don’t get why they would base it on a sidereal day instead of a normal day and I don’t want to spend the 17 minutes it’ll take to find out—if someone knows, please tell me in the comments. It also annoys me because sidereal is just an annoying word.↩

 6. Am I supposed to capitalize GIF? Unclear.↩

 7. The movie Gravity illustrated exactly what sucks about space debris.↩

 8. I’m not sure people realize that before this, we had never actually seen what Pluto looks like—it’s too small and too far away for even our best telescopes to get a decent photo. Before these new images came in, everything that looked like a good photo of Pluto was actually an artist’s rendition. That changed on July 14th.↩

 9. Pluto, discovered in 1930, was originally given planet status, but as we discovered more and more outer Solar System objects, we started to realize that Pluto was just the largest object in the crowded Kuiper belt, and that it kind of made no sense for it to be a planet. If it were alone out there, that would be one thing, but if none of the huge dwarf planets in the asteroid or Kuiper belts were planets (including Pluto’s newly-discovered and almost-as-large neighbor, Eris), then there was no good reason Pluto should randomly be one. So the dramatically nerdy International Astronomical Union got together and, amidst tantrums on both sides, settled on an official definition for a planet: 1) Had to orbit the sun, 2) Had to be big enough to become spherical-ish under its own gravity, 3) Had to have cleared out its own orbit. Pluto failed on #3, since there are many other objects in its orbit, which is part of the Kuiper belt. One other fun fact while we’re here: after Uranus was discovered and named, chemists soon after named a newly-invented element after it—uranium. They did the same thing with neptunian (apparently there’s a neptunium) after Neptune was named, and the newly-named Pluto turned into the naming of the element plutonium.↩

 10. An AU is an “astronomical unit”—the distance from the Earth to the sun—which is about 93 million miles (150 million km).↩

 11. To get a feel for the size of Ceres and Vesta, here’s what they’d look like next to our moon.↩

 12. Awkwardly, after almost 20 years of battling for a Hubble budget and creating the telescope, and after finally having launched a risky and difficult Space Shuttle mission to put it into orbit, NASA received the first Hubble photos, only to find that they were blurry. Turns out the mirror curvature was off by 2.2 thousandth of a millimeter. An almost imperceptible error, but with the vast distances the telescope needed to take in, it was enough to ruin everything. It wasn’t until almost four years later that another Space Shuttle mission was able to get back to the telescope to make a fix. The fix had to be worked out perfectly on Earth first, and the astronauts had to implement it perfectly in space—the mirror shape is so precise that if an astronaut even brushed up against it by accident during the repair process, it would ruin it. Luckily, everything went well and from 1994 on, the Hubble has worked flawlessly.↩

 13. The Hubble is expected to fail at some point not too far from now, maybe before 2020. Without anyone up there to repair system failures since 2009, it’s inevitable—and its orbit will decay slowly until between 2030-2040, when it’s expected to burn up in the Earth’s atmosphere. This is kind of sad—the original plan was to have a Space Shuttle retrieve it and safely return it to Earth, where it could be a celebrity in the Smithsonian. But the Space Shuttle program ended, and now the Hubble will die a horrifying death instead. On the bright side, Hubble has an exciting successor—the James Webb Space Telescope—which is scheduled to be flown into orbit in 2018 and can detect objects that are 10 to 100 times fainter than the best Hubble can do.↩

 14. I’m gettin reallllll sick of this miles (km) thing. But I have no choice because 58% of WBW readers are from the US and kilometers measurements don’t mean much to them, and the other 42% are from the rest of the planet that doesn’t get mile measurements. How dare the US be on this inane system for no apparent reason.↩

 15. Cool video showing what it would look like if the moon orbited at the same altitude as the ISS.↩

 16. Two Russians, one American. The American is Scott Kelly, identical twin of astronaut Mark Kelly, who’s the husband of Congresswoman Gabrielle Giffords. These are the only three humans in space currently—you can see the total “people in space” count at any given time here.↩

 1. Small gray square footnotes are boring and when you click on one of these, you’ll end up bored. They’re for sources and citations.↩

 2. Image: Wikimedia Commons↩

 3. GIF: http://acciolacquer.com/swatches/glam-polish-youre-never-too-old-to-be-young-pt-1/↩

 4. Image: Wikimedia Commons↩

 5. Image: Wikimedia Commons↩

 6. Graph: Wikimedia Commons↩

 7. SIA 2014 Report↩

 8. Graph source: SIA 2014 Report↩

 9. Image: Wikimedia Commons↩

 10. Image: Wikimedia Commons↩

 11. GIF: http://ucresearch.tumblr.com/post/124673707676/blasting-space-junk-with-a-laser-its-getting↩

 12. Image: One of those images that is everywhere and it’s hard to find the original source. Here’s one source for it: http://bizlifes.net/img/2015/07/1437578125_bozhthc.jpg↩

 13. ESA, Astronomy Cafe↩

 14. Astronomy Cafe↩

 15. ERAU Scholarly Commons, The History of Space Debris↩

 16. Image made by Michael Paukner.↩

 17. Probe image sources are hyperlinked when you click the image.↩

 18. Image: NASA↩

 19. Image: Wikimedia Commons.↩

 20. Images: NASA↩

 21. Image: Wikimedia Commons↩

 22. Images: NASA (Uranus, Neptune)↩

 23. Space.com↩

 24. Image: Couldn’t find this on the National Geographic website, but it’s on this random blog so http://cosmicdiary.org/fmarchis/2014/05/19/54_years_of_exploration/↩

 25. Image: Wikimedia Commons↩

 26. Facts and quote: NASA.↩

 27. Hubble Images: NASA↩

 28. Image: Couldn’t find the original source, so I’ll just put someone else who stole the image as the source—Imgur↩

 29. Facts and image: NASA↩

 30. Image: https://www.nasa.gov/mission_pages/station/main/onthestation/facts_and_figures.html↩

 31. Image: How It Works Daily↩

 32. Sources: first quote, second two quotes.↩

 Pages: 1 2 3 4 5
 Tweet

 About Tim Urban

 View all posts by Tim Urban →
 Previous Post
 Next Post

 RECOMMENDED POSTS

 * The Big and the Small

 September 22, 2020

 * A Sick Giant

 January 8, 2020

 * Neuralink and the Brain’s Magical Future

 April 20, 2017
 <!­­-- END TAG ­­-->
 Home Archive

 Join 608,367 other humans and have new posts emailed to you.

 Follow these special men
 412,637
 735,280
 130,240

 * Popular Posts

 * The AI Revolution

 January 22, 2015 1,107

 * Why Procrastinators Procrastinate

 October 30, 2013 566

 * The Fermi Paradox

 May 21, 2014 1,391

 * The Elon Musk Series

 May 7, 2015 511

 * How to Pick a Life Partner

 February 12, 2014 341

 * Why Gen Y Yuppies Are Unhappy

 September 9, 2013 1,197

 * 7 Ways to be Insufferable on Facebook

 July 8, 2013 973

 * Putting Time In Perspective

 August 22, 2013 559

 * How to Name a Baby

 December 11, 2013 327

 * Why You Should Stop Caring What Other People Think

 June 13, 2014 351

 * Religion for the Nonreligious

 October 19, 2014 830

 * 10 Types of 30-Year-Old Single Guys

 October 9, 2013 661

 * The Tail End

 December 11, 2015 303

 Follow these special men
 412,637
 735,280
 130,240

 Popular Posts

 * Hi.
 * Why Going to the Doctor Sucks
 * Mailbag #2
 * The Trump-Biden Debate
 * The Big and the Small
 * You Won’t Believe My Morning
 * A Sick Giant
 * It’s 2020 and you’re in the future
 * Political Disney World
 * Idea Labs and Echo Chambers

 Links

 * Homepage
 * about
 + wait but who
 + faq
 + contact
 * archive
 * minis
 * the shed
 * dinner table
 * store
 + store home
 + new releases
 + posters
 + phone cases
 + cards & wrapping paper
 + squishy things
 + men’s tees
 + women’s tees
 + coffee mugs
 + store support
 * support wbw
 * #7246 (no title)
 * Finn’s Cave
 © WaitButWhy 2023 ContactPartner With Us
 Privacy Policy
 * 
 * 
This is Part 3 of a four-part series on Elon Musk’s companies. For an explanation of why this series is happening and how Musk is involved,   start with Part 1. 
 Pre-Post Note: I started working on this post ten weeks ago. When I started, I never intended for it to become such an ordeal. But like the Tesla post, I decided as I researched that this was A) a supremely important topic that will only become more important in the years to come, and B) something most people don’t know nearly enough about. My weeks of research and discussions with Musk and others built me an in-depth, tree-trunk understanding of what’s happening in what I’m calling The Story of Humans and Space—one that has totally reframed my mental picture of the future (yet again). And as I planned out what to include in the post, I wanted to make sure every Wait But Why reader ended up with the same foundation moving forward—because with everything that’s coming, we’re gonna need it. So like the Tesla post, this post became a full situation. Even the progress updates leading up to its publication became a full situation.
Thanks for your patience. I know you’d prefer this not to be a site that updates every two months, and I would too. The Tesla and SpaceX posts were special cases, and you can expect a return to more normal-length WBW posts now that they’re done.
About the post itself: There are three main parts. Part 1 provides the context and background, Part 2 explores the “Why” part of colonizing Mars, and Part 3 digs into the “How.” To make reading this post as accessible as possible, it’s broken into five pages, each about the length of a normal WBW post, and you can jump to any part of the post easily by clicking the links in the Table of Contents below. We’re also trying two new things, both coming in the next couple days:
 1) PDF and ebook options: We made a fancy PDF of this post for printing and offline viewing (see a preview here), and an ebook containing the whole four-part Elon Musk series:
     
 2) An audio version  . You can find an unabridged audio version of the post, read by me, as well as a discussion about the post between Andrew and me here.
___________
Contents 
  Part 1: The Story of Humans and Space  
  Part 2: Musk’s Mission  
  Part 3: How to Colonize Mars   
→ Phase 1: Figure out how to put things into space 
→  Phase 2: Revolutionize the cost of space travel 
→  Phase 3: Colonize Mars 
  A SpaceX Future  
  
2365 AD, Ganymede 
 One more day until departure. It was so surreal to picture actually being there that she still didn’t really believe it would happen. All those things she had always heard about—buildings that were constructed hundreds of years before the first human set foot on Ganymede; animals the size of a house; oceans the size of her whole world; tropical beaches; the famous blue sky; the giant sun that’s so close it can burn your skin; and the weirdest part—no Jupiter hovering overhead. Having seen it all in so many movies, she felt like she was going to visit a legendary movie set. It was too much to think about all at once. For now, she just had to focus on making sure she had everything she needed and saying goodbye to everyone—it would be a long time before she would see them again…
___________
   Part 1: The Story of Humans and Space 
About six million years ago, a very important female great ape had two children. One of her children would go on to become the common ancestor of all chimpanzees. The other would give birth to a line that would one day include the entire human race. While the descendants of her first child would end up being pretty normal and monkey-ish, as time passed, strange things began to happen with the lineage of the other.1 1  ← click these  
We’re not quite sure why, but over the next six million years, our ancestral line started to do something no creatures on Earth had ever done before—they woke up.
It happened slowly and gradually through the thousands of generations the same way your brain slowly comes to in the first few seconds after you rouse from sleep. But as the clarity increased, our ancestors started to look around and, for the very first time, wonder.
Emerging from a 3.6-billion-year dream, life on Earth had its first questions.
 What is this big room we’re in, and who put us here? What is that bright yellow circle on the ceiling and where does it go every night? Where does the ocean end and what happens when you get there? Where are all the dead people now that they’re not here anymore?  
We had discovered our species’ great mystery novel—Where Are We?—and we wanted to learn how to read it.
As the light of human consciousness grew brighter and brighter, we began to arrive at answers that seemed to make sense. Maybe we were on top of a floating disk, and maybe that disk was on top of a huge turtle. Maybe the pinpricks of light above us at night are a glimpse into what lies beyond this big room—and maybe that’s where we go when we die. Maybe if we can find the place where the ceiling meets the floor, we can poke our heads through and see all the super fun stuff on the other side.2
    
Around 10,000 years ago, isolated tribes of humans began to merge together and form the first cities. In larger communities, people were able to talk to each other about this mystery novel we had found, comparing notes across tribes and through the generations. As the techniques for learning became more sophisticated and the clues piled up, new discoveries surfaced.
The world was apparently a ball, not a disk. Which meant that the ceiling was actually a larger sphere surrounding us. The sizes of the other objects floating out there in the sphere with us, and the distances between them, were vaster than we had ever imagined. And then, something upsetting:
The sun wasn’t revolving around us. We were revolving around the sun.
This was a super unwarm, unfuzzy discovery. Why the hell weren’t we in the center of things? What did that mean?
 Where are we? 
The sphere was already unpleasantly big—if we weren’t in the center of it, were we just on a random ball inside of it, kind of for no apparent reason? Could this really be what was happening?
   
Scary.
Then things got worse.
It seemed that the pinpricks of light on the edge of the sphere weren’t what we thought they were—they were other suns like ours. And they were out there floating just like our sun—which means we weren’t inside of a sphere at all. Not only was our planet not the center of things, even our sun was just a random dude out there, in the middle of nowhere, surrounded by nothingness.
   
 Scary. 
Our sun turned out to be a little piece of something much bigger. A beautiful, vast cloud of billions of suns. The everything of everything.
   
At least we had that. Until we realized that it wasn’t everything, it was this:
   
Darkness.
The better our tools and understanding became, the more we could zoom out, and the more we zoomed out, the more things sucked. We were deciphering the pages of Where Are We? at our own peril, and we had deciphered our way right into the knowledge that we’re unbelievably alone, living on a lonely island inside a lonely island inside a lonely island, buried in layers of isolation, with no one to talk to.
That’s our situation.
In the most recent 1% of our species’ short existence, we have become the first life on Earth to know about the Situation—and we’ve been having a collective existential crisis ever since.
You really can’t blame us. Imagine not realizing that the universe is a thing and then realizing the universe is a thing. It’s a lot to take in.
Most of us handle it by living in a pleasant delusion, pretending that the only place we live is in an endless land of colors and warmth. We’re like this guy, who’s doing everything he possibly can to ignore the Situation:3
    
And our best friend for this activity? The clear blue sky. The blue sky seems like it was invented to help humans pretend the Situation doesn’t exist, serving as the perfect whimsical backdrop to shield us from reality.
Then nighttime happens, and there’s the Situation, staring us right in the face.
   
 Oh yeah… 
This la-di-da → oh yeah… → la-di-da → oh yeah… merry-go-round of psychosis was, for most of recent history, the extent of our relationship with the Situation.
But in the last 60 years, that relationship has vaulted to a whole new level. During World War II, missile technology leapt forward,2 and for the first time, a new, mind-blowing concept was possible—
 Space travel. 
For thousands of years, The Story of Humans and Space had been the story of staring out and wondering. The possibility of people leaving our Earth island and venturing out into space burst open the human spirit of adventure.
I imagine a similar feeling in the people of the 15th century, during the Age of Discovery, when we were working our way through the world map chapter of Where Are We? and the notion of cross-ocean voyages dazzled people’s imaginations. If you asked a child in 1495 what they wanted to be when they grew up, “an ocean explorer” would probably have been a common response.
In 1970, if you asked a child the same question, the answer would be, “an astronaut”—i.e. a Situation explorer.
WWII advanced the possibility of human space travel, but it was in late 1957, when the Soviets launched the first man-made object into orbit, the adorable Sputnik 1, that space travel became the defining quest of the world’s great powers.
At the time, the Cold War was in full throttle, and the US and Soviets had their measuring sticks out for an internationally-televised penis-measuring contest. With the successful launch of Sputnik, the Soviet penis bolted out by a few centimeters, horrifying the Americans.
   
To the Soviets, putting a satellite into space before the US was proof that Soviet technology was superior to American technology, which in turn was put forward as proof, for all the world to see, that communism was a system superior to capitalism.
Eight months later, NASA was born.
The Space Race had begun, and NASA’s first order of business would be to get a man into space, and then a man into full orbit, preferably both before the Soviets. The US was not to be shown up again.
In 1959, NASA launched Project Mercury to carry out the mission. They were on the verge of success when in April of 1961, the Soviets launched Yuri Gagarin into a full orbit around the Earth, making the first human in space and in orbit a Soviet.
   
It was time for drastic measures. John F. Kennedy’s advisors told him that the Soviets had too big a lead for the US to beat them at any near-term achievements—but that the prospect of a manned moon landing was far enough in the future that the US had a fighting chance to get there first. So Kennedy gave his famous “we choose to go to the moon, not because it is easy, but because it is hahhd” speech, and directed an outrageous amount of funding at the mission ($20 billion, or $205 billion in today’s dollars).
The result was Project Apollo. Apollo’s mission was to land an American on the moon—and to do it first. The Soviets answered with Soyuz, their own moon program, and the race was on.
As the early phases of Apollo started coming together, Project Mercury finally hit its stride. Just a month after Yuri Gagarin became the first man in space, American astronaut Alan Shepard became the second man in space, completing a little arc that didn’t put him in full orbit but allowed him to give space a high-five at the top of the arc. A few months later, in February of 1962, John Glenn became the first American to orbit the Earth.
The next seven years saw 22 US and Soviet manned launches as the superpowers honed their skills and technology. By late 1968, the furiously-sprinting US had more total launches under their belt (17) than the Soviets (10), and together, the two nations had mastered what we call Low Earth Orbit (LEO).
   
But LEO hadn’t really excited anyone since the early ’60s. Both powers had their sights firmly set on the moon. The Apollo program was making quick leaps, and in December of 1968, the US became the first nation to soar outside of LEO. Apollo 8 made it all the way to the moon’s orbit and circled around 10 times before returning home safely. The crew, which included James Lovell (who a few months later played the role of Tom Hanks on the Apollo 13 mission), shattered the human altitude record and became the first people to see the moon up close, the first to see the “dark” side of the moon, and the first to see the Earth as a whole planet, snapping this iconic photo:4
    
Upon return, the crew became America’s most celebrated heroes—which I hope they enjoyed for eight months. Three Apollo missions later, in July of 1969, Apollo 11 made Americans Neil Armstrong3 and Buzz Aldrin the first humans on the moon, and Armstrong took this famous photo of Aldrin looking all puffy:5
    
It’s hard to fully emphasize what a big deal this was. Ever since life on Earth began 3.6 billion years ago, no earthly creature had set foot on any celestial body other than the Earth. Suddenly, there are Armstrong and Aldrin, bouncing around another sphere, looking up in the sky where the moon is supposed to be and seeing the Earth instead. Insane.
Project Apollo proved to be a smashing success. Not only did Apollo get a man on the moon before the Soviets, the program sent 10 more men to the moon over the next 3.5 years on five other Apollo missions. There were six successful moon trips in seven tries, with the famous exception being Apollo 13, which was safely aborted after an explosion in the oxygen tank.4
 The Soviet Soyuz program kept running into technical problems, and it never ended up putting someone on the moon.
The final Apollo moonwalk took place in late 1972. In only one decade, we had conquered nearby space, and progress was accelerating. If at that time you had asked any American, or any other human, what the coming decades of space travel would bring, they’d have made big, bold predictions. Many more people on the moon, a permanent moon base, people on Mars, and beyond.
So you can only imagine how surprised they’d be if you told them in 1972, after just watching 12 humans walk on the moon, that 43 years later, in the impossibly futuristic-sounding year 2015, the number of people to set foot on the moon would still be 12. Or that after leaving Low Earth Orbit in the dust years earlier and using it now as our pre-moon trip parking lot, 2015 would roll around and LEO would be the farthest out humans would ever go.
1972 people would be blown away by our smart phones and our internet, but they’d be just as shocked that we gave up on pushing our boundaries in space.
So what happened? After such a wildly exciting decade of human space adventure, why did we just stop? 
Well, like we found in the Tesla post, “Why did we stop?” is the wrong question. Instead, we should ask:
 Why were we ever adventurous about sending humans into space in the first place? 
Space travel is unbelievably expensive. National budgets are incredibly tight. The fact is, it’s kind of surprising that a nation ever ponied up a sizable chunk of its budget for the sake of adventure and inspiration and pushing our boundaries.
And that’s actually because no nation did blow their budget for the sake of adventure and inspiration and pushing our boundaries—two nations blew their budgets because of a penis–length contest. In the face of international embarrassment at a time when everyone was trying to figure out whose economic system was better, the US government agreed to drop the usual rules for a few years to pour whatever resources were necessary on the problem to make sure they won that argument—
   
And once they won it, the contest was over and so were the special rules. And the US went back to spending money like a normal person.6
    
Instead of continuing to push the limits at all costs, the US and the Soviets got a grip, put their pants back on, shook hands, and started working together like adults on far more practical projects, like setting up a joint space station in LEO.
In the four decades since then, the Story of Humans and Space has again become confined to Earth, where we find ourselves with two primary reasons to interact with space (Note: the next whole chunk of the post is a slight diversion for an overview on satellites, space probes, and space telescopes. If that doesn’t excite you, I won’t be hurt if you skip down to the International Space Station section):
 1) Support for Earth Industries 
The first and primary reason humans have interacted with space since the Apollo program isn’t about human interest in space. It’s about using space for practical purposes in support of industries on Earth—mostly in the form of satellites. The bulk of today’s rocket launches into space are simply putting things into LEO whose purpose is to look back down at Earth, not to the great expanses in the other direction.
Here’s a little satellite overview:
 Satellites Blue Box 
We don’t think about them that often, but above us are hundreds of flying robots that play a large part in our lives on Earth. In 1957, lonely Sputnik circled the Earth by itself, but today, the worlds of communication, weather forecasting, television, navigation, and aerial photography all rely heavily on satellites, as do many national militaries and government intelligence agencies.
The total market for satellite manufacturing, the launches that carry them to space, and related equipment and services has ballooned from $60 billion in 2004 to over $200 billion in 2015. Satellite industry revenue today makes up only 4% of the global telecommunications industry but accounts for over 60% of space industry revenue.7
 Here’s how the world’s satellites break down by role (in 2013):8 
   
Of the 1,265 active satellites in orbit at the beginning of 2015, the US owns by far the largest number at 528—over 40% of the total—but over 50 countries own at least one orbiting satellite.
As for where all of these satellites are, most of them fall into two distinct “layers” of space:
About two-thirds of active satellites are in Low Earth Orbit. LEO starts up at 99 miles (160 km) above the Earth, the lowest altitude at which an object can orbit without atmospheric drag messing things up. The top of LEO is 1,240 miles (2,000 km) up. Typically, the lowest satellites are at around 220 miles (350 km) up or higher.
Most of the rest (about one-third) of the satellites are much farther out, in a place called geostationary orbit (GEO). It’s right at 22,236 miles (35,786 km) above the Earth, and it’s called geostationary because something orbiting in it rotates at the exact speed that the Earth turns, making its position in the sky stationary relative to a point on the Earth. It’ll seem to be motionless to an observer on the ground.9
    
GEO is ideal for something like a TV satellite because a dish on the Earth can aim at the same fixed spot all the time.
A small percentage of other satellites are in medium Earth orbit (MEO), which is everything in between LEO and GEO. One notable resident of MEO is the GPS system that most Americans, and people from many other countries, use every day. I never realized that the entire GPS system, a US Department of Defense project that went live in 1995, only uses 32 satellites total. And until 2012, the number was only 24—six orbits, each with four satellites. But you can see in the GIF below that even with 24, a given point on the Earth can be seen by at least six of the satellites at any given time, and usually it’s nine or higher (in the GIF, the blue dot on the Earth is a hypothetical person on the ground, and whichever satellites can see him at a given time are blue, with the green lines showing their line of sight to the person):10
    
This is why your phone’s map can still show your location even when you’re somewhere with no cellular service—because it has nothing to do with cellular service. The system is also set up to be redundant—only four satellites need to simultaneously see you in order for the system to pinpoint your location. GPS satellites have an orbital period of about 12 hours, making two full rotations of the Earth each day.5
 You can see satellite locations using Google Earth (here’s a cool video of Google Earth showing the satellites).
 Space Debris Bluer Box 
There’s a big problem happening in the world of satellites. In addition to the 1,265 active satellites up in orbit, there are thousands more inactive satellites, as well as a bunch of spent rockets from previous missions. And once in a while, one of them explodes, or two of them collide, creating a ton of tiny fragments called space debris. The number of objects in space has risen quickly over recent decades, as a GIF6 made by the ESA shows (with exaggerated-sized objects relative to the Earth’s size):11
    
The majority of satellites and debris are bunched around the Earth in LEO, and the outer ring of objects is what’s located in GEO.
Earth space agencies track about 17,000 objects in space, only 7% of which are active satellites. Here’s a map showing every known object in space today.12
    
But the crazy thing is they only track the large objects, and that’s what we’re seeing in that image. Estimates for the number of smaller debris objects (1 – 10 cm) range from 150,000 to 500,000, and there are over a million total pieces of debris larger than 2 mm.13
 The issue is that at the incredible speeds at which space objects move (most LEO objects zip along at over 17,000 mph), a collision with even a tiny object can cause devastating damage to an active satellite or spacecraft. An object of only 1 cm at those speeds will cause the same damage in a collision as a small hand grenade.7 14
 Over a third of all space debris originated from just two events: China’s 2007 anti-satellite test, when China shat on the world’s face by intentionally blowing up one of its own satellites, creating 3,000 new pieces of debris large enough to be trackable, and a 2009 collision between two satellites that exploded into 2,000 debris chunks.15 Each collision increases the amount of debris, which in turn increases the likelihood of more collisions, and there’s danger of a domino effect situation, which scientists call the Kessler Syndrome. A bunch of parties are proposing ways to mitigate the amount of debris in LEO—everything from harpooning the debris to laser blasting it to intercepting it with a cloud of gas.
Here’s a chart that sums up each nation’s “space footprint,” showing the quantity of active satellites, inactive satellites, and space debris caused by each country:16
    
There are a few other space activities in the “Support for Earth Industries” category of human/space interaction—like space mining, space burial, and space tourism—but at least for now, satellites account for almost the entire category.
 2) Looking and Learning 
The second reason humans have interacted with space in the last four decades proves that while we may have stopped sending people into The Situation, we never lost our hunger to learn about what’s out there. As society moved on from space and turned its attention elsewhere, astronomers have kept busy at work deciphering their way through page after page of the old mystery novel, Where Are We?   
Astronomers learn best with their eyes, and a side effect of the Space Race was the development of far better technology for seeing what’s out there. There are two high-tech ways modern astronomers see things:
 Looking and Learning Tool #1: Sending probes around the Solar System 
Basically, scientists fire a fancy robot toward some distant planet, moon, or asteroid, and the robot spends months or years flying through space, bored, until it finally arrives. Then, depending on the plan, it either just flies by the object, taking some pictures on the way, orbits the object to get more detailed information, or lands on the object for a full inspection. Everything it learns, it sends back to us, and one day, when its job is done, we either kill the probe by crashing it into the object or let it just fly out into deep space to be depressed.
I often use myself as a litmus test for what the public probably knows about or doesn’t know about. As I’ve mentioned before on this blog, I’ve been seriously dating astronomy ever since I was three years old—so if I don’t know something going on in the world of space, I assume that most people don’t. And when it comes to space probes, I’ve felt pretty disoriented. Are there 200 of them flying around out there? 50? 9? Why are they out there, who sent them, and what are they doing? All I’d know is that sometimes there would be a random story about some probe sending back stunning pictures—I’d open the cnn.com gallery, click through them, be thrilled for a second, send the link to the three friends of mine who are also dating astronomy, and then try to close the page but instead see some trashy CNN clickbait headline on the side of the page, click that, and ruin my life for the next three hateful hours. That’s my relationship with humanity’s space probes.
But in researching this post, I quickly realized there’s not that much to know, and it doesn’t take too big an effort to get fully oriented. Here are what I consider the eight key space robots to know about right now:17
  1) New Horizons (Pluto, NASA)
   
New Horizons goes first because its big moment just happened. Launched in 2006 on a decade-long trip to Pluto (sped up on its way by a Jupiter fly-by in 2007 that gravity-zinged it to a much faster speed), New Horizons finally reached Pluto on July 14th, 2015. It didn’t land on Pluto, but it flew very near to it and showed us Pluto for the first time:8 18
    
Next, New Horizons will be on its way further outwards into the Kuiper belt to send back images of comets and dwarf planets. You can track New Horizons’ location here.
Awkwardly, Pluto was still a planet when New Horizons launched, and everyone spent the years following Pluto’s demotion avoiding making eye contact with the New Horizons team. While I agree with the common sentiment that it’s sad that Pluto’s sad about its demotion,9 the truth is, Pluto should probably appreciate that it got away with 76 illegitimate years as a planet celebrity, pulling in a ton of Kuiper belt ass in the process, given that fellow Kuiper belt dwarf planet Eris spent that whole time living its life in total obscurity, only discovered in 2005.
 2) Curiosity  (Mars, NASA)
   
Curiosity is a now-famous rover. A car-sized lovable lander robot dropped down on Mars’s surface in 2012, Curiosity is studying a bunch of things inside a large crater, with its primary objective being to figure out if there’s ever been life on Mars. The last two Mars rovers, Opportunity and Spirit, landed in 2004 with a planned mission of 90 days. Both lasted way past their expiry date, and Opportunity is still active. Such a good boy.
There are a bunch of other probes orbiting around Mars as well, but Curiosity is the main event there.
In my research, I came across this video from an IMAX movie about getting the rover Spirit from Earth to the surface of Mars and thought it was the coolest video ever. Until I found this video about getting Curiosity on Mars, which was even cooler.
 3) Juno (Jupiter, NASA)
   
Juno left Earth in 2011, made a big loop and came back to Earth in 2013 to get a gravity zing (during which it captured a cool video of the moon circling the Earth), and is now on its way to Jupiter, where it’ll arrive in July of 2016.19
    
Once it arrives, Juno will orbit Jupiter, taking pictures and using sensors to try to figure out what’s going on in there underneath all the succulent-looking cloud tops. It’ll die by falling into Jupiter, hopefully snapping and relaying some quick photos of what it looks like inside Jupiter’s atmosphere before burning up so that someone can make a virtual reality video that lets you descend into Jupiter’s surface.
 4) Cassini (Saturn, NASA / European Space Agency / Italian Space Agency collaboration)
   
Launched in 1997, Cassini set off towards Saturn, the only planet in the Solar System who decided it was okay to wear a tutu. Reaching Saturn in 2004, Cassini became the first probe in history to orbit the planet, sending back some jaw-dropping pictures, like this one:20
    
And this one:
   
And this close-up of the rings:
   
And this absurdly cool picture of Saturn with the sun behind it:
   
In 2005, Cassini dropped its attached lander, the upsettingly-named Huygens, down onto the largest of Saturn’s moons, Titan. Here’s a real image of the surface of Titan, taken by Huygens (it’s creepily fascinating seeing the actual surface of something as far away and mysterious as a Saturn moon):21
    
 5 and 6) Voyager 1 and 2 (Jupiter, Saturn, Uranus, Neptune; NASA)
   
Launched in 1977, the two Voyager probes were the first probes to collect images of the four outer giants of the Solar System. Voyager 2 is still the only probe to visit Uranus and Neptune, taking these eerie photos of the two, respectively:22
    
The cool thing about the Voyagers is that even though their original missions are now long over, they’re still zooming outward. They’re both ridiculously far away now and going super fast. Voyager 1 is the faster of the two, going 38,000 mph (61,000 km/h)—so fast that it would cross the Atlantic Ocean in five minutes—and it’s the farthest man-made object from Earth, currently 131 AU10 away from Earth. It was also the first man-made object to leave the Solar System. At this rate, Voyager 1 will reach Proxima Centauri, the closest star to us, in about 73,000 years.
Another cool thing about the Voyagers is that before they launched, a NASA committee, led by Carl Sagan, loaded them each up with a time capsule, full of symbols, sounds, and images of Earth (and symbol instructions about how to play and view the media), so the probes can one day tell aliens what our deal is. Probably a waste of everyone’s time, but who knows.
 7) Rosetta (comet, ESA)
   
Launched in 2004, Rosetta got a lot of attention last year when it reached comet 67P in August 2014 and successfully dropped its little lander, Philae, onto the comet a couple months later. Comet 67P turned out to kind of just be a big rock (2.7 mi/4.3 km long), but the images taken by Rosetta were cool:
   
 8) Dawn  (Vesta and Ceres, NASA)
   
Dawn can’t believe it made the cut on this list. The reason I included it is that I’m not sure people realize that there are huge, almost planet-size objects in the asteroid belt. The asteroid belt, a huge ring of millions of asteroids, including over 750,000 that are at least 1 km in diameter,23 lies between the orbits of Mars and Jupiter (not to be confused with the much larger Kuiper belt that surrounds the outer Solar System). Among the many asteroids in the asteroid belt is Ceres, a dwarf planet 27% the diameter of the moon that makes up one-third of the asteroid belt’s total mass, and Vesta, the second largest object in the belt after Ceres and the brightest belt object in our night sky.11 I didn’t really know Ceres and Vesta were things. Anyway, Dawn, which was launched in 2007, spent nine months orbiting Vesta in 2011 before heading off to Ceres, where it arrived in March 2015 (making it the first probe to orbit two different bodies).
There’s another handful of probes out there as well. Like Messenger, which orbited Mercury for seven years until intentionally crashing into it in April 2015; Akatsuki, a Japanese probe that was supposed to start orbiting Venus in 2010 but botched it, and will try again this year; a bunch of probes uneventfully circling the moon, including China’s Chang’e 3, which dropped the first lander on the moon since 1976; and a group of others taking measurements from the sun. Here’s an exhaustive list of all past and present probes, and an awesome National Geographic visualization that sums it all up (click the graphic for a larger view):24
    
  
Looking and Learning Tool #2: Telescopes 
Telescopes have been around since the early 17th century, and as they got more and more powerful over the next 400 years, they became humanity’s primary tool for turning the pages of Where Are We? 
But there came a point when ground telescopes ran into a limit on what they’d be able to see, no matter how advanced they became. You know when you look at a light through a glass of water and the light is all bendy and silly? That’s what’s happening when stars twinkle, except instead of water, we’re looking at them through the Earth’s atmosphere. The atmosphere doesn’t distort light as much as water does, but stars and galaxies are tiny pinpricks of light in our sky, so any level of blur is a big problem—it’s like being underwater in a swimming pool and looking upwards, trying to examine a bunch of birds flying in the sky above.
In the 1960s, humans gained the ability to put telescopes in space, where they’d show us the first crystal-clear view of the stars in history. In 1990, NASA launched the first truly badass space telescope, the Hubble.12 25
    
The 13-ton, school bus-length Hubble Space Telescope’s 7.9 foot (2.4 m) lens is accurate enough to shine a laser beam on a dime 200 miles away and powerful enough to see a pair of fireflies in Tokyo from your home in Boston (if the Earth were flat). And in its position in orbit 340 miles above Earth, where there’s no atmosphere or light pollution in the way, the Hubble is on what NASA calls “the ultimate mountaintop.”26 All of this gives the Hubble an unprecedented view of the universe, allowing it to spend the last 25 years sending us the most astounding photographs of things I can’t really believe are real. Like this epic galaxy:27
    
Or these two galaxies, which are in the slow process of merging:
   
Or the inconceivably huge Pillars of Creation (the left finger is so big, at four light years from top to bottom, that if you started at the knuckle and flew in an airplane upwards, it would take 4.5 million years to get to the fingertip):
   
Or the time Hubble aimed its lens at a tiny, seemingly empty square of the sky (seen here next to the moon to show the size of the square):
   
And found thousands of galaxies:
   
What Hubble and other space telescopes13 have shown us has revealed worlds of new information about where we are and how we got here, expanding our knowledge about everything from dark energy to the origin and age and size of the universe to the number of planets out there like ours that might have life on them.
For over 40 years now, those two objectives—supporting Earth industries and continuing to learn and discover—have been the extent of our relationship with space.
And because those two goals are both best accomplished by machine space travelers, the most recent chapter of The Story of Humans and Space has been all about space faring machines, with the human role taking place on or very near Earth, controlling things with joysticks.
The only reason any humans have gone to space since Apollo 17 returned to Earth in 1972 is that sometimes, the machines aren’t yet advanced enough to do a certain task, so we need to send a human up to do it instead. Of the roughly 550 people who have ever been in space, over 400 of them have gone there in the post-Space Race era. But since Apollo, the reasons have been practical—scientists and technicians going to space to do a job. That’s why each and every manned mission of the past four decades has kept within the thin blanket of space surrounding the Earth—Low Earth Orbit.
 The International Space Station 
Today, the purpose of almost every manned space mission is to take astronauts to and from the International Space Station (ISS). 28
    
The ISS is an international collaboration among 16 countries, started in 1998 and constructed over the span of a decade. The space station orbits the Earth in the lowest strip of LEO at an altitude of between 205 and 255 miles (330–410 km14), about the distance across Iceland—close enough to the ground that you can easily see it at night with your naked eye.15 And it’s bigger than people realize, weighing as much as 320 cars and spanning the full length of an American football field:29
    
 What the Hell Does Anyone Do in the ISS? Blue Box 
As I began working on this post, I realized I didn’t really know what the ISS was for or what anyone did while they were there. Every time I see a video of what goes on inside the space station, it’s just some adult floating around having playtime.
Conveniently, there’s such a thing as an ISS conference, and it happened to take place last month, in Boston. So I went. The conference was run by the Center for the Advancement of Science in Space (CASIS), which manages the US portion of the ISS. Here’s what I learned at the conference:
The ISS is a science laboratory. It’s kind of like other labs, except with the party trick that it’s soaring through space, so it’s the one lab where you can test things in zero gravity (it’s not actually zero gravity—it’s microgravity—something I’ll explain later in the post).
What most ISS experiments have in common is that they’re there for the gravity situation, but beyond that, they span a wide range of purposes—everything from learning about osteoporosis as astronauts’ bones atrophy (because they don’t have to fight against gravity), to testing how equipment holds up in space, to analyzing how fluids behave and interact without the influence of any other forces, to using the change in gravity to trick bacteria into revealing which genes make them immune to certain medicines.
Astronauts in the ISS have a tight and controlled schedule during the week. At all times, they’re either sleeping (8.5 hours), eating (1.5 hours for breakfast/dinner, 1 hour for lunch) exercising (mandatory 2.5 hours a day), or working on experiments (9 hours a day)—I took this photo of the current schedule of the three astronauts on the ISS.16 Weekends are off, which could not possibly sound more fun—you get to spend the whole time floating around and looking out the window.
I’m not the only one who badly wants to play on the ISS—there’s a furiously competitive process to be selected by NASA to go. Thousands apply, 100 are picked for a final round interview and physical examination, and only one or two end up getting the nod. On rare occasion, a private company or individual can buy a spot on the station for a few days, but it costs around $60 million.
If you want to get a better feel for what it’s like to live on the ISS, here’s a video tour of the space station by a floaty astronaut.
So far, 216 people have gotten to play on the ISS, from 15 countries:30
    
  
How Stuff Gets to Space
 
We’ve gone over what’s in space, but how does all that stuff get to space? Have you ever asked yourself how something like the GPS satellite gets up there in the first place? The answer is that there are nine countries that have the ability to launch something into orbit: Russia, the US, France, Japan, China, India, Israel, Iran and, um, North Korea—along with one non-national entity, the European Space Agency (ESA). If a satellite goes up into space, it’s because someone paid one of those ten entities to bring it there atop a massive, expensive rocket (or because a country is putting one up there for its own uses).
As for launching humans into space, only three countries in history have done it—Russia, the US, and China (who is a fast-growing newcomer to the space industry). Since the 60s, Russia has used its Soyuz rockets to launch people into space, and the US, after wrapping up the Apollo program in 1972, regained the ability to put people in orbit in 1981 with the Space Shuttle program.31
    
Over the next 30 years, the US launched 135 Space Shuttles into LEO, with 133 successes. The two exceptions are fairly traumatizing parts of American history— Challenger in 1986 and  Columbia in 2003.
The Space Shuttle Program retired in 2011. Today, only two countries can launch a human into orbit—Russia and China. With no capability themselves, the US—the country that once triumphantly put a man on the moon while the world watched—now has to launch their astronauts on Russian rockets, at Russia’s whim.
___________
So what are we to make of The Story of Humans and Space? It’s a bit of an odd tale. In 1970, the story looked like this:
   
So the assumption about where the story was headed was this:
   
But now it’s 2015, and it turns out that this is what was happening:
   
When I look at what’s going on with humans and space today, I should think it’s incredible. Just 58 years after the Soviets put the first man-made object into orbit, we now have a swarm of high-tech equipment soaring around our planet, giving humans magical capabilities in vision and communication. There’s a team of flying robot messengers spread out through the Solar System, reporting back to us with their findings. There’s a huge flying telescope high above Earth, showing us exactly what the observable universe looks like. There’s a football field-sized science lab 250 miles above our heads with people in it.
Everything I just said is amazing.
And if only The Story of Humans and Space looked like this—
   
—I would be marveling at the things we’re currently doing out in The Situation.
But unfortunately, the 60s happened. So instead, it’s like this:
                 
   
     
A good magic show follows a simple rule—make the act get better as it goes along. If you can’t continue to stay a step ahead of the increasingly-jaded crowd, they’ll quickly tune you out.
In some areas, the Humans and Space magic show has continued steadily upward. In our quest for knowledge and understanding, for example, we continue to outdo ourselves, learning significantly more about the universe every decade. The human spirit of discovery is alive and well, having thrived in space in the years since Apollo.
But as fascinated as we are by discovery—as much as we yearn to know all the secrets hidden in the pages of Where Are We?—when it comes to filling us with true excitement and inspiration and getting our adrenaline pumping, discovery doesn’t hold a candle to adventure. Probes and telescopes may fill us with wonder and light up our curiosity, but nothing gets us in our animal core like watching our species go where no man has gone before. And in that arena, the last four decades have left us feeling empty. After watching people land on the moon, following manned missions to and from the ISS is, as Ross Andersen said, “about as thrilling as watching Columbus sail to Ibiza.”
And that’s why, in today’s world, The Story of Humans and Space has drifted off the front page of our consciousness. The topic that should drop all of us to our knees has become a geeky sideshow. Ask 10 well-educated people you know about what’s going on with Solar System probes or the ISS or NASA or SpaceX and most won’t be able to tell you very much. Some won’t even know that people ever go to space anymore. People don’t know because people don’t care. Because of the way it played out, The Story of Humans and Space feels like a disappointment. And looking at the world around us today, it’s intuitive to predict that future chapters of the space story will continue to putter along as they do today:
   
Many people don’t think this is a bad thing. “Why spend exorbitant amounts of money sending people to the far reaches of space when we have so many problems right here on Earth?” they ask. Massachusetts Congressman Barney Frank, who spent three decades playing a key role in US budget decision-making, calls ambitious manned space travel “at best a luxury that the country ought not to be indulging in” and “a complete and total waste of money” and “pure boondoggle.”32 And the dramatic slashes to NASA’s budget since the Space Race ended suggest that Frank isn’t the only US politician to hold this view.
Upon first assessment, Frank is being perfectly rational—after all, in the face of concerns like healthcare, national security, education, and poverty, should we really make room for an “adventure budget”? And in that light, the graph projection above for The Future of Humans and Space seems all the more likely to continue on its current course.
I’ve spent the last couple months reading, talking, and thinking almost non-stop about what the coming chapters of this story will look like—and my assumptions about the future have now changed dramatically.
I think we’re all in for a big surprise.
 Part 2: Musk’s Mission →
How (and Why) SpaceX Will Colonize Mars — Wait But Why
https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html
https://distill.pub/2021/understanding-gnns/
Understanding Convolutions on Graphs
2021-09-02
Ameya Daigavane; Balaraman Ravindran; Gaurav Aggarwal; Daigavane; Ameya; Ravindran; Balaraman; Aggarwal; Gaurav
<div><div>
 
 <nav>
 <h3>Contents</h3>
 
 
 <ul>
 <li><a href="#lack-of-consistent-structure">Lack of Consistent Structure</a></li>
 <li><a href="#node-order">Node-Order Equivariance</a></li>
 <li><a href="#scalability">Scalability</a></li>
 </ul>
 
 
 
 
 
 
 <ul>
 <li><a href="#spectral">Spectral Convolutions</a></li>
 <li><a href="#graph-embeddings">Global Propagation via Graph Embeddings</a></li>
 </ul>
 
 
 
 <ul>
 <li><a href="#practical-techniques">GNNs in Practice</a></li>
 <li><a href="#different-kinds-of-graphs">Different Kinds of Graphs</a></li>
 <li><a href="#pooling">Pooling</a></li>
 </ul>
 
 <ul>
 <li><a href="#experiments-notebooks">Reproducing Experiments</a></li>
 <li><a href="#visualizations-notebooks">Recreating Visualizations</a></li>
 </ul>
 </nav>
 
 <div>
 <p><em>
 This article is one of two Distill publications about graph neural networks.
 Take a look at
 <a href="https://distill.pub/2021/gnn-intro/">A Gentle Introduction to Graph Neural Networks</a>
 
 for a companion view on many things graph and neural network related.
 </em></p>
 <p>
 Many systems and interactions - social networks, molecules, organizations, citations, physical models, transactions - can be represented quite naturally as graphs.
 How can we reason about and make predictions within these systems?
 </p>
 <p>
 One idea is to look at tools that have worked well in other domains: neural networks have shown immense predictive power in a variety of learning tasks.
 However, neural networks have been traditionally used to operate on fixed-size and/or regular-structured inputs (such as sentences, images and video).
 This makes them unable to elegantly process graph-structured data.
 </p>
 <figure>
 
 </figure>
 </div>
 
 <p>
 Graph neural networks (GNNs) are a family of neural networks that can operate naturally on graph-structured data. 
 By extracting and utilizing features from the underlying graph,
 GNNs can make more informed predictions about entities in these interactions,
 as compared to models that consider individual entities in isolation.
 </p>
 <p>
 GNNs are not the only tools available to model graph-structured data:
 graph kernels 
 and random-walk methods 
 were some of the most popular ones.
 Today, however, GNNs have largely replaced these techniques
 because of their inherent flexibility to model the underlying systems
 better.
 </p>
 <p>
 In this article, we will illustrate
 the challenges of computing over graphs, 
 describe the origin and design of graph neural networks,
 and explore the most popular GNN variants in recent times.
 Particularly, we will see that many of these variants
 are composed of similar building blocks.
 </p>
 <p>
 First, let’s discuss some of the complications that graphs come with.
 </p>
 <h2>
 The Challenges of Computation on Graphs
 </h2>
 <h3>
 Lack of Consistent Structure
 </h3>
 <p>
 Graphs are extremely flexible mathematical models; but this means they lack consistent structure across instances.
 Consider the task of predicting whether a given chemical molecule is toxic  :
 </p>
 <figure>
 
 
 </figure>
 <figure>
 <figure>
 <figcaption><b>Left:</b> A non-toxic 1,2,6-trigalloyl-glucose molecule.</figcaption>
 </figure>
 <figure>
 <figcaption><b>Right:</b> A toxic caramboxin molecule.</figcaption>
 </figure>
 </figure>
 <p>
 Looking at a few examples, the following issues quickly become apparent:
 </p>
 <ul>
 <li>Molecules may have different numbers of atoms.</li>
 <li>The atoms in a molecule may be of different types.</li>
 <li>Each of these atoms may have different number of connections.</li>
 <li>These connections can have different strengths.</li>
 </ul>
 <p>
 Representing graphs in a format that can be computed over is non-trivial,
 and the final representation chosen often depends significantly on the actual problem.
 </p>
 <h3>
 Node-Order Equivariance
 </h3>
 <p>
 Extending the point above: graphs often have no inherent ordering present amongst the nodes.
 Compare this to images, where every pixel is uniquely determined by its absolute position within the image!
 </p>
 <figure>
 
 <figcaption>
 Representing the graph as one vector requires us to fix an order on the nodes.
 But what do we do when the nodes have no inherent order?
 <b>Above:</b> 
 The same graph labelled in two different ways. The alphabets indicate the ordering of the nodes.
 </figcaption>
 </figure>
 
 <p>
 As a result, we would like our algorithms to be node-order equivariant:
 they should not depend on the ordering of the nodes of the graph.
 If we permute the nodes in some way, the resulting representations of 
 the nodes as computed by our algorithms should also be permuted in the same way.
 </p>
 <h3>
 Scalability
 </h3>
 <p>
 Graphs can be really large! Think about social networks like Facebook and Twitter, which have over a billion users. 
 Operating on data this large is not easy.
 </p>
 <p>
 Luckily, most naturally occuring graphs are ‘sparse’:
 they tend to have their number of edges linear in their number of vertices.
 We will see that this allows the use of clever methods
 to efficiently compute representations of nodes within the graph.
 Further, the methods that we look at here will have significantly fewer parameters
 in comparison to the size of the graphs they operate on.
 </p>
 <h2>
 Problem Setting and Notation
 </h2>
 <p>
 There are many useful problems that can be formulated over graphs:
 </p>
 <ul>
 <li><b>Node Classification:</b> Classifying individual nodes.</li>
 <li><b>Graph Classification:</b> Classifying entire graphs. </li>
 <li><b>Node Clustering:</b> Grouping together similar nodes based on connectivity.</li>
 <li><b>Link Prediction:</b> Predicting missing links.</li>
 <li><b>Influence Maximization:</b> Identifying influential nodes.</li>
 </ul>
 <figure>
 
 <figcaption>
 Examples of problems that can be defined over graphs.
 This list is not exhaustive!
 </figcaption>
 </figure>
 <p>
 A common precursor in solving many of these problems is <b>node representation learning</b>:
 learning to map individual nodes to fixed-size real-valued vectors (called ‘representations’ or ‘embeddings’).
 </p>
 <p>
 In <a href="#learning">Learning GNN Parameters</a>, we will see how the learnt embeddings can be used for these tasks.
 </p>
 <p>
 Different GNN variants are distinguished by the way these representations are computed.
 Generally, however, GNNs compute node representations in an iterative process.
 We will use the notation hv(k)h_v^{(k)} to indicate the representation of node vv after the kthk^{\text{th}} iteration.
 Each iteration can be thought of as the equivalent of a ‘layer’ in standard neural networks.
 </p>
 <p>
 We will define a graph GG as a set of nodes, VV, with a set of edges EE connecting them.
 Nodes can have individual features as part of the input: we will denote by xvx_v the individual feature for node v∈Vv \in V.
 For example, the ‘node features’ for a pixel in a color image
 would be the red, green and blue channel (RGB) values at that pixel.
 </p>
 <p>
 For ease of exposition, we will assume GG is undirected, and all nodes are of the same type.
 These kinds of graphs are called ‘homogeneous’.
 Many of the same ideas we will see here 
 apply to other kinds of graphs:
 we will discuss this later in <a href="#different-kinds-of-graphs">Different Kinds of Graphs</a>.
 </p>
 <p>
 Sometimes we will need to denote a graph property by a matrix MM,
 where each row MvM_v represents a property corresponding to a particular vertex vv.
 </p>
 <h2>
 Extending Convolutions to Graphs
 </h2>
 <p>
 Convolutional Neural Networks have been seen to be quite powerful in extracting features from images.
 However, images themselves can be seen as graphs with a very regular grid-like structure,
 where the individual pixels are nodes, and the RGB channel values at each pixel as the node features.
 </p>
 <p>
 A natural idea, then, is to consider generalizing convolutions to arbitrary graphs. Recall, however, the challenges
 listed out in the <a href="#challenges">previous section</a>: in particular, ordinary convolutions are not node-order invariant, because
 they depend on the absolute positions of pixels.
 It is initially unclear as how to generalize convolutions over grids to convolutions over general graphs,
 where the neighbourhood structure differs from node to node.
 
 The curious reader may wonder if performing some sort of padding and ordering
 could be done to ensure the consistency of neighbourhood structure across nodes.
 This has been attempted with some success ,
 but the techniques we will look at here are more general and powerful.
 
 </p>
 
 <figure>
 
 <figure>
 <figcaption>
 Convolutions in CNNs are inherently localized.
 Neighbours participating in the convolution at the center pixel are highlighted in gray.
 </figcaption>
 </figure>
 
 <figure>
 <figcaption>
 GNNs can perform localized convolutions mimicking CNNs.
 Hover over a node to see its immediate neighbourhood highlighted on the left.
 The structure of this neighbourhood changes from node to node.
 </figcaption>
 </figure>
 
 </figure>
 <p>
 We begin by introducing the idea of constructing polynomial filters over node neighbourhoods,
 much like how CNNs compute localized filters over neighbouring pixels.
 Then, we will see how more recent approaches extend on this idea with more powerful mechanisms.
 Finally, we will discuss alternative methods
 that can use ‘global’ graph-level information for computing node representations.
 </p>
 <h2>
 Polynomial Filters on Graphs
 </h2>
 <h3>
 The Graph Laplacian
 </h3>
 <p>
 Given a graph GG, let us fix an arbitrary ordering of the nn nodes of GG.
 We denote the 0−10-1 adjacency matrix of GG by AA, we can construct the diagonal degree matrix DD of GG as: 
 </p>
 <div>
 <p>Dv=∑uAvu.
 D_v = \sum_u A_{vu}.
 </p><figure>
 <figcaption>
 The degree of node vv is the number of edges incident at vv.
 </figcaption>
 </figure>
 </div>
 <p>
 where AvuA_{vu} denotes the entry in the row corresponding to vv and the column corresponding to uu
 in the matrix AA. We will use this notation throughout this section.
 </p>
 <p>
 Then, the graph Laplacian LL is the square n×nn \times n matrix defined as:
 L=D−A.
 L = D - A.
 
 </p>
 <figure>
 
 <figcaption>
 The Laplacian LL for an undirected graph GG, with the row corresponding to node C\textsf{C} highlighted.
 Zeros in LL are not displayed above.
 The Laplacian LL depends only on the structure of the graph GG, not on any node features.
 </figcaption>
 </figure>
 <p>
 The graph Laplacian gets its name from being the discrete analog of the
 <a href="https://mathworld.wolfram.com/Laplacian.html">Laplacian operator</a>
 from calculus.
 </p>
 <p>
 Although it encodes precisely the same information as the adjacency matrix AA
 
 In the sense that given either of the matrices AA or LL, you can construct the other.
 ,
 the graph Laplacian has many interesting properties of its own.
 
 The graph Laplacian shows up in many mathematical problems involving graphs:
 <a href="https://people.math.sc.edu/lu/talks/nankai_2014/spec_nankai_2.pdf">random walks</a>,
 <a href="https://arxiv.org/abs/0711.0189">spectral clustering</a>,
 and
 <a href="https://www.math.fsu.edu/~bertram/lectures/Diffusion.pdf">diffusion</a>, to name a few.
 
 We will see some of these properties
 in <a href="#spectral">a later section</a>,
 but will instead point readers to
 <a href="https://csustan.csustan.edu/~tom/Clustering/GraphLaplacian-tutorial.pdf">this tutorial</a>
 for greater insight into the graph Laplacian.
 </p>
 <h3>
 Polynomials of the Laplacian
 </h3>
 <p>
 Now that we have understood what the graph Laplacian is,
 we can build polynomials of the form:
 pw(L)=w0In+w1L+w2L2+…+wdLd=∑i=0dwiLi.
 p_w(L) = w_0 I_n + w_1 L + w_2 L^2 + \ldots + w_d L^d = \sum_{i = 0}^d w_i L^i.
 
 Each polynomial of this form can alternately be represented by
 its vector of coefficients w=[w0,…,wd]w = [w_0, \ldots, w_d].
 Note that for every ww, pw(L)p_w(L) is an n×nn \times n matrix, just like LL.
 </p>
 <p>
 These polynomials can be thought of as the equivalent of ‘filters’ in CNNs,
 and the coefficients ww as the weights of the ‘filters’.
 </p>
 <p>
 For ease of exposition, we will focus on the case where nodes have one-dimensional features:
 each of the xvx_v for v∈Vv \in V is just a real number. 
 The same ideas hold when each of the xvx_v are higher-dimensional vectors, as well.
 </p>
 <p>
 Using the previously chosen ordering of the nodes,
 we can stack all of the node features xvx_v
 to get a vector x∈Rnx \in \mathbb{R}^n.
 </p>
 <figure>
 
 <figcaption>
 Fixing a node order (indicated by the alphabets) and collecting all node features into a single vector xx.
 </figcaption>
 </figure>
 <p>
 Once we have constructed the feature vector xx,
 we can define its convolution with a polynomial filter pwp_w as:
 x′=pw(L) x
 x’ = p_w(L) \ x
 
 To understand how the coefficients ww affect the convolution,
 let us begin by considering the ‘simplest’ polynomial:
 when w0=1w_0 = 1 and all of the other coefficients are 00.
 In this case, x′x’ is just xx:
 x′=pw(L) x=∑i=0dwiLix=w0Inx=x.
 x’ = p_w(L) \ x = \sum_{i = 0}^d w_i L^ix = w_0 I_n x = x.
 
 Now, if we increase the degree, and consider the case where
 instead w1=1w_1 = 1 and and all of the other coefficients are 00.
 Then, x′x’ is just LxLx, and so:
 xv′=(Lx)v=Lvx=∑u∈GLvuxu=∑u∈G(Dvu−Avu)xu=Dv xv−∑u∈N(v)xu
 \begin{aligned}
 x’_v = (Lx)_v &amp;= L_v x \\ 
 &amp;= \sum_{u \in G} L_{vu} x_u \\ 
 &amp;= \sum_{u \in G} (D_{vu} - A_{vu}) x_u \\ 
 &amp;= D_v \ x_v - \sum_{u \in \mathcal{N}(v)} x_u
 \end{aligned}
 
 We see that the features at each node vv are combined
 with the features of its immediate neighbours u∈N(v)u \in \mathcal{N}(v).
 
 For readers familiar with
 <a href="https://docs.opencv.org/3.4/d5/db5/tutorial_laplace_operator.html">Laplacian filtering of images</a>,
 this is the exact same idea. When xx is an image, 
 x′=Lxx’ = Lx is exactly the result of applying a ‘Laplacian filter’ to xx.
 
 </p>
 <p>
 At this point, a natural question to ask is:
 How does the degree dd of the polynomial influence the behaviour of the convolution?
 Indeed, it is not too hard to show that:
 This is Lemma 5.2 from .
 distG(v,u)&gt;i⟹Lvui=0.
 \text{dist}_G(v, u) &gt; i \quad \Longrightarrow \quad L_{vu}^i = 0.
 
 
 This implies, when we convolve xx with pw(L)p_w(L) of degree dd to get x′x’:
 xv′=(pw(L)x)v=(pw(L))vx=∑i=0dwiLvix=∑i=0dwi∑u∈GLvuixu=∑i=0dwi∑u∈GdistG(v,u)≤iLvuixu.
 \begin{aligned}
 x’_v = (p_w(L)x)_v &amp;= (p_w(L))_v x \\
 &amp;= \sum_{i = 0}^d w_i L_v^i x \\
 &amp;= \sum_{i = 0}^d w_i \sum_{u \in G} L_{vu}^i x_u \\
 &amp;= \sum_{i = 0}^d w_i \sum_{u \in G \atop \text{dist}_G(v, u) \leq i} L_{vu}^i x_u.
 \end{aligned}
 
 </p><p>
 Effectively, the convolution at node vv occurs only with nodes uu which are not more than dd hops away.
 Thus, these polynomial filters are localized. The degree of the localization is governed completely by dd.
 </p>
 <p>
 To help you understand these ‘polynomial-based’ convolutions better, we have created the visualization below.
 Vary the polynomial coefficients and the input grid xx to see how the result x′x’ of the convolution changes.
 The grid under the arrow shows the equivalent convolutional kernel applied at the highlighted pixel in xx to get
 the resulting pixel in x′x’.
 The kernel corresponds to the row of pw(L)p_w(L) for the highlighted pixel.
 Note that even after adjusting for position,
 this kernel is different for different pixels, depending on their position within the grid.
 </p>
 
 <figure>
 <figcaption>
 <p>
 Hover over a pixel in the input grid (left, representing xx)
 to highlight it and see the equivalent convolutional kernel
 for that pixel under the arrow.
 The result x′x’ of the convolution is shown on the right:
 note that different convolutional kernels are applied at different pixels,
 depending on their location.
 </p>
 <p>
 Click on the input grid to toggle pixel values between 00 (white) and 11 (blue).
 To randomize the input grid, press ‘Randomize Grid’. To reset all pixels to 00, press ‘Reset Grid’.
 Use the sliders at the bottom to change the coefficients ww.
 To reset all coefficients ww to 00, press ‘Reset Coefficients.’
 </p>
 </figcaption>
 </figure>
 <h3>
 ChebNet
 </h3>
 
 ChebNet refines this idea of polynomial filters by looking at polynomial filters of the form:
 
 pw(L)=∑i=1dwiTi(L~)
 p_w(L) = \sum_{i = 1}^d w_i T_i(\tilde{L})
 
 
 where TiT_i is the degree-ii
 <a href="https://en.wikipedia.org/wiki/Chebyshev_polynomials">Chebyshev polynomial of the first kind</a> and
 L~\tilde{L} is the normalized Laplacian defined using the largest eigenvalue of LL:
 
 We discuss the eigenvalues of the Laplacian LL in more detail in <a href="#spectral">a later section</a>.
 
 
 L~=2Lλmax(L)−In.
 \tilde{L} = \frac{2L}{\lambda_{\max}(L)} - I_n.
 
 <p>
 What is the motivation behind these choices?
 </p>
 <ul>
 <li>
 LL is actually positive semi-definite: all of the eigenvalues of LL are not lesser than 00.
 If λmax(L)&gt;1\lambda_{\max}(L) &gt; 1, the entries in the powers of LL rapidly increase in size.
 L~\tilde{L} is effectively a scaled-down version of LL, with eigenvalues guaranteed to be in the range [−1,1][-1, 1].
 This prevents the entries of powers of L~\tilde{L} from blowing up.
 Indeed, in the <a href="#polynomial-convolutions">visualization above</a>: we restrict the higher-order coefficients
 when the unnormalized Laplacian LL is selected, but allow larger values when the normalized Laplacian L~\tilde{L} is selected,
 in order to show the result x′x’ on the same color scale.
 </li>
 <li>
 The Chebyshev polynomials have certain interesting properties that make interpolation more numerically stable.
 We won’t talk about this in more depth here,
 but will advise interested readers to take a look at as a definitive resource.
 </li>
 </ul>
 
 <h3>
 Polynomial Filters are Node-Order Equivariant
 </h3>
 <p>
 The polynomial filters we considered here are actually independent of the ordering of the nodes.
 This is particularly easy to see when the degree of the polynomial pwp_w is 11:
 where each node’s feature is aggregated with the sum of its neighbour’s features.
 Clearly, this sum does not depend on the order of the neighbours.
 A similar proof follows for higher degree polynomials:
 the entries in the powers of LL are equivariant to the ordering of the nodes.
 </p>
 <div>
 <p><b>Details for the Interested Reader</b></p><p>
 As above, let’s assume an arbitrary node-order over the nn nodes of our graph.
 Any other node-order can be thought of as a permutation of this original node-order.
 We can represent any permutation by a
 <a href="https://en.wikipedia.org/wiki/Permutation_matrix">permutation matrix</a> PP.
 PP will always be an orthogonal 0−10-1 matrix:
 PPT=PTP=In.
 PP^T = P^TP = I_n.
 
 Then, we call a function ff node-order equivariant iff for all permutations PP:
 f(Px)=Pf(x).
 f(Px) = P f(x).
 
 When switching to the new node-order using the permutation PP,
 the quantities below transform in the following way:
 x→PxL→PLPTLi→PLiPT
 \begin{aligned}
 x &amp;\to Px \\
 L &amp;\to PLP^T \\
 L^i &amp;\to PL^iP^T
 \end{aligned}
 
 and so, for the case of polynomial filters where f(x)=pw(L) xf(x) = p_w(L) \ x, we can see that:
 f(Px)=∑i=0dwi(PLiPT)(Px)=P∑i=0dwiLix=Pf(x).
 \begin{aligned}
 f(Px) &amp; = \sum_{i = 0}^d w_i (PL^iP^T) (Px) \\
 &amp; = P \sum_{i = 0}^d w_i L^i x \\
 &amp; = P f(x).
 \end{aligned}
 
 as claimed.
 </p>
 </div>
 <h3>
 Embedding Computation
 </h3>
 <p>
 We now describe how we can build a graph neural network
 by stacking ChebNet (or any polynomial filter) layers
 one after the other with non-linearities,
 much like a standard CNN.
 In particular, if we have KK different polynomial filter layers,
 the kthk^{\text{th}} of which has its own learnable weights w(k)w^{(k)},
 we would perform the following computation:
 </p>
 
 
 
 
 
 <p>
 Note that these networks
 reuse the same filter weights across different nodes,
 exactly mimicking weight-sharing in Convolutional Neural Networks (CNNs)
 which reuse weights for convolutional filters across a grid.
 </p>
 <h2>
 Modern Graph Neural Networks
 </h2>
 <p>
 ChebNet was a breakthrough in learning localized filters over graphs,
 and it motivated many to think of graph convolutions from a different perspective.
 </p>
 
 We return back to the result of convolving xx by the polynomial kernel pw(L)=Lp_w(L) = L,
 focussing on a particular vertex vv:
 
 (Lx)v=Lvx=∑u∈GLvuxu=∑u∈G(Dvu−Avu)xu=Dv xv−∑u∈N(v)xu
 \begin{aligned}
 (Lx)_v &amp;= L_v x \\ 
 &amp;= \sum_{u \in G} L_{vu} x_u \\ 
 &amp;= \sum_{u \in G} (D_{vu} - A_{vu}) x_u \\ 
 &amp;= D_v \ x_v - \sum_{u \in \mathcal{N}(v)} x_u
 \end{aligned}
 
 <p> 
 As we noted before, this is a 11-hop localized convolution.
 But more importantly, we can think of this convolution as arising of two steps:
 </p>
 <ul>
 <li>Aggregating over immediate neighbour features xux_u.</li>
 <li>Combining with the node’s own feature xvx_v.</li>
 </ul>
 <p>
 <b>Key Idea:</b>
 What if we consider different kinds of ‘aggregation’ and ‘combination’ steps,
 beyond what are possible using polynomial filters?
 </p>
 <p>
 By ensuring that the aggregation is node-order equivariant,
 the overall convolution becomes node-order equivariant.
 </p>
 <p>
 These convolutions can be thought of as ‘message-passing’ between adjacent nodes:
 after each step, every node receives some ‘information’ from its neighbours.
 </p>
 <p>
 By iteratively repeating the 11-hop localized convolutions KK times (i.e., repeatedly ‘passing messages’),
 the receptive field of the convolution effectively includes all nodes upto KK hops away.
 </p>
 <h3>
 Embedding Computation
 </h3>
 <p>
 Message-passing forms the backbone of many GNN architectures today.
 We describe the most popular ones in depth below:
 </p>
 <ul>
 <li>Graph Convolutional Networks (GCN)</li>
 <li>Graph Attention Networks (GAT)</li>
 <li>Graph Sample and Aggregate (GraphSAGE)</li>
 <li>Graph Isomorphism Network (GIN)</li>
 </ul>
 
 
 
 <h3>
 Thoughts
 </h3>
 <p>
 An interesting point is to assess different aggregation functions: are some better and others worse?
 demonstrates that aggregation functions indeed can be compared on how well
 they can uniquely preserve node neighbourhood features;
 we recommend the interested reader take a look at the detailed theoretical analysis there.
 </p>
 <p>
 Here, we’ve talk about GNNs where the computation only occurs at the nodes.
 More recent GNN models
 such as Message-Passing Neural Networks 
 and Graph Networks 
 perform computation over the edges as well;
 they compute edge embeddings together with node embeddings.
 This is an even more general framework -
 but the same ‘message passing’ ideas from this section apply.
 </p>
 <h2>
 Interactive Graph Neural Networks
 </h2>
 <p>
 Below is an interactive visualization of these GNN models on small graphs.
 For clarity, the node features are just real numbers here, shown inside the squares next to each node,
 but the same equations hold when the node features are vectors.
 </p>
 
 
 <figure>
 <figcaption>
 Choose a GNN model using the tabs at the top. Click on a node to see the update equation at that node for the next iteration.
 Use the sliders on the left to change the weights for the current iteration, and watch how the update equation changes. 
 </figcaption>
 </figure>
 <p>
 In practice, each iteration above is generally thought of as a single ‘neural network layer’.
 This ideology is followed by many popular Graph Neural Network libraries,
 
 For example: <a href="https://pytorch-geometric.readthedocs.io/en/latest/modules/nn.html">PyTorch Geometric</a>
 and <a href="https://stellargraph.readthedocs.io/en/stable/api.html#module-stellargraph.layer">StellarGraph</a>.
 
 allowing one to compose different types of graph convolutions in the same model.
 </p>
 <h2>
 From Local to Global Convolutions 
 </h2>
 <p>
 The methods we’ve seen so far perform ‘local’ convolutions:
 every node’s feature is updated using a function of its local neighbours’ features.
 </p>
 <p>
 While performing enough steps of message-passing will eventually ensure that
 information from all nodes in the graph is passed,
 one may wonder if there are more direct ways to perform ‘global’ convolutions.
 </p>
 <p>
 The answer is yes; we will now describe an approach that was actually first put forward
 in the context of neural networks by ,
 much before any of the GNN models we looked at above.
 </p> 
 <h3>
 Spectral Convolutions 
 </h3>
 <p>
 As before, we will focus on the case where nodes have one-dimensional features.
 After choosing an arbitrary node-order, we can stack all of the node features to get a
 ‘feature vector’ x∈Rnx \in \mathbb{R}^n.
 </p>
 <p>
 <b>Key Idea:</b>
 Given a feature vector xx, 
 the Laplacian LL allows us to quantify how smooth xx is, with respect to GG.
 </p>
 <p>
 How?
 </p>
 <p>
 After normalizing xx such that ∑i=1nxi2=1\sum_{i = 1}^n x_i^2 = 1,
 if we look at the following quantity involving LL:
 
 RLR_L is formally called the <a href="https://en.wikipedia.org/wiki/Rayleigh_quotient">Rayleigh quotient</a>.
 
 RL(x)=xTLxxTx=∑(i,j)∈E(xi−xj)2∑ixi2=∑(i,j)∈E(xi−xj)2.
 R_L(x) = \frac{x^T L x}{x^T x} = \frac{\sum_{(i, j) \in E} (x_i - x_j)^2}{\sum_i x_i^2} = \sum_{(i, j) \in E} (x_i - x_j)^2.
 
 we immediately see that feature vectors xx that assign similar values to 
 adjacent nodes in GG (hence, are smooth) would have smaller values of RL(x)R_L(x).
 </p>
 <p>
 LL is a real, symmetric matrix, which means it has all real eigenvalues λ1≤…≤λn\lambda_1 \leq \ldots \leq \lambda_{n}.
 
 An eigenvalue λ\lambda of a matrix AA is a value
 satisfying the equation Au=λuAu = \lambda u for a certain vector uu, called an eigenvector.
 For a friendly introduction to eigenvectors,
 please see <a href="http://www.sosmath.com/matrix/eigen0/eigen0.html">this tutorial</a>.
 
 Further, the corresponding eigenvectors u1,…,unu_1, \ldots, u_{n} can be taken to be orthonormal:
 uk1Tuk2={1 if k1=k2.0 if k1≠k2.
 u_{k_1}^T u_{k_2} =
 \begin{cases}
 1 \quad \text{ if } {k_1} = {k_2}. \\
 0 \quad \text{ if } {k_1} \neq {k_2}.
 \end{cases}
 
 It turns out that these eigenvectors of LL are successively less smooth, as RLR_L indicates:
 This is the <a href="https://en.wikipedia.org/wiki/Min-max_theorem">min-max theorem for eigenvalues.</a>
 argminx, x⊥{u1,…,ui−1}RL(x)=ui.minx, x⊥{u1,…,ui−1}RL(x)=λi.
 \underset{x, \ x \perp \{u_1, \ldots, u_{i - 1}\}}{\text{argmin}} R_L(x) = u_i.
 \qquad
 \qquad
 \qquad
 \min_{x, \ x \perp \{u_1, \ldots, u_{i - 1}\}} R_L(x) = \lambda_i.
 
 The set of eigenvalues of LL are called its ‘spectrum’, hence the name!
 We denote the ‘spectral’ decomposition of LL as:
 L=UΛUT.
 L = U \Lambda U^T.
 
 where Λ\Lambda is the diagonal matrix of sorted eigenvalues,
 and UU denotes the matrix of the eigenvectors (sorted corresponding to increasing eigenvalues):
 Λ=[λ1⋱λn]U=[u1 ⋯ un].
 \Lambda = \begin{bmatrix}
 \lambda_{1} &amp; &amp; \\
 &amp; \ddots &amp; \\
 &amp; &amp; \lambda_{n}
 \end{bmatrix}
 \qquad
 \qquad
 \qquad
 \qquad
 U = \begin{bmatrix} \\ u_1 \ \cdots \ u_n \\ \end{bmatrix}.
 
 The orthonormality condition between eigenvectors gives us that UTU=IU^T U = I, the identity matrix.
 As these nn eigenvectors form a basis for Rn\mathbb{R}^n,
 any feature vector xx can be represented as a linear combination of these eigenvectors:
 x=∑i=1nxi^ui=Ux^.
 x = \sum_{i = 1}^n \hat{x_i} u_i = U \hat{x}.
 
 where x^\hat{x} is the vector of coefficients [x0,…xn][x_0, \ldots x_n].
 We call x^\hat{x} as the spectral representation of the feature vector xx.
 The orthonormality condition allows us to state:
 x=Ux^⟺UTx=x^.
 x = U \hat{x} \quad \Longleftrightarrow \quad U^T x = \hat{x}.
 
 This pair of equations allows us to interconvert
 between the ‘natural’ representation xx and the ‘spectral’ representation x^\hat{x}
 for any vector x∈Rnx \in \mathbb{R}^n.
 </p>
 <h3>
 Spectral Representations of Natural Images
 </h3>
 <p>
 As discussed before, we can consider any image as a grid graph, where each pixel is a node,
 connected by edges to adjacent pixels.
 Thus, a pixel can have either 3,5,3, 5, or 88 neighbours, depending on its location within the image grid.
 Each pixel gets a value as part of the image. If the image is grayscale, each value will be a single 
 real number indicating how dark the pixel is. If the image is colored, each value will be a 33-dimensional
 vector, indicating the values for the red, green and blue (RGB) channels.
 We use the alpha channel as well in the visualization below, so this is actually RGBA.
 </p>
 <p>
 This construction allows us to compute the graph Laplacian and the eigenvector matrix UU.
 Given an image, we can then investigate what its spectral representation looks like.
 </p>
 <p>
 To shed some light on what the spectral representation actually encodes,
 we perform the following experiment over each channel of the image independently: 
 </p>
 <ul>
 <li>
 We first collect all pixel values across a channel into a feature vector xx.
 </li>
 <li>
 Then, we obtain its spectral representation x^\hat{x}.
 x^=UTx
 \hat{x} = U^T x
 
 </li>
 <li>
 We truncate this to the first mm components to get x^m\hat{x}_m.
 By truncation, we mean zeroing out all of the remaining n−mn - m components of x^\hat{x}.
 This truncation is equivalent to using only the first mm eigenvectors to compute the spectral representation.
 x^m=Truncatem(x^)
 \hat{x}_m = \text{Truncate}_m(\hat{x})
 
 </li>
 <li>
 Then, we convert this truncated representation x^m\hat{x}_m back to the natural basis to get xmx_m.
 xm=Ux^m
 x_m = U \hat{x}_m
 
 </li>
 </ul>
 <p>
 Finally, we stack the resulting channels back together to get back an image.
 We can now see how the resulting image changes with choices of mm.
 Note that when m=nm = n, the resulting image is identical to the original image,
 as we can reconstruct each channel exactly.
 </p>
 
 <figure>
 <figcaption>
 Use the radio buttons at the top to chose one of the four sample images.
 Each of these images has been taken from the ImageNet 
 dataset and downsampled to 5050 pixels wide and 4040 pixels tall.
 As there are n=50×40=2000n = 50 \times 40 = 2000 pixels in each image, there are 20002000 Laplacian eigenvectors.
 Use the slider at the bottom to change the number of spectral components to keep, noting how
 images get progressively blurrier as the number of components decrease.
 </figcaption>
 </figure>
 <p>
 As mm decreases, we see that the output image xmx_m gets blurrier.
 If we decrease mm to 11, the output image xmx_m is entirely the same color throughout.
 We see that we do not need to keep all nn components;
 we can retain a lot of the information in the image with significantly fewer components.
 We can relate this to the Fourier decomposition of images:
 the more eigenvectors we use, the higher frequencies we can represent on the grid.
 </p>
 <p>
 To complement the visualization above,
 we additionally visualize the first few eigenvectors on a smaller 8×88 \times 8 grid below.
 We change the coefficients of the first 1010 out of 6464 eigenvectors
 in the spectral representation
 and see how the resulting image changes:
 </p>
 
 
 <figure>
 <figcaption>
 Move the sliders to change the spectral representation x^\hat{x} (right),
 and see how xx itself changes on the image (left).
 Note how the first eigenvectors are much ‘smoother’ than the later ones,
 and the many patterns we can make with only 1010 eigenvectors.
 </figcaption>
 </figure>
 <p>
 These visualizations should convince you that the first eigenvectors are indeed smooth,
 and the smoothness correspondingly decreases as we consider later eigenvectors.
 </p>
 <p>
 For any image xx, we can think of
 the initial entries of the spectral representation x^\hat{x}
 as capturing ‘global’ image-wide trends, which are the low-frequency components,
 while the later entries as capturing ‘local’ details, which are the high-frequency components.
 </p>
 <h3>Embedding Computation</h3>
 <p>
 We now have the background to understand spectral convolutions
 and how they can be used to compute embeddings/feature representations of nodes.
 </p>
 <p>
 As before, the model we describe below has KK layers:
 each layer kk has learnable parameters w^(k)\hat{w}^{(k)},
 called the ‘filter weights’.
 These weights will be convolved with the spectral representations of the node features.
 As a result, the number of weights needed in each layer is equal to mm, the number of 
 eigenvectors used to compute the spectral representations.
 We had shown in the previous section that we can take m≪nm \ll n
 and still not lose out on significant amounts of information.
 </p>
 <p>
 Thus, convolution in the spectral domain enables the use of significantly fewer parameters
 than just direct convolution in the natural domain.
 Further, by virtue of the smoothness of the Laplacian eigenvectors across the graph,
 using spectral representations automatically enforces an inductive bias for
 neighbouring nodes to get similar representations.
 </p>
 <p>
 Assuming one-dimensional node features for now,
 the output of each layer is a vector of node representations h(k)h^{(k)},
 where each node’s representation corresponds to a row
 of the vector.
 </p>
 
 <p>
 We fix an ordering of the nodes in GG. This gives us the adjacency matrix AA and the graph Laplacian LL,
 allowing us to compute UmU_m.
 Finally, we can describe the computation that the layers perform, one after the other:
 </p>
 
 
 
 <p>
 The method above generalizes easily to the case where each h(k)∈Rdkh^{(k)} \in \mathbb{R}^{d_k}, as well:
 see for details.
 </p>
 <p>
 With the insights from the previous section, we see that convolution in the spectral-domain of graphs
 can be thought of as the generalization of convolution in the frequency-domain of images.
 </p>
 <h3>
 Spectral Convolutions are Node-Order Equivariant
 </h3>
 <p>
 We can show spectral convolutions are node-order equivariant using a similar approach
 as for Laplacian polynomial filters. 
 </p>
 <div>
 <p><b>
 Details for the Interested Reader
 </b></p><p>
 As in <a href="#poly-filters-equivariance">our proof before</a>,
 let’s fix an arbitrary node-order.
 Then, any other node-order can be represented by a
 permutation of this original node-order.
 We can associate this permutation with its permutation matrix PP.
 Under this new node-order,
 the quantities below transform in the following way:
 x→PxA→PAPTL→PLPTUm→PUm
 \begin{aligned}
 x &amp;\to Px \\
 A &amp;\to PAP^T \\
 L &amp;\to PLP^T \\
 U_m &amp;\to PU_m
 \end{aligned}
 
 which implies that, in the embedding computation:
 x^→(PUm)T(Px)=UmTx=x^w^→(PUm)T(Pw)=UmTw=w^g^→g^g→(PUm)g^=P(Umg^)=Pg
 \begin{aligned}
 \hat{x} &amp;\to \left(PU_m\right)^T (Px) = U_m^T x = \hat{x} \\
 \hat{w} &amp;\to \left(PU_m\right)^T (Pw) = U_m^T w = \hat{w} \\
 \hat{g} &amp;\to \hat{g} \\
 g &amp;\to (PU_m)\hat{g} = P(U_m\hat{g}) = Pg
 \end{aligned}
 
 Hence, as σ\sigma is applied elementwise:
 f(Px)=σ(Pg)=Pσ(g)=Pf(x)
 f(Px) = \sigma(Pg) = P \sigma(g) = P f(x)
 
 as required.
 Further, we see that the spectral quantities x^,w^\hat{x}, \hat{w} and g^\hat{g}
 are unchanged by permutations of the nodes.
 
 Formally, they are what we would call node-order invariant.
 
 </p>
 </div>
 <p>
 The theory of spectral convolutions is mathematically well-grounded;
 however, there are some key disadvantages that we must talk about:
 </p>
 <ul>
 <li>
 We need to compute the eigenvector matrix UmU_m from LL. For large graphs, this becomes quite infeasible.
 </li>
 <li>
 Even if we can compute UmU_m, global convolutions themselves are inefficient to compute,
 because of the repeated
 multiplications with UmU_m and UmTU_m^T.
 </li>
 <li>
 The learned filters are specific to the input graphs,
 as they are represented in terms
 of the spectral decomposition of input graph Laplacian LL.
 This means they do not transfer well to new graphs
 which have significantly different structure (and hence, significantly
 different eigenvalues) .
 </li>
 </ul>
 <p>
 While spectral convolutions have largely been superseded by
 ‘local’ convolutions for the reasons discussed above,
 there is still much merit to understanding the ideas behind them.
 Indeed, a recently proposed GNN model called Directional Graph Networks
 
 actually uses the Laplacian eigenvectors
 and their mathematical properties
 extensively.
 </p>
 <h3>
 Global Propagation via Graph Embeddings
 </h3>
 <p>
 A simpler way to incorporate graph-level information
 is to compute embeddings of the entire graph by pooling node
 (and possibly edge) embeddings,
 and then using the graph embedding to update node embeddings,
 following an iterative scheme similar to what we have looked at here.
 This is an approach used by Graph Networks
 .
 We will briefly discuss how graph-level embeddings
 can be constructed in <a href="#pooling">Pooling</a>.
 However, such approaches tend to ignore the underlying
 topology of the graph that spectral convolutions can capture.
 </p>
 <h2>
 Learning GNN Parameters
 </h2>
 <p>
 All of the embedding computations we’ve described here, whether spectral or spatial, are completely differentiable.
 This allows GNNs to be trained in an end-to-end fashion, just like a standard neural network,
 once a suitable loss function L\mathcal{L} is defined:
 </p>
 <ul>
 <li>
 <b>Node Classification</b>: By minimizing any of the standard losses for classification tasks,
 such as categorical cross-entropy when multiple classes are present:
 L(yv,yv^)=−∑cyvclogyvc^.
 \mathcal{L}(y_v, \hat{y_v}) = -\sum_{c} y_{vc} \log{\hat{y_{vc}}}.
 
 where yvc^\hat{y_{vc}} is the predicted probability that node vv is in class cc.
 GNNs adapt well to the semi-supervised setting, which is when only some nodes in the graph are labelled.
 In this setting, one way to define a loss LG\mathcal{L}_{G} over an input graph GG is:
 LG=∑v∈Lab(G)L(yv,yv^)∣Lab(G)∣
 \mathcal{L}_{G} = \frac{\sum\limits_{v \in \text{Lab}(G)} \mathcal{L}(y_v, \hat{y_v})}{| \text{Lab}(G) |}
 
 where, we only compute losses over labelled nodes Lab(G)\text{Lab}(G).
 </li>
 <li>
 <b>Graph Classification</b>: By aggregating node representations,
 one can construct a vector representation of the entire graph.
 This graph representation can be used for any graph-level task, even beyond classification.
 See <a href="#pooling">Pooling</a> for how representations of graphs can be constructed.
 </li>
 <li>
 <b>Link Prediction</b>: By sampling pairs of adjacent and non-adjacent nodes,
 and use these vector pairs as inputs to predict the presence/absence of an edge.
 For a concrete example, by minimizing the following ‘logistic regression’-like loss:
 L(yv,yu,evu)=−evulog(pvu)−(1−evu)log(1−pvu)pvu=σ(yvTyu)
 \begin{aligned}
 \mathcal{L}(y_v, y_u, e_{vu}) &amp;= -e_{vu} \log(p_{vu}) - (1 - e_{vu}) \log(1 - p_{vu}) \\
 p_{vu} &amp;= \sigma(y_v^Ty_u)
 \end{aligned}
 
 where σ\sigma is the <a href="https://en.wikipedia.org/wiki/Sigmoid_function">sigmoid function</a>,
 and evu=1e_{vu} = 1 iff there is an edge between nodes vv and uu, being 00 otherwise.
 </li>
 <li>
 <b>Node Clustering</b>: By simply clustering the learned node representations.
 </li>
 </ul>
 <p>
 The broad success of pre-training for natural language processing models
 such as ELMo and BERT 
 has sparked interest in similar techniques for GNNs
 .
 The key idea in each of these papers is to train GNNs to predict
 local (eg. node degrees, clustering coefficient, masked node attributes)
 and/or global graph properties (eg. pairwise distances, masked global attributes).
 </p>
 <p>
 Another self-supervised technique is to enforce that neighbouring nodes get similar embeddings,
 mimicking random-walk approaches such as node2vec and DeepWalk :
 </p>
 LG=∑v∑u∈NR(v)logexpzvTzu∑u′expzu′Tzu.
 L_{G} = \sum_{v} \sum_{u \in N_R(v)} \log\frac{\exp{z_v^T z_u}}{\sum\limits_{u’} \exp{z_{u’}^T z_u}}.
 
 <p>
 where NR(v)N_R(v) is a multi-set of nodes visited when random walks are started from vv.
 For large graphs, where computing the sum over all nodes may be computationally expensive,
 techniques such as Noise Contrastive Estimation are especially useful.
 </p>
 
 <h2>
 Conclusion and Further Reading
 </h2>
 
 <p>
 While we have looked at many techniques and ideas in this article,
 the field of Graph Neural Networks is extremely vast.
 We have been forced to restrict our discussion to a small subset of the entire literature,
 while still communicating the key ideas and design principles behind GNNs.
 We recommend the interested reader take a look at
 for a more comprehensive survey.
 </p>
 <p>
 We end with pointers and references for additional concepts readers might be interested in:
 </p>
 <h3>
 GNNs in Practice
 </h3>
 <p>
 It turns out that accomodating the different structures of graphs is often hard to do efficiently,
 but we can still represent many GNN update equations using
 as sparse matrix-vector products (since generally, the adjacency matrix is sparse for most real-world graph datasets.)
 For example, the GCN variant discussed here can be represented as:
 h(k)=D−1A⋅h(k−1)W(k)T+h(k−1)B(k)T.
 h^{(k)} = D^{-1} A \cdot h^{(k - 1)} {W^{(k)}}^T + h^{(k - 1)} {B^{(k)}}^T.
 
 Restructuring the update equations in this way allows for efficient vectorized implementations of GNNs on accelerators
 such as GPUs.
 </p>
 <p>
 Regularization techniques for standard neural networks,
 such as Dropout ,
 can be applied in a straightforward manner to the parameters
 (for example, zero out entire rows of W(k)W^{(k)} above).
 However, there are graph-specific techniques such as DropEdge 
 that removes entire edges at random from the graph,
 that also boost the performance of many GNN models.
 </p>
 <h3>
 Different Kinds of Graphs
 </h3>
 <p>
 Here, we have focused on undirected graphs, to avoid going into too many unnecessary details.
 However, there are some simple variants of spatial convolutions for:
 </p>
 <ul>
 <li>Directed graphs: Aggregate across in-neighbourhood and/or out-neighbourhood features. </li>
 <li>Temporal graphs: Aggregate across previous and/or future node features.</li>
 <li>Heterogeneous graphs: Learn different aggregation functions for each node/edge type.</li>
 </ul>
 <p>
 There do exist more sophisticated techniques that can take advantage of the different structures of these graphs:
 see for more discussion.
 </p>
 <h3>
 Pooling
 </h3>
 <p>
 This article discusses how GNNs compute useful representations of nodes.
 But what if we wanted to compute representations of graphs for graph-level tasks (for example, predicting the toxicity of a molecule)?
 </p>
 <p>
 A simple solution is to just aggregate the final node embeddings and pass them through another neural network PREDICTG\text{PREDICT}_G:
 hG=PREDICTG(AGGv∈G({hv}))
 h_G = \text{PREDICT}_G \Big( \text{AGG}_{v \in G}\left(\{ h_v \} \right) \Big)
 
 However, there do exist more powerful techniques for ‘pooling’ together node representations:
 </p>
 <ul>
 <li>SortPool: Sort vertices of the graph to get a fixed-size node-order invariant representation of the graph, and then apply any standard neural network architecture.</li>
 <li>DiffPool: Learn to cluster vertices, build a coarser graph over clusters instead of nodes, then apply a GNN over the coarser graph. Repeat until only one cluster is left. </li>
 <li>SAGPool: Apply a GNN to learn node scores, then keep only the nodes with the top scores, throwing away the rest. Repeat until only one node is left.</li>
 </ul>
 
 <h2>
 Supplementary Material
 </h2>
 <h3>
 Reproducing Experiments
 </h3>
 <p>
 The experiments from
 <a href="#spectral-decompositions-of-natural-images">Spectral Representations of Natural Images</a>
 can be reproduced using the following
 Colab notebook:
 <a href="https://colab.research.google.com/github/google-research/google-research/blob/master/understanding_convolutions_on_graphs/SpectralRepresentations.ipynb">Spectral Representations of Natural Images</a>.
 </p>
 <h3>
 Recreating Visualizations
 </h3>
 <p>
 To aid in the creation of future interactive articles,
 we have created ObservableHQ
 
 notebooks for each of the interactive visualizations here:
 </p>
 <ul>
 <li><a href="https://observablehq.com/@ameyasd/neighbourhoods-for-cnns-and-gnns">Neighbourhood Definitions for CNNs and GNNs</a></li>
 <li><a href="https://observablehq.com/@ameyasd/cleaner-interactive-graph-polynomial-convolutions">Graph Polynomial Convolutions on a Grid</a></li>
 <li><a href="https://observablehq.com/@ameyasd/updated-chebnet-equations">Graph Polynomial Convolutions: Equations</a></li>
 <li><a href="https://observablehq.com/@ameyasd/interactive-gnn-equations">Modern Graph Neural Networks: Equations</a></li>
 <li>
 <a href="https://observablehq.com/@ameyasd/interactive-gnn-visualizations">Modern Graph Neural Networks: Interactive Models</a>
 which pulls together the following standalone notebooks:
 <ul>
 <li><a href="https://observablehq.com/@ameyasd/graph-convolutional-networks">Graph Convolutional Networks</a></li>
 <li><a href="https://observablehq.com/@ameyasd/graph-attention-networks">Graph Attention Networks</a></li>
 <li><a href="https://observablehq.com/@ameyasd/graph-sample-and-aggregate-graphsage">GraphSAGE</a></li>
 <li><a href="https://observablehq.com/@ameyasd/graph-isomorphism-networks">Graph Isomorphism Networks</a></li>
 </ul>
 </li>
 <li><a href="https://observablehq.com/@ameyasd/interactive-spectral-conversions">Laplacian Eigenvectors for Grids</a></li>
 <li><a href="https://observablehq.com/@ameyasd/spectral-decompositions-of-natural-images">Spectral Decomposition of Natural Images</a></li>
 <li><a href="https://observablehq.com/@ameyasd/spectral-convolutions-equation">Spectral Convolutions: Equations</a></li>
 
 </ul>
 </div></div>||||I|||| Distill
 About Prize Submit

Understanding Convolutions on Graphs

Understanding the building blocks and design choices of graph neural networks.

 Authors

 Affiliations

 Ameya Daigavane

 Google Research

 Balaraman Ravindran

 Google Research

 Gaurav Aggarwal

 Google Research

 Published

 Sept. 2, 2021

 DOI

 10.23915/distill.00032

Contents

 Introduction
 The Challenges of Computation on Graphs
 * Lack of Consistent Structure
 * Node-Order Equivariance
 * Scalability
 Problem Setting and Notation
 Extending Convolutions to Graphs
 Polynomial Filters on Graphs
 Modern Graph Neural Networks
 Interactive Graph Neural Networks
 From Local to Global Convolutions
 * Spectral Convolutions
 * Global Propagation via Graph Embeddings
 Learning GNN Parameters
 Conclusions and Further Reading
 * GNNs in Practice
 * Different Kinds of Graphs
 * Pooling
 Supplementary Material
 * Reproducing Experiments
 * Recreating Visualizations

 This article is one of two Distill publications about graph neural networks. Take a look at A Gentle Introduction to Graph Neural Networks for a companion view on many things graph and neural network related.

 Many systems and interactions - social networks, molecules, organizations, citations, physical models, transactions - can be represented quite naturally as graphs. How can we reason about and make predictions within these systems?

 One idea is to look at tools that have worked well in other domains: neural networks have shown immense predictive power in a variety of learning tasks. However, neural networks have been traditionally used to operate on fixed-size and/or regular-structured inputs (such as sentences, images and video). This makes them unable to elegantly process graph-structured data.

Graph neural networks (GNNs) are a family of neural networks that can operate naturally on graph-structured data. By extracting and utilizing features from the underlying graph, GNNs can make more informed predictions about entities in these interactions, as compared to models that consider individual entities in isolation.

GNNs are not the only tools available to model graph-structured data: graph kernels and random-walk methods were some of the most popular ones. Today, however, GNNs have largely replaced these techniques because of their inherent flexibility to model the underlying systems better.

In this article, we will illustrate the challenges of computing over graphs, describe the origin and design of graph neural networks, and explore the most popular GNN variants in recent times. Particularly, we will see that many of these variants are composed of similar building blocks.

First, let’s discuss some of the complications that graphs come with.

The Challenges of Computation on Graphs

Lack of Consistent Structure

Graphs are extremely flexible mathematical models; but this means they lack consistent structure across instances. Consider the task of predicting whether a given chemical molecule is toxic :

Left: A non-toxic 1,2,6-trigalloyl-glucose molecule.

Right: A toxic caramboxin molecule.

Looking at a few examples, the following issues quickly become apparent:

 * Molecules may have different numbers of atoms.
 * The atoms in a molecule may be of different types.
 * Each of these atoms may have different number of connections.
 * These connections can have different strengths.

Representing graphs in a format that can be computed over is non-trivial, and the final representation chosen often depends significantly on the actual problem.

Node-Order Equivariance

Extending the point above: graphs often have no inherent ordering present amongst the nodes. Compare this to images, where every pixel is uniquely determined by its absolute position within the image!

Representing the graph as one vector requires us to fix an order on the nodes. But what do we do when the nodes have no inherent order? Above: The same graph labelled in two different ways. The alphabets indicate the ordering of the nodes.

As a result, we would like our algorithms to be node-order equivariant: they should not depend on the ordering of the nodes of the graph. If we permute the nodes in some way, the resulting representations of the nodes as computed by our algorithms should also be permuted in the same way.

Scalability

Graphs can be really large! Think about social networks like Facebook and Twitter, which have over a billion users. Operating on data this large is not easy.

Luckily, most naturally occuring graphs are ‘sparse’: they tend to have their number of edges linear in their number of vertices. We will see that this allows the use of clever methods to efficiently compute representations of nodes within the graph. Further, the methods that we look at here will have significantly fewer parameters in comparison to the size of the graphs they operate on.

Problem Setting and Notation

There are many useful problems that can be formulated over graphs:

 * Node Classification: Classifying individual nodes.
 * Graph Classification: Classifying entire graphs.
 * Node Clustering: Grouping together similar nodes based on connectivity.
 * Link Prediction: Predicting missing links.
 * Influence Maximization: Identifying influential nodes.

Examples of problems that can be defined over graphs. This list is not exhaustive!

A common precursor in solving many of these problems is node representation learning: learning to map individual nodes to fixed-size real-valued vectors (called ‘representations’ or ‘embeddings’).

In Learning GNN Parameters, we will see how the learnt embeddings can be used for these tasks.

Different GNN variants are distinguished by the way these representations are computed. Generally, however, GNNs compute node representations in an iterative process. We will use the notation hv(k)h_v^{(k)} h v ( k ) ​ to indicate the representation of node vv v after the kthk^{\text{th}} k th iteration. Each iteration can be thought of as the equivalent of a ‘layer’ in standard neural networks.

We will define a graph GG G as a set of nodes, VV V , with a set of edges EE E connecting them. Nodes can have individual features as part of the input: we will denote by xvx_v x v ​ the individual feature for node v∈Vv \in V v ∈ V . For example, the ‘node features’ for a pixel in a color image would be the red, green and blue channel (RGB) values at that pixel.

For ease of exposition, we will assume GG G is undirected, and all nodes are of the same type. These kinds of graphs are called ‘homogeneous’. Many of the same ideas we will see here apply to other kinds of graphs: we will discuss this later in Different Kinds of Graphs.

Sometimes we will need to denote a graph property by a matrix MM M , where each row MvM_v M v ​ represents a property corresponding to a particular vertex vv v .

Extending Convolutions to Graphs

Convolutional Neural Networks have been seen to be quite powerful in extracting features from images. However, images themselves can be seen as graphs with a very regular grid-like structure, where the individual pixels are nodes, and the RGB channel values at each pixel as the node features.

A natural idea, then, is to consider generalizing convolutions to arbitrary graphs. Recall, however, the challenges listed out in the previous section: in particular, ordinary convolutions are not node-order invariant, because they depend on the absolute positions of pixels. It is initially unclear as how to generalize convolutions over grids to convolutions over general graphs, where the neighbourhood structure differs from node to node. The curious reader may wonder if performing some sort of padding and ordering could be done to ensure the consistency of neighbourhood structure across nodes. This has been attempted with some success , but the techniques we will look at here are more general and powerful.

Convolutions in CNNs are inherently localized. Neighbours participating in the convolution at the center pixel are highlighted in gray.

GNNs can perform localized convolutions mimicking CNNs. Hover over a node to see its immediate neighbourhood highlighted on the left. The structure of this neighbourhood changes from node to node.

We begin by introducing the idea of constructing polynomial filters over node neighbourhoods, much like how CNNs compute localized filters over neighbouring pixels. Then, we will see how more recent approaches extend on this idea with more powerful mechanisms. Finally, we will discuss alternative methods that can use ‘global’ graph-level information for computing node representations.

Polynomial Filters on Graphs

The Graph Laplacian

Given a graph GG G , let us fix an arbitrary ordering of the nn n nodes of GG G . We denote the 0−10-1 0 − 1 adjacency matrix of GG G by AA A , we can construct the diagonal degree matrix DD D of GG G as:

Dv=∑uAvu. D_v = \sum_u A_{vu}. D v ​ = u ∑ ​ A v u ​ .

The degree of node vv v is the number of edges incident at vv v .

where AvuA_{vu} A v u ​ denotes the entry in the row corresponding to vv v and the column corresponding to uu u in the matrix AA A . We will use this notation throughout this section.

Then, the graph Laplacian LL L is the square n×nn \times n n × n matrix defined as: L=D−A. L = D - A. L = D − A .

The Laplacian LL L for an undirected graph GG G , with the row corresponding to node C\textsf{C} C highlighted. Zeros in LL L are not displayed above. The Laplacian LL L depends only on the structure of the graph GG G , not on any node features.

The graph Laplacian gets its name from being the discrete analog of the Laplacian operator from calculus.

Although it encodes precisely the same information as the adjacency matrix AA A In the sense that given either of the matrices AA A or LL L , you can construct the other. , the graph Laplacian has many interesting properties of its own. The graph Laplacian shows up in many mathematical problems involving graphs: random walks, spectral clustering, and diffusion, to name a few. We will see some of these properties in a later section, but will instead point readers to this tutorial for greater insight into the graph Laplacian.

Polynomials of the Laplacian

Now that we have understood what the graph Laplacian is, we can build polynomials of the form: pw(L)=w0In+w1L+w2L2+…+wdLd=∑i=0dwiLi. p_w(L) = w_0 I_n + w_1 L + w_2 L^2 + \ldots + w_d L^d = \sum_{i = 0}^d w_i L^i. p w ​ ( L ) = w 0 ​ I n ​ + w 1 ​ L + w 2 ​ L 2 + … + w d ​ L d = i = 0 ∑ d ​ w i ​ L i . Each polynomial of this form can alternately be represented by its vector of coefficients w=[w0,…,wd]w = [w_0, \ldots, w_d] w = [ w 0 ​ , … , w d ​ ] . Note that for every ww w , pw(L)p_w(L) p w ​ ( L ) is an n×nn \times n n × n matrix, just like LL L .

These polynomials can be thought of as the equivalent of ‘filters’ in CNNs, and the coefficients ww w as the weights of the ‘filters’.

For ease of exposition, we will focus on the case where nodes have one-dimensional features: each of the xvx_v x v ​ for v∈Vv \in V v ∈ V is just a real number. The same ideas hold when each of the xvx_v x v ​ are higher-dimensional vectors, as well.

Using the previously chosen ordering of the nodes, we can stack all of the node features xvx_v x v ​ to get a vector x∈Rnx \in \mathbb{R}^n x ∈ R n .

Fixing a node order (indicated by the alphabets) and collecting all node features into a single vector xx x .

Once we have constructed the feature vector xx x , we can define its convolution with a polynomial filter pwp_w p w ​ as: x′=pw(L) x x’ = p_w(L) \ x x ′ = p w ​ ( L ) x To understand how the coefficients ww w affect the convolution, let us begin by considering the ‘simplest’ polynomial: when w0=1w_0 = 1 w 0 ​ = 1 and all of the other coefficients are 00 0 . In this case, x′x’ x ′ is just xx x : x′=pw(L) x=∑i=0dwiLix=w0Inx=x. x’ = p_w(L) \ x = \sum_{i = 0}^d w_i L^ix = w_0 I_n x = x. x ′ = p w ​ ( L ) x = i = 0 ∑ d ​ w i ​ L i x = w 0 ​ I n ​ x = x . Now, if we increase the degree, and consider the case where instead w1=1w_1 = 1 w 1 ​ = 1 and and all of the other coefficients are 00 0 . Then, x′x’ x ′ is just LxLx L x , and so: xv′=(Lx)v=Lvx=∑u∈GLvuxu=∑u∈G(Dvu−Avu)xu=Dv xv−∑u∈N(v)xu \begin{aligned} x’_v = (Lx)_v &= L_v x \\ &= \sum_{u \in G} L_{vu} x_u \\ &= \sum_{u \in G} (D_{vu} - A_{vu}) x_u \\ &= D_v \ x_v - \sum_{u \in \mathcal{N}(v)} x_u \end{aligned} x v ′ ​ = ( L x ) v ​ ​ = L v ​ x = u ∈ G ∑ ​ L v u ​ x u ​ = u ∈ G ∑ ​ ( D v u ​ − A v u ​ ) x u ​ = D v ​ x v ​ − u ∈ N ( v ) ∑ ​ x u ​ ​ We see that the features at each node vv v are combined with the features of its immediate neighbours u∈N(v)u \in \mathcal{N}(v) u ∈ N ( v ) . For readers familiar with Laplacian filtering of images, this is the exact same idea. When xx x is an image, x′=Lxx’ = Lx x ′ = L x is exactly the result of applying a ‘Laplacian filter’ to xx x .

At this point, a natural question to ask is: How does the degree dd d of the polynomial influence the behaviour of the convolution? Indeed, it is not too hard to show that: This is Lemma 5.2 from . distG(v,u)>i⟹Lvui=0. \text{dist}_G(v, u) > i \quad \Longrightarrow \quad L_{vu}^i = 0. dist G ​ ( v , u ) > i ⟹ L v u i ​ = 0 . This implies, when we convolve xx x with pw(L)p_w(L) p w ​ ( L ) of degree dd d to get x′x’ x ′ : xv′=(pw(L)x)v=(pw(L))vx=∑i=0dwiLvix=∑i=0dwi∑u∈GLvuixu=∑i=0dwi∑u∈GdistG(v,u)≤iLvuixu. \begin{aligned} x’_v = (p_w(L)x)_v &= (p_w(L))_v x \\ &= \sum_{i = 0}^d w_i L_v^i x \\ &= \sum_{i = 0}^d w_i \sum_{u \in G} L_{vu}^i x_u \\ &= \sum_{i = 0}^d w_i \sum_{u \in G \atop \text{dist}_G(v, u) \leq i} L_{vu}^i x_u. \end{aligned} x v ′ ​ = ( p w ​ ( L ) x ) v ​ ​ = ( p w ​ ( L ) ) v ​ x = i = 0 ∑ d ​ w i ​ L v i ​ x = i = 0 ∑ d ​ w i ​ u ∈ G ∑ ​ L v u i ​ x u ​ = i = 0 ∑ d ​ w i ​ dist G ​ ( v , u ) ≤ i u ∈ G ​ ∑ ​ L v u i ​ x u ​ . ​

Effectively, the convolution at node vv v occurs only with nodes uu u which are not more than dd d hops away. Thus, these polynomial filters are localized. The degree of the localization is governed completely by dd d .

To help you understand these ‘polynomial-based’ convolutions better, we have created the visualization below. Vary the polynomial coefficients and the input grid xx x to see how the result x′x’ x ′ of the convolution changes. The grid under the arrow shows the equivalent convolutional kernel applied at the highlighted pixel in xx x to get the resulting pixel in x′x’ x ′ . The kernel corresponds to the row of pw(L)p_w(L) p w ​ ( L ) for the highlighted pixel. Note that even after adjusting for position, this kernel is different for different pixels, depending on their position within the grid.



Hover over a pixel in the input grid (left, representing xx x ) to highlight it and see the equivalent convolutional kernel for that pixel under the arrow. The result x′x’ x ′ of the convolution is shown on the right: note that different convolutional kernels are applied at different pixels, depending on their location.

Click on the input grid to toggle pixel values between 00 0 (white) and 11 1 (blue). To randomize the input grid, press ‘Randomize Grid’. To reset all pixels to 00 0 , press ‘Reset Grid’. Use the sliders at the bottom to change the coefficients ww w . To reset all coefficients ww w to 00 0 , press ‘Reset Coefficients.’

ChebNet

ChebNet refines this idea of polynomial filters by looking at polynomial filters of the form: pw(L)=∑i=1dwiTi(L~) p_w(L) = \sum_{i = 1}^d w_i T_i(\tilde{L}) p w ​ ( L ) = i = 1 ∑ d ​ w i ​ T i ​ ( L ~ ) where TiT_i T i ​ is the degree- ii i Chebyshev polynomial of the first kind and L~\tilde{L} L ~ is the normalized Laplacian defined using the largest eigenvalue of LL L : We discuss the eigenvalues of the Laplacian LL L in more detail in a later section. L~=2Lλmax(L)−In. \tilde{L} = \frac{2L}{\lambda_{\max}(L)} - I_n. L ~ = λ max ​ ( L ) 2 L ​ − I n ​ .

What is the motivation behind these choices?

 * LL L is actually positive semi-definite: all of the eigenvalues of LL L are not lesser than 00 0 . If λmax(L)>1\lambda_{\max}(L) > 1 λ max ​ ( L ) > 1 , the entries in the powers of LL L rapidly increase in size. L~\tilde{L} L ~ is effectively a scaled-down version of LL L , with eigenvalues guaranteed to be in the range [−1,1][-1, 1] [ − 1 , 1 ] . This prevents the entries of powers of L~\tilde{L} L ~ from blowing up. Indeed, in the visualization above: we restrict the higher-order coefficients when the unnormalized Laplacian LL L is selected, but allow larger values when the normalized Laplacian L~\tilde{L} L ~ is selected, in order to show the result x′x’ x ′ on the same color scale.
 * The Chebyshev polynomials have certain interesting properties that make interpolation more numerically stable. We won’t talk about this in more depth here, but will advise interested readers to take a look at as a definitive resource.

Polynomial Filters are Node-Order Equivariant

The polynomial filters we considered here are actually independent of the ordering of the nodes. This is particularly easy to see when the degree of the polynomial pwp_w p w ​ is 11 1 : where each node’s feature is aggregated with the sum of its neighbour’s features. Clearly, this sum does not depend on the order of the neighbours. A similar proof follows for higher degree polynomials: the entries in the powers of LL L are equivariant to the ordering of the nodes.

 Details for the Interested Reader

 As above, let’s assume an arbitrary node-order over the nn n nodes of our graph. Any other node-order can be thought of as a permutation of this original node-order. We can represent any permutation by a permutation matrix PP P . PP P will always be an orthogonal 0−10-1 0 − 1 matrix: PPT=PTP=In. PP^T = P^TP = I_n. P P T = P T P = I n ​ . Then, we call a function ff f node-order equivariant iff for all permutations PP P : f(Px)=Pf(x). f(Px) = P f(x). f ( P x ) = P f ( x ) . When switching to the new node-order using the permutation PP P , the quantities below transform in the following way: x→PxL→PLPTLi→PLiPT \begin{aligned} x &\to Px \\ L &\to PLP^T \\ L^i &\to PL^iP^T \end{aligned} x L L i ​ → P x → P L P T → P L i P T ​ and so, for the case of polynomial filters where f(x)=pw(L) xf(x) = p_w(L) \ x f ( x ) = p w ​ ( L ) x , we can see that: f(Px)=∑i=0dwi(PLiPT)(Px)=P∑i=0dwiLix=Pf(x). \begin{aligned} f(Px) & = \sum_{i = 0}^d w_i (PL^iP^T) (Px) \\ & = P \sum_{i = 0}^d w_i L^i x \\ & = P f(x). \end{aligned} f ( P x ) ​ = i = 0 ∑ d ​ w i ​ ( P L i P T ) ( P x ) = P i = 0 ∑ d ​ w i ​ L i x = P f ( x ) . ​ as claimed.

Embedding Computation

We now describe how we can build a graph neural network by stacking ChebNet (or any polynomial filter) layers one after the other with non-linearities, much like a standard CNN. In particular, if we have KK K different polynomial filter layers, the kthk^{\text{th}} k th of which has its own learnable weights w(k)w^{(k)} w ( k ) , we would perform the following computation:

Note that these networks reuse the same filter weights across different nodes, exactly mimicking weight-sharing in Convolutional Neural Networks (CNNs) which reuse weights for convolutional filters across a grid.

Modern Graph Neural Networks

ChebNet was a breakthrough in learning localized filters over graphs, and it motivated many to think of graph convolutions from a different perspective.

We return back to the result of convolving xx x by the polynomial kernel pw(L)=Lp_w(L) = L p w ​ ( L ) = L , focussing on a particular vertex vv v : (Lx)v=Lvx=∑u∈GLvuxu=∑u∈G(Dvu−Avu)xu=Dv xv−∑u∈N(v)xu \begin{aligned} (Lx)_v &= L_v x \\ &= \sum_{u \in G} L_{vu} x_u \\ &= \sum_{u \in G} (D_{vu} - A_{vu}) x_u \\ &= D_v \ x_v - \sum_{u \in \mathcal{N}(v)} x_u \end{aligned} ( L x ) v ​ ​ = L v ​ x = u ∈ G ∑ ​ L v u ​ x u ​ = u ∈ G ∑ ​ ( D v u ​ − A v u ​ ) x u ​ = D v ​ x v ​ − u ∈ N ( v ) ∑ ​ x u ​ ​

As we noted before, this is a 11 1 -hop localized convolution. But more importantly, we can think of this convolution as arising of two steps:

 * Aggregating over immediate neighbour features xux_u x u ​ .
 * Combining with the node’s own feature xvx_v x v ​ .

 Key Idea: What if we consider different kinds of ‘aggregation’ and ‘combination’ steps, beyond what are possible using polynomial filters?

By ensuring that the aggregation is node-order equivariant, the overall convolution becomes node-order equivariant.

These convolutions can be thought of as ‘message-passing’ between adjacent nodes: after each step, every node receives some ‘information’ from its neighbours.

By iteratively repeating the 11 1 -hop localized convolutions KK K times (i.e., repeatedly ‘passing messages’), the receptive field of the convolution effectively includes all nodes upto KK K hops away.

Embedding Computation

Message-passing forms the backbone of many GNN architectures today. We describe the most popular ones in depth below:

 * Graph Convolutional Networks (GCN)
 * Graph Attention Networks (GAT)
 * Graph Sample and Aggregate (GraphSAGE)
 * Graph Isomorphism Network (GIN)

Thoughts

An interesting point is to assess different aggregation functions: are some better and others worse? demonstrates that aggregation functions indeed can be compared on how well they can uniquely preserve node neighbourhood features; we recommend the interested reader take a look at the detailed theoretical analysis there.

Here, we’ve talk about GNNs where the computation only occurs at the nodes. More recent GNN models such as Message-Passing Neural Networks and Graph Networks perform computation over the edges as well; they compute edge embeddings together with node embeddings. This is an even more general framework - but the same ‘message passing’ ideas from this section apply.

Interactive Graph Neural Networks

Below is an interactive visualization of these GNN models on small graphs. For clarity, the node features are just real numbers here, shown inside the squares next to each node, but the same equations hold when the node features are vectors.

Choose a GNN model using the tabs at the top. Click on a node to see the update equation at that node for the next iteration. Use the sliders on the left to change the weights for the current iteration, and watch how the update equation changes.

In practice, each iteration above is generally thought of as a single ‘neural network layer’. This ideology is followed by many popular Graph Neural Network libraries, For example: PyTorch Geometric and StellarGraph. allowing one to compose different types of graph convolutions in the same model.

From Local to Global Convolutions

The methods we’ve seen so far perform ‘local’ convolutions: every node’s feature is updated using a function of its local neighbours’ features.

While performing enough steps of message-passing will eventually ensure that information from all nodes in the graph is passed, one may wonder if there are more direct ways to perform ‘global’ convolutions.

The answer is yes; we will now describe an approach that was actually first put forward in the context of neural networks by , much before any of the GNN models we looked at above.

Spectral Convolutions

As before, we will focus on the case where nodes have one-dimensional features. After choosing an arbitrary node-order, we can stack all of the node features to get a ‘feature vector’ x∈Rnx \in \mathbb{R}^n x ∈ R n .

 Key Idea: Given a feature vector xx x , the Laplacian LL L allows us to quantify how smooth xx x is, with respect to GG G .

How?

After normalizing xx x such that ∑i=1nxi2=1\sum_{i = 1}^n x_i^2 = 1 ∑ i = 1 n ​ x i 2 ​ = 1 , if we look at the following quantity involving LL L : RLR_L R L ​ is formally called the Rayleigh quotient. RL(x)=xTLxxTx=∑(i,j)∈E(xi−xj)2∑ixi2=∑(i,j)∈E(xi−xj)2. R_L(x) = \frac{x^T L x}{x^T x} = \frac{\sum_{(i, j) \in E} (x_i - x_j)^2}{\sum_i x_i^2} = \sum_{(i, j) \in E} (x_i - x_j)^2. R L ​ ( x ) = x T x x T L x ​ = ∑ i ​ x i 2 ​ ∑ ( i , j ) ∈ E ​ ( x i ​ − x j ​ ) 2 ​ = ( i , j ) ∈ E ∑ ​ ( x i ​ − x j ​ ) 2 . we immediately see that feature vectors xx x that assign similar values to adjacent nodes in GG G (hence, are smooth) would have smaller values of RL(x)R_L(x) R L ​ ( x ) .

LL L is a real, symmetric matrix, which means it has all real eigenvalues λ1≤…≤λn\lambda_1 \leq \ldots \leq \lambda_{n} λ 1 ​ ≤ … ≤ λ n ​ . An eigenvalue λ\lambda λ of a matrix AA A is a value satisfying the equation Au=λuAu = \lambda u A u = λ u for a certain vector uu u , called an eigenvector. For a friendly introduction to eigenvectors, please see this tutorial. Further, the corresponding eigenvectors u1,…,unu_1, \ldots, u_{n} u 1 ​ , … , u n ​ can be taken to be orthonormal: uk1Tuk2={1 if k1=k2.0 if k1≠k2. u_{k_1}^T u_{k_2} = \begin{cases} 1 \quad \text{ if } {k_1} = {k_2}. \\ 0 \quad \text{ if } {k_1} \neq {k_2}. \end{cases} u k 1 ​ T ​ u k 2 ​ ​ = { 1 if k 1 ​ = k 2 ​ . 0 if k 1 ​ ≠ k 2 ​ . ​ It turns out that these eigenvectors of LL L are successively less smooth, as RLR_L R L ​ indicates: This is the min-max theorem for eigenvalues. argminx, x⊥{u1,…,ui−1}RL(x)=ui.minx, x⊥{u1,…,ui−1}RL(x)=λi. \underset{x, \ x \perp \{u_1, \ldots, u_{i - 1}\}}{\text{argmin}} R_L(x) = u_i. \qquad \qquad \qquad \min_{x, \ x \perp \{u_1, \ldots, u_{i - 1}\}} R_L(x) = \lambda_i. x , x ⊥ { u 1 ​ , … , u i − 1 ​ } argmin ​ R L ​ ( x ) = u i ​ . x , x ⊥ { u 1 ​ , … , u i − 1 ​ } min ​ R L ​ ( x ) = λ i ​ . The set of eigenvalues of LL L are called its ‘spectrum’, hence the name! We denote the ‘spectral’ decomposition of LL L as: L=UΛUT. L = U \Lambda U^T. L = U Λ U T . where Λ\Lambda Λ is the diagonal matrix of sorted eigenvalues, and UU U denotes the matrix of the eigenvectors (sorted corresponding to increasing eigenvalues): Λ=[λ1⋱λn]U=[u1 ⋯ un]. \Lambda = \begin{bmatrix} \lambda_{1} & & \\ & \ddots & \\ & & \lambda_{n} \end{bmatrix} \qquad \qquad \qquad \qquad U = \begin{bmatrix} \\ u_1 \ \cdots \ u_n \\ \end{bmatrix}. Λ = ⎣ ⎡ ​ λ 1 ​ ​ ⋱ ​ λ n ​ ​ ⎦ ⎤ ​ U = ⎣ ⎡ ​ u 1 ​ ⋯ u n ​ ​ ⎦ ⎤ ​ . The orthonormality condition between eigenvectors gives us that UTU=IU^T U = I U T U = I , the identity matrix. As these nn n eigenvectors form a basis for Rn\mathbb{R}^n R n , any feature vector xx x can be represented as a linear combination of these eigenvectors: x=∑i=1nxi^ui=Ux^. x = \sum_{i = 1}^n \hat{x_i} u_i = U \hat{x}. x = i = 1 ∑ n ​ x i ​ ^ ​ u i ​ = U x ^ . where x^\hat{x} x ^ is the vector of coefficients [x0,…xn][x_0, \ldots x_n] [ x 0 ​ , … x n ​ ] . We call x^\hat{x} x ^ as the spectral representation of the feature vector xx x . The orthonormality condition allows us to state: x=Ux^⟺UTx=x^. x = U \hat{x} \quad \Longleftrightarrow \quad U^T x = \hat{x}. x = U x ^ ⟺ U T x = x ^ . This pair of equations allows us to interconvert between the ‘natural’ representation xx x and the ‘spectral’ representation x^\hat{x} x ^ for any vector x∈Rnx \in \mathbb{R}^n x ∈ R n .

Spectral Representations of Natural Images

As discussed before, we can consider any image as a grid graph, where each pixel is a node, connected by edges to adjacent pixels. Thus, a pixel can have either 3,5,3, 5, 3 , 5 , or 88 8 neighbours, depending on its location within the image grid. Each pixel gets a value as part of the image. If the image is grayscale, each value will be a single real number indicating how dark the pixel is. If the image is colored, each value will be a 33 3 -dimensional vector, indicating the values for the red, green and blue (RGB) channels. We use the alpha channel as well in the visualization below, so this is actually RGBA.

This construction allows us to compute the graph Laplacian and the eigenvector matrix UU U . Given an image, we can then investigate what its spectral representation looks like.

To shed some light on what the spectral representation actually encodes, we perform the following experiment over each channel of the image independently:

 * We first collect all pixel values across a channel into a feature vector xx x .
 * Then, we obtain its spectral representation x^\hat{x} x ^ . x^=UTx \hat{x} = U^T x x ^ = U T x
 * We truncate this to the first mm m components to get x^m\hat{x}_m x ^ m ​ . By truncation, we mean zeroing out all of the remaining n−mn - m n − m components of x^\hat{x} x ^ . This truncation is equivalent to using only the first mm m eigenvectors to compute the spectral representation. x^m=Truncatem(x^) \hat{x}_m = \text{Truncate}_m(\hat{x}) x ^ m ​ = Truncate m ​ ( x ^ )
 * Then, we convert this truncated representation x^m\hat{x}_m x ^ m ​ back to the natural basis to get xmx_m x m ​ . xm=Ux^m x_m = U \hat{x}_m x m ​ = U x ^ m ​

Finally, we stack the resulting channels back together to get back an image. We can now see how the resulting image changes with choices of mm m . Note that when m=nm = n m = n , the resulting image is identical to the original image, as we can reconstruct each channel exactly.

Use the radio buttons at the top to chose one of the four sample images. Each of these images has been taken from the ImageNet dataset and downsampled to 5050 5 0 pixels wide and 4040 4 0 pixels tall. As there are n=50×40=2000n = 50 \times 40 = 2000 n = 5 0 × 4 0 = 2 0 0 0 pixels in each image, there are 20002000 2 0 0 0 Laplacian eigenvectors. Use the slider at the bottom to change the number of spectral components to keep, noting how images get progressively blurrier as the number of components decrease.

As mm m decreases, we see that the output image xmx_m x m ​ gets blurrier. If we decrease mm m to 11 1 , the output image xmx_m x m ​ is entirely the same color throughout. We see that we do not need to keep all nn n components; we can retain a lot of the information in the image with significantly fewer components. We can relate this to the Fourier decomposition of images: the more eigenvectors we use, the higher frequencies we can represent on the grid.

To complement the visualization above, we additionally visualize the first few eigenvectors on a smaller 8×88 \times 8 8 × 8 grid below. We change the coefficients of the first 1010 1 0 out of 6464 6 4 eigenvectors in the spectral representation and see how the resulting image changes:


Move the sliders to change the spectral representation x^\hat{x} x ^ (right), and see how xx x itself changes on the image (left). Note how the first eigenvectors are much ‘smoother’ than the later ones, and the many patterns we can make with only 1010 1 0 eigenvectors.

These visualizations should convince you that the first eigenvectors are indeed smooth, and the smoothness correspondingly decreases as we consider later eigenvectors.

For any image xx x , we can think of the initial entries of the spectral representation x^\hat{x} x ^ as capturing ‘global’ image-wide trends, which are the low-frequency components, while the later entries as capturing ‘local’ details, which are the high-frequency components.

Embedding Computation

We now have the background to understand spectral convolutions and how they can be used to compute embeddings/feature representations of nodes.

As before, the model we describe below has KK K layers: each layer kk k has learnable parameters w^(k)\hat{w}^{(k)} w ^ ( k ) , called the ‘filter weights’. These weights will be convolved with the spectral representations of the node features. As a result, the number of weights needed in each layer is equal to mm m , the number of eigenvectors used to compute the spectral representations. We had shown in the previous section that we can take m≪nm \ll n m ≪ n and still not lose out on significant amounts of information.

Thus, convolution in the spectral domain enables the use of significantly fewer parameters than just direct convolution in the natural domain. Further, by virtue of the smoothness of the Laplacian eigenvectors across the graph, using spectral representations automatically enforces an inductive bias for neighbouring nodes to get similar representations.

Assuming one-dimensional node features for now, the output of each layer is a vector of node representations h(k)h^{(k)} h ( k ) , where each node’s representation corresponds to a row of the vector.

We fix an ordering of the nodes in GG G . This gives us the adjacency matrix AA A and the graph Laplacian LL L , allowing us to compute UmU_m U m ​ . Finally, we can describe the computation that the layers perform, one after the other:

The method above generalizes easily to the case where each h(k)∈Rdkh^{(k)} \in \mathbb{R}^{d_k} h ( k ) ∈ R d k ​ , as well: see for details.

With the insights from the previous section, we see that convolution in the spectral-domain of graphs can be thought of as the generalization of convolution in the frequency-domain of images.

Spectral Convolutions are Node-Order Equivariant

We can show spectral convolutions are node-order equivariant using a similar approach as for Laplacian polynomial filters.

 Details for the Interested Reader

 As in our proof before, let’s fix an arbitrary node-order. Then, any other node-order can be represented by a permutation of this original node-order. We can associate this permutation with its permutation matrix PP P . Under this new node-order, the quantities below transform in the following way: x→PxA→PAPTL→PLPTUm→PUm \begin{aligned} x &\to Px \\ A &\to PAP^T \\ L &\to PLP^T \\ U_m &\to PU_m \end{aligned} x A L U m ​ ​ → P x → P A P T → P L P T → P U m ​ ​ which implies that, in the embedding computation: x^→(PUm)T(Px)=UmTx=x^w^→(PUm)T(Pw)=UmTw=w^g^→g^g→(PUm)g^=P(Umg^)=Pg \begin{aligned} \hat{x} &\to \left(PU_m\right)^T (Px) = U_m^T x = \hat{x} \\ \hat{w} &\to \left(PU_m\right)^T (Pw) = U_m^T w = \hat{w} \\ \hat{g} &\to \hat{g} \\ g &\to (PU_m)\hat{g} = P(U_m\hat{g}) = Pg \end{aligned} x ^ w ^ g ^ ​ g ​ → ( P U m ​ ) T ( P x ) = U m T ​ x = x ^ → ( P U m ​ ) T ( P w ) = U m T ​ w = w ^ → g ^ ​ → ( P U m ​ ) g ^ ​ = P ( U m ​ g ^ ​ ) = P g ​ Hence, as σ\sigma σ is applied elementwise: f(Px)=σ(Pg)=Pσ(g)=Pf(x) f(Px) = \sigma(Pg) = P \sigma(g) = P f(x) f ( P x ) = σ ( P g ) = P σ ( g ) = P f ( x ) as required. Further, we see that the spectral quantities x^,w^\hat{x}, \hat{w} x ^ , w ^ and g^\hat{g} g ^ ​ are unchanged by permutations of the nodes. Formally, they are what we would call node-order invariant.

The theory of spectral convolutions is mathematically well-grounded; however, there are some key disadvantages that we must talk about:

 * We need to compute the eigenvector matrix UmU_m U m ​ from LL L . For large graphs, this becomes quite infeasible.
 * Even if we can compute UmU_m U m ​ , global convolutions themselves are inefficient to compute, because of the repeated multiplications with UmU_m U m ​ and UmTU_m^T U m T ​ .
 * The learned filters are specific to the input graphs, as they are represented in terms of the spectral decomposition of input graph Laplacian LL L . This means they do not transfer well to new graphs which have significantly different structure (and hence, significantly different eigenvalues) .

While spectral convolutions have largely been superseded by ‘local’ convolutions for the reasons discussed above, there is still much merit to understanding the ideas behind them. Indeed, a recently proposed GNN model called Directional Graph Networks actually uses the Laplacian eigenvectors and their mathematical properties extensively.

Global Propagation via Graph Embeddings

A simpler way to incorporate graph-level information is to compute embeddings of the entire graph by pooling node (and possibly edge) embeddings, and then using the graph embedding to update node embeddings, following an iterative scheme similar to what we have looked at here. This is an approach used by Graph Networks . We will briefly discuss how graph-level embeddings can be constructed in Pooling. However, such approaches tend to ignore the underlying topology of the graph that spectral convolutions can capture.

Learning GNN Parameters

All of the embedding computations we’ve described here, whether spectral or spatial, are completely differentiable. This allows GNNs to be trained in an end-to-end fashion, just like a standard neural network, once a suitable loss function L\mathcal{L} L is defined:

 * Node Classification: By minimizing any of the standard losses for classification tasks, such as categorical cross-entropy when multiple classes are present: L(yv,yv^)=−∑cyvclogyvc^. \mathcal{L}(y_v, \hat{y_v}) = -\sum_{c} y_{vc} \log{\hat{y_{vc}}}. L ( y v ​ , y v ​ ^ ​ ) = − c ∑ ​ y v c ​ lo g y v c ​ ^ ​ . where yvc^\hat{y_{vc}} y v c ​ ^ ​ is the predicted probability that node vv v is in class cc c . GNNs adapt well to the semi-supervised setting, which is when only some nodes in the graph are labelled. In this setting, one way to define a loss LG\mathcal{L}_{G} L G ​ over an input graph GG G is: LG=∑v∈Lab(G)L(yv,yv^)∣Lab(G)∣ \mathcal{L}_{G} = \frac{\sum\limits_{v \in \text{Lab}(G)} \mathcal{L}(y_v, \hat{y_v})}{| \text{Lab}(G) |} L G ​ = ∣ Lab ( G ) ∣ v ∈ Lab ( G ) ∑ ​ L ( y v ​ , y v ​ ^ ​ ) ​ where, we only compute losses over labelled nodes Lab(G)\text{Lab}(G) Lab ( G ) .
 * Graph Classification: By aggregating node representations, one can construct a vector representation of the entire graph. This graph representation can be used for any graph-level task, even beyond classification. See Pooling for how representations of graphs can be constructed.
 * Link Prediction: By sampling pairs of adjacent and non-adjacent nodes, and use these vector pairs as inputs to predict the presence/absence of an edge. For a concrete example, by minimizing the following ‘logistic regression’-like loss: L(yv,yu,evu)=−evulog(pvu)−(1−evu)log(1−pvu)pvu=σ(yvTyu) \begin{aligned} \mathcal{L}(y_v, y_u, e_{vu}) &= -e_{vu} \log(p_{vu}) - (1 - e_{vu}) \log(1 - p_{vu}) \\ p_{vu} &= \sigma(y_v^Ty_u) \end{aligned} L ( y v ​ , y u ​ , e v u ​ ) p v u ​ ​ = − e v u ​ lo g ( p v u ​ ) − ( 1 − e v u ​ ) lo g ( 1 − p v u ​ ) = σ ( y v T ​ y u ​ ) ​ where σ\sigma σ is the sigmoid function, and evu=1e_{vu} = 1 e v u ​ = 1 iff there is an edge between nodes vv v and uu u , being 00 0 otherwise.
 * Node Clustering: By simply clustering the learned node representations.

The broad success of pre-training for natural language processing models such as ELMo and BERT has sparked interest in similar techniques for GNNs . The key idea in each of these papers is to train GNNs to predict local (eg. node degrees, clustering coefficient, masked node attributes) and/or global graph properties (eg. pairwise distances, masked global attributes).

Another self-supervised technique is to enforce that neighbouring nodes get similar embeddings, mimicking random-walk approaches such as node2vec and DeepWalk :

LG=∑v∑u∈NR(v)logexpzvTzu∑u′expzu′Tzu. L_{G} = \sum_{v} \sum_{u \in N_R(v)} \log\frac{\exp{z_v^T z_u}}{\sum\limits_{u’} \exp{z_{u’}^T z_u}}. L G ​ = v ∑ ​ u ∈ N R ​ ( v ) ∑ ​ lo g u ′ ∑ ​ exp z u ′ T ​ z u ​ exp z v T ​ z u ​ ​ .

where NR(v)N_R(v) N R ​ ( v ) is a multi-set of nodes visited when random walks are started from vv v . For large graphs, where computing the sum over all nodes may be computationally expensive, techniques such as Noise Contrastive Estimation are especially useful.

Conclusion and Further Reading

While we have looked at many techniques and ideas in this article, the field of Graph Neural Networks is extremely vast. We have been forced to restrict our discussion to a small subset of the entire literature, while still communicating the key ideas and design principles behind GNNs. We recommend the interested reader take a look at for a more comprehensive survey.

We end with pointers and references for additional concepts readers might be interested in:

GNNs in Practice

It turns out that accomodating the different structures of graphs is often hard to do efficiently, but we can still represent many GNN update equations using as sparse matrix-vector products (since generally, the adjacency matrix is sparse for most real-world graph datasets.) For example, the GCN variant discussed here can be represented as: h(k)=D−1A⋅h(k−1)W(k)T+h(k−1)B(k)T. h^{(k)} = D^{-1} A \cdot h^{(k - 1)} {W^{(k)}}^T + h^{(k - 1)} {B^{(k)}}^T. h ( k ) = D − 1 A ⋅ h ( k − 1 ) W ( k ) T + h ( k − 1 ) B ( k ) T . Restructuring the update equations in this way allows for efficient vectorized implementations of GNNs on accelerators such as GPUs.

Regularization techniques for standard neural networks, such as Dropout , can be applied in a straightforward manner to the parameters (for example, zero out entire rows of W(k)W^{(k)} W ( k ) above). However, there are graph-specific techniques such as DropEdge that removes entire edges at random from the graph, that also boost the performance of many GNN models.

Different Kinds of Graphs

Here, we have focused on undirected graphs, to avoid going into too many unnecessary details. However, there are some simple variants of spatial convolutions for:

 * Directed graphs: Aggregate across in-neighbourhood and/or out-neighbourhood features.
 * Temporal graphs: Aggregate across previous and/or future node features.
 * Heterogeneous graphs: Learn different aggregation functions for each node/edge type.

There do exist more sophisticated techniques that can take advantage of the different structures of these graphs: see for more discussion.

Pooling

This article discusses how GNNs compute useful representations of nodes. But what if we wanted to compute representations of graphs for graph-level tasks (for example, predicting the toxicity of a molecule)?

A simple solution is to just aggregate the final node embeddings and pass them through another neural network PREDICTG\text{PREDICT}_G PREDICT G ​ : hG=PREDICTG(AGGv∈G({hv})) h_G = \text{PREDICT}_G \Big( \text{AGG}_{v \in G}\left(\{ h_v \} \right) \Big) h G ​ = PREDICT G ​ ( AGG v ∈ G ​ ( { h v ​ } ) ) However, there do exist more powerful techniques for ‘pooling’ together node representations:

 * SortPool: Sort vertices of the graph to get a fixed-size node-order invariant representation of the graph, and then apply any standard neural network architecture.
 * DiffPool: Learn to cluster vertices, build a coarser graph over clusters instead of nodes, then apply a GNN over the coarser graph. Repeat until only one cluster is left.
 * SAGPool: Apply a GNN to learn node scores, then keep only the nodes with the top scores, throwing away the rest. Repeat until only one node is left.

Supplementary Material

Reproducing Experiments

The experiments from Spectral Representations of Natural Images can be reproduced using the following Colab notebook: Spectral Representations of Natural Images.

Recreating Visualizations

To aid in the creation of future interactive articles, we have created ObservableHQ notebooks for each of the interactive visualizations here:

 * Neighbourhood Definitions for CNNs and GNNs
 * Graph Polynomial Convolutions on a Grid
 * Graph Polynomial Convolutions: Equations
 * Modern Graph Neural Networks: Equations
 * Modern Graph Neural Networks: Interactive Models which pulls together the following standalone notebooks:
 + Graph Convolutional Networks
 + Graph Attention Networks
 + GraphSAGE
 + Graph Isomorphism Networks
 * Laplacian Eigenvectors for Grids
 * Spectral Decomposition of Natural Images
 * Spectral Convolutions: Equations

Acknowledgments

We are deeply grateful to ObservableHQ, a wonderful platform for developing interactive visualizations. The static visualizations would not have been possible without Inkscape and Alexander Lenail’s Neural Network SVG Generator. The molecule diagrams depicted above were obtained and modified from Wikimedia Commons, available in the public domain.

We would like to acknowledge the following Distill articles for inspiration on article design:

 * Visualizing memorization in RNNs
 * Understanding RL Vision

We would like to thank Thomas Kipf for his valuable feedback on the technical content within this article.

We would like to thank David Nichols for creating Coloring for Colorblindness which helped us improve the accessibility of this article’s color scheme.

We would also like to acknowledge CS224W: Machine Learning with Graphs as an excellent reference from which the authors benefitted significantly.

Ashish Tendulkar from Google Research India provided significant feedback on the content within this article, helping its readability. He also helped with identifying the topics this article should cover, and with brainstorming the experiments here.

Adam Pearce from Google Research helped us immensely with article hosting and rendering.

Finally, we would like to thank Anirban Santara, Sujoy Paul and Ansh Khurana from Google Research India for their help with setting up and running experiments.

Author Contributions

Ameya Daigavane drafted most of the text, designed experiments and created the interactive visualizations in this article. Balaraman Ravindran and Gaurav Aggarwal extensively guided the overall direction of the article, deliberated over the design and scope of experiments, provided much feedback on the interactive visualizations, edited the text, and described improvements to make the article more accessible to readers.

Discussion and Review

Review 1 - Chaitanya K. Joshi
Review 2 - Nick Moran
Review 3 - Anonymous

References

 1. A Gentle Introduction to Graph Neural Networks
 Sanchez-Lengeling, B., Reif, E., Pearce, A. and Wiltschko, A., 2021. Distill. DOI: 10.23915/distill.00033
 2. Graph Kernels [HTML]
 Vishwanathan, S., Schraudolph, N.N., Kondor, R. and Borgwardt, K.M., 2010. Journal of Machine Learning Research, Vol 11(40), pp. 1201-1242.
 3. Node2vec: Scalable Feature Learning for Networks [link]
 Grover, A. and Leskovec, J., 2016. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864. Association for Computing Machinery. DOI: 10.1145/2939672.2939754
 4. DeepWalk: Online Learning of Social Representations [link]
 Perozzi, B., Al-Rfou, R. and Skiena, S., 2014. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710. Association for Computing Machinery. DOI: 10.1145/2623330.2623732
 5. Convolutional Networks on Graphs for Learning Molecular Fingerprints [PDF]
 Duvenaud, D.K., Maclaurin, D., Iparraguirre, J., Bombarell, R., Hirzel, T., Aspuru-Guzik, A. and Adams, R.P., 2015. Advances in Neural Information Processing Systems, Vol 28, pp. 2224-2232. Curran Associates, Inc.
 6. Neural Message Passing for Quantum Chemistry [HTML]
 Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O. and Dahl, G.E., 2017. Proceedings of the 34th International Conference on Machine Learning, Vol 70, pp. 1263-1272. PMLR.
 7. Learning Convolutional Neural Networks for Graphs
 Niepert, M., Ahmed, M. and Kutzkov, K., 2016. Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, pp. 2014–2023. JMLR.org.
 8. A Tutorial on Spectral Clustering [PDF]
 Luxburg, U.v., 2007. CoRR, Vol abs/0711.0189.
 9. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering [PDF]
 Defferrard, M., Bresson, X. and Vandergheynst, P., 2016. Advances in Neural Information Processing Systems, Vol 29, pp. 3844-3852. Curran Associates, Inc.
10. Wavelets on Graphs via Spectral Graph Theory [link]
 Hammond, D.K., Vandergheynst, P. and Gribonval, R., 2011. Applied and Computational Harmonic Analysis, Vol 30(2), pp. 129 - 150. DOI: https://doi.org/10.1016/j.acha.2010.04.005
11. Chebyshev Polynomials [link]
 Mason, J. and Handscomb, D., 2002. CRC Press.
12. Semi-Supervised Classification with Graph Convolutional Networks [link]
 Kipf, T.N. and Welling, M., 2017. 5th International Conference on Learning Representations (ICLR) 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
13. Graph Attention Networks [link]
 Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P. and Bengio, Y., 2018. International Conference on Learning Representations.
14. Inductive Representation Learning on Large Graphs [PDF]
 Hamilton, W., Ying, Z. and Leskovec, J., 2017. Advances in Neural Information Processing Systems, Vol 30, pp. 1024-1034. Curran Associates, Inc.
15. How Powerful are Graph Neural Networks? [link]
 Xu, K., Hu, W., Leskovec, J. and Jegelka, S., 2019. International Conference on Learning Representations.
16. Relational inductive biases, deep learning, and graph networks [PDF]
 Battaglia, P.W., Hamrick, J.B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V.F., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., Gulcehre, C., Song, H.F., Ballard, A.J., Gilmer, J., Dahl, G.E., Vaswani, A., Allen, K.R., Nash, C., Langston, V., Dyer, C., Heess, N., Wierstra, D., Kohli, P., Botvinick, M., Vinyals, O., Li, Y. and Pascanu, R., 2018. CoRR, Vol abs/1806.01261.
17. Spectral Networks and Locally Connected Networks on Graphs [PDF]
 Bruna, J., Zaremba, W., Szlam, A. and LeCun, Y., 2014. International Conference on Learning Representations (ICLR 2014), CBLS, April 2014.
18. ImageNet: A Large-Scale Hierarchical Image Database
 Deng, J., Dong, W., Socher, R., Li, L., Li, K. and Fei-Fei, L., 2009. CVPR09.
19. On the Transferability of Spectral Graph Filters
 Levie, R., Isufi, E. and Kutyniok, G., 2019. 2019 13th International conference on Sampling Theory and Applications (SampTA), Vol (), pp. 1-5. DOI: 10.1109/SampTA45681.2019.9030932
20. Directional Graph Networks
 Beaini, D., Passaro, S., Létourneau, V., Hamilton, W.L., Corso, G. and Liò, P., 2021.
21. Deep contextualized word representations
 Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K. and Zettlemoyer, L., 2018. Proc. of NAACL.
22. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [link]
 Devlin, J., Chang, M., Lee, K. and Toutanova, K., 2019. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186. Association for Computational Linguistics. DOI: 10.18653/v1/N19-1423
23. Strategies for Pre-training Graph Neural Networks [link]
 Hu*, W., Liu*, B., Gomes, J., Zitnik, M., Liang, P., Pande, V. and Leskovec, J., 2020. International Conference on Learning Representations.
24. Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labeled Nodes [link]
 Sun, K., Lin, Z. and Zhu, Z., 2020. The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pp. 5892-5899. AAAI Press.
25. When Does Self-Supervision Help Graph Convolutional Networks? [PDF]
 You, Y., Chen, T., Wang, Z. and Shen, Y., 2020.
26. Self-supervised Learning on Graphs: Deep Insights and New Direction [PDF]
 Jin, W., Derr, T., Liu, H., Wang, Y., Wang, S., Liu, Z. and Tang, J., 2020.
27. Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics [HTML]
 Gutmann, M.U. and Hyvärinen, A., 2012. Journal of Machine Learning Research, Vol 13(11), pp. 307-361.
28. Learning word embeddings efficiently with noise-contrastive estimation [PDF]
 Mnih, A. and Kavukcuoglu, K., 2013. Advances in Neural Information Processing Systems, Vol 26, pp. 2265-2273. Curran Associates, Inc.
29. A Comprehensive Survey on Graph Neural Networks [link]
 Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C. and Yu, P.S., 2020. IEEE Transactions on Neural Networks and Learning Systems, pp. 1-21. DOI: 10.1109/TNNLS.2020.2978386
30. Graph Neural Networks: A Review of Methods and Applications [PDF]
 Zhou, J., Cui, G., Zhang, Z., Yang, C., Liu, Z. and Sun, M., 2018. CoRR, Vol abs/1812.08434.
31. Dropout: A Simple Way to Prevent Neural Networks from Overfitting [HTML]
 Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. and Salakhutdinov, R., 2014. Journal of Machine Learning Research, Vol 15(56), pp. 1929-1958.
32. DropEdge: Towards Deep Graph Convolutional Networks on Node Classification [link]
 Rong, Y., Huang, W., Xu, T. and Huang, J., 2020. International Conference on Learning Representations.
33. An End-to-End Deep Learning Architecture for Graph Classification [link]
 Zhang, M., Cui, Z., Neumann, M. and Chen, Y., 2018. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pp. 4438-4445. AAAI Press.
34. Hierarchical Graph Representation Learning with Differentiable Pooling [PDF]
 Ying, Z., You, J., Morris, C., Ren, X., Hamilton, W. and Leskovec, J., 2018. Advances in Neural Information Processing Systems, Vol 31, pp. 4800-4810. Curran Associates, Inc.
35. Self-Attention Graph Pooling [HTML]
 Lee, J., Lee, I. and Kang, J., 2019. Proceedings of the 36th International Conference on Machine Learning, Vol 97, pp. 3734-3743. PMLR.

Updates and Corrections

If you see mistakes or want to suggest changes, please create an issue on GitHub.

Reuse

Diagrams and text are licensed under Creative Commons Attribution CC-BY 4.0 with the source available on GitHub, unless noted otherwise. The figures that have been reused from other sources don’t fall under this license and can be recognized by a note in their caption: “Figure from …”.

Citation

For attribution in academic contexts, please cite this work as

Daigavane, et al., "Understanding Convolutions on Graphs", Distill, 2021.

BibTeX citation

@article{daigavane2021understanding,
 author = {Daigavane, Ameya and Ravindran, Balaraman and Aggarwal, Gaurav},
 title = {Understanding Convolutions on Graphs},
 journal = {Distill},
 year = {2021},
 note = {https://distill.pub/2021/understanding-gnns},
 doi = {10.23915/distill.00032}
}
 Distill is dedicated to clear explanations of machine learning
 About Submit Prize Archive RSS GitHub Twitter ISSN 2476-0757
Contents
 
 
 
  Lack of Consistent Structure 
  Node-Order Equivariance 
  Scalability 
 
 
 
 
 
 
 
 
  Spectral Convolutions 
  Global Propagation via Graph Embeddings 
 
 
 
 
 
  GNNs in Practice 
  Different Kinds of Graphs 
  Pooling 
 
 
 
  Reproducing Experiments 
  Recreating Visualizations 
 
 
 
 
  
 This article is one of two Distill publications about graph neural networks.
 Take a look at
 A Gentle Introduction to Graph Neural Networks
 
 for a companion view on many things graph and neural network related.
  
 
 Many systems and interactions - social networks, molecules, organizations, citations, physical models, transactions - can be represented quite naturally as graphs.
 How can we reason about and make predictions within these systems?
 
 
 One idea is to look at tools that have worked well in other domains: neural networks have shown immense predictive power in a variety of learning tasks.
 However, neural networks have been traditionally used to operate on fixed-size and/or regular-structured inputs (such as sentences, images and video).
 This makes them unable to elegantly process graph-structured data.
 
 
 
 
 
 
 
 Graph neural networks (GNNs) are a family of neural networks that can operate naturally on graph-structured data. 
 By extracting and utilizing features from the underlying graph,
 GNNs can make more informed predictions about entities in these interactions,
 as compared to models that consider individual entities in isolation.
 
 
 GNNs are not the only tools available to model graph-structured data:
 graph kernels 
 and random-walk methods 
 were some of the most popular ones.
 Today, however, GNNs have largely replaced these techniques
 because of their inherent flexibility to model the underlying systems
 better.
 
 
 In this article, we will illustrate
 the challenges of computing over graphs, 
 describe the origin and design of graph neural networks,
 and explore the most popular GNN variants in recent times.
 Particularly, we will see that many of these variants
 are composed of similar building blocks.
 
 
 First, let’s discuss some of the complications that graphs come with.
 
 
 The Challenges of Computation on Graphs
 
 
 Lack of Consistent Structure
 
 
 Graphs are extremely flexible mathematical models; but this means they lack consistent structure across instances.
 Consider the task of predicting whether a given chemical molecule is toxic  :
 
 
 
 
 
 
 
  Left: A non-toxic 1,2,6-trigalloyl-glucose molecule.
 
 
  Right: A toxic caramboxin molecule.
 
 
 
 Looking at a few examples, the following issues quickly become apparent:
 
 
 Molecules may have different numbers of atoms.
 The atoms in a molecule may be of different types.
 Each of these atoms may have different number of connections.
 These connections can have different strengths.
 
 
 Representing graphs in a format that can be computed over is non-trivial,
 and the final representation chosen often depends significantly on the actual problem.
 
 
 Node-Order Equivariance
 
 
 Extending the point above: graphs often have no inherent ordering present amongst the nodes.
 Compare this to images, where every pixel is uniquely determined by its absolute position within the image!
 
 
 
 
 Representing the graph as one vector requires us to fix an order on the nodes.
 But what do we do when the nodes have no inherent order?
 Above: 
 The same graph labelled in two different ways. The alphabets indicate the ordering of the nodes.
 
 
 
 
 As a result, we would like our algorithms to be node-order equivariant:
 they should not depend on the ordering of the nodes of the graph.
 If we permute the nodes in some way, the resulting representations of 
 the nodes as computed by our algorithms should also be permuted in the same way.
 
 
 Scalability
 
 
 Graphs can be really large! Think about social networks like Facebook and Twitter, which have over a billion users. 
 Operating on data this large is not easy.
 
 
 Luckily, most naturally occuring graphs are ‘sparse’:
 they tend to have their number of edges linear in their number of vertices.
 We will see that this allows the use of clever methods
 to efficiently compute representations of nodes within the graph.
 Further, the methods that we look at here will have significantly fewer parameters
 in comparison to the size of the graphs they operate on.
 
 
 Problem Setting and Notation
 
 
 There are many useful problems that can be formulated over graphs:
 
 
  Node Classification: Classifying individual nodes.
  Graph Classification: Classifying entire graphs. 
  Node Clustering: Grouping together similar nodes based on connectivity.
  Link Prediction: Predicting missing links.
  Influence Maximization: Identifying influential nodes.
 
 
 
 
 Examples of problems that can be defined over graphs.
 This list is not exhaustive!
 
 
 
 A common precursor in solving many of these problems is node representation learning:
 learning to map individual nodes to fixed-size real-valued vectors (called ‘representations’ or ‘embeddings’).
 
 
 In Learning GNN Parameters, we will see how the learnt embeddings can be used for these tasks.
 
 
 Different GNN variants are distinguished by the way these representations are computed.
 Generally, however, GNNs compute node representations in an iterative process.
 We will use the notation hv(k)h_v^{(k)} to indicate the representation of node vv after the kthk^{\text{th}} iteration.
 Each iteration can be thought of as the equivalent of a ‘layer’ in standard neural networks.
 
 
 We will define a graph GG as a set of nodes, VV, with a set of edges EE connecting them.
 Nodes can have individual features as part of the input: we will denote by xvx_v the individual feature for node v∈Vv \in V.
 For example, the ‘node features’ for a pixel in a color image
 would be the red, green and blue channel (RGB) values at that pixel.
 
 
 For ease of exposition, we will assume GG is undirected, and all nodes are of the same type.
 These kinds of graphs are called ‘homogeneous’.
 Many of the same ideas we will see here 
 apply to other kinds of graphs:
 we will discuss this later in Different Kinds of Graphs.
 
 
 Sometimes we will need to denote a graph property by a matrix MM,
 where each row MvM_v represents a property corresponding to a particular vertex vv.
 
 
 Extending Convolutions to Graphs
 
 
 Convolutional Neural Networks have been seen to be quite powerful in extracting features from images.
 However, images themselves can be seen as graphs with a very regular grid-like structure,
 where the individual pixels are nodes, and the RGB channel values at each pixel as the node features.
 
 
 A natural idea, then, is to consider generalizing convolutions to arbitrary graphs. Recall, however, the challenges
 listed out in the previous section: in particular, ordinary convolutions are not node-order invariant, because
 they depend on the absolute positions of pixels.
 It is initially unclear as how to generalize convolutions over grids to convolutions over general graphs,
 where the neighbourhood structure differs from node to node.
 
 The curious reader may wonder if performing some sort of padding and ordering
 could be done to ensure the consistency of neighbourhood structure across nodes.
 This has been attempted with some success ,
 but the techniques we will look at here are more general and powerful.
 
 
 
 
 
 
 
 Convolutions in CNNs are inherently localized.
 Neighbours participating in the convolution at the center pixel are highlighted in gray.
 
 
 
 
 
 GNNs can perform localized convolutions mimicking CNNs.
 Hover over a node to see its immediate neighbourhood highlighted on the left.
 The structure of this neighbourhood changes from node to node.
 
 
 
 
 
 We begin by introducing the idea of constructing polynomial filters over node neighbourhoods,
 much like how CNNs compute localized filters over neighbouring pixels.
 Then, we will see how more recent approaches extend on this idea with more powerful mechanisms.
 Finally, we will discuss alternative methods
 that can use ‘global’ graph-level information for computing node representations.
 
 
 Polynomial Filters on Graphs
 
 
 The Graph Laplacian
 
 
 Given a graph GG, let us fix an arbitrary ordering of the nn nodes of GG.
 We denote the 0−10-1 adjacency matrix of GG by AA, we can construct the diagonal degree matrix DD of GG as: 
 
 
 Dv=∑uAvu.
 D_v = \sum_u A_{vu}.
  
 
 The degree of node vv is the number of edges incident at vv.
 
 
 
 
 where AvuA_{vu} denotes the entry in the row corresponding to vv and the column corresponding to uu
 in the matrix AA. We will use this notation throughout this section.
 
 
 Then, the graph Laplacian LL is the square n×nn \times n matrix defined as:
 L=D−A.
 L = D - A.
 
 
 
 
 
 The Laplacian LL for an undirected graph GG, with the row corresponding to node C\textsf{C} highlighted.
 Zeros in LL are not displayed above.
 The Laplacian LL depends only on the structure of the graph GG, not on any node features.
 
 
 
 The graph Laplacian gets its name from being the discrete analog of the
 Laplacian operator
 from calculus.
 
 
 Although it encodes precisely the same information as the adjacency matrix AA
 
 In the sense that given either of the matrices AA or LL, you can construct the other.
 ,
 the graph Laplacian has many interesting properties of its own.
 
 The graph Laplacian shows up in many mathematical problems involving graphs:
 random walks,
 spectral clustering,
 and
 diffusion, to name a few.
 
 We will see some of these properties
 in a later section,
 but will instead point readers to
 this tutorial
 for greater insight into the graph Laplacian.
 
 
 Polynomials of the Laplacian
 
 
 Now that we have understood what the graph Laplacian is,
 we can build polynomials of the form:
 pw(L)=w0In+w1L+w2L2+…+wdLd=∑i=0dwiLi.
 p_w(L) = w_0 I_n + w_1 L + w_2 L^2 + \ldots + w_d L^d = \sum_{i = 0}^d w_i L^i.
 
 Each polynomial of this form can alternately be represented by
 its vector of coefficients w=[w0,…,wd]w = [w_0, \ldots, w_d].
 Note that for every ww, pw(L)p_w(L) is an n×nn \times n matrix, just like LL.
 
 
 These polynomials can be thought of as the equivalent of ‘filters’ in CNNs,
 and the coefficients ww as the weights of the ‘filters’.
 
 
 For ease of exposition, we will focus on the case where nodes have one-dimensional features:
 each of the xvx_v for v∈Vv \in V is just a real number. 
 The same ideas hold when each of the xvx_v are higher-dimensional vectors, as well.
 
 
 Using the previously chosen ordering of the nodes,
 we can stack all of the node features xvx_v
 to get a vector x∈Rnx \in \mathbb{R}^n.
 
 
 
 
 Fixing a node order (indicated by the alphabets) and collecting all node features into a single vector xx.
 
 
 
 Once we have constructed the feature vector xx,
 we can define its convolution with a polynomial filter pwp_w as:
 x′=pw(L) x
 x’ = p_w(L) \ x
 
 To understand how the coefficients ww affect the convolution,
 let us begin by considering the ‘simplest’ polynomial:
 when w0=1w_0 = 1 and all of the other coefficients are 00.
 In this case, x′x’ is just xx:
 x′=pw(L) x=∑i=0dwiLix=w0Inx=x.
 x’ = p_w(L) \ x = \sum_{i = 0}^d w_i L^ix = w_0 I_n x = x.
 
 Now, if we increase the degree, and consider the case where
 instead w1=1w_1 = 1 and and all of the other coefficients are 00.
 Then, x′x’ is just LxLx, and so:
 xv′=(Lx)v=Lvx=∑u∈GLvuxu=∑u∈G(Dvu−Avu)xu=Dv xv−∑u∈N(v)xu
 \begin{aligned}
 x’_v = (Lx)_v &amp;= L_v x \\ 
 &amp;= \sum_{u \in G} L_{vu} x_u \\ 
 &amp;= \sum_{u \in G} (D_{vu} - A_{vu}) x_u \\ 
 &amp;= D_v \ x_v - \sum_{u \in \mathcal{N}(v)} x_u
 \end{aligned}
 
 We see that the features at each node vv are combined
 with the features of its immediate neighbours u∈N(v)u \in \mathcal{N}(v).
 
 For readers familiar with
 Laplacian filtering of images,
 this is the exact same idea. When xx is an image, 
 x′=Lxx’ = Lx is exactly the result of applying a ‘Laplacian filter’ to xx.
 
 
 
 At this point, a natural question to ask is:
 How does the degree dd of the polynomial influence the behaviour of the convolution?
 Indeed, it is not too hard to show that:
 This is Lemma 5.2 from .
 distG(v,u)&gt;i⟹Lvui=0.
 \text{dist}_G(v, u) &gt; i \quad \Longrightarrow \quad L_{vu}^i = 0.
 
 
 This implies, when we convolve xx with pw(L)p_w(L) of degree dd to get x′x’:
 xv′=(pw(L)x)v=(pw(L))vx=∑i=0dwiLvix=∑i=0dwi∑u∈GLvuixu=∑i=0dwi∑u∈GdistG(v,u)≤iLvuixu.
 \begin{aligned}
 x’_v = (p_w(L)x)_v &amp;= (p_w(L))_v x \\
 &amp;= \sum_{i = 0}^d w_i L_v^i x \\
 &amp;= \sum_{i = 0}^d w_i \sum_{u \in G} L_{vu}^i x_u \\
 &amp;= \sum_{i = 0}^d w_i \sum_{u \in G \atop \text{dist}_G(v, u) \leq i} L_{vu}^i x_u.
 \end{aligned}
 
  
 Effectively, the convolution at node vv occurs only with nodes uu which are not more than dd hops away.
 Thus, these polynomial filters are localized. The degree of the localization is governed completely by dd.
 
 
 To help you understand these ‘polynomial-based’ convolutions better, we have created the visualization below.
 Vary the polynomial coefficients and the input grid xx to see how the result x′x’ of the convolution changes.
 The grid under the arrow shows the equivalent convolutional kernel applied at the highlighted pixel in xx to get
 the resulting pixel in x′x’.
 The kernel corresponds to the row of pw(L)p_w(L) for the highlighted pixel.
 Note that even after adjusting for position,
 this kernel is different for different pixels, depending on their position within the grid.
 
 
 
 
 
 Hover over a pixel in the input grid (left, representing xx)
 to highlight it and see the equivalent convolutional kernel
 for that pixel under the arrow.
 The result x′x’ of the convolution is shown on the right:
 note that different convolutional kernels are applied at different pixels,
 depending on their location.
 
 
 Click on the input grid to toggle pixel values between 00 (white) and 11 (blue).
 To randomize the input grid, press ‘Randomize Grid’. To reset all pixels to 00, press ‘Reset Grid’.
 Use the sliders at the bottom to change the coefficients ww.
 To reset all coefficients ww to 00, press ‘Reset Coefficients.’
 
 
 
 
 ChebNet
 
 
 ChebNet refines this idea of polynomial filters by looking at polynomial filters of the form:
 
 pw(L)=∑i=1dwiTi(L~)
 p_w(L) = \sum_{i = 1}^d w_i T_i(\tilde{L})
 
 
 where TiT_i is the degree-ii
 Chebyshev polynomial of the first kind and
 L~\tilde{L} is the normalized Laplacian defined using the largest eigenvalue of LL:
 
 We discuss the eigenvalues of the Laplacian LL in more detail in a later section.
 
 
 L~=2Lλmax(L)−In.
 \tilde{L} = \frac{2L}{\lambda_{\max}(L)} - I_n.
 
 
 What is the motivation behind these choices?
 
 
 
 LL is actually positive semi-definite: all of the eigenvalues of LL are not lesser than 00.
 If λmax(L)&gt;1\lambda_{\max}(L) &gt; 1, the entries in the powers of LL rapidly increase in size.
 L~\tilde{L} is effectively a scaled-down version of LL, with eigenvalues guaranteed to be in the range [−1,1][-1, 1].
 This prevents the entries of powers of L~\tilde{L} from blowing up.
 Indeed, in the visualization above: we restrict the higher-order coefficients
 when the unnormalized Laplacian LL is selected, but allow larger values when the normalized Laplacian L~\tilde{L} is selected,
 in order to show the result x′x’ on the same color scale.
 
 
 The Chebyshev polynomials have certain interesting properties that make interpolation more numerically stable.
 We won’t talk about this in more depth here,
 but will advise interested readers to take a look at as a definitive resource.
 
 
 
 
 Polynomial Filters are Node-Order Equivariant
 
 
 The polynomial filters we considered here are actually independent of the ordering of the nodes.
 This is particularly easy to see when the degree of the polynomial pwp_w is 11:
 where each node’s feature is aggregated with the sum of its neighbour’s features.
 Clearly, this sum does not depend on the order of the neighbours.
 A similar proof follows for higher degree polynomials:
 the entries in the powers of LL are equivariant to the ordering of the nodes.
 
 
  Details for the Interested Reader  
 As above, let’s assume an arbitrary node-order over the nn nodes of our graph.
 Any other node-order can be thought of as a permutation of this original node-order.
 We can represent any permutation by a
 permutation matrix PP.
 PP will always be an orthogonal 0−10-1 matrix:
 PPT=PTP=In.
 PP^T = P^TP = I_n.
 
 Then, we call a function ff node-order equivariant iff for all permutations PP:
 f(Px)=Pf(x).
 f(Px) = P f(x).
 
 When switching to the new node-order using the permutation PP,
 the quantities below transform in the following way:
 x→PxL→PLPTLi→PLiPT
 \begin{aligned}
 x &amp;\to Px \\
 L &amp;\to PLP^T \\
 L^i &amp;\to PL^iP^T
 \end{aligned}
 
 and so, for the case of polynomial filters where f(x)=pw(L) xf(x) = p_w(L) \ x, we can see that:
 f(Px)=∑i=0dwi(PLiPT)(Px)=P∑i=0dwiLix=Pf(x).
 \begin{aligned}
 f(Px) &amp; = \sum_{i = 0}^d w_i (PL^iP^T) (Px) \\
 &amp; = P \sum_{i = 0}^d w_i L^i x \\
 &amp; = P f(x).
 \end{aligned}
 
 as claimed.
 
 
 
 Embedding Computation
 
 
 We now describe how we can build a graph neural network
 by stacking ChebNet (or any polynomial filter) layers
 one after the other with non-linearities,
 much like a standard CNN.
 In particular, if we have KK different polynomial filter layers,
 the kthk^{\text{th}} of which has its own learnable weights w(k)w^{(k)},
 we would perform the following computation:
 
 
 
 
 
 
 
 Note that these networks
 reuse the same filter weights across different nodes,
 exactly mimicking weight-sharing in Convolutional Neural Networks (CNNs)
 which reuse weights for convolutional filters across a grid.
 
 
 Modern Graph Neural Networks
 
 
 ChebNet was a breakthrough in learning localized filters over graphs,
 and it motivated many to think of graph convolutions from a different perspective.
 
 
 We return back to the result of convolving xx by the polynomial kernel pw(L)=Lp_w(L) = L,
 focussing on a particular vertex vv:
 
 (Lx)v=Lvx=∑u∈GLvuxu=∑u∈G(Dvu−Avu)xu=Dv xv−∑u∈N(v)xu
 \begin{aligned}
 (Lx)_v &amp;= L_v x \\ 
 &amp;= \sum_{u \in G} L_{vu} x_u \\ 
 &amp;= \sum_{u \in G} (D_{vu} - A_{vu}) x_u \\ 
 &amp;= D_v \ x_v - \sum_{u \in \mathcal{N}(v)} x_u
 \end{aligned}
 
  
 As we noted before, this is a 11-hop localized convolution.
 But more importantly, we can think of this convolution as arising of two steps:
 
 
 Aggregating over immediate neighbour features xux_u.
 Combining with the node’s own feature xvx_v.
 
 
 Key Idea:
 What if we consider different kinds of ‘aggregation’ and ‘combination’ steps,
 beyond what are possible using polynomial filters?
 
 
 By ensuring that the aggregation is node-order equivariant,
 the overall convolution becomes node-order equivariant.
 
 
 These convolutions can be thought of as ‘message-passing’ between adjacent nodes:
 after each step, every node receives some ‘information’ from its neighbours.
 
 
 By iteratively repeating the 11-hop localized convolutions KK times (i.e., repeatedly ‘passing messages’),
 the receptive field of the convolution effectively includes all nodes upto KK hops away.
 
 
 Embedding Computation
 
 
 Message-passing forms the backbone of many GNN architectures today.
 We describe the most popular ones in depth below:
 
 
 Graph Convolutional Networks (GCN)
 Graph Attention Networks (GAT)
 Graph Sample and Aggregate (GraphSAGE)
 Graph Isomorphism Network (GIN)
 
 
 
 
 
 Thoughts
 
 
 An interesting point is to assess different aggregation functions: are some better and others worse?
 demonstrates that aggregation functions indeed can be compared on how well
 they can uniquely preserve node neighbourhood features;
 we recommend the interested reader take a look at the detailed theoretical analysis there.
 
 
 Here, we’ve talk about GNNs where the computation only occurs at the nodes.
 More recent GNN models
 such as Message-Passing Neural Networks 
 and Graph Networks 
 perform computation over the edges as well;
 they compute edge embeddings together with node embeddings.
 This is an even more general framework -
 but the same ‘message passing’ ideas from this section apply.
 
 
 Interactive Graph Neural Networks
 
 
 Below is an interactive visualization of these GNN models on small graphs.
 For clarity, the node features are just real numbers here, shown inside the squares next to each node,
 but the same equations hold when the node features are vectors.
 
 
 
 
 
 Choose a GNN model using the tabs at the top. Click on a node to see the update equation at that node for the next iteration.
 Use the sliders on the left to change the weights for the current iteration, and watch how the update equation changes. 
 
 
 
 In practice, each iteration above is generally thought of as a single ‘neural network layer’.
 This ideology is followed by many popular Graph Neural Network libraries,
 
 For example: PyTorch Geometric
 and StellarGraph.
 
 allowing one to compose different types of graph convolutions in the same model.
 
 
 From Local to Global Convolutions 
 
 
 The methods we’ve seen so far perform ‘local’ convolutions:
 every node’s feature is updated using a function of its local neighbours’ features.
 
 
 While performing enough steps of message-passing will eventually ensure that
 information from all nodes in the graph is passed,
 one may wonder if there are more direct ways to perform ‘global’ convolutions.
 
 
 The answer is yes; we will now describe an approach that was actually first put forward
 in the context of neural networks by ,
 much before any of the GNN models we looked at above.
  
 
 Spectral Convolutions 
 
 
 As before, we will focus on the case where nodes have one-dimensional features.
 After choosing an arbitrary node-order, we can stack all of the node features to get a
 ‘feature vector’ x∈Rnx \in \mathbb{R}^n.
 
 
 Key Idea:
 Given a feature vector xx, 
 the Laplacian LL allows us to quantify how smooth xx is, with respect to GG.
 
 
 How?
 
 
 After normalizing xx such that ∑i=1nxi2=1\sum_{i = 1}^n x_i^2 = 1,
 if we look at the following quantity involving LL:
 
 RLR_L is formally called the Rayleigh quotient.
 
 RL(x)=xTLxxTx=∑(i,j)∈E(xi−xj)2∑ixi2=∑(i,j)∈E(xi−xj)2.
 R_L(x) = \frac{x^T L x}{x^T x} = \frac{\sum_{(i, j) \in E} (x_i - x_j)^2}{\sum_i x_i^2} = \sum_{(i, j) \in E} (x_i - x_j)^2.
 
 we immediately see that feature vectors xx that assign similar values to 
 adjacent nodes in GG (hence, are smooth) would have smaller values of RL(x)R_L(x).
 
 
 LL is a real, symmetric matrix, which means it has all real eigenvalues λ1≤…≤λn\lambda_1 \leq \ldots \leq \lambda_{n}.
 
 An eigenvalue λ\lambda of a matrix AA is a value
 satisfying the equation Au=λuAu = \lambda u for a certain vector uu, called an eigenvector.
 For a friendly introduction to eigenvectors,
 please see this tutorial.
 
 Further, the corresponding eigenvectors u1,…,unu_1, \ldots, u_{n} can be taken to be orthonormal:
 uk1Tuk2={1 if k1=k2.0 if k1≠k2.
 u_{k_1}^T u_{k_2} =
 \begin{cases}
 1 \quad \text{ if } {k_1} = {k_2}. \\
 0 \quad \text{ if } {k_1} \neq {k_2}.
 \end{cases}
 
 It turns out that these eigenvectors of LL are successively less smooth, as RLR_L indicates:
 This is the min-max theorem for eigenvalues.
 argminx, x⊥{u1,…,ui−1}RL(x)=ui.minx, x⊥{u1,…,ui−1}RL(x)=λi.
 \underset{x, \ x \perp \{u_1, \ldots, u_{i - 1}\}}{\text{argmin}} R_L(x) = u_i.
 \qquad
 \qquad
 \qquad
 \min_{x, \ x \perp \{u_1, \ldots, u_{i - 1}\}} R_L(x) = \lambda_i.
 
 The set of eigenvalues of LL are called its ‘spectrum’, hence the name!
 We denote the ‘spectral’ decomposition of LL as:
 L=UΛUT.
 L = U \Lambda U^T.
 
 where Λ\Lambda is the diagonal matrix of sorted eigenvalues,
 and UU denotes the matrix of the eigenvectors (sorted corresponding to increasing eigenvalues):
 Λ=[λ1⋱λn]U=[u1 ⋯ un].
 \Lambda = \begin{bmatrix}
 \lambda_{1} &amp; &amp; \\
 &amp; \ddots &amp; \\
 &amp; &amp; \lambda_{n}
 \end{bmatrix}
 \qquad
 \qquad
 \qquad
 \qquad
 U = \begin{bmatrix} \\ u_1 \ \cdots \ u_n \\ \end{bmatrix}.
 
 The orthonormality condition between eigenvectors gives us that UTU=IU^T U = I, the identity matrix.
 As these nn eigenvectors form a basis for Rn\mathbb{R}^n,
 any feature vector xx can be represented as a linear combination of these eigenvectors:
 x=∑i=1nxi^ui=Ux^.
 x = \sum_{i = 1}^n \hat{x_i} u_i = U \hat{x}.
 
 where x^\hat{x} is the vector of coefficients [x0,…xn][x_0, \ldots x_n].
 We call x^\hat{x} as the spectral representation of the feature vector xx.
 The orthonormality condition allows us to state:
 x=Ux^⟺UTx=x^.
 x = U \hat{x} \quad \Longleftrightarrow \quad U^T x = \hat{x}.
 
 This pair of equations allows us to interconvert
 between the ‘natural’ representation xx and the ‘spectral’ representation x^\hat{x}
 for any vector x∈Rnx \in \mathbb{R}^n.
 
 
 Spectral Representations of Natural Images
 
 
 As discussed before, we can consider any image as a grid graph, where each pixel is a node,
 connected by edges to adjacent pixels.
 Thus, a pixel can have either 3,5,3, 5, or 88 neighbours, depending on its location within the image grid.
 Each pixel gets a value as part of the image. If the image is grayscale, each value will be a single 
 real number indicating how dark the pixel is. If the image is colored, each value will be a 33-dimensional
 vector, indicating the values for the red, green and blue (RGB) channels.
 We use the alpha channel as well in the visualization below, so this is actually RGBA.
 
 
 This construction allows us to compute the graph Laplacian and the eigenvector matrix UU.
 Given an image, we can then investigate what its spectral representation looks like.
 
 
 To shed some light on what the spectral representation actually encodes,
 we perform the following experiment over each channel of the image independently: 
 
 
 
 We first collect all pixel values across a channel into a feature vector xx.
 
 
 Then, we obtain its spectral representation x^\hat{x}.
 x^=UTx
 \hat{x} = U^T x
 
 
 
 We truncate this to the first mm components to get x^m\hat{x}_m.
 By truncation, we mean zeroing out all of the remaining n−mn - m components of x^\hat{x}.
 This truncation is equivalent to using only the first mm eigenvectors to compute the spectral representation.
 x^m=Truncatem(x^)
 \hat{x}_m = \text{Truncate}_m(\hat{x})
 
 
 
 Then, we convert this truncated representation x^m\hat{x}_m back to the natural basis to get xmx_m.
 xm=Ux^m
 x_m = U \hat{x}_m
 
 
 
 
 Finally, we stack the resulting channels back together to get back an image.
 We can now see how the resulting image changes with choices of mm.
 Note that when m=nm = n, the resulting image is identical to the original image,
 as we can reconstruct each channel exactly.
 
 
 
 
 Use the radio buttons at the top to chose one of the four sample images.
 Each of these images has been taken from the ImageNet 
 dataset and downsampled to 5050 pixels wide and 4040 pixels tall.
 As there are n=50×40=2000n = 50 \times 40 = 2000 pixels in each image, there are 20002000 Laplacian eigenvectors.
 Use the slider at the bottom to change the number of spectral components to keep, noting how
 images get progressively blurrier as the number of components decrease.
 
 
 
 As mm decreases, we see that the output image xmx_m gets blurrier.
 If we decrease mm to 11, the output image xmx_m is entirely the same color throughout.
 We see that we do not need to keep all nn components;
 we can retain a lot of the information in the image with significantly fewer components.
 We can relate this to the Fourier decomposition of images:
 the more eigenvectors we use, the higher frequencies we can represent on the grid.
 
 
 To complement the visualization above,
 we additionally visualize the first few eigenvectors on a smaller 8×88 \times 8 grid below.
 We change the coefficients of the first 1010 out of 6464 eigenvectors
 in the spectral representation
 and see how the resulting image changes:
 
 
 
 
 
 Move the sliders to change the spectral representation x^\hat{x} (right),
 and see how xx itself changes on the image (left).
 Note how the first eigenvectors are much ‘smoother’ than the later ones,
 and the many patterns we can make with only 1010 eigenvectors.
 
 
 
 These visualizations should convince you that the first eigenvectors are indeed smooth,
 and the smoothness correspondingly decreases as we consider later eigenvectors.
 
 
 For any image xx, we can think of
 the initial entries of the spectral representation x^\hat{x}
 as capturing ‘global’ image-wide trends, which are the low-frequency components,
 while the later entries as capturing ‘local’ details, which are the high-frequency components.
 
 Embedding Computation
 
 We now have the background to understand spectral convolutions
 and how they can be used to compute embeddings/feature representations of nodes.
 
 
 As before, the model we describe below has KK layers:
 each layer kk has learnable parameters w^(k)\hat{w}^{(k)},
 called the ‘filter weights’.
 These weights will be convolved with the spectral representations of the node features.
 As a result, the number of weights needed in each layer is equal to mm, the number of 
 eigenvectors used to compute the spectral representations.
 We had shown in the previous section that we can take m≪nm \ll n
 and still not lose out on significant amounts of information.
 
 
 Thus, convolution in the spectral domain enables the use of significantly fewer parameters
 than just direct convolution in the natural domain.
 Further, by virtue of the smoothness of the Laplacian eigenvectors across the graph,
 using spectral representations automatically enforces an inductive bias for
 neighbouring nodes to get similar representations.
 
 
 Assuming one-dimensional node features for now,
 the output of each layer is a vector of node representations h(k)h^{(k)},
 where each node’s representation corresponds to a row
 of the vector.
 
 
 
 We fix an ordering of the nodes in GG. This gives us the adjacency matrix AA and the graph Laplacian LL,
 allowing us to compute UmU_m.
 Finally, we can describe the computation that the layers perform, one after the other:
 
 
 
 
 
 The method above generalizes easily to the case where each h(k)∈Rdkh^{(k)} \in \mathbb{R}^{d_k}, as well:
 see for details.
 
 
 With the insights from the previous section, we see that convolution in the spectral-domain of graphs
 can be thought of as the generalization of convolution in the frequency-domain of images.
 
 
 Spectral Convolutions are Node-Order Equivariant
 
 
 We can show spectral convolutions are node-order equivariant using a similar approach
 as for Laplacian polynomial filters. 
 
 
  
 Details for the Interested Reader
   
 As in our proof before,
 let’s fix an arbitrary node-order.
 Then, any other node-order can be represented by a
 permutation of this original node-order.
 We can associate this permutation with its permutation matrix PP.
 Under this new node-order,
 the quantities below transform in the following way:
 x→PxA→PAPTL→PLPTUm→PUm
 \begin{aligned}
 x &amp;\to Px \\
 A &amp;\to PAP^T \\
 L &amp;\to PLP^T \\
 U_m &amp;\to PU_m
 \end{aligned}
 
 which implies that, in the embedding computation:
 x^→(PUm)T(Px)=UmTx=x^w^→(PUm)T(Pw)=UmTw=w^g^→g^g→(PUm)g^=P(Umg^)=Pg
 \begin{aligned}
 \hat{x} &amp;\to \left(PU_m\right)^T (Px) = U_m^T x = \hat{x} \\
 \hat{w} &amp;\to \left(PU_m\right)^T (Pw) = U_m^T w = \hat{w} \\
 \hat{g} &amp;\to \hat{g} \\
 g &amp;\to (PU_m)\hat{g} = P(U_m\hat{g}) = Pg
 \end{aligned}
 
 Hence, as σ\sigma is applied elementwise:
 f(Px)=σ(Pg)=Pσ(g)=Pf(x)
 f(Px) = \sigma(Pg) = P \sigma(g) = P f(x)
 
 as required.
 Further, we see that the spectral quantities x^,w^\hat{x}, \hat{w} and g^\hat{g}
 are unchanged by permutations of the nodes.
 
 Formally, they are what we would call node-order invariant.
 
 
 
 
 The theory of spectral convolutions is mathematically well-grounded;
 however, there are some key disadvantages that we must talk about:
 
 
 
 We need to compute the eigenvector matrix UmU_m from LL. For large graphs, this becomes quite infeasible.
 
 
 Even if we can compute UmU_m, global convolutions themselves are inefficient to compute,
 because of the repeated
 multiplications with UmU_m and UmTU_m^T.
 
 
 The learned filters are specific to the input graphs,
 as they are represented in terms
 of the spectral decomposition of input graph Laplacian LL.
 This means they do not transfer well to new graphs
 which have significantly different structure (and hence, significantly
 different eigenvalues) .
 
 
 
 While spectral convolutions have largely been superseded by
 ‘local’ convolutions for the reasons discussed above,
 there is still much merit to understanding the ideas behind them.
 Indeed, a recently proposed GNN model called Directional Graph Networks
 
 actually uses the Laplacian eigenvectors
 and their mathematical properties
 extensively.
 
 
 Global Propagation via Graph Embeddings
 
 
 A simpler way to incorporate graph-level information
 is to compute embeddings of the entire graph by pooling node
 (and possibly edge) embeddings,
 and then using the graph embedding to update node embeddings,
 following an iterative scheme similar to what we have looked at here.
 This is an approach used by Graph Networks
 .
 We will briefly discuss how graph-level embeddings
 can be constructed in Pooling.
 However, such approaches tend to ignore the underlying
 topology of the graph that spectral convolutions can capture.
 
 
 Learning GNN Parameters
 
 
 All of the embedding computations we’ve described here, whether spectral or spatial, are completely differentiable.
 This allows GNNs to be trained in an end-to-end fashion, just like a standard neural network,
 once a suitable loss function L\mathcal{L} is defined:
 
 
 
 Node Classification: By minimizing any of the standard losses for classification tasks,
 such as categorical cross-entropy when multiple classes are present:
 L(yv,yv^)=−∑cyvclogyvc^.
 \mathcal{L}(y_v, \hat{y_v}) = -\sum_{c} y_{vc} \log{\hat{y_{vc}}}.
 
 where yvc^\hat{y_{vc}} is the predicted probability that node vv is in class cc.
 GNNs adapt well to the semi-supervised setting, which is when only some nodes in the graph are labelled.
 In this setting, one way to define a loss LG\mathcal{L}_{G} over an input graph GG is:
 LG=∑v∈Lab(G)L(yv,yv^)∣Lab(G)∣
 \mathcal{L}_{G} = \frac{\sum\limits_{v \in \text{Lab}(G)} \mathcal{L}(y_v, \hat{y_v})}{| \text{Lab}(G) |}
 
 where, we only compute losses over labelled nodes Lab(G)\text{Lab}(G).
 
 
 Graph Classification: By aggregating node representations,
 one can construct a vector representation of the entire graph.
 This graph representation can be used for any graph-level task, even beyond classification.
 See Pooling for how representations of graphs can be constructed.
 
 
 Link Prediction: By sampling pairs of adjacent and non-adjacent nodes,
 and use these vector pairs as inputs to predict the presence/absence of an edge.
 For a concrete example, by minimizing the following ‘logistic regression’-like loss:
 L(yv,yu,evu)=−evulog(pvu)−(1−evu)log(1−pvu)pvu=σ(yvTyu)
 \begin{aligned}
 \mathcal{L}(y_v, y_u, e_{vu}) &amp;= -e_{vu} \log(p_{vu}) - (1 - e_{vu}) \log(1 - p_{vu}) \\
 p_{vu} &amp;= \sigma(y_v^Ty_u)
 \end{aligned}
 
 where σ\sigma is the sigmoid function,
 and evu=1e_{vu} = 1 iff there is an edge between nodes vv and uu, being 00 otherwise.
 
 
 Node Clustering: By simply clustering the learned node representations.
 
 
 
 The broad success of pre-training for natural language processing models
 such as ELMo and BERT 
 has sparked interest in similar techniques for GNNs
 .
 The key idea in each of these papers is to train GNNs to predict
 local (eg. node degrees, clustering coefficient, masked node attributes)
 and/or global graph properties (eg. pairwise distances, masked global attributes).
 
 
 Another self-supervised technique is to enforce that neighbouring nodes get similar embeddings,
 mimicking random-walk approaches such as node2vec and DeepWalk :
 
 LG=∑v∑u∈NR(v)logexpzvTzu∑u′expzu′Tzu.
 L_{G} = \sum_{v} \sum_{u \in N_R(v)} \log\frac{\exp{z_v^T z_u}}{\sum\limits_{u’} \exp{z_{u’}^T z_u}}.
 
 
 where NR(v)N_R(v) is a multi-set of nodes visited when random walks are started from vv.
 For large graphs, where computing the sum over all nodes may be computationally expensive,
 techniques such as Noise Contrastive Estimation are especially useful.
 
 
 
 Conclusion and Further Reading
 
 
 
 While we have looked at many techniques and ideas in this article,
 the field of Graph Neural Networks is extremely vast.
 We have been forced to restrict our discussion to a small subset of the entire literature,
 while still communicating the key ideas and design principles behind GNNs.
 We recommend the interested reader take a look at
 for a more comprehensive survey.
 
 
 We end with pointers and references for additional concepts readers might be interested in:
 
 
 GNNs in Practice
 
 
 It turns out that accomodating the different structures of graphs is often hard to do efficiently,
 but we can still represent many GNN update equations using
 as sparse matrix-vector products (since generally, the adjacency matrix is sparse for most real-world graph datasets.)
 For example, the GCN variant discussed here can be represented as:
 h(k)=D−1A⋅h(k−1)W(k)T+h(k−1)B(k)T.
 h^{(k)} = D^{-1} A \cdot h^{(k - 1)} {W^{(k)}}^T + h^{(k - 1)} {B^{(k)}}^T.
 
 Restructuring the update equations in this way allows for efficient vectorized implementations of GNNs on accelerators
 such as GPUs.
 
 
 Regularization techniques for standard neural networks,
 such as Dropout ,
 can be applied in a straightforward manner to the parameters
 (for example, zero out entire rows of W(k)W^{(k)} above).
 However, there are graph-specific techniques such as DropEdge 
 that removes entire edges at random from the graph,
 that also boost the performance of many GNN models.
 
 
 Different Kinds of Graphs
 
 
 Here, we have focused on undirected graphs, to avoid going into too many unnecessary details.
 However, there are some simple variants of spatial convolutions for:
 
 
 Directed graphs: Aggregate across in-neighbourhood and/or out-neighbourhood features. 
 Temporal graphs: Aggregate across previous and/or future node features.
 Heterogeneous graphs: Learn different aggregation functions for each node/edge type.
 
 
 There do exist more sophisticated techniques that can take advantage of the different structures of these graphs:
 see for more discussion.
 
 
 Pooling
 
 
 This article discusses how GNNs compute useful representations of nodes.
 But what if we wanted to compute representations of graphs for graph-level tasks (for example, predicting the toxicity of a molecule)?
 
 
 A simple solution is to just aggregate the final node embeddings and pass them through another neural network PREDICTG\text{PREDICT}_G:
 hG=PREDICTG(AGGv∈G({hv}))
 h_G = \text{PREDICT}_G \Big( \text{AGG}_{v \in G}\left(\{ h_v \} \right) \Big)
 
 However, there do exist more powerful techniques for ‘pooling’ together node representations:
 
 
 SortPool: Sort vertices of the graph to get a fixed-size node-order invariant representation of the graph, and then apply any standard neural network architecture.
 DiffPool: Learn to cluster vertices, build a coarser graph over clusters instead of nodes, then apply a GNN over the coarser graph. Repeat until only one cluster is left. 
 SAGPool: Apply a GNN to learn node scores, then keep only the nodes with the top scores, throwing away the rest. Repeat until only one node is left.
 
 
 
 Supplementary Material
 
 
 Reproducing Experiments
 
 
 The experiments from
 Spectral Representations of Natural Images
 can be reproduced using the following
 Colab notebook:
 Spectral Representations of Natural Images.
 
 
 Recreating Visualizations
 
 
 To aid in the creation of future interactive articles,
 we have created ObservableHQ
 
 notebooks for each of the interactive visualizations here:
 
 
  Neighbourhood Definitions for CNNs and GNNs 
  Graph Polynomial Convolutions on a Grid 
  Graph Polynomial Convolutions: Equations 
  Modern Graph Neural Networks: Equations 
 
 Modern Graph Neural Networks: Interactive Models
 which pulls together the following standalone notebooks:
 
  Graph Convolutional Networks 
  Graph Attention Networks 
  GraphSAGE 
  Graph Isomorphism Networks 
 
 
  Laplacian Eigenvectors for Grids 
  Spectral Decomposition of Natural Images 
  Spectral Convolutions: Equations
Understanding Convolutions on Graphs
https://distill.pub/2021/understanding-gnns/
http://karpathy.github.io/2015/05/21/rnn-effectiveness/
The Unreasonable Effectiveness of Recurrent Neural Networks
2015-05-21
Karpathy
<div><article>
 <p>There’s something magical about Recurrent Neural Networks (RNNs). I still remember when I trained my first recurrent network for <a href="http://cs.stanford.edu/people/karpathy/deepimagesent/">Image Captioning</a>. Within a few dozen minutes of training my first baby model (with rather arbitrarily-chosen hyperparameters) started to generate very nice looking descriptions of images that were on the edge of making sense. Sometimes the ratio of how simple your model is to the quality of the results you get out of it blows past your expectations, and this was one of those times. What made this result so shocking at the time was that the common wisdom was that RNNs were supposed to be difficult to train (with more experience I’ve in fact reached the opposite conclusion). Fast forward about a year: I’m training RNNs all the time and I’ve witnessed their power and robustness many times, and yet their magical outputs still find ways of amusing me. This post is about sharing some of that magic with you.</p>
<blockquote>
 <p>We’ll train RNNs to generate text character by character and ponder the question “how is that even possible?”</p>
</blockquote>
<p>By the way, together with this post I am also releasing <a href="https://github.com/karpathy/char-rnn">code on Github</a> that allows you to train character-level language models based on multi-layer LSTMs. You give it a large chunk of text and it will learn to generate text like it one character at a time. You can also use it to reproduce my experiments below. But we’re getting ahead of ourselves; What are RNNs anyway?</p>
<p><strong>Sequences</strong>. Depending on your background you might be wondering: <em>What makes Recurrent Networks so special</em>? A glaring limitation of Vanilla Neural Networks (and also Convolutional Networks) is that their API is too constrained: they accept a fixed-sized vector as input (e.g. an image) and produce a fixed-sized vector as output (e.g. probabilities of different classes). Not only that: These models perform this mapping using a fixed amount of computational steps (e.g. the number of layers in the model). The core reason that recurrent nets are more exciting is that they allow us to operate over <em>sequences</em> of vectors: Sequences in the input, the output, or in the most general case both. A few examples may make this more concrete:</p>
<div>
<p></p><p>Each rectangle is a vector and arrows represent functions (e.g. matrix multiply). Input vectors are in red, output vectors are in blue and green vectors hold the RNN's state (more on this soon). From left to right: <b>(1)</b> Vanilla mode of processing without RNN, from fixed-sized input to fixed-sized output (e.g. image classification). <b>(2)</b> Sequence output (e.g. image captioning takes an image and outputs a sentence of words). <b>(3)</b> Sequence input (e.g. sentiment analysis where a given sentence is classified as expressing positive or negative sentiment). <b>(4)</b> Sequence input and sequence output (e.g. Machine Translation: an RNN reads a sentence in English and then outputs a sentence in French). <b>(5)</b> Synced sequence input and output (e.g. video classification where we wish to label each frame of the video). Notice that in every case are no pre-specified constraints on the lengths sequences because the recurrent transformation (green) is fixed and can be applied as many times as we like.</p>
</div>
<p>As you might expect, the sequence regime of operation is much more powerful compared to fixed networks that are doomed from the get-go by a fixed number of computational steps, and hence also much more appealing for those of us who aspire to build more intelligent systems. Moreover, as we’ll see in a bit, RNNs combine the input vector with their state vector with a fixed (but learned) function to produce a new state vector. This can in programming terms be interpreted as running a fixed program with certain inputs and some internal variables. Viewed this way, RNNs essentially describe programs. In fact, it is known that <a href="http://binds.cs.umass.edu/papers/1995_Siegelmann_Science.pdf">RNNs are Turing-Complete</a> in the sense that they can to simulate arbitrary programs (with proper weights). But similar to universal approximation theorems for neural nets you shouldn’t read too much into this. In fact, forget I said anything.</p>
<blockquote>
 <p>If training vanilla neural nets is optimization over functions, training recurrent nets is optimization over programs.</p>
</blockquote>
<p><strong>Sequential processing in absence of sequences</strong>. You might be thinking that having sequences as inputs or outputs could be relatively rare, but an important point to realize is that even if your inputs/outputs are fixed vectors, it is still possible to use this powerful formalism to <em>process</em> them in a sequential manner. For instance, the figure below shows results from two very nice papers from <a href="http://deepmind.com/">DeepMind</a>. On the left, an algorithm learns a recurrent network policy that steers its attention around an image; In particular, it learns to read out house numbers from left to right (<a href="http://arxiv.org/abs/1412.7755">Ba et al.</a>). On the right, a recurrent network <em>generates</em> images of digits by learning to sequentially add color to a canvas (<a href="http://arxiv.org/abs/1502.04623">Gregor et al.</a>):</p>
<div>
<p>
</p>
<p>Left: RNN learns to read house numbers. Right: RNN learns to paint house numbers.</p>
</div>
<p>The takeaway is that even if your data is not in form of sequences, you can still formulate and train powerful models that learn to process it sequentially. You’re learning stateful programs that process your fixed-sized data.</p>
<p><strong>RNN computation.</strong> So how do these things work? At the core, RNNs have a deceptively simple API: They accept an input vector <code>x</code> and give you an output vector <code>y</code>. However, crucially this output vector’s contents are influenced not only by the input you just fed in, but also on the entire history of inputs you’ve fed in in the past. Written as a class, the RNN’s API consists of a single <code>step</code> function:</p>
<div><pre><code>rnn = RNN()
y = rnn.step(x) # x is an input vector, y is the RNN's output vector
</code></pre></div>
<p>The RNN class has some internal state that it gets to update every time <code>step</code> is called. In the simplest case this state consists of a single <em>hidden</em> vector <code>h</code>. Here is an implementation of the step function in a Vanilla RNN:</p>
<div><pre><code>class RNN:
 # ...
 def step(self, x):
 # update the hidden state
 self.h = np.tanh(np.dot(self.W_hh, self.h) + np.dot(self.W_xh, x))
 # compute the output vector
 y = np.dot(self.W_hy, self.h)
 return y
</code></pre></div>
<p>The above specifies the forward pass of a vanilla RNN. This RNN’s parameters are the three matrices <code>W_hh, W_xh, W_hy</code>. The hidden state <code>self.h</code> is initialized with the zero vector. The <code>np.tanh</code> function implements a non-linearity that squashes the activations to the range <code>[-1, 1]</code>. Notice briefly how this works: There are two terms inside of the tanh: one is based on the previous hidden state and one is based on the current input. In numpy <code>np.dot</code> is matrix multiplication. The two intermediates interact with addition, and then get squashed by the tanh into the new state vector. If you’re more comfortable with math notation, we can also write the hidden state update as \( h_t = \tanh ( W_{hh} h_{t-1} + W_{xh} x_t ) \), where tanh is applied elementwise.</p>
<p>We initialize the matrices of the RNN with random numbers and the bulk of work during training goes into finding the matrices that give rise to desirable behavior, as measured with some loss function that expresses your preference to what kinds of outputs <code>y</code> you’d like to see in response to your input sequences <code>x</code>.</p>
<p><strong>Going deep</strong>. RNNs are neural networks and everything works monotonically better (if done right) if you put on your deep learning hat and start stacking models up like pancakes. For instance, we can form a 2-layer recurrent network as follows:</p>
<div><pre><code>y1 = rnn1.step(x)
y = rnn2.step(y1)
</code></pre></div>
<p>In other words we have two separate RNNs: One RNN is receiving the input vectors and the second RNN is receiving the output of the first RNN as its input. Except neither of these RNNs know or care - it’s all just vectors coming in and going out, and some gradients flowing through each module during backpropagation.</p>
<p><strong>Getting fancy</strong>. I’d like to briefly mention that in practice most of us use a slightly different formulation than what I presented above called a <em>Long Short-Term Memory</em> (LSTM) network. The LSTM is a particular type of recurrent network that works slightly better in practice, owing to its more powerful update equation and some appealing backpropagation dynamics. I won’t go into details, but everything I’ve said about RNNs stays exactly the same, except the mathematical form for computing the update (the line <code>self.h = ... </code>) gets a little more complicated. From here on I will use the terms “RNN/LSTM” interchangeably but all experiments in this post use an LSTM.</p>
<h2>Character-Level Language Models</h2>
<p>Okay, so we have an idea about what RNNs are, why they are super exciting, and how they work. We’ll now ground this in a fun application: We’ll train RNN character-level language models. That is, we’ll give the RNN a huge chunk of text and ask it to model the probability distribution of the next character in the sequence given a sequence of previous characters. This will then allow us to generate new text one character at a time.</p>
<p>As a working example, suppose we only had a vocabulary of four possible letters “helo”, and wanted to train an RNN on the training sequence “hello”. This training sequence is in fact a source of 4 separate training examples: 1. The probability of “e” should be likely given the context of “h”, 2. “l” should be likely in the context of “he”, 3. “l” should also be likely given the context of “hel”, and finally 4. “o” should be likely given the context of “hell”.</p>
<p>Concretely, we will encode each character into a vector using 1-of-k encoding (i.e. all zero except for a single one at the index of the character in the vocabulary), and feed them into the RNN one at a time with the <code>step</code> function. We will then observe a sequence of 4-dimensional output vectors (one dimension per character), which we interpret as the confidence the RNN currently assigns to each character coming next in the sequence. Here’s a diagram:</p>
<div>
<p></p><p>An example RNN with 4-dimensional input and output layers, and a hidden layer of 3 units (neurons). This diagram shows the activations in the forward pass when the RNN is fed the characters "hell" as input. The output layer contains confidences the RNN assigns for the next character (vocabulary is "h,e,l,o"); We want the green numbers to be high and red numbers to be low.</p>
</div>
<p>For example, we see that in the first time step when the RNN saw the character “h” it assigned confidence of 1.0 to the next letter being “h”, 2.2 to letter “e”, -3.0 to “l”, and 4.1 to “o”. Since in our training data (the string “hello”) the next correct character is “e”, we would like to increase its confidence (green) and decrease the confidence of all other letters (red). Similarly, we have a desired target character at every one of the 4 time steps that we’d like the network to assign a greater confidence to. Since the RNN consists entirely of differentiable operations we can run the backpropagation algorithm (this is just a recursive application of the chain rule from calculus) to figure out in what direction we should adjust every one of its weights to increase the scores of the correct targets (green bold numbers). We can then perform a <em>parameter update</em>, which nudges every weight a tiny amount in this gradient direction. If we were to feed the same inputs to the RNN after the parameter update we would find that the scores of the correct characters (e.g. “e” in the first time step) would be slightly higher (e.g. 2.3 instead of 2.2), and the scores of incorrect characters would be slightly lower. We then repeat this process over and over many times until the network converges and its predictions are eventually consistent with the training data in that correct characters are always predicted next.</p>
<p>A more technical explanation is that we use the standard Softmax classifier (also commonly referred to as the cross-entropy loss) on every output vector simultaneously. The RNN is trained with mini-batch Stochastic Gradient Descent and I like to use <a href="http://arxiv.org/abs/1502.04390">RMSProp</a> or Adam (per-parameter adaptive learning rate methods) to stablilize the updates.</p>
<p>Notice also that the first time the character “l” is input, the target is “l”, but the second time the target is “o”. The RNN therefore cannot rely on the input alone and must use its recurrent connection to keep track of the context to achieve this task.</p>
<p>At <strong>test time</strong>, we feed a character into the RNN and get a distribution over what characters are likely to come next. We sample from this distribution, and feed it right back in to get the next letter. Repeat this process and you’re sampling text! Lets now train an RNN on different datasets and see what happens.</p>
<p>To further clarify, for educational purposes I also wrote a <a href="https://gist.github.com/karpathy/d4dee566867f8291f086">minimal character-level RNN language model in Python/numpy</a>. It is only about 100 lines long and hopefully it gives a concise, concrete and useful summary of the above if you’re better at reading code than text. We’ll now dive into example results, produced with the much more efficient Lua/Torch codebase.</p>
<h2>Fun with RNNs</h2>
<p>All 5 example character models below were trained with the <a href="https://github.com/karpathy/char-rnn">code</a> I’m releasing on Github. The input in each case is a single file with some text, and we’re training an RNN to predict the next character in the sequence.</p>
<h3>Paul Graham generator</h3>
<p>Lets first try a small dataset of English as a sanity check. My favorite fun dataset is the concatenation of <a href="http://www.paulgraham.com/articles.html">Paul Graham’s essays</a>. The basic idea is that there’s a lot of wisdom in these essays, but unfortunately Paul Graham is a relatively slow generator. Wouldn’t it be great if we could sample startup wisdom on demand? That’s where an RNN comes in.</p>
<p>Concatenating all pg essays over the last ~5 years we get approximately 1MB text file, or about 1 million characters (this is considered a very small dataset by the way). <em>Technical:</em> Lets train a 2-layer LSTM with 512 hidden nodes (approx. 3.5 million parameters), and with dropout of 0.5 after each layer. We’ll train with batches of 100 examples and truncated backpropagation through time of length 100 characters. With these settings one batch on a TITAN Z GPU takes about 0.46 seconds (this can be cut in half with 50 character BPTT at negligible cost in performance). Without further ado, lets see a sample from the RNN:</p>
<p><em>“The surprised in investors weren’t going to raise money. I’m not the company with the time there are all interesting quickly, don’t have to get off the same programmers. There’s a super-angel round fundraising, why do you can do. If you have a different physical investment are become in people who reduced in a startup with the way to argument the acquirer could see them just that you’re also the founders will part of users’ affords that and an alternation to the idea. [2] Don’t work at first member to see the way kids will seem in advance of a bad successful startup. And if you have to act the big company too.”</em></p>
<p>Okay, clearly the above is unfortunately not going to replace Paul Graham anytime soon, but remember that the RNN had to learn English completely from scratch and with a small dataset (including where you put commas, apostrophes and spaces). I also like that it learns to support its own arguments (e.g. [2], above). Sometimes it says something that offers a glimmer of insight, such as <em>“a company is a meeting to think to investors”</em>. <a href="http://cs.stanford.edu/people/karpathy/char-rnn/pg.txt">Here’s</a> a link to 50K character sample if you’d like to see more.</p>
<p><strong>Temperature.</strong> We can also play with the temperature of the Softmax during sampling. Decreasing the temperature from 1 to some lower number (e.g. 0.5) makes the RNN more confident, but also more conservative in its samples. Conversely, higher temperatures will give more diversity but at cost of more mistakes (e.g. spelling mistakes, etc). In particular, setting temperature very near zero will give the most likely thing that Paul Graham might say:</p>
<p><em>“is that they were all the same thing that was a startup is that they were all the same thing that was a startup is that they were all the same thing that was a startup is that they were all the same”</em></p>
<p>looks like we’ve reached an infinite loop about startups.</p>
<h3>Shakespeare</h3>
<p>It looks like we can learn to spell English words. But how about if there is more structure and style in the data? To examine this I downloaded all the works of Shakespeare and concatenated them into a single (4.4MB) file. We can now afford to train a larger network, in this case lets try a 3-layer RNN with 512 hidden nodes on each layer. After we train the network for a few hours we obtain samples such as:</p>
<div><pre><code>PANDARUS:
Alas, I think he shall be come approached and the day
When little srain would be attain'd into being never fed,
And who is but a chain and subjects of his death,
I should not sleep.
Second Senator:
They are away this miseries, produced upon my soul,
Breaking and strongly should be buried, when I perish
The earth and thoughts of many states.
DUKE VINCENTIO:
Well, your wit is in the care of side and that.
Second Lord:
They would be ruled after this chamber, and
my fair nues begun out of the fact, to be conveyed,
Whose noble souls I'll have the heart of the wars.
Clown:
Come, sir, I will make did behold your worship.
VIOLA:
I'll drink it.
</code></pre></div>
<p>Remember, all the RNN knows are characters, so in particular it samples both speaker’s names and the contents. Sometimes we also get relatively extented monologue passages, such as:</p>
<div><pre><code>VIOLA:
Why, Salisbury must find his flesh and thought
That which I am not aps, not a man and in fire,
To show the reining of the raven and the wars
To grace my hand reproach within, and not a fair are hand,
That Caesar and my goodly father's world;
When I was heaven of presence and our fleets,
We spare with hours, but cut thy council I am great,
Murdered and by thy master's ready there
My power to give thee but so much as hell:
Some service in the noble bondman here,
Would show him to her wine.
KING LEAR:
O, if you were a feeble sight, the courtesy of your law,
Your sight and several breath, will wear the gods
With his heads, and my hands are wonder'd at the deeds,
So drop upon your lordship's head, and your opinion
Shall be against your honour.
</code></pre></div>
<p>I can barely recognize these samples from actual Shakespeare :) If you like Shakespeare, you might appreciate this <a href="http://cs.stanford.edu/people/karpathy/char-rnn/shakespear.txt">100,000 character sample</a>. Of course, you can also generate an infinite amount of your own samples at different temperatures with the provided code.</p>
<h3>Wikipedia</h3>
<p>We saw that the LSTM can learn to spell words and copy general syntactic structures. Lets further increase the difficulty and train on structured markdown. In particular, lets take the <a href="http://prize.hutter1.net/">Hutter Prize</a> 100MB dataset of raw Wikipedia and train an LSTM. Following <a href="http://arxiv.org/abs/1308.0850">Graves et al.</a>, I used the first 96MB for training, the rest for validation and ran a few models overnight. We can now sample Wikipedia articles! Below are a few fun excerpts. First, some basic markdown output:</p>
<div><pre><code>Naturalism and decision for the majority of Arab countries' capitalide was grounded
by the Irish language by [[John Clair]], [[An Imperial Japanese Revolt]], associated 
with Guangzham's sovereignty. His generals were the powerful ruler of the Portugal 
in the [[Protestant Immineners]], which could be said to be directly in Cantonese 
Communication, which followed a ceremony and set inspired prison, training. The 
emperor travelled back to [[Antioch, Perth, October 25|21]] to note, the Kingdom 
of Costa Rica, unsuccessful fashioned the [[Thrales]], [[Cynth's Dajoard]], known 
in western [[Scotland]], near Italy to the conquest of India with the conflict. 
Copyright was the succession of independence in the slop of Syrian influence that 
was a famous German movement based on a more popular servicious, non-doctrinal 
and sexual power post. Many governments recognize the military housing of the 
[[Civil Liberalization and Infantry Resolution 265 National Party in Hungary]], 
that is sympathetic to be to the [[Punjab Resolution]]
(PJS)[http://www.humah.yahoo.com/guardian.
cfm/7754800786d17551963s89.htm Official economics Adjoint for the Nazism, Montgomery 
was swear to advance to the resources for those Socialism's rule, 
was starting to signing a major tripad of aid exile.]]
</code></pre></div>
<p>In case you were wondering, the yahoo url above doesn’t actually exist, the model just hallucinated it. Also, note that the model learns to open and close the parenthesis correctly. There’s also quite a lot of structured markdown that the model learns, for example sometimes it creates headings, lists, etc.:</p>
<div><pre><code>{ { cite journal | id=Cerling Nonforest Department|format=Newlymeslated|none } }
''www.e-complete''.
'''See also''': [[List of ethical consent processing]]
== See also ==
*[[Iender dome of the ED]]
*[[Anti-autism]]
===[[Religion|Religion]]===
*[[French Writings]]
*[[Maria]]
*[[Revelation]]
*[[Mount Agamul]]
== External links==
* [http://www.biblegateway.nih.gov/entrepre/ Website of the World Festival. The labour of India-county defeats at the Ripper of California Road.]
==External links==
* [http://www.romanology.com/ Constitution of the Netherlands and Hispanic Competition for Bilabial and Commonwealth Industry (Republican Constitution of the Extent of the Netherlands)]
</code></pre></div>
<p>Sometimes the model snaps into a mode of generating random but valid XML:</p>
<div><pre><code>&lt;page&gt;
 &lt;title&gt;Antichrist&lt;/title&gt;
 &lt;id&gt;865&lt;/id&gt;
 &lt;revision&gt;
 &lt;id&gt;15900676&lt;/id&gt;
 &lt;timestamp&gt;2002-08-03T18:14:12Z&lt;/timestamp&gt;
 &lt;contributor&gt;
 &lt;username&gt;Paris&lt;/username&gt;
 &lt;id&gt;23&lt;/id&gt;
 &lt;/contributor&gt;
 &lt;minor /&gt;
 &lt;comment&gt;Automated conversion&lt;/comment&gt;
 &lt;text xml:space="preserve"&gt;#REDIRECT [[Christianity]]&lt;/text&gt;
 &lt;/revision&gt;
&lt;/page&gt;
</code></pre></div>
<p>The model completely makes up the timestamp, id, and so on. Also, note that it closes the correct tags appropriately and in the correct nested order. Here are <a href="http://cs.stanford.edu/people/karpathy/char-rnn/wiki.txt">100,000 characters of sampled wikipedia</a> if you’re interested to see more.</p>
<h3>Algebraic Geometry (Latex)</h3>
<p>The results above suggest that the model is actually quite good at learning complex syntactic structures. Impressed by these results, my labmate (<a href="http://cs.stanford.edu/people/jcjohns/">Justin Johnson</a>) and I decided to push even further into structured territories and got a hold of <a href="http://stacks.math.columbia.edu/">this book</a> on algebraic stacks/geometry. We downloaded the raw Latex source file (a 16MB file) and trained a multilayer LSTM. Amazingly, the resulting sampled Latex <em>almost</em> compiles. We had to step in and fix a few issues manually but then you get plausible looking math, it’s quite astonishing:</p>
<div>
<p></p>
</div>
<p>Here’s another sample:</p>
<div>
<p></p><p>More hallucinated algebraic geometry. Nice try on the diagram (right).</p>
</div>
<p>As you can see above, sometimes the model tries to generate latex diagrams, but clearly it hasn’t really figured them out. I also like the part where it chooses to skip a proof (<em>“Proof omitted.”</em>, top left). Of course, keep in mind that latex has a relatively difficult structured syntactic format that I haven’t even fully mastered myself. For instance, here is a raw sample from the model (unedited):</p>
<div><pre><code>\begin{proof}
We may assume that $\mathcal{I}$ is an abelian sheaf on $\mathcal{C}$.
\item Given a morphism $\Delta : \mathcal{F} \to \mathcal{I}$
is an injective and let $\mathfrak q$ be an abelian sheaf on $X$.
Let $\mathcal{F}$ be a fibered complex. Let $\mathcal{F}$ be a category.
\begin{enumerate}
\item \hyperref[setain-construction-phantom]{Lemma}
\label{lemma-characterize-quasi-finite}
Let $\mathcal{F}$ be an abelian quasi-coherent sheaf on $\mathcal{C}$.
Let $\mathcal{F}$ be a coherent $\mathcal{O}_X$-module. Then
$\mathcal{F}$ is an abelian catenary over $\mathcal{C}$.
\item The following are equivalent
\begin{enumerate}
\item $\mathcal{F}$ is an $\mathcal{O}_X$-module.
\end{lemma}
</code></pre></div>
<p>This sample from a relatively decent model illustrates a few common mistakes. For example, the model opens a <code>\begin{proof}</code> environment but then ends it with a <code>\end{lemma}</code>. This is an example of a problem we’d have to fix manually, and is likely due to the fact that the dependency is too long-term: By the time the model is done with the proof it has forgotten whether it was doing a proof or a lemma. Similarly, it opens an <code>\begin{enumerate}</code> but then forgets to close it. We observed that these became less common with larger/better models, but nonetheless, these are the kinds of mistakes that come up.</p>
<h3>Linux Source Code</h3>
<p>I wanted to push structured data to its limit, so for the final challenge I decided to use code. In particular, I took all the source and header files found in the <a href="https://github.com/torvalds/linux">Linux repo on Github</a>, concatenated all of them in a single giant file (474MB of C code) (I was originally going to train only on the kernel but that by itself is only ~16MB). Then I trained several as-large-as-fits-on-my-GPU 3-layer LSTMs over a period of a few days. These models have about 10 million parameters, which is still on the lower end for RNN models. The results are superfun:</p>
<div><pre><code>/*
 * Increment the size file of the new incorrect UI_FILTER group information
 * of the size generatively.
 */
static int indicate_policy(void)
{
 int error;
 if (fd == MARN_EPT) {
 /*
 * The kernel blank will coeld it to userspace.
 */
 if (ss-&gt;segment &lt; mem_total)
 unblock_graph_and_set_blocked();
 else
 ret = 1;
 goto bail;
 }
 segaddr = in_SB(in.addr);
 selector = seg / 16;
 setup_works = true;
 for (i = 0; i &lt; blocks; i++) {
 seq = buf[i++];
 bpf = bd-&gt;bd.next + i * search;
 if (fd) {
 current = blocked;
 }
 }
 rw-&gt;name = "Getjbbregs";
 bprm_self_clearl(&amp;iv-&gt;version);
 regs-&gt;new = blocks[(BPF_STATS &lt;&lt; info-&gt;historidac)] | PFMR_CLOBATHINC_SECONDS &lt;&lt; 12;
 return segtable;
}
</code></pre></div>
<p>The code looks really quite great overall. Of course, I don’t think it compiles but when you scroll through the generate code it feels very much like a giant C code base. Notice that the RNN peppers its code with comments here and there at random. It is also very good at making very few syntactic errors. For example, it uses strings properly, pointer notation, etc. It also opens and closes brackets <code>{[</code> correctly and learns to indent its code very well. A common error is that it can’t keep track of variable names: It often uses undefined variables (e.g. <code>rw</code> above), declares variables it never uses (e.g. <code>int error</code>), or returns non-existing variables. Lets see a few more examples. Here’s another snippet that shows a wider array of operations that the RNN learns:</p>
<div><pre><code>/*
 * If this error is set, we will need anything right after that BSD.
 */
static void action_new_function(struct s_stat_info *wb)
{
 unsigned long flags;
 int lel_idx_bit = e-&gt;edd, *sys &amp; ~((unsigned long) *FIRST_COMPAT);
 buf[0] = 0xFFFFFFFF &amp; (bit &lt;&lt; 4);
 min(inc, slist-&gt;bytes);
 printk(KERN_WARNING "Memory allocated %02x/%02x, "
 "original MLL instead\n"),
 min(min(multi_run - s-&gt;len, max) * num_data_in),
 frame_pos, sz + first_seg);
 div_u64_w(val, inb_p);
 spin_unlock(&amp;disk-&gt;queue_lock);
 mutex_unlock(&amp;s-&gt;sock-&gt;mutex);
 mutex_unlock(&amp;func-&gt;mutex);
 return disassemble(info-&gt;pending_bh);
}
static void num_serial_settings(struct tty_struct *tty)
{
 if (tty == tty)
 disable_single_st_p(dev);
 pci_disable_spool(port);
 return 0;
}
static void do_command(struct seq_file *m, void *v)
{
 int column = 32 &lt;&lt; (cmd[2] &amp; 0x80);
 if (state)
 cmd = (int)(int_state ^ (in_8(&amp;ch-&gt;ch_flags) &amp; Cmd) ? 2 : 1);
 else
 seq = 1;
 for (i = 0; i &lt; 16; i++) {
 if (k &amp; (1 &lt;&lt; 1))
 pipe = (in_use &amp; UMXTHREAD_UNCCA) +
 ((count &amp; 0x00000000fffffff8) &amp; 0x000000f) &lt;&lt; 8;
 if (count == 0)
 sub(pid, ppc_md.kexec_handle, 0x20000000);
 pipe_set_bytes(i, 0);
 }
 /* Free our user pages pointer to place camera if all dash */
 subsystem_info = &amp;of_changes[PAGE_SIZE];
 rek_controls(offset, idx, &amp;soffset);
 /* Now we want to deliberately put it to device */
 control_check_polarity(&amp;context, val, 0);
 for (i = 0; i &lt; COUNTER; i++)
 seq_puts(s, "policy ");
}
</code></pre></div>
<p>Notice that in the second function the model compares <code>tty == tty</code>, which is vacuously true. On the other hand, at least the variable <code>tty</code> exists in the scope this time! In the last function, notice that the code does not return anything, which happens to be correct since the function signature is <code>void</code>. However, the first two functions were also declared <code>void</code> and did return values. This is again a form of a common mistake due to long-term interactions.</p>
<p>Sometimes the model decides that it’s time to sample a new file. This is usually a very amusing part: The model first recites the GNU license character by character, samples a few includes, generates some macros and then dives into the code:</p>
<div><pre><code>/*
 * Copyright (c) 2006-2010, Intel Mobile Communications. All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify it
 * under the terms of the GNU General Public License version 2 as published by
 * the Free Software Foundation.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 *
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software Foundation,
 * Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 */
#include &lt;linux/kexec.h&gt;
#include &lt;linux/errno.h&gt;
#include &lt;linux/io.h&gt;
#include &lt;linux/platform_device.h&gt;
#include &lt;linux/multi.h&gt;
#include &lt;linux/ckevent.h&gt;
#include &lt;asm/io.h&gt;
#include &lt;asm/prom.h&gt;
#include &lt;asm/e820.h&gt;
#include &lt;asm/system_info.h&gt;
#include &lt;asm/setew.h&gt;
#include &lt;asm/pgproto.h&gt;
#define REG_PG vesa_slot_addr_pack
#define PFM_NOCOMP AFSR(0, load)
#define STACK_DDR(type) (func)
#define SWAP_ALLOCATE(nr) (e)
#define emulate_sigs() arch_get_unaligned_child()
#define access_rw(TST) asm volatile("movd %%esp, %0, %3" : : "r" (0)); \
 if (__type &amp; DO_READ)
static void stat_PC_SEC __read_mostly offsetof(struct seq_argsqueue, \
 pC&gt;[1]);
static void
os_prefix(unsigned long sys)
{
#ifdef CONFIG_PREEMPT
 PUT_PARAM_RAID(2, sel) = get_state_state();
 set_pid_sum((unsigned long)state, current_state_str(),
 (unsigned long)-1-&gt;lr_full; low;
}
</code></pre></div>
<p>There are too many fun parts to cover- I could probably write an entire blog post on just this part. I’ll cut it short for now, but here is <a href="http://cs.stanford.edu/people/karpathy/char-rnn/linux.txt">1MB of sampled Linux code</a> for your viewing pleasure.</p>
<h3>Generating Baby Names</h3>
<p>Lets try one more for fun. Lets feed the RNN a large text file that contains 8000 baby names listed out, one per line (names obtained from <a href="http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/areas/nlp/corpora/names/">here</a>). We can feed this to the RNN and then generate new names! Here are some example names, only showing the ones that do not occur in the training data (90% don’t):</p>
<p><em>Rudi
Levette
Berice
Lussa
Hany
Mareanne
Chrestina
Carissy
Marylen
Hammine
Janye
Marlise
Jacacrie
Hendred
Romand
Charienna
Nenotto
Ette
Dorane
Wallen
Marly
Darine
Salina
Elvyn
Ersia
Maralena
Minoria
Ellia
Charmin
Antley
Nerille
Chelon
Walmor
Evena
Jeryly
Stachon
Charisa
Allisa
Anatha
Cathanie
Geetra
Alexie
Jerin
Cassen
Herbett
Cossie
Velen
Daurenge
Robester
Shermond
Terisa
Licia
Roselen
Ferine
Jayn
Lusine
Charyanne
Sales
Sanny
Resa
Wallon
Martine
Merus
Jelen
Candica
Wallin
Tel
Rachene
Tarine
Ozila
Ketia
Shanne
Arnande
Karella
Roselina
Alessia
Chasty
Deland
Berther
Geamar
Jackein
Mellisand
Sagdy
Nenc
Lessie
Rasemy
Guen
Gavi
Milea
Anneda
Margoris
Janin
Rodelin
Zeanna
Elyne
Janah
Ferzina
Susta
Pey
Castina</em></p>
<p>You can see many more <a href="http://cs.stanford.edu/people/karpathy/namesGenUnique.txt">here</a>. Some of my favorites include “Baby” (haha), “Killie”, “Char”, “R”, “More”, “Mars”, “Hi”, “Saddie”, “With” and “Ahbort”. Well that was fun.﻿ Of course, you can imagine this being quite useful inspiration when writing a novel, or naming a new startup :)</p>
<h2>Understanding what’s going on</h2>
<p>We saw that the results at the end of training can be impressive, but how does any of this work? Lets run two quick experiments to briefly peek under the hood.</p>
<h3>The evolution of samples while training</h3>
<p>First, it’s fun to look at how the sampled text evolves while the model trains. For example, I trained an LSTM of Leo Tolstoy’s War and Peace and then generated samples every 100 iterations of training. At iteration 100 the model samples random jumbles:</p>
<div><pre><code>tyntd-iafhatawiaoihrdemot lytdws e ,tfti, astai f ogoh eoase rrranbyne 'nhthnee e 
plia tklrgd t o idoe ns,smtt h ne etie h,hregtrs nigtike,aoaenns lng
</code></pre></div>
<p>However, notice that at least it is starting to get an idea about words separated by spaces. Except sometimes it inserts two spaces. It also doesn’t know that comma is amost always followed by a space. At 300 iterations we see that the model starts to get an idea about quotes and periods:</p>
<div><pre><code>"Tmont thithey" fomesscerliund
Keushey. Thom here
sheulke, anmerenith ol sivh I lalterthend Bleipile shuwy fil on aseterlome
coaniogennc Phe lism thond hon at. MeiDimorotion in ther thize."
</code></pre></div>
<p>The words are now also separated with spaces and the model starts to get the idea about periods at the end of a sentence. At iteration 500:</p>
<div><pre><code>we counter. He stutn co des. His stanted out one ofler that concossions and was 
to gearang reay Jotrets and with fre colt otf paitt thin wall. Which das stimn 
</code></pre></div>
<p>the model has now learned to spell the shortest and most common words such as “we”, “He”, “His”, “Which”, “and”, etc. At iteration 700 we’re starting to see more and more English-like text emerge:</p>
<div><pre><code>Aftair fall unsuch that the hall for Prince Velzonski's that me of
her hearly, and behs to so arwage fiving were to it beloge, pavu say falling misfort 
how, and Gogition is so overelical and ofter.
</code></pre></div>
<p>At iteration 1200 we’re now seeing use of quotations and question/exclamation marks. Longer words have now been learned as well:</p>
<div><pre><code>"Kite vouch!" he repeated by her
door. "But I would be done and quarts, feeling, then, son is people...."
</code></pre></div>
<p>Until at last we start to get properly spelled words, quotations, names, and so on by about iteration 2000:</p>
<div><pre><code>"Why do what that day," replied Natasha, and wishing to himself the fact the
princess, Princess Mary was easier, fed in had oftened him.
Pierre aking his soul came to the packs and drove up his father-in-law women.
</code></pre></div>
<p>The picture that emerges is that the model first discovers the general word-space structure and then rapidly starts to learn the words; First starting with the short words and then eventually the longer ones. Topics and themes that span multiple words (and in general longer-term dependencies) start to emerge only much later.</p>
<h3>Visualizing the predictions and the “neuron” firings in the RNN</h3>
<p>Another fun visualization is to look at the predicted distributions over characters. In the visualizations below we feed a Wikipedia RNN model character data from the validation set (shown along the blue/green rows) and under every character we visualize (in red) the top 5 guesses that the model assigns for the next character. The guesses are colored by their probability (so dark red = judged as very likely, white = not very likely). For example, notice that there are stretches of characters where the model is extremely confident about the next letter (e.g., the model is very confident about characters during the <em>http://www.</em> sequence).</p>
<p>The input character sequence (blue/green) is colored based on the <em>firing</em> of a randomly chosen neuron in the hidden representation of the RNN. Think about it as green = very excited and blue = not very excited (for those familiar with details of LSTMs, these are values between [-1,1] in the hidden state vector, which is just the gated and tanh’d LSTM cell state). Intuitively, this is visualizing the firing rate of some neuron in the “brain” of the RNN while it reads the input sequence. Different neurons might be looking for different patterns; Below we’ll look at 4 different ones that I found and thought were interesting or interpretable (many also aren’t):</p>
<div>
<p></p><p>
The neuron highlighted in this image seems to get very excited about URLs and turns off outside of the URLs. The LSTM is likely using this neuron to remember if it is inside a URL or not.
</p>
</div>
<div>
<p></p><p>
The highlighted neuron here gets very excited when the RNN is inside the [[ ]] markdown environment and turns off outside of it. Interestingly, the neuron can't turn on right after it sees the character "[", it must wait for the second "[" and then activate. This task of counting whether the model has seen one or two "[" is likely done with a different neuron.
</p>
</div>
<div>
<p></p><p>
Here we see a neuron that varies seemingly linearly across the [[ ]] environment. In other words its activation is giving the RNN a time-aligned coordinate system across the [[ ]] scope. The RNN can use this information to make different characters more or less likely depending on how early/late it is in the [[ ]] scope (perhaps?).
</p>
</div>
<div>
<p></p><p>
Here is another neuron that has very local behavior: it is relatively silent but sharply turns off right after the first "w" in the "www" sequence. The RNN might be using this neuron to count up how far in the "www" sequence it is, so that it can know whether it should emit another "w", or if it should start the URL.
</p>
</div>
<p>Of course, a lot of these conclusions are slightly hand-wavy as the hidden state of the RNN is a huge, high-dimensional and largely distributed representation. These visualizations were produced with custom HTML/CSS/Javascript, you can see a sketch of what’s involved <a href="http://cs.stanford.edu/people/karpathy/viscode.zip">here</a> if you’d like to create something similar.</p>
<p>We can also condense this visualization by excluding the most likely predictions and only visualize the text, colored by activations of a cell. We can see that in addition to a large portion of cells that do not do anything interpretible, about 5% of them turn out to have learned quite interesting and interpretible algorithms:</p>
<div>
<p>
</p>
</div>
<p>Again, what is beautiful about this is that we didn’t have to hardcode at any point that if you’re trying to predict the next character it might, for example, be useful to keep track of whether or not you are currently inside or outside of quote. We just trained the LSTM on raw data and it decided that this is a useful quantitity to keep track of. In other words one of its cells gradually tuned itself during training to become a quote detection cell, since this helps it better perform the final task. This is one of the cleanest and most compelling examples of where the power in Deep Learning models (and more generally end-to-end training) is coming from.</p>
<h2>Source Code</h2>
<p>I hope I’ve convinced you that training character-level language models is a very fun exercise. You can train your own models using the <a href="https://github.com/karpathy/char-rnn">char-rnn code</a> I released on Github (under MIT license). It takes one large text file and trains a character-level model that you can then sample from. Also, it helps if you have a GPU or otherwise training on CPU will be about a factor of 10x slower. In any case, if you end up training on some data and getting fun results let me know! And if you get lost in the Torch/Lua codebase remember that all it is is just a more fancy version of this <a href="https://gist.github.com/karpathy/d4dee566867f8291f086">100-line gist</a>.</p>
<p><em>Brief digression.</em> The code is written in <a href="http://torch.ch/">Torch 7</a>, which has recently become my favorite deep learning framework. I’ve only started working with Torch/LUA over the last few months and it hasn’t been easy (I spent a good amount of time digging through the raw Torch code on Github and asking questions on their <em>gitter</em> to get things done), but once you get a hang of things it offers a lot of flexibility and speed. I’ve also worked with Caffe and Theano in the past and I believe Torch, while not perfect, gets its levels of abstraction and philosophy right better than others. In my view the desirable features of an effective framework are:</p>
<ol>
 <li>CPU/GPU transparent Tensor library with a lot of functionality (slicing, array/matrix operations, etc. )</li>
 <li>An entirely separate code base in a scripting language (ideally Python) that operates over Tensors and implements all Deep Learning stuff (forward/backward, computation graphs, etc)</li>
 <li>It should be possible to easily share pretrained models (Caffe does this well, others don’t), and crucially</li>
 <li>NO compilation step (or at least not as currently done in Theano). The trend in Deep Learning is towards larger, more complex networks that are are time-unrolled in complex graphs. It is critical that these do not compile for a long time or development time greatly suffers. Second, by compiling one gives up interpretability and the ability to log/debug effectively. If there is an <em>option</em> to compile the graph once it has been developed for efficiency in prod that’s fine.</li>
</ol>
<h2>Further Reading</h2>
<p>Before the end of the post I also wanted to position RNNs in a wider context and provide a sketch of the current research directions. RNNs have recently generated a significant amount of buzz and excitement in the field of Deep Learning. Similar to Convolutional Networks they have been around for decades but their full potential has only recently started to get widely recognized, in large part due to our growing computational resources. Here’s a brief sketch of a few recent developments (definitely not complete list, and a lot of this work draws from research back to 1990s, see related work sections):</p>
<p>In the domain of <strong>NLP/Speech</strong>, RNNs <a href="http://www.jmlr.org/proceedings/papers/v32/graves14.pdf">transcribe speech to text</a>, perform <a href="http://arxiv.org/abs/1409.3215">machine translation</a>, <a href="http://www.cs.toronto.edu/~graves/handwriting.html">generate handwritten text</a>, and of course, they have been used as powerful language models <a href="http://www.cs.utoronto.ca/~ilya/pubs/2011/LANG-RNN.pdf">(Sutskever et al.)</a> <a href="http://arxiv.org/abs/1308.0850">(Graves)</a> <a href="http://www.rnnlm.org/">(Mikolov et al.)</a> (both on the level of characters and words). Currently it seems that word-level models work better than character-level models, but this is surely a temporary thing.</p>
<p><strong>Computer Vision.</strong> RNNs are also quickly becoming pervasive in Computer Vision. For example, we’re seeing RNNs in frame-level <a href="http://arxiv.org/abs/1411.4389">video classification</a>, <a href="http://arxiv.org/abs/1411.4555">image captioning</a> (also including my own work and many others), <a href="http://arxiv.org/abs/1505.00487">video captioning</a> and very recently <a href="http://arxiv.org/abs/1505.02074">visual question answering</a>. My personal favorite RNNs in Computer Vision paper is <a href="http://arxiv.org/abs/1406.6247">Recurrent Models of Visual Attention</a>, both due to its high-level direction (sequential processing of images with glances) and the low-level modeling (REINFORCE learning rule that is a special case of policy gradient methods in Reinforcement Learning, which allows one to train models that perform non-differentiable computation (taking glances around the image in this case)). I’m confident that this type of hybrid model that consists of a blend of CNN for raw perception coupled with an RNN glance policy on top will become pervasive in perception, especially for more complex tasks that go beyond classifying some objects in plain view.</p>
<p><strong>Inductive Reasoning, Memories and Attention.</strong> Another extremely exciting direction of research is oriented towards addressing the limitations of vanilla recurrent networks. One problem is that RNNs are not inductive: They memorize sequences extremely well, but they don’t necessarily always show convincing signs of generalizing in the <em>correct</em> way (I’ll provide pointers in a bit that make this more concrete). A second issue is they unnecessarily couple their representation size to the amount of computation per step. For instance, if you double the size of the hidden state vector you’d quadruple the amount of FLOPS at each step due to the matrix multiplication. Ideally, we’d like to maintain a huge representation/memory (e.g. containing all of Wikipedia or many intermediate state variables), while maintaining the ability to keep computation per time step fixed.</p>
<p>The first convincing example of moving towards these directions was developed in DeepMind’s <a href="http://arxiv.org/abs/1410.5401">Neural Turing Machines</a> paper. This paper sketched a path towards models that can perform read/write operations between large, external memory arrays and a smaller set of memory registers (think of these as our working memory) where the computation happens. Crucially, the NTM paper also featured very interesting memory addressing mechanisms that were implemented with a (soft, and fully-differentiable) attention model. The concept of <strong>soft attention</strong> has turned out to be a powerful modeling feature and was also featured in <a href="http://arxiv.org/abs/1409.0473">Neural Machine Translation by Jointly Learning to Align and Translate</a> for Machine Translation and <a href="http://arxiv.org/abs/1503.08895">Memory Networks</a> for (toy) Question Answering. In fact, I’d go as far as to say that</p>
<blockquote>
 <p>The concept of <strong>attention</strong> is the most interesting recent architectural innovation in neural networks.</p>
</blockquote>
<p>Now, I don’t want to dive into too many details but a soft attention scheme for memory addressing is convenient because it keeps the model fully-differentiable, but unfortunately one sacrifices efficiency because everything that can be attended to is attended to (but softly). Think of this as declaring a pointer in C that doesn’t point to a specific address but instead defines an entire distribution over all addresses in the entire memory, and dereferencing the pointer returns a weighted sum of the pointed content (that would be an expensive operation!). This has motivated multiple authors to swap soft attention models for <strong>hard attention</strong> where one samples a particular chunk of memory to attend to (e.g. a read/write action for some memory cell instead of reading/writing from all cells to some degree). This model is significantly more philosophically appealing, scalable and efficient, but unfortunately it is also non-differentiable. This then calls for use of techniques from the Reinforcement Learning literature (e.g. REINFORCE) where people are perfectly used to the concept of non-differentiable interactions. This is very much ongoing work but these hard attention models have been explored, for example, in <a href="http://arxiv.org/abs/1503.01007">Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets</a>, <a href="http://arxiv.org/abs/1505.00521">Reinforcement Learning Neural Turing Machines</a>, and <a href="http://arxiv.org/abs/1502.03044">Show Attend and Tell</a>.</p>
<p><strong>People</strong>. If you’d like to read up on RNNs I recommend theses from <a href="http://www.cs.toronto.edu/~graves/">Alex Graves</a>, <a href="http://www.cs.toronto.edu/~ilya/">Ilya Sutskever</a> and <a href="http://www.rnnlm.org/">Tomas Mikolov</a>. For more about REINFORCE and more generally Reinforcement Learning and policy gradient methods (which REINFORCE is a special case of) <a href="http://www0.cs.ucl.ac.uk/staff/d.silver/web/Home.html">David Silver</a>’s class, or one of <a href="http://www.cs.berkeley.edu/~pabbeel/">Pieter Abbeel</a>’s classes.</p>
<p><strong>Code</strong>. If you’d like to play with training RNNs I hear good things about <a href="https://github.com/fchollet/keras">keras</a> or <a href="https://github.com/IndicoDataSolutions/Passage">passage</a> for Theano, the <a href="https://github.com/karpathy/char-rnn">code</a> released with this post for Torch, or <a href="https://gist.github.com/karpathy/587454dc0146a6ae21fc">this gist</a> for raw numpy code I wrote a while ago that implements an efficient, batched LSTM forward and backward pass. You can also have a look at my numpy-based <a href="https://github.com/karpathy/neuraltalk">NeuralTalk</a> which uses an RNN/LSTM to caption images, or maybe this <a href="http://jeffdonahue.com/lrcn/">Caffe</a> implementation by Jeff Donahue.</p>
<h2>Conclusion</h2>
<p>We’ve learned about RNNs, how they work, why they have become a big deal, we’ve trained an RNN character-level language model on several fun datasets, and we’ve seen where RNNs are going. You can confidently expect a large amount of innovation in the space of RNNs, and I believe they will become a pervasive and critical component to intelligent systems.</p>
<p>Lastly, to add some <strong>meta</strong> to this post, I trained an RNN on the source file of this blog post. Unfortunately, at about 46K characters I haven’t written enough data to properly feed the RNN, but the returned sample (generated with low temperature to get a more typical sample) is:</p>
<div><pre><code>I've the RNN with and works, but the computed with program of the 
RNN with and the computed of the RNN with with and the code
</code></pre></div>
<p>Yes, the post was about RNN and how well it works, so clearly this works :). See you next time!</p>
<p><strong>EDIT (extra links):</strong></p>
<p>Videos:</p>
<ul>
 <li>I gave a talk on this work at the <a href="https://skillsmatter.com/skillscasts/6611-visualizing-and-understanding-recurrent-networks">London Deep Learning meetup (video)</a>.</li>
</ul>
<p>Discussions:</p>
<ul>
 <li><a href="https://news.ycombinator.com/item?id=9584325">HN discussion</a></li>
 <li>Reddit discussion on <a href="http://www.reddit.com/r/MachineLearning/comments/36s673/the_unreasonable_effectiveness_of_recurrent/">r/machinelearning</a></li>
 <li>Reddit discussion on <a href="http://www.reddit.com/r/programming/comments/36su8d/the_unreasonable_effectiveness_of_recurrent/">r/programming</a></li>
</ul>
<p>Replies:</p>
<ul>
 <li><a href="https://twitter.com/yoavgo">Yoav Goldberg</a> compared these RNN results to <a href="http://nbviewer.ipython.org/gist/yoavg/d76121dfde2618422139">n-gram maximum likelihood (counting) baseline</a></li>
 <li><a href="https://twitter.com/nylk">@nylk</a> trained char-rnn on <a href="https://gist.github.com/nylki/1efbaa36635956d35bcc">cooking recipes</a>. They look great!</li>
 <li><a href="https://twitter.com/MrChrisJohnson">@MrChrisJohnson</a> trained char-rnn on Eminem lyrics and then synthesized a rap song with robotic voice reading it out. Hilarious :)</li>
 <li><a href="https://twitter.com/samim">@samim</a> trained char-rnn on <a href="https://medium.com/@samim/obama-rnn-machine-generated-political-speeches-c8abd18a2ea0">Obama Speeches</a>. They look fun!</li>
 <li><a href="https://twitter.com/seaandsailor">João Felipe</a> trained char-rnn irish folk music and <a href="https://soundcloud.com/seaandsailor/sets/char-rnn-composes-irish-folk-music">sampled music</a></li>
 <li><a href="https://twitter.com/boblsturm">Bob Sturm</a> also trained char-rnn on <a href="https://highnoongmt.wordpress.com/2015/05/22/lisls-stis-recurrent-neural-networks-for-folk-music-generation/">music in ABC notation</a></li>
 <li><a href="https://twitter.com/RNN_Bible">RNN Bible bot</a> by <a href="https://twitter.com/the__glu/with_replies">Maximilien</a></li>
 <li><a href="http://cpury.github.io/learning-holiness/">Learning Holiness</a> learning the Bible</li>
 <li><a href="https://www.terminal.com/tiny/ZMcqdkWGOM">Terminal.com snapshot</a> that has char-rnn set up and ready to go in a browser-based virtual machine (thanks <a href="https://www.twitter.com/samim">@samim</a>)</li>
</ul>
 </article>||||I|||| Andrej Karpathy blog
 About

 The Unreasonable Effectiveness of Recurrent Neural Networks

 May 21, 2015

 There’s something magical about Recurrent Neural Networks (RNNs). I still remember when I trained my first recurrent network for Image Captioning. Within a few dozen minutes of training my first baby model (with rather arbitrarily-chosen hyperparameters) started to generate very nice looking descriptions of images that were on the edge of making sense. Sometimes the ratio of how simple your model is to the quality of the results you get out of it blows past your expectations, and this was one of those times. What made this result so shocking at the time was that the common wisdom was that RNNs were supposed to be difficult to train (with more experience I’ve in fact reached the opposite conclusion). Fast forward about a year: I’m training RNNs all the time and I’ve witnessed their power and robustness many times, and yet their magical outputs still find ways of amusing me. This post is about sharing some of that magic with you.

 We’ll train RNNs to generate text character by character and ponder the question “how is that even possible?”

 By the way, together with this post I am also releasing code on Github that allows you to train character-level language models based on multi-layer LSTMs. You give it a large chunk of text and it will learn to generate text like it one character at a time. You can also use it to reproduce my experiments below. But we’re getting ahead of ourselves; What are RNNs anyway?

 Recurrent Neural Networks

 Sequences. Depending on your background you might be wondering: What makes Recurrent Networks so special? A glaring limitation of Vanilla Neural Networks (and also Convolutional Networks) is that their API is too constrained: they accept a fixed-sized vector as input (e.g. an image) and produce a fixed-sized vector as output (e.g. probabilities of different classes). Not only that: These models perform this mapping using a fixed amount of computational steps (e.g. the number of layers in the model). The core reason that recurrent nets are more exciting is that they allow us to operate over sequences of vectors: Sequences in the input, the output, or in the most general case both. A few examples may make this more concrete:

 Each rectangle is a vector and arrows represent functions (e.g. matrix multiply). Input vectors are in red, output vectors are in blue and green vectors hold the RNN's state (more on this soon). From left to right: (1) Vanilla mode of processing without RNN, from fixed-sized input to fixed-sized output (e.g. image classification). (2) Sequence output (e.g. image captioning takes an image and outputs a sentence of words). (3) Sequence input (e.g. sentiment analysis where a given sentence is classified as expressing positive or negative sentiment). (4) Sequence input and sequence output (e.g. Machine Translation: an RNN reads a sentence in English and then outputs a sentence in French). (5) Synced sequence input and output (e.g. video classification where we wish to label each frame of the video). Notice that in every case are no pre-specified constraints on the lengths sequences because the recurrent transformation (green) is fixed and can be applied as many times as we like.

 As you might expect, the sequence regime of operation is much more powerful compared to fixed networks that are doomed from the get-go by a fixed number of computational steps, and hence also much more appealing for those of us who aspire to build more intelligent systems. Moreover, as we’ll see in a bit, RNNs combine the input vector with their state vector with a fixed (but learned) function to produce a new state vector. This can in programming terms be interpreted as running a fixed program with certain inputs and some internal variables. Viewed this way, RNNs essentially describe programs. In fact, it is known that RNNs are Turing-Complete in the sense that they can to simulate arbitrary programs (with proper weights). But similar to universal approximation theorems for neural nets you shouldn’t read too much into this. In fact, forget I said anything.

 If training vanilla neural nets is optimization over functions, training recurrent nets is optimization over programs.

 Sequential processing in absence of sequences. You might be thinking that having sequences as inputs or outputs could be relatively rare, but an important point to realize is that even if your inputs/outputs are fixed vectors, it is still possible to use this powerful formalism to process them in a sequential manner. For instance, the figure below shows results from two very nice papers from DeepMind. On the left, an algorithm learns a recurrent network policy that steers its attention around an image; In particular, it learns to read out house numbers from left to right (Ba et al.). On the right, a recurrent network generates images of digits by learning to sequentially add color to a canvas (Gregor et al.):

 Left: RNN learns to read house numbers. Right: RNN learns to paint house numbers.

 The takeaway is that even if your data is not in form of sequences, you can still formulate and train powerful models that learn to process it sequentially. You’re learning stateful programs that process your fixed-sized data.

 RNN computation. So how do these things work? At the core, RNNs have a deceptively simple API: They accept an input vector x and give you an output vector y. However, crucially this output vector’s contents are influenced not only by the input you just fed in, but also on the entire history of inputs you’ve fed in in the past. Written as a class, the RNN’s API consists of a single step function:

 rnn = RNN()
 y = rnn.step(x) # x is an input vector, y is the RNN's output vector
 

 The RNN class has some internal state that it gets to update every time step is called. In the simplest case this state consists of a single hidden vector h. Here is an implementation of the step function in a Vanilla RNN:

 class RNN:
 # ...
 def step(self, x):
 # update the hidden state
 self.h = np.tanh(np.dot(self.W_hh, self.h) + np.dot(self.W_xh, x))
 # compute the output vector
 y = np.dot(self.W_hy, self.h)
 return y
 

 The above specifies the forward pass of a vanilla RNN. This RNN’s parameters are the three matrices W_hh, W_xh, W_hy. The hidden state self.h is initialized with the zero vector. The np.tanh function implements a non-linearity that squashes the activations to the range [-1, 1]. Notice briefly how this works: There are two terms inside of the tanh: one is based on the previous hidden state and one is based on the current input. In numpy np.dot is matrix multiplication. The two intermediates interact with addition, and then get squashed by the tanh into the new state vector. If you’re more comfortable with math notation, we can also write the hidden state update as \( h_t = \tanh ( W_{hh} h_{t-1} + W_{xh} x_t ) \), where tanh is applied elementwise.

 We initialize the matrices of the RNN with random numbers and the bulk of work during training goes into finding the matrices that give rise to desirable behavior, as measured with some loss function that expresses your preference to what kinds of outputs y you’d like to see in response to your input sequences x.

 Going deep. RNNs are neural networks and everything works monotonically better (if done right) if you put on your deep learning hat and start stacking models up like pancakes. For instance, we can form a 2-layer recurrent network as follows:

 y1 = rnn1.step(x)
 y = rnn2.step(y1)
 

 In other words we have two separate RNNs: One RNN is receiving the input vectors and the second RNN is receiving the output of the first RNN as its input. Except neither of these RNNs know or care - it’s all just vectors coming in and going out, and some gradients flowing through each module during backpropagation.

 Getting fancy. I’d like to briefly mention that in practice most of us use a slightly different formulation than what I presented above called a Long Short-Term Memory (LSTM) network. The LSTM is a particular type of recurrent network that works slightly better in practice, owing to its more powerful update equation and some appealing backpropagation dynamics. I won’t go into details, but everything I’ve said about RNNs stays exactly the same, except the mathematical form for computing the update (the line self.h = ... ) gets a little more complicated. From here on I will use the terms “RNN/LSTM” interchangeably but all experiments in this post use an LSTM.

 Character-Level Language Models

 Okay, so we have an idea about what RNNs are, why they are super exciting, and how they work. We’ll now ground this in a fun application: We’ll train RNN character-level language models. That is, we’ll give the RNN a huge chunk of text and ask it to model the probability distribution of the next character in the sequence given a sequence of previous characters. This will then allow us to generate new text one character at a time.

 As a working example, suppose we only had a vocabulary of four possible letters “helo”, and wanted to train an RNN on the training sequence “hello”. This training sequence is in fact a source of 4 separate training examples: 1. The probability of “e” should be likely given the context of “h”, 2. “l” should be likely in the context of “he”, 3. “l” should also be likely given the context of “hel”, and finally 4. “o” should be likely given the context of “hell”.

 Concretely, we will encode each character into a vector using 1-of-k encoding (i.e. all zero except for a single one at the index of the character in the vocabulary), and feed them into the RNN one at a time with the step function. We will then observe a sequence of 4-dimensional output vectors (one dimension per character), which we interpret as the confidence the RNN currently assigns to each character coming next in the sequence. Here’s a diagram:

 An example RNN with 4-dimensional input and output layers, and a hidden layer of 3 units (neurons). This diagram shows the activations in the forward pass when the RNN is fed the characters "hell" as input. The output layer contains confidences the RNN assigns for the next character (vocabulary is "h,e,l,o"); We want the green numbers to be high and red numbers to be low.

 For example, we see that in the first time step when the RNN saw the character “h” it assigned confidence of 1.0 to the next letter being “h”, 2.2 to letter “e”, -3.0 to “l”, and 4.1 to “o”. Since in our training data (the string “hello”) the next correct character is “e”, we would like to increase its confidence (green) and decrease the confidence of all other letters (red). Similarly, we have a desired target character at every one of the 4 time steps that we’d like the network to assign a greater confidence to. Since the RNN consists entirely of differentiable operations we can run the backpropagation algorithm (this is just a recursive application of the chain rule from calculus) to figure out in what direction we should adjust every one of its weights to increase the scores of the correct targets (green bold numbers). We can then perform a parameter update, which nudges every weight a tiny amount in this gradient direction. If we were to feed the same inputs to the RNN after the parameter update we would find that the scores of the correct characters (e.g. “e” in the first time step) would be slightly higher (e.g. 2.3 instead of 2.2), and the scores of incorrect characters would be slightly lower. We then repeat this process over and over many times until the network converges and its predictions are eventually consistent with the training data in that correct characters are always predicted next.

 A more technical explanation is that we use the standard Softmax classifier (also commonly referred to as the cross-entropy loss) on every output vector simultaneously. The RNN is trained with mini-batch Stochastic Gradient Descent and I like to use RMSProp or Adam (per-parameter adaptive learning rate methods) to stablilize the updates.

 Notice also that the first time the character “l” is input, the target is “l”, but the second time the target is “o”. The RNN therefore cannot rely on the input alone and must use its recurrent connection to keep track of the context to achieve this task.

 At test time, we feed a character into the RNN and get a distribution over what characters are likely to come next. We sample from this distribution, and feed it right back in to get the next letter. Repeat this process and you’re sampling text! Lets now train an RNN on different datasets and see what happens.

 To further clarify, for educational purposes I also wrote a minimal character-level RNN language model in Python/numpy. It is only about 100 lines long and hopefully it gives a concise, concrete and useful summary of the above if you’re better at reading code than text. We’ll now dive into example results, produced with the much more efficient Lua/Torch codebase.

 Fun with RNNs

 All 5 example character models below were trained with the code I’m releasing on Github. The input in each case is a single file with some text, and we’re training an RNN to predict the next character in the sequence.

 Paul Graham generator

 Lets first try a small dataset of English as a sanity check. My favorite fun dataset is the concatenation of Paul Graham’s essays. The basic idea is that there’s a lot of wisdom in these essays, but unfortunately Paul Graham is a relatively slow generator. Wouldn’t it be great if we could sample startup wisdom on demand? That’s where an RNN comes in.

 Concatenating all pg essays over the last ~5 years we get approximately 1MB text file, or about 1 million characters (this is considered a very small dataset by the way). Technical: Lets train a 2-layer LSTM with 512 hidden nodes (approx. 3.5 million parameters), and with dropout of 0.5 after each layer. We’ll train with batches of 100 examples and truncated backpropagation through time of length 100 characters. With these settings one batch on a TITAN Z GPU takes about 0.46 seconds (this can be cut in half with 50 character BPTT at negligible cost in performance). Without further ado, lets see a sample from the RNN:

 “The surprised in investors weren’t going to raise money. I’m not the company with the time there are all interesting quickly, don’t have to get off the same programmers. There’s a super-angel round fundraising, why do you can do. If you have a different physical investment are become in people who reduced in a startup with the way to argument the acquirer could see them just that you’re also the founders will part of users’ affords that and an alternation to the idea. [2] Don’t work at first member to see the way kids will seem in advance of a bad successful startup. And if you have to act the big company too.”

 Okay, clearly the above is unfortunately not going to replace Paul Graham anytime soon, but remember that the RNN had to learn English completely from scratch and with a small dataset (including where you put commas, apostrophes and spaces). I also like that it learns to support its own arguments (e.g. [2], above). Sometimes it says something that offers a glimmer of insight, such as “a company is a meeting to think to investors”. Here’s a link to 50K character sample if you’d like to see more.

 Temperature. We can also play with the temperature of the Softmax during sampling. Decreasing the temperature from 1 to some lower number (e.g. 0.5) makes the RNN more confident, but also more conservative in its samples. Conversely, higher temperatures will give more diversity but at cost of more mistakes (e.g. spelling mistakes, etc). In particular, setting temperature very near zero will give the most likely thing that Paul Graham might say:

 “is that they were all the same thing that was a startup is that they were all the same thing that was a startup is that they were all the same thing that was a startup is that they were all the same”

 looks like we’ve reached an infinite loop about startups.

 Shakespeare

 It looks like we can learn to spell English words. But how about if there is more structure and style in the data? To examine this I downloaded all the works of Shakespeare and concatenated them into a single (4.4MB) file. We can now afford to train a larger network, in this case lets try a 3-layer RNN with 512 hidden nodes on each layer. After we train the network for a few hours we obtain samples such as:

 PANDARUS:
 Alas, I think he shall be come approached and the day
 When little srain would be attain'd into being never fed,
 And who is but a chain and subjects of his death,
 I should not sleep.
 
 Second Senator:
 They are away this miseries, produced upon my soul,
 Breaking and strongly should be buried, when I perish
 The earth and thoughts of many states.
 
 DUKE VINCENTIO:
 Well, your wit is in the care of side and that.
 
 Second Lord:
 They would be ruled after this chamber, and
 my fair nues begun out of the fact, to be conveyed,
 Whose noble souls I'll have the heart of the wars.
 
 Clown:
 Come, sir, I will make did behold your worship.
 
 VIOLA:
 I'll drink it.
 

 Remember, all the RNN knows are characters, so in particular it samples both speaker’s names and the contents. Sometimes we also get relatively extented monologue passages, such as:

 VIOLA:
 Why, Salisbury must find his flesh and thought
 That which I am not aps, not a man and in fire,
 To show the reining of the raven and the wars
 To grace my hand reproach within, and not a fair are hand,
 That Caesar and my goodly father's world;
 When I was heaven of presence and our fleets,
 We spare with hours, but cut thy council I am great,
 Murdered and by thy master's ready there
 My power to give thee but so much as hell:
 Some service in the noble bondman here,
 Would show him to her wine.
 
 KING LEAR:
 O, if you were a feeble sight, the courtesy of your law,
 Your sight and several breath, will wear the gods
 With his heads, and my hands are wonder'd at the deeds,
 So drop upon your lordship's head, and your opinion
 Shall be against your honour.
 

 I can barely recognize these samples from actual Shakespeare :) If you like Shakespeare, you might appreciate this 100,000 character sample. Of course, you can also generate an infinite amount of your own samples at different temperatures with the provided code.

 Wikipedia

 We saw that the LSTM can learn to spell words and copy general syntactic structures. Lets further increase the difficulty and train on structured markdown. In particular, lets take the Hutter Prize 100MB dataset of raw Wikipedia and train an LSTM. Following Graves et al., I used the first 96MB for training, the rest for validation and ran a few models overnight. We can now sample Wikipedia articles! Below are a few fun excerpts. First, some basic markdown output:

 Naturalism and decision for the majority of Arab countries' capitalide was grounded
 by the Irish language by [[John Clair]], [[An Imperial Japanese Revolt]], associated 
 with Guangzham's sovereignty. His generals were the powerful ruler of the Portugal 
 in the [[Protestant Immineners]], which could be said to be directly in Cantonese 
 Communication, which followed a ceremony and set inspired prison, training. The 
 emperor travelled back to [[Antioch, Perth, October 25|21]] to note, the Kingdom 
 of Costa Rica, unsuccessful fashioned the [[Thrales]], [[Cynth's Dajoard]], known 
 in western [[Scotland]], near Italy to the conquest of India with the conflict. 
 Copyright was the succession of independence in the slop of Syrian influence that 
 was a famous German movement based on a more popular servicious, non-doctrinal 
 and sexual power post. Many governments recognize the military housing of the 
 [[Civil Liberalization and Infantry Resolution 265 National Party in Hungary]], 
 that is sympathetic to be to the [[Punjab Resolution]]
 (PJS)[http://www.humah.yahoo.com/guardian.
 cfm/7754800786d17551963s89.htm Official economics Adjoint for the Nazism, Montgomery 
 was swear to advance to the resources for those Socialism's rule, 
 was starting to signing a major tripad of aid exile.]]
 

 In case you were wondering, the yahoo url above doesn’t actually exist, the model just hallucinated it. Also, note that the model learns to open and close the parenthesis correctly. There’s also quite a lot of structured markdown that the model learns, for example sometimes it creates headings, lists, etc.:

 { { cite journal | id=Cerling Nonforest Department|format=Newlymeslated|none } }
 ''www.e-complete''.
 
 '''See also''': [[List of ethical consent processing]]
 
 == See also ==
 *[[Iender dome of the ED]]
 *[[Anti-autism]]
 
 ===[[Religion|Religion]]===
 *[[French Writings]]
 *[[Maria]]
 *[[Revelation]]
 *[[Mount Agamul]]
 
 == External links==
 * [http://www.biblegateway.nih.gov/entrepre/ Website of the World Festival. The labour of India-county defeats at the Ripper of California Road.]
 
 ==External links==
 * [http://www.romanology.com/ Constitution of the Netherlands and Hispanic Competition for Bilabial and Commonwealth Industry (Republican Constitution of the Extent of the Netherlands)]
 
 

 Sometimes the model snaps into a mode of generating random but valid XML:

 <page>
 <title>Antichrist</title>
 <id>865</id>
 <revision>
 <id>15900676</id>
 <timestamp>2002-08-03T18:14:12Z</timestamp>
 <contributor>
 <username>Paris</username>
 <id>23</id>
 </contributor>
 <minor />
 <comment>Automated conversion</comment>
 <text xml:space="preserve">#REDIRECT [[Christianity]]</text>
 </revision>
 </page>
 

 The model completely makes up the timestamp, id, and so on. Also, note that it closes the correct tags appropriately and in the correct nested order. Here are 100,000 characters of sampled wikipedia if you’re interested to see more.

 Algebraic Geometry (Latex)

 The results above suggest that the model is actually quite good at learning complex syntactic structures. Impressed by these results, my labmate (Justin Johnson) and I decided to push even further into structured territories and got a hold of this book on algebraic stacks/geometry. We downloaded the raw Latex source file (a 16MB file) and trained a multilayer LSTM. Amazingly, the resulting sampled Latex almost compiles. We had to step in and fix a few issues manually but then you get plausible looking math, it’s quite astonishing:

 Sampled (fake) algebraic geometry. Here's the actual pdf.

 Here’s another sample:

 More hallucinated algebraic geometry. Nice try on the diagram (right).

 As you can see above, sometimes the model tries to generate latex diagrams, but clearly it hasn’t really figured them out. I also like the part where it chooses to skip a proof (“Proof omitted.”, top left). Of course, keep in mind that latex has a relatively difficult structured syntactic format that I haven’t even fully mastered myself. For instance, here is a raw sample from the model (unedited):

 \begin{proof}
 We may assume that $\mathcal{I}$ is an abelian sheaf on $\mathcal{C}$.
 \item Given a morphism $\Delta : \mathcal{F} \to \mathcal{I}$
 is an injective and let $\mathfrak q$ be an abelian sheaf on $X$.
 Let $\mathcal{F}$ be a fibered complex. Let $\mathcal{F}$ be a category.
 \begin{enumerate}
 \item \hyperref[setain-construction-phantom]{Lemma}
 \label{lemma-characterize-quasi-finite}
 Let $\mathcal{F}$ be an abelian quasi-coherent sheaf on $\mathcal{C}$.
 Let $\mathcal{F}$ be a coherent $\mathcal{O}_X$-module. Then
 $\mathcal{F}$ is an abelian catenary over $\mathcal{C}$.
 \item The following are equivalent
 \begin{enumerate}
 \item $\mathcal{F}$ is an $\mathcal{O}_X$-module.
 \end{lemma}
 

 This sample from a relatively decent model illustrates a few common mistakes. For example, the model opens a \begin{proof} environment but then ends it with a \end{lemma}. This is an example of a problem we’d have to fix manually, and is likely due to the fact that the dependency is too long-term: By the time the model is done with the proof it has forgotten whether it was doing a proof or a lemma. Similarly, it opens an \begin{enumerate} but then forgets to close it. We observed that these became less common with larger/better models, but nonetheless, these are the kinds of mistakes that come up.

 Linux Source Code

 I wanted to push structured data to its limit, so for the final challenge I decided to use code. In particular, I took all the source and header files found in the Linux repo on Github, concatenated all of them in a single giant file (474MB of C code) (I was originally going to train only on the kernel but that by itself is only ~16MB). Then I trained several as-large-as-fits-on-my-GPU 3-layer LSTMs over a period of a few days. These models have about 10 million parameters, which is still on the lower end for RNN models. The results are superfun:

 /*
 * Increment the size file of the new incorrect UI_FILTER group information
 * of the size generatively.
 */
 static int indicate_policy(void)
 {
 int error;
 if (fd == MARN_EPT) {
 /*
 * The kernel blank will coeld it to userspace.
 */
 if (ss->segment < mem_total)
 unblock_graph_and_set_blocked();
 else
 ret = 1;
 goto bail;
 }
 segaddr = in_SB(in.addr);
 selector = seg / 16;
 setup_works = true;
 for (i = 0; i < blocks; i++) {
 seq = buf[i++];
 bpf = bd->bd.next + i * search;
 if (fd) {
 current = blocked;
 }
 }
 rw->name = "Getjbbregs";
 bprm_self_clearl(&iv->version);
 regs->new = blocks[(BPF_STATS << info->historidac)] | PFMR_CLOBATHINC_SECONDS << 12;
 return segtable;
 }
 

 The code looks really quite great overall. Of course, I don’t think it compiles but when you scroll through the generate code it feels very much like a giant C code base. Notice that the RNN peppers its code with comments here and there at random. It is also very good at making very few syntactic errors. For example, it uses strings properly, pointer notation, etc. It also opens and closes brackets {[ correctly and learns to indent its code very well. A common error is that it can’t keep track of variable names: It often uses undefined variables (e.g. rw above), declares variables it never uses (e.g. int error), or returns non-existing variables. Lets see a few more examples. Here’s another snippet that shows a wider array of operations that the RNN learns:

 /*
 * If this error is set, we will need anything right after that BSD.
 */
 static void action_new_function(struct s_stat_info *wb)
 {
 unsigned long flags;
 int lel_idx_bit = e->edd, *sys & ~((unsigned long) *FIRST_COMPAT);
 buf[0] = 0xFFFFFFFF & (bit << 4);
 min(inc, slist->bytes);
 printk(KERN_WARNING "Memory allocated %02x/%02x, "
 "original MLL instead\n"),
 min(min(multi_run - s->len, max) * num_data_in),
 frame_pos, sz + first_seg);
 div_u64_w(val, inb_p);
 spin_unlock(&disk->queue_lock);
 mutex_unlock(&s->sock->mutex);
 mutex_unlock(&func->mutex);
 return disassemble(info->pending_bh);
 }
 
 static void num_serial_settings(struct tty_struct *tty)
 {
 if (tty == tty)
 disable_single_st_p(dev);
 pci_disable_spool(port);
 return 0;
 }
 
 static void do_command(struct seq_file *m, void *v)
 {
 int column = 32 << (cmd[2] & 0x80);
 if (state)
 cmd = (int)(int_state ^ (in_8(&ch->ch_flags) & Cmd) ? 2 : 1);
 else
 seq = 1;
 for (i = 0; i < 16; i++) {
 if (k & (1 << 1))
 pipe = (in_use & UMXTHREAD_UNCCA) +
 ((count & 0x00000000fffffff8) & 0x000000f) << 8;
 if (count == 0)
 sub(pid, ppc_md.kexec_handle, 0x20000000);
 pipe_set_bytes(i, 0);
 }
 /* Free our user pages pointer to place camera if all dash */
 subsystem_info = &of_changes[PAGE_SIZE];
 rek_controls(offset, idx, &soffset);
 /* Now we want to deliberately put it to device */
 control_check_polarity(&context, val, 0);
 for (i = 0; i < COUNTER; i++)
 seq_puts(s, "policy ");
 }
 

 Notice that in the second function the model compares tty == tty, which is vacuously true. On the other hand, at least the variable tty exists in the scope this time! In the last function, notice that the code does not return anything, which happens to be correct since the function signature is void. However, the first two functions were also declared void and did return values. This is again a form of a common mistake due to long-term interactions.

 Sometimes the model decides that it’s time to sample a new file. This is usually a very amusing part: The model first recites the GNU license character by character, samples a few includes, generates some macros and then dives into the code:

 /*
 * Copyright (c) 2006-2010, Intel Mobile Communications. All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify it
 * under the terms of the GNU General Public License version 2 as published by
 * the Free Software Foundation.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 *
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software Foundation,
 * Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 */
 
 #include <linux/kexec.h>
 #include <linux/errno.h>
 #include <linux/io.h>
 #include <linux/platform_device.h>
 #include <linux/multi.h>
 #include <linux/ckevent.h>
 
 #include <asm/io.h>
 #include <asm/prom.h>
 #include <asm/e820.h>
 #include <asm/system_info.h>
 #include <asm/setew.h>
 #include <asm/pgproto.h>
 
 #define REG_PG vesa_slot_addr_pack
 #define PFM_NOCOMP AFSR(0, load)
 #define STACK_DDR(type) (func)
 
 #define SWAP_ALLOCATE(nr) (e)
 #define emulate_sigs() arch_get_unaligned_child()
 #define access_rw(TST) asm volatile("movd %%esp, %0, %3" : : "r" (0)); \
 if (__type & DO_READ)
 
 static void stat_PC_SEC __read_mostly offsetof(struct seq_argsqueue, \
 pC>[1]);
 
 static void
 os_prefix(unsigned long sys)
 {
 #ifdef CONFIG_PREEMPT
 PUT_PARAM_RAID(2, sel) = get_state_state();
 set_pid_sum((unsigned long)state, current_state_str(),
 (unsigned long)-1->lr_full; low;
 }
 

 There are too many fun parts to cover- I could probably write an entire blog post on just this part. I’ll cut it short for now, but here is 1MB of sampled Linux code for your viewing pleasure.

 Generating Baby Names

 Lets try one more for fun. Lets feed the RNN a large text file that contains 8000 baby names listed out, one per line (names obtained from here). We can feed this to the RNN and then generate new names! Here are some example names, only showing the ones that do not occur in the training data (90% don’t):

 Rudi Levette Berice Lussa Hany Mareanne Chrestina Carissy Marylen Hammine Janye Marlise Jacacrie Hendred Romand Charienna Nenotto Ette Dorane Wallen Marly Darine Salina Elvyn Ersia Maralena Minoria Ellia Charmin Antley Nerille Chelon Walmor Evena Jeryly Stachon Charisa Allisa Anatha Cathanie Geetra Alexie Jerin Cassen Herbett Cossie Velen Daurenge Robester Shermond Terisa Licia Roselen Ferine Jayn Lusine Charyanne Sales Sanny Resa Wallon Martine Merus Jelen Candica Wallin Tel Rachene Tarine Ozila Ketia Shanne Arnande Karella Roselina Alessia Chasty Deland Berther Geamar Jackein Mellisand Sagdy Nenc Lessie Rasemy Guen Gavi Milea Anneda Margoris Janin Rodelin Zeanna Elyne Janah Ferzina Susta Pey Castina

 You can see many more here. Some of my favorites include “Baby” (haha), “Killie”, “Char”, “R”, “More”, “Mars”, “Hi”, “Saddie”, “With” and “Ahbort”. Well that was fun.﻿ Of course, you can imagine this being quite useful inspiration when writing a novel, or naming a new startup :)

 Understanding what’s going on

 We saw that the results at the end of training can be impressive, but how does any of this work? Lets run two quick experiments to briefly peek under the hood.

 The evolution of samples while training

 First, it’s fun to look at how the sampled text evolves while the model trains. For example, I trained an LSTM of Leo Tolstoy’s War and Peace and then generated samples every 100 iterations of training. At iteration 100 the model samples random jumbles:

 tyntd-iafhatawiaoihrdemot lytdws e ,tfti, astai f ogoh eoase rrranbyne 'nhthnee e 
 plia tklrgd t o idoe ns,smtt h ne etie h,hregtrs nigtike,aoaenns lng
 

 However, notice that at least it is starting to get an idea about words separated by spaces. Except sometimes it inserts two spaces. It also doesn’t know that comma is amost always followed by a space. At 300 iterations we see that the model starts to get an idea about quotes and periods:

 "Tmont thithey" fomesscerliund
 Keushey. Thom here
 sheulke, anmerenith ol sivh I lalterthend Bleipile shuwy fil on aseterlome
 coaniogennc Phe lism thond hon at. MeiDimorotion in ther thize."
 

 The words are now also separated with spaces and the model starts to get the idea about periods at the end of a sentence. At iteration 500:

 we counter. He stutn co des. His stanted out one ofler that concossions and was 
 to gearang reay Jotrets and with fre colt otf paitt thin wall. Which das stimn 
 

 the model has now learned to spell the shortest and most common words such as “we”, “He”, “His”, “Which”, “and”, etc. At iteration 700 we’re starting to see more and more English-like text emerge:

 Aftair fall unsuch that the hall for Prince Velzonski's that me of
 her hearly, and behs to so arwage fiving were to it beloge, pavu say falling misfort 
 how, and Gogition is so overelical and ofter.
 

 At iteration 1200 we’re now seeing use of quotations and question/exclamation marks. Longer words have now been learned as well:

 "Kite vouch!" he repeated by her
 door. "But I would be done and quarts, feeling, then, son is people...."
 

 Until at last we start to get properly spelled words, quotations, names, and so on by about iteration 2000:

 "Why do what that day," replied Natasha, and wishing to himself the fact the
 princess, Princess Mary was easier, fed in had oftened him.
 Pierre aking his soul came to the packs and drove up his father-in-law women.
 

 The picture that emerges is that the model first discovers the general word-space structure and then rapidly starts to learn the words; First starting with the short words and then eventually the longer ones. Topics and themes that span multiple words (and in general longer-term dependencies) start to emerge only much later.

 Visualizing the predictions and the “neuron” firings in the RNN

 Another fun visualization is to look at the predicted distributions over characters. In the visualizations below we feed a Wikipedia RNN model character data from the validation set (shown along the blue/green rows) and under every character we visualize (in red) the top 5 guesses that the model assigns for the next character. The guesses are colored by their probability (so dark red = judged as very likely, white = not very likely). For example, notice that there are stretches of characters where the model is extremely confident about the next letter (e.g., the model is very confident about characters during the http://www. sequence).

 The input character sequence (blue/green) is colored based on the firing of a randomly chosen neuron in the hidden representation of the RNN. Think about it as green = very excited and blue = not very excited (for those familiar with details of LSTMs, these are values between [-1,1] in the hidden state vector, which is just the gated and tanh’d LSTM cell state). Intuitively, this is visualizing the firing rate of some neuron in the “brain” of the RNN while it reads the input sequence. Different neurons might be looking for different patterns; Below we’ll look at 4 different ones that I found and thought were interesting or interpretable (many also aren’t):

 The neuron highlighted in this image seems to get very excited about URLs and turns off outside of the URLs. The LSTM is likely using this neuron to remember if it is inside a URL or not.
 The highlighted neuron here gets very excited when the RNN is inside the [[ ]] markdown environment and turns off outside of it. Interestingly, the neuron can't turn on right after it sees the character "[", it must wait for the second "[" and then activate. This task of counting whether the model has seen one or two "[" is likely done with a different neuron.
 Here we see a neuron that varies seemingly linearly across the [[ ]] environment. In other words its activation is giving the RNN a time-aligned coordinate system across the [[ ]] scope. The RNN can use this information to make different characters more or less likely depending on how early/late it is in the [[ ]] scope (perhaps?).
 Here is another neuron that has very local behavior: it is relatively silent but sharply turns off right after the first "w" in the "www" sequence. The RNN might be using this neuron to count up how far in the "www" sequence it is, so that it can know whether it should emit another "w", or if it should start the URL.

 Of course, a lot of these conclusions are slightly hand-wavy as the hidden state of the RNN is a huge, high-dimensional and largely distributed representation. These visualizations were produced with custom HTML/CSS/Javascript, you can see a sketch of what’s involved here if you’d like to create something similar.

 We can also condense this visualization by excluding the most likely predictions and only visualize the text, colored by activations of a cell. We can see that in addition to a large portion of cells that do not do anything interpretible, about 5% of them turn out to have learned quite interesting and interpretible algorithms:

 Again, what is beautiful about this is that we didn’t have to hardcode at any point that if you’re trying to predict the next character it might, for example, be useful to keep track of whether or not you are currently inside or outside of quote. We just trained the LSTM on raw data and it decided that this is a useful quantitity to keep track of. In other words one of its cells gradually tuned itself during training to become a quote detection cell, since this helps it better perform the final task. This is one of the cleanest and most compelling examples of where the power in Deep Learning models (and more generally end-to-end training) is coming from.

 Source Code

 I hope I’ve convinced you that training character-level language models is a very fun exercise. You can train your own models using the char-rnn code I released on Github (under MIT license). It takes one large text file and trains a character-level model that you can then sample from. Also, it helps if you have a GPU or otherwise training on CPU will be about a factor of 10x slower. In any case, if you end up training on some data and getting fun results let me know! And if you get lost in the Torch/Lua codebase remember that all it is is just a more fancy version of this 100-line gist.

 Brief digression. The code is written in Torch 7, which has recently become my favorite deep learning framework. I’ve only started working with Torch/LUA over the last few months and it hasn’t been easy (I spent a good amount of time digging through the raw Torch code on Github and asking questions on their gitter to get things done), but once you get a hang of things it offers a lot of flexibility and speed. I’ve also worked with Caffe and Theano in the past and I believe Torch, while not perfect, gets its levels of abstraction and philosophy right better than others. In my view the desirable features of an effective framework are:

 1. CPU/GPU transparent Tensor library with a lot of functionality (slicing, array/matrix operations, etc. )
 2. An entirely separate code base in a scripting language (ideally Python) that operates over Tensors and implements all Deep Learning stuff (forward/backward, computation graphs, etc)
 3. It should be possible to easily share pretrained models (Caffe does this well, others don’t), and crucially
 4. NO compilation step (or at least not as currently done in Theano). The trend in Deep Learning is towards larger, more complex networks that are are time-unrolled in complex graphs. It is critical that these do not compile for a long time or development time greatly suffers. Second, by compiling one gives up interpretability and the ability to log/debug effectively. If there is an option to compile the graph once it has been developed for efficiency in prod that’s fine.

 Further Reading

 Before the end of the post I also wanted to position RNNs in a wider context and provide a sketch of the current research directions. RNNs have recently generated a significant amount of buzz and excitement in the field of Deep Learning. Similar to Convolutional Networks they have been around for decades but their full potential has only recently started to get widely recognized, in large part due to our growing computational resources. Here’s a brief sketch of a few recent developments (definitely not complete list, and a lot of this work draws from research back to 1990s, see related work sections):

 In the domain of NLP/Speech, RNNs transcribe speech to text, perform machine translation, generate handwritten text, and of course, they have been used as powerful language models (Sutskever et al.) (Graves) (Mikolov et al.) (both on the level of characters and words). Currently it seems that word-level models work better than character-level models, but this is surely a temporary thing.

 Computer Vision. RNNs are also quickly becoming pervasive in Computer Vision. For example, we’re seeing RNNs in frame-level video classification, image captioning (also including my own work and many others), video captioning and very recently visual question answering. My personal favorite RNNs in Computer Vision paper is Recurrent Models of Visual Attention, both due to its high-level direction (sequential processing of images with glances) and the low-level modeling (REINFORCE learning rule that is a special case of policy gradient methods in Reinforcement Learning, which allows one to train models that perform non-differentiable computation (taking glances around the image in this case)). I’m confident that this type of hybrid model that consists of a blend of CNN for raw perception coupled with an RNN glance policy on top will become pervasive in perception, especially for more complex tasks that go beyond classifying some objects in plain view.

 Inductive Reasoning, Memories and Attention. Another extremely exciting direction of research is oriented towards addressing the limitations of vanilla recurrent networks. One problem is that RNNs are not inductive: They memorize sequences extremely well, but they don’t necessarily always show convincing signs of generalizing in the correct way (I’ll provide pointers in a bit that make this more concrete). A second issue is they unnecessarily couple their representation size to the amount of computation per step. For instance, if you double the size of the hidden state vector you’d quadruple the amount of FLOPS at each step due to the matrix multiplication. Ideally, we’d like to maintain a huge representation/memory (e.g. containing all of Wikipedia or many intermediate state variables), while maintaining the ability to keep computation per time step fixed.

 The first convincing example of moving towards these directions was developed in DeepMind’s Neural Turing Machines paper. This paper sketched a path towards models that can perform read/write operations between large, external memory arrays and a smaller set of memory registers (think of these as our working memory) where the computation happens. Crucially, the NTM paper also featured very interesting memory addressing mechanisms that were implemented with a (soft, and fully-differentiable) attention model. The concept of soft attention has turned out to be a powerful modeling feature and was also featured in Neural Machine Translation by Jointly Learning to Align and Translate for Machine Translation and Memory Networks for (toy) Question Answering. In fact, I’d go as far as to say that

 The concept of attention is the most interesting recent architectural innovation in neural networks.

 Now, I don’t want to dive into too many details but a soft attention scheme for memory addressing is convenient because it keeps the model fully-differentiable, but unfortunately one sacrifices efficiency because everything that can be attended to is attended to (but softly). Think of this as declaring a pointer in C that doesn’t point to a specific address but instead defines an entire distribution over all addresses in the entire memory, and dereferencing the pointer returns a weighted sum of the pointed content (that would be an expensive operation!). This has motivated multiple authors to swap soft attention models for hard attention where one samples a particular chunk of memory to attend to (e.g. a read/write action for some memory cell instead of reading/writing from all cells to some degree). This model is significantly more philosophically appealing, scalable and efficient, but unfortunately it is also non-differentiable. This then calls for use of techniques from the Reinforcement Learning literature (e.g. REINFORCE) where people are perfectly used to the concept of non-differentiable interactions. This is very much ongoing work but these hard attention models have been explored, for example, in Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets, Reinforcement Learning Neural Turing Machines, and Show Attend and Tell.

 People. If you’d like to read up on RNNs I recommend theses from Alex Graves, Ilya Sutskever and Tomas Mikolov. For more about REINFORCE and more generally Reinforcement Learning and policy gradient methods (which REINFORCE is a special case of) David Silver’s class, or one of Pieter Abbeel’s classes.

 Code. If you’d like to play with training RNNs I hear good things about keras or passage for Theano, the code released with this post for Torch, or this gist for raw numpy code I wrote a while ago that implements an efficient, batched LSTM forward and backward pass. You can also have a look at my numpy-based NeuralTalk which uses an RNN/LSTM to caption images, or maybe this Caffe implementation by Jeff Donahue.

 Conclusion

 We’ve learned about RNNs, how they work, why they have become a big deal, we’ve trained an RNN character-level language model on several fun datasets, and we’ve seen where RNNs are going. You can confidently expect a large amount of innovation in the space of RNNs, and I believe they will become a pervasive and critical component to intelligent systems.

 Lastly, to add some meta to this post, I trained an RNN on the source file of this blog post. Unfortunately, at about 46K characters I haven’t written enough data to properly feed the RNN, but the returned sample (generated with low temperature to get a more typical sample) is:

 I've the RNN with and works, but the computed with program of the 
 RNN with and the computed of the RNN with with and the code
 

 Yes, the post was about RNN and how well it works, so clearly this works :). See you next time!

 EDIT (extra links):

 Videos:

 * I gave a talk on this work at the London Deep Learning meetup (video).

 Discussions:

 * HN discussion
 * Reddit discussion on r/machinelearning
 * Reddit discussion on r/programming

 Replies:

 * Yoav Goldberg compared these RNN results to n-gram maximum likelihood (counting) baseline
 * @nylk trained char-rnn on cooking recipes. They look great!
 * @MrChrisJohnson trained char-rnn on Eminem lyrics and then synthesized a rap song with robotic voice reading it out. Hilarious :)
 * @samim trained char-rnn on Obama Speeches. They look fun!
 * João Felipe trained char-rnn irish folk music and sampled music
 * Bob Sturm also trained char-rnn on music in ABC notation
 * RNN Bible bot by Maximilien
 * Learning Holiness learning the Bible
 * Terminal.com snapshot that has char-rnn set up and ready to go in a browser-based virtual machine (thanks @samim)
 Please enable JavaScript to view the comments powered by Disqus. comments powered by Disqus
 * Andrej Karpathy blog
 * karpathy
 * karpathy

 Musings of a Computer Scientist.

There’s something magical about Recurrent Neural Networks (RNNs). I still remember when I trained my first recurrent network for Image Captioning. Within a few dozen minutes of training my first baby model (with rather arbitrarily-chosen hyperparameters) started to generate very nice looking descriptions of images that were on the edge of making sense. Sometimes the ratio of how simple your model is to the quality of the results you get out of it blows past your expectations, and this was one of those times. What made this result so shocking at the time was that the common wisdom was that RNNs were supposed to be difficult to train (with more experience I’ve in fact reached the opposite conclusion). Fast forward about a year: I’m training RNNs all the time and I’ve witnessed their power and robustness many times, and yet their magical outputs still find ways of amusing me. This post is about sharing some of that magic with you.
 We’ll train RNNs to generate text character by character and ponder the question “how is that even possible?”
By the way, together with this post I am also releasing code on Github that allows you to train character-level language models based on multi-layer LSTMs. You give it a large chunk of text and it will learn to generate text like it one character at a time. You can also use it to reproduce my experiments below. But we’re getting ahead of ourselves; What are RNNs anyway?
 Sequences. Depending on your background you might be wondering: What makes Recurrent Networks so special? A glaring limitation of Vanilla Neural Networks (and also Convolutional Networks) is that their API is too constrained: they accept a fixed-sized vector as input (e.g. an image) and produce a fixed-sized vector as output (e.g. probabilities of different classes). Not only that: These models perform this mapping using a fixed amount of computational steps (e.g. the number of layers in the model). The core reason that recurrent nets are more exciting is that they allow us to operate over sequences of vectors: Sequences in the input, the output, or in the most general case both. A few examples may make this more concrete:
  Each rectangle is a vector and arrows represent functions (e.g. matrix multiply). Input vectors are in red, output vectors are in blue and green vectors hold the RNN's state (more on this soon). From left to right: (1) Vanilla mode of processing without RNN, from fixed-sized input to fixed-sized output (e.g. image classification). (2) Sequence output (e.g. image captioning takes an image and outputs a sentence of words). (3) Sequence input (e.g. sentiment analysis where a given sentence is classified as expressing positive or negative sentiment). (4) Sequence input and sequence output (e.g. Machine Translation: an RNN reads a sentence in English and then outputs a sentence in French). (5) Synced sequence input and output (e.g. video classification where we wish to label each frame of the video). Notice that in every case are no pre-specified constraints on the lengths sequences because the recurrent transformation (green) is fixed and can be applied as many times as we like.
As you might expect, the sequence regime of operation is much more powerful compared to fixed networks that are doomed from the get-go by a fixed number of computational steps, and hence also much more appealing for those of us who aspire to build more intelligent systems. Moreover, as we’ll see in a bit, RNNs combine the input vector with their state vector with a fixed (but learned) function to produce a new state vector. This can in programming terms be interpreted as running a fixed program with certain inputs and some internal variables. Viewed this way, RNNs essentially describe programs. In fact, it is known that RNNs are Turing-Complete in the sense that they can to simulate arbitrary programs (with proper weights). But similar to universal approximation theorems for neural nets you shouldn’t read too much into this. In fact, forget I said anything.
 If training vanilla neural nets is optimization over functions, training recurrent nets is optimization over programs.
 Sequential processing in absence of sequences. You might be thinking that having sequences as inputs or outputs could be relatively rare, but an important point to realize is that even if your inputs/outputs are fixed vectors, it is still possible to use this powerful formalism to process them in a sequential manner. For instance, the figure below shows results from two very nice papers from DeepMind. On the left, an algorithm learns a recurrent network policy that steers its attention around an image; In particular, it learns to read out house numbers from left to right (Ba et al.). On the right, a recurrent network generates images of digits by learning to sequentially add color to a canvas (Gregor et al.):
Left: RNN learns to read house numbers. Right: RNN learns to paint house numbers.
The takeaway is that even if your data is not in form of sequences, you can still formulate and train powerful models that learn to process it sequentially. You’re learning stateful programs that process your fixed-sized data.
 RNN computation. So how do these things work? At the core, RNNs have a deceptively simple API: They accept an input vector x and give you an output vector y. However, crucially this output vector’s contents are influenced not only by the input you just fed in, but also on the entire history of inputs you’ve fed in in the past. Written as a class, the RNN’s API consists of a single step function:
  rnn = RNN()
y = rnn.step(x) # x is an input vector, y is the RNN's output vector
  
The RNN class has some internal state that it gets to update every time step is called. In the simplest case this state consists of a single hidden vector h. Here is an implementation of the step function in a Vanilla RNN:
  class RNN:
 # ...
 def step(self, x):
 # update the hidden state
 self.h = np.tanh(np.dot(self.W_hh, self.h) + np.dot(self.W_xh, x))
 # compute the output vector
 y = np.dot(self.W_hy, self.h)
 return y
  
The above specifies the forward pass of a vanilla RNN. This RNN’s parameters are the three matrices W_hh, W_xh, W_hy. The hidden state self.h is initialized with the zero vector. The np.tanh function implements a non-linearity that squashes the activations to the range [-1, 1]. Notice briefly how this works: There are two terms inside of the tanh: one is based on the previous hidden state and one is based on the current input. In numpy np.dot is matrix multiplication. The two intermediates interact with addition, and then get squashed by the tanh into the new state vector. If you’re more comfortable with math notation, we can also write the hidden state update as \( h_t = \tanh ( W_{hh} h_{t-1} + W_{xh} x_t ) \), where tanh is applied elementwise.
We initialize the matrices of the RNN with random numbers and the bulk of work during training goes into finding the matrices that give rise to desirable behavior, as measured with some loss function that expresses your preference to what kinds of outputs y you’d like to see in response to your input sequences x.
 Going deep. RNNs are neural networks and everything works monotonically better (if done right) if you put on your deep learning hat and start stacking models up like pancakes. For instance, we can form a 2-layer recurrent network as follows:
  y1 = rnn1.step(x)
y = rnn2.step(y1)
  
In other words we have two separate RNNs: One RNN is receiving the input vectors and the second RNN is receiving the output of the first RNN as its input. Except neither of these RNNs know or care - it’s all just vectors coming in and going out, and some gradients flowing through each module during backpropagation.
 Getting fancy. I’d like to briefly mention that in practice most of us use a slightly different formulation than what I presented above called a Long Short-Term Memory (LSTM) network. The LSTM is a particular type of recurrent network that works slightly better in practice, owing to its more powerful update equation and some appealing backpropagation dynamics. I won’t go into details, but everything I’ve said about RNNs stays exactly the same, except the mathematical form for computing the update (the line self.h = ... ) gets a little more complicated. From here on I will use the terms “RNN/LSTM” interchangeably but all experiments in this post use an LSTM.
Character-Level Language Models
Okay, so we have an idea about what RNNs are, why they are super exciting, and how they work. We’ll now ground this in a fun application: We’ll train RNN character-level language models. That is, we’ll give the RNN a huge chunk of text and ask it to model the probability distribution of the next character in the sequence given a sequence of previous characters. This will then allow us to generate new text one character at a time.
As a working example, suppose we only had a vocabulary of four possible letters “helo”, and wanted to train an RNN on the training sequence “hello”. This training sequence is in fact a source of 4 separate training examples: 1. The probability of “e” should be likely given the context of “h”, 2. “l” should be likely in the context of “he”, 3. “l” should also be likely given the context of “hel”, and finally 4. “o” should be likely given the context of “hell”.
Concretely, we will encode each character into a vector using 1-of-k encoding (i.e. all zero except for a single one at the index of the character in the vocabulary), and feed them into the RNN one at a time with the step function. We will then observe a sequence of 4-dimensional output vectors (one dimension per character), which we interpret as the confidence the RNN currently assigns to each character coming next in the sequence. Here’s a diagram:
  An example RNN with 4-dimensional input and output layers, and a hidden layer of 3 units (neurons). This diagram shows the activations in the forward pass when the RNN is fed the characters "hell" as input. The output layer contains confidences the RNN assigns for the next character (vocabulary is "h,e,l,o"); We want the green numbers to be high and red numbers to be low.
For example, we see that in the first time step when the RNN saw the character “h” it assigned confidence of 1.0 to the next letter being “h”, 2.2 to letter “e”, -3.0 to “l”, and 4.1 to “o”. Since in our training data (the string “hello”) the next correct character is “e”, we would like to increase its confidence (green) and decrease the confidence of all other letters (red). Similarly, we have a desired target character at every one of the 4 time steps that we’d like the network to assign a greater confidence to. Since the RNN consists entirely of differentiable operations we can run the backpropagation algorithm (this is just a recursive application of the chain rule from calculus) to figure out in what direction we should adjust every one of its weights to increase the scores of the correct targets (green bold numbers). We can then perform a parameter update, which nudges every weight a tiny amount in this gradient direction. If we were to feed the same inputs to the RNN after the parameter update we would find that the scores of the correct characters (e.g. “e” in the first time step) would be slightly higher (e.g. 2.3 instead of 2.2), and the scores of incorrect characters would be slightly lower. We then repeat this process over and over many times until the network converges and its predictions are eventually consistent with the training data in that correct characters are always predicted next.
A more technical explanation is that we use the standard Softmax classifier (also commonly referred to as the cross-entropy loss) on every output vector simultaneously. The RNN is trained with mini-batch Stochastic Gradient Descent and I like to use RMSProp or Adam (per-parameter adaptive learning rate methods) to stablilize the updates.
Notice also that the first time the character “l” is input, the target is “l”, but the second time the target is “o”. The RNN therefore cannot rely on the input alone and must use its recurrent connection to keep track of the context to achieve this task.
At test time, we feed a character into the RNN and get a distribution over what characters are likely to come next. We sample from this distribution, and feed it right back in to get the next letter. Repeat this process and you’re sampling text! Lets now train an RNN on different datasets and see what happens.
To further clarify, for educational purposes I also wrote a minimal character-level RNN language model in Python/numpy. It is only about 100 lines long and hopefully it gives a concise, concrete and useful summary of the above if you’re better at reading code than text. We’ll now dive into example results, produced with the much more efficient Lua/Torch codebase.
Fun with RNNs
All 5 example character models below were trained with the code I’m releasing on Github. The input in each case is a single file with some text, and we’re training an RNN to predict the next character in the sequence.
Paul Graham generator
Lets first try a small dataset of English as a sanity check. My favorite fun dataset is the concatenation of Paul Graham’s essays. The basic idea is that there’s a lot of wisdom in these essays, but unfortunately Paul Graham is a relatively slow generator. Wouldn’t it be great if we could sample startup wisdom on demand? That’s where an RNN comes in.
Concatenating all pg essays over the last ~5 years we get approximately 1MB text file, or about 1 million characters (this is considered a very small dataset by the way). Technical: Lets train a 2-layer LSTM with 512 hidden nodes (approx. 3.5 million parameters), and with dropout of 0.5 after each layer. We’ll train with batches of 100 examples and truncated backpropagation through time of length 100 characters. With these settings one batch on a TITAN Z GPU takes about 0.46 seconds (this can be cut in half with 50 character BPTT at negligible cost in performance). Without further ado, lets see a sample from the RNN:
 “The surprised in investors weren’t going to raise money. I’m not the company with the time there are all interesting quickly, don’t have to get off the same programmers. There’s a super-angel round fundraising, why do you can do. If you have a different physical investment are become in people who reduced in a startup with the way to argument the acquirer could see them just that you’re also the founders will part of users’ affords that and an alternation to the idea. [2] Don’t work at first member to see the way kids will seem in advance of a bad successful startup. And if you have to act the big company too.” 
Okay, clearly the above is unfortunately not going to replace Paul Graham anytime soon, but remember that the RNN had to learn English completely from scratch and with a small dataset (including where you put commas, apostrophes and spaces). I also like that it learns to support its own arguments (e.g. [2], above). Sometimes it says something that offers a glimmer of insight, such as “a company is a meeting to think to investors”. Here’s a link to 50K character sample if you’d like to see more.
 Temperature. We can also play with the temperature of the Softmax during sampling. Decreasing the temperature from 1 to some lower number (e.g. 0.5) makes the RNN more confident, but also more conservative in its samples. Conversely, higher temperatures will give more diversity but at cost of more mistakes (e.g. spelling mistakes, etc). In particular, setting temperature very near zero will give the most likely thing that Paul Graham might say:
 “is that they were all the same thing that was a startup is that they were all the same thing that was a startup is that they were all the same thing that was a startup is that they were all the same” 
looks like we’ve reached an infinite loop about startups.
Shakespeare
It looks like we can learn to spell English words. But how about if there is more structure and style in the data? To examine this I downloaded all the works of Shakespeare and concatenated them into a single (4.4MB) file. We can now afford to train a larger network, in this case lets try a 3-layer RNN with 512 hidden nodes on each layer. After we train the network for a few hours we obtain samples such as:
  PANDARUS:
Alas, I think he shall be come approached and the day
When little srain would be attain'd into being never fed,
And who is but a chain and subjects of his death,
I should not sleep.
Second Senator:
They are away this miseries, produced upon my soul,
Breaking and strongly should be buried, when I perish
The earth and thoughts of many states.
DUKE VINCENTIO:
Well, your wit is in the care of side and that.
Second Lord:
They would be ruled after this chamber, and
my fair nues begun out of the fact, to be conveyed,
Whose noble souls I'll have the heart of the wars.
Clown:
Come, sir, I will make did behold your worship.
VIOLA:
I'll drink it.
  
Remember, all the RNN knows are characters, so in particular it samples both speaker’s names and the contents. Sometimes we also get relatively extented monologue passages, such as:
  VIOLA:
Why, Salisbury must find his flesh and thought
That which I am not aps, not a man and in fire,
To show the reining of the raven and the wars
To grace my hand reproach within, and not a fair are hand,
That Caesar and my goodly father's world;
When I was heaven of presence and our fleets,
We spare with hours, but cut thy council I am great,
Murdered and by thy master's ready there
My power to give thee but so much as hell:
Some service in the noble bondman here,
Would show him to her wine.
KING LEAR:
O, if you were a feeble sight, the courtesy of your law,
Your sight and several breath, will wear the gods
With his heads, and my hands are wonder'd at the deeds,
So drop upon your lordship's head, and your opinion
Shall be against your honour.
  
I can barely recognize these samples from actual Shakespeare :) If you like Shakespeare, you might appreciate this 100,000 character sample. Of course, you can also generate an infinite amount of your own samples at different temperatures with the provided code.
Wikipedia
We saw that the LSTM can learn to spell words and copy general syntactic structures. Lets further increase the difficulty and train on structured markdown. In particular, lets take the Hutter Prize 100MB dataset of raw Wikipedia and train an LSTM. Following Graves et al., I used the first 96MB for training, the rest for validation and ran a few models overnight. We can now sample Wikipedia articles! Below are a few fun excerpts. First, some basic markdown output:
  Naturalism and decision for the majority of Arab countries' capitalide was grounded
by the Irish language by [[John Clair]], [[An Imperial Japanese Revolt]], associated 
with Guangzham's sovereignty. His generals were the powerful ruler of the Portugal 
in the [[Protestant Immineners]], which could be said to be directly in Cantonese 
Communication, which followed a ceremony and set inspired prison, training. The 
emperor travelled back to [[Antioch, Perth, October 25|21]] to note, the Kingdom 
of Costa Rica, unsuccessful fashioned the [[Thrales]], [[Cynth's Dajoard]], known 
in western [[Scotland]], near Italy to the conquest of India with the conflict. 
Copyright was the succession of independence in the slop of Syrian influence that 
was a famous German movement based on a more popular servicious, non-doctrinal 
and sexual power post. Many governments recognize the military housing of the 
[[Civil Liberalization and Infantry Resolution 265 National Party in Hungary]], 
that is sympathetic to be to the [[Punjab Resolution]]
(PJS)[http://www.humah.yahoo.com/guardian.
cfm/7754800786d17551963s89.htm Official economics Adjoint for the Nazism, Montgomery 
was swear to advance to the resources for those Socialism's rule, 
was starting to signing a major tripad of aid exile.]]
  
In case you were wondering, the yahoo url above doesn’t actually exist, the model just hallucinated it. Also, note that the model learns to open and close the parenthesis correctly. There’s also quite a lot of structured markdown that the model learns, for example sometimes it creates headings, lists, etc.:
  { { cite journal | id=Cerling Nonforest Department|format=Newlymeslated|none } }
''www.e-complete''.
'''See also''': [[List of ethical consent processing]]
== See also ==
*[[Iender dome of the ED]]
*[[Anti-autism]]
===[[Religion|Religion]]===
*[[French Writings]]
*[[Maria]]
*[[Revelation]]
*[[Mount Agamul]]
== External links==
* [http://www.biblegateway.nih.gov/entrepre/ Website of the World Festival. The labour of India-county defeats at the Ripper of California Road.]
==External links==
* [http://www.romanology.com/ Constitution of the Netherlands and Hispanic Competition for Bilabial and Commonwealth Industry (Republican Constitution of the Extent of the Netherlands)]
  
Sometimes the model snaps into a mode of generating random but valid XML:
  &lt;page&gt;
 &lt;title&gt;Antichrist&lt;/title&gt;
 &lt;id&gt;865&lt;/id&gt;
 &lt;revision&gt;
 &lt;id&gt;15900676&lt;/id&gt;
 &lt;timestamp&gt;2002-08-03T18:14:12Z&lt;/timestamp&gt;
 &lt;contributor&gt;
 &lt;username&gt;Paris&lt;/username&gt;
 &lt;id&gt;23&lt;/id&gt;
 &lt;/contributor&gt;
 &lt;minor /&gt;
 &lt;comment&gt;Automated conversion&lt;/comment&gt;
 &lt;text xml:space="preserve"&gt;#REDIRECT [[Christianity]]&lt;/text&gt;
 &lt;/revision&gt;
&lt;/page&gt;
  
The model completely makes up the timestamp, id, and so on. Also, note that it closes the correct tags appropriately and in the correct nested order. Here are 100,000 characters of sampled wikipedia if you’re interested to see more.
Algebraic Geometry (Latex)
The results above suggest that the model is actually quite good at learning complex syntactic structures. Impressed by these results, my labmate (Justin Johnson) and I decided to push even further into structured territories and got a hold of this book on algebraic stacks/geometry. We downloaded the raw Latex source file (a 16MB file) and trained a multilayer LSTM. Amazingly, the resulting sampled Latex almost compiles. We had to step in and fix a few issues manually but then you get plausible looking math, it’s quite astonishing:
 
Here’s another sample:
  More hallucinated algebraic geometry. Nice try on the diagram (right).
As you can see above, sometimes the model tries to generate latex diagrams, but clearly it hasn’t really figured them out. I also like the part where it chooses to skip a proof (“Proof omitted.”, top left). Of course, keep in mind that latex has a relatively difficult structured syntactic format that I haven’t even fully mastered myself. For instance, here is a raw sample from the model (unedited):
  \begin{proof}
We may assume that $\mathcal{I}$ is an abelian sheaf on $\mathcal{C}$.
\item Given a morphism $\Delta : \mathcal{F} \to \mathcal{I}$
is an injective and let $\mathfrak q$ be an abelian sheaf on $X$.
Let $\mathcal{F}$ be a fibered complex. Let $\mathcal{F}$ be a category.
\begin{enumerate}
\item \hyperref[setain-construction-phantom]{Lemma}
\label{lemma-characterize-quasi-finite}
Let $\mathcal{F}$ be an abelian quasi-coherent sheaf on $\mathcal{C}$.
Let $\mathcal{F}$ be a coherent $\mathcal{O}_X$-module. Then
$\mathcal{F}$ is an abelian catenary over $\mathcal{C}$.
\item The following are equivalent
\begin{enumerate}
\item $\mathcal{F}$ is an $\mathcal{O}_X$-module.
\end{lemma}
  
This sample from a relatively decent model illustrates a few common mistakes. For example, the model opens a \begin{proof} environment but then ends it with a \end{lemma}. This is an example of a problem we’d have to fix manually, and is likely due to the fact that the dependency is too long-term: By the time the model is done with the proof it has forgotten whether it was doing a proof or a lemma. Similarly, it opens an \begin{enumerate} but then forgets to close it. We observed that these became less common with larger/better models, but nonetheless, these are the kinds of mistakes that come up.
Linux Source Code
I wanted to push structured data to its limit, so for the final challenge I decided to use code. In particular, I took all the source and header files found in the Linux repo on Github, concatenated all of them in a single giant file (474MB of C code) (I was originally going to train only on the kernel but that by itself is only ~16MB). Then I trained several as-large-as-fits-on-my-GPU 3-layer LSTMs over a period of a few days. These models have about 10 million parameters, which is still on the lower end for RNN models. The results are superfun:
  /*
 * Increment the size file of the new incorrect UI_FILTER group information
 * of the size generatively.
 */
static int indicate_policy(void)
{
 int error;
 if (fd == MARN_EPT) {
 /*
 * The kernel blank will coeld it to userspace.
 */
 if (ss-&gt;segment &lt; mem_total)
 unblock_graph_and_set_blocked();
 else
 ret = 1;
 goto bail;
 }
 segaddr = in_SB(in.addr);
 selector = seg / 16;
 setup_works = true;
 for (i = 0; i &lt; blocks; i++) {
 seq = buf[i++];
 bpf = bd-&gt;bd.next + i * search;
 if (fd) {
 current = blocked;
 }
 }
 rw-&gt;name = "Getjbbregs";
 bprm_self_clearl(&amp;iv-&gt;version);
 regs-&gt;new = blocks[(BPF_STATS &lt;&lt; info-&gt;historidac)] | PFMR_CLOBATHINC_SECONDS &lt;&lt; 12;
 return segtable;
}
  
The code looks really quite great overall. Of course, I don’t think it compiles but when you scroll through the generate code it feels very much like a giant C code base. Notice that the RNN peppers its code with comments here and there at random. It is also very good at making very few syntactic errors. For example, it uses strings properly, pointer notation, etc. It also opens and closes brackets {[ correctly and learns to indent its code very well. A common error is that it can’t keep track of variable names: It often uses undefined variables (e.g. rw above), declares variables it never uses (e.g. int error), or returns non-existing variables. Lets see a few more examples. Here’s another snippet that shows a wider array of operations that the RNN learns:
  /*
 * If this error is set, we will need anything right after that BSD.
 */
static void action_new_function(struct s_stat_info *wb)
{
 unsigned long flags;
 int lel_idx_bit = e-&gt;edd, *sys &amp; ~((unsigned long) *FIRST_COMPAT);
 buf[0] = 0xFFFFFFFF &amp; (bit &lt;&lt; 4);
 min(inc, slist-&gt;bytes);
 printk(KERN_WARNING "Memory allocated %02x/%02x, "
 "original MLL instead\n"),
 min(min(multi_run - s-&gt;len, max) * num_data_in),
 frame_pos, sz + first_seg);
 div_u64_w(val, inb_p);
 spin_unlock(&amp;disk-&gt;queue_lock);
 mutex_unlock(&amp;s-&gt;sock-&gt;mutex);
 mutex_unlock(&amp;func-&gt;mutex);
 return disassemble(info-&gt;pending_bh);
}
static void num_serial_settings(struct tty_struct *tty)
{
 if (tty == tty)
 disable_single_st_p(dev);
 pci_disable_spool(port);
 return 0;
}
static void do_command(struct seq_file *m, void *v)
{
 int column = 32 &lt;&lt; (cmd[2] &amp; 0x80);
 if (state)
 cmd = (int)(int_state ^ (in_8(&amp;ch-&gt;ch_flags) &amp; Cmd) ? 2 : 1);
 else
 seq = 1;
 for (i = 0; i &lt; 16; i++) {
 if (k &amp; (1 &lt;&lt; 1))
 pipe = (in_use &amp; UMXTHREAD_UNCCA) +
 ((count &amp; 0x00000000fffffff8) &amp; 0x000000f) &lt;&lt; 8;
 if (count == 0)
 sub(pid, ppc_md.kexec_handle, 0x20000000);
 pipe_set_bytes(i, 0);
 }
 /* Free our user pages pointer to place camera if all dash */
 subsystem_info = &amp;of_changes[PAGE_SIZE];
 rek_controls(offset, idx, &amp;soffset);
 /* Now we want to deliberately put it to device */
 control_check_polarity(&amp;context, val, 0);
 for (i = 0; i &lt; COUNTER; i++)
 seq_puts(s, "policy ");
}
  
Notice that in the second function the model compares tty == tty, which is vacuously true. On the other hand, at least the variable tty exists in the scope this time! In the last function, notice that the code does not return anything, which happens to be correct since the function signature is void. However, the first two functions were also declared void and did return values. This is again a form of a common mistake due to long-term interactions.
Sometimes the model decides that it’s time to sample a new file. This is usually a very amusing part: The model first recites the GNU license character by character, samples a few includes, generates some macros and then dives into the code:
  /*
 * Copyright (c) 2006-2010, Intel Mobile Communications. All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify it
 * under the terms of the GNU General Public License version 2 as published by
 * the Free Software Foundation.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 *
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software Foundation,
 * Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 */
#include &lt;linux/kexec.h&gt;
#include &lt;linux/errno.h&gt;
#include &lt;linux/io.h&gt;
#include &lt;linux/platform_device.h&gt;
#include &lt;linux/multi.h&gt;
#include &lt;linux/ckevent.h&gt;
#include &lt;asm/io.h&gt;
#include &lt;asm/prom.h&gt;
#include &lt;asm/e820.h&gt;
#include &lt;asm/system_info.h&gt;
#include &lt;asm/setew.h&gt;
#include &lt;asm/pgproto.h&gt;
#define REG_PG vesa_slot_addr_pack
#define PFM_NOCOMP AFSR(0, load)
#define STACK_DDR(type) (func)
#define SWAP_ALLOCATE(nr) (e)
#define emulate_sigs() arch_get_unaligned_child()
#define access_rw(TST) asm volatile("movd %%esp, %0, %3" : : "r" (0)); \
 if (__type &amp; DO_READ)
static void stat_PC_SEC __read_mostly offsetof(struct seq_argsqueue, \
 pC&gt;[1]);
static void
os_prefix(unsigned long sys)
{
#ifdef CONFIG_PREEMPT
 PUT_PARAM_RAID(2, sel) = get_state_state();
 set_pid_sum((unsigned long)state, current_state_str(),
 (unsigned long)-1-&gt;lr_full; low;
}
  
There are too many fun parts to cover- I could probably write an entire blog post on just this part. I’ll cut it short for now, but here is 1MB of sampled Linux code for your viewing pleasure.
Generating Baby Names
Lets try one more for fun. Lets feed the RNN a large text file that contains 8000 baby names listed out, one per line (names obtained from here). We can feed this to the RNN and then generate new names! Here are some example names, only showing the ones that do not occur in the training data (90% don’t):
 Rudi
Levette
Berice
Lussa
Hany
Mareanne
Chrestina
Carissy
Marylen
Hammine
Janye
Marlise
Jacacrie
Hendred
Romand
Charienna
Nenotto
Ette
Dorane
Wallen
Marly
Darine
Salina
Elvyn
Ersia
Maralena
Minoria
Ellia
Charmin
Antley
Nerille
Chelon
Walmor
Evena
Jeryly
Stachon
Charisa
Allisa
Anatha
Cathanie
Geetra
Alexie
Jerin
Cassen
Herbett
Cossie
Velen
Daurenge
Robester
Shermond
Terisa
Licia
Roselen
Ferine
Jayn
Lusine
Charyanne
Sales
Sanny
Resa
Wallon
Martine
Merus
Jelen
Candica
Wallin
Tel
Rachene
Tarine
Ozila
Ketia
Shanne
Arnande
Karella
Roselina
Alessia
Chasty
Deland
Berther
Geamar
Jackein
Mellisand
Sagdy
Nenc
Lessie
Rasemy
Guen
Gavi
Milea
Anneda
Margoris
Janin
Rodelin
Zeanna
Elyne
Janah
Ferzina
Susta
Pey
Castina 
You can see many more here. Some of my favorites include “Baby” (haha), “Killie”, “Char”, “R”, “More”, “Mars”, “Hi”, “Saddie”, “With” and “Ahbort”. Well that was fun.﻿ Of course, you can imagine this being quite useful inspiration when writing a novel, or naming a new startup :)
Understanding what’s going on
We saw that the results at the end of training can be impressive, but how does any of this work? Lets run two quick experiments to briefly peek under the hood.
The evolution of samples while training
First, it’s fun to look at how the sampled text evolves while the model trains. For example, I trained an LSTM of Leo Tolstoy’s War and Peace and then generated samples every 100 iterations of training. At iteration 100 the model samples random jumbles:
  tyntd-iafhatawiaoihrdemot lytdws e ,tfti, astai f ogoh eoase rrranbyne 'nhthnee e 
plia tklrgd t o idoe ns,smtt h ne etie h,hregtrs nigtike,aoaenns lng
  
However, notice that at least it is starting to get an idea about words separated by spaces. Except sometimes it inserts two spaces. It also doesn’t know that comma is amost always followed by a space. At 300 iterations we see that the model starts to get an idea about quotes and periods:
  "Tmont thithey" fomesscerliund
Keushey. Thom here
sheulke, anmerenith ol sivh I lalterthend Bleipile shuwy fil on aseterlome
coaniogennc Phe lism thond hon at. MeiDimorotion in ther thize."
  
The words are now also separated with spaces and the model starts to get the idea about periods at the end of a sentence. At iteration 500:
  we counter. He stutn co des. His stanted out one ofler that concossions and was 
to gearang reay Jotrets and with fre colt otf paitt thin wall. Which das stimn 
  
the model has now learned to spell the shortest and most common words such as “we”, “He”, “His”, “Which”, “and”, etc. At iteration 700 we’re starting to see more and more English-like text emerge:
  Aftair fall unsuch that the hall for Prince Velzonski's that me of
her hearly, and behs to so arwage fiving were to it beloge, pavu say falling misfort 
how, and Gogition is so overelical and ofter.
  
At iteration 1200 we’re now seeing use of quotations and question/exclamation marks. Longer words have now been learned as well:
  "Kite vouch!" he repeated by her
door. "But I would be done and quarts, feeling, then, son is people...."
  
Until at last we start to get properly spelled words, quotations, names, and so on by about iteration 2000:
  "Why do what that day," replied Natasha, and wishing to himself the fact the
princess, Princess Mary was easier, fed in had oftened him.
Pierre aking his soul came to the packs and drove up his father-in-law women.
  
The picture that emerges is that the model first discovers the general word-space structure and then rapidly starts to learn the words; First starting with the short words and then eventually the longer ones. Topics and themes that span multiple words (and in general longer-term dependencies) start to emerge only much later.
Visualizing the predictions and the “neuron” firings in the RNN
Another fun visualization is to look at the predicted distributions over characters. In the visualizations below we feed a Wikipedia RNN model character data from the validation set (shown along the blue/green rows) and under every character we visualize (in red) the top 5 guesses that the model assigns for the next character. The guesses are colored by their probability (so dark red = judged as very likely, white = not very likely). For example, notice that there are stretches of characters where the model is extremely confident about the next letter (e.g., the model is very confident about characters during the http://www. sequence).
The input character sequence (blue/green) is colored based on the firing of a randomly chosen neuron in the hidden representation of the RNN. Think about it as green = very excited and blue = not very excited (for those familiar with details of LSTMs, these are values between [-1,1] in the hidden state vector, which is just the gated and tanh’d LSTM cell state). Intuitively, this is visualizing the firing rate of some neuron in the “brain” of the RNN while it reads the input sequence. Different neurons might be looking for different patterns; Below we’ll look at 4 different ones that I found and thought were interesting or interpretable (many also aren’t):
  
The neuron highlighted in this image seems to get very excited about URLs and turns off outside of the URLs. The LSTM is likely using this neuron to remember if it is inside a URL or not.
  
The highlighted neuron here gets very excited when the RNN is inside the [[ ]] markdown environment and turns off outside of it. Interestingly, the neuron can't turn on right after it sees the character "[", it must wait for the second "[" and then activate. This task of counting whether the model has seen one or two "[" is likely done with a different neuron.
  
Here we see a neuron that varies seemingly linearly across the [[ ]] environment. In other words its activation is giving the RNN a time-aligned coordinate system across the [[ ]] scope. The RNN can use this information to make different characters more or less likely depending on how early/late it is in the [[ ]] scope (perhaps?).
  
Here is another neuron that has very local behavior: it is relatively silent but sharply turns off right after the first "w" in the "www" sequence. The RNN might be using this neuron to count up how far in the "www" sequence it is, so that it can know whether it should emit another "w", or if it should start the URL.
Of course, a lot of these conclusions are slightly hand-wavy as the hidden state of the RNN is a huge, high-dimensional and largely distributed representation. These visualizations were produced with custom HTML/CSS/Javascript, you can see a sketch of what’s involved here if you’d like to create something similar.
We can also condense this visualization by excluding the most likely predictions and only visualize the text, colored by activations of a cell. We can see that in addition to a large portion of cells that do not do anything interpretible, about 5% of them turn out to have learned quite interesting and interpretible algorithms:
Again, what is beautiful about this is that we didn’t have to hardcode at any point that if you’re trying to predict the next character it might, for example, be useful to keep track of whether or not you are currently inside or outside of quote. We just trained the LSTM on raw data and it decided that this is a useful quantitity to keep track of. In other words one of its cells gradually tuned itself during training to become a quote detection cell, since this helps it better perform the final task. This is one of the cleanest and most compelling examples of where the power in Deep Learning models (and more generally end-to-end training) is coming from.
Source Code
I hope I’ve convinced you that training character-level language models is a very fun exercise. You can train your own models using the char-rnn code I released on Github (under MIT license). It takes one large text file and trains a character-level model that you can then sample from. Also, it helps if you have a GPU or otherwise training on CPU will be about a factor of 10x slower. In any case, if you end up training on some data and getting fun results let me know! And if you get lost in the Torch/Lua codebase remember that all it is is just a more fancy version of this 100-line gist.
 Brief digression. The code is written in Torch 7, which has recently become my favorite deep learning framework. I’ve only started working with Torch/LUA over the last few months and it hasn’t been easy (I spent a good amount of time digging through the raw Torch code on Github and asking questions on their gitter to get things done), but once you get a hang of things it offers a lot of flexibility and speed. I’ve also worked with Caffe and Theano in the past and I believe Torch, while not perfect, gets its levels of abstraction and philosophy right better than others. In my view the desirable features of an effective framework are:
 CPU/GPU transparent Tensor library with a lot of functionality (slicing, array/matrix operations, etc. )
 An entirely separate code base in a scripting language (ideally Python) that operates over Tensors and implements all Deep Learning stuff (forward/backward, computation graphs, etc)
 It should be possible to easily share pretrained models (Caffe does this well, others don’t), and crucially
 NO compilation step (or at least not as currently done in Theano). The trend in Deep Learning is towards larger, more complex networks that are are time-unrolled in complex graphs. It is critical that these do not compile for a long time or development time greatly suffers. Second, by compiling one gives up interpretability and the ability to log/debug effectively. If there is an option to compile the graph once it has been developed for efficiency in prod that’s fine.
Further Reading
Before the end of the post I also wanted to position RNNs in a wider context and provide a sketch of the current research directions. RNNs have recently generated a significant amount of buzz and excitement in the field of Deep Learning. Similar to Convolutional Networks they have been around for decades but their full potential has only recently started to get widely recognized, in large part due to our growing computational resources. Here’s a brief sketch of a few recent developments (definitely not complete list, and a lot of this work draws from research back to 1990s, see related work sections):
In the domain of NLP/Speech, RNNs transcribe speech to text, perform machine translation, generate handwritten text, and of course, they have been used as powerful language models (Sutskever et al.) (Graves) (Mikolov et al.) (both on the level of characters and words). Currently it seems that word-level models work better than character-level models, but this is surely a temporary thing.
 Computer Vision. RNNs are also quickly becoming pervasive in Computer Vision. For example, we’re seeing RNNs in frame-level video classification, image captioning (also including my own work and many others), video captioning and very recently visual question answering. My personal favorite RNNs in Computer Vision paper is Recurrent Models of Visual Attention, both due to its high-level direction (sequential processing of images with glances) and the low-level modeling (REINFORCE learning rule that is a special case of policy gradient methods in Reinforcement Learning, which allows one to train models that perform non-differentiable computation (taking glances around the image in this case)). I’m confident that this type of hybrid model that consists of a blend of CNN for raw perception coupled with an RNN glance policy on top will become pervasive in perception, especially for more complex tasks that go beyond classifying some objects in plain view.
 Inductive Reasoning, Memories and Attention. Another extremely exciting direction of research is oriented towards addressing the limitations of vanilla recurrent networks. One problem is that RNNs are not inductive: They memorize sequences extremely well, but they don’t necessarily always show convincing signs of generalizing in the correct way (I’ll provide pointers in a bit that make this more concrete). A second issue is they unnecessarily couple their representation size to the amount of computation per step. For instance, if you double the size of the hidden state vector you’d quadruple the amount of FLOPS at each step due to the matrix multiplication. Ideally, we’d like to maintain a huge representation/memory (e.g. containing all of Wikipedia or many intermediate state variables), while maintaining the ability to keep computation per time step fixed.
The first convincing example of moving towards these directions was developed in DeepMind’s Neural Turing Machines paper. This paper sketched a path towards models that can perform read/write operations between large, external memory arrays and a smaller set of memory registers (think of these as our working memory) where the computation happens. Crucially, the NTM paper also featured very interesting memory addressing mechanisms that were implemented with a (soft, and fully-differentiable) attention model. The concept of soft attention has turned out to be a powerful modeling feature and was also featured in Neural Machine Translation by Jointly Learning to Align and Translate for Machine Translation and Memory Networks for (toy) Question Answering. In fact, I’d go as far as to say that
 The concept of attention is the most interesting recent architectural innovation in neural networks.
Now, I don’t want to dive into too many details but a soft attention scheme for memory addressing is convenient because it keeps the model fully-differentiable, but unfortunately one sacrifices efficiency because everything that can be attended to is attended to (but softly). Think of this as declaring a pointer in C that doesn’t point to a specific address but instead defines an entire distribution over all addresses in the entire memory, and dereferencing the pointer returns a weighted sum of the pointed content (that would be an expensive operation!). This has motivated multiple authors to swap soft attention models for hard attention where one samples a particular chunk of memory to attend to (e.g. a read/write action for some memory cell instead of reading/writing from all cells to some degree). This model is significantly more philosophically appealing, scalable and efficient, but unfortunately it is also non-differentiable. This then calls for use of techniques from the Reinforcement Learning literature (e.g. REINFORCE) where people are perfectly used to the concept of non-differentiable interactions. This is very much ongoing work but these hard attention models have been explored, for example, in Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets, Reinforcement Learning Neural Turing Machines, and Show Attend and Tell.
 People. If you’d like to read up on RNNs I recommend theses from Alex Graves, Ilya Sutskever and Tomas Mikolov. For more about REINFORCE and more generally Reinforcement Learning and policy gradient methods (which REINFORCE is a special case of) David Silver’s class, or one of Pieter Abbeel’s classes.
 Code. If you’d like to play with training RNNs I hear good things about keras or passage for Theano, the code released with this post for Torch, or this gist for raw numpy code I wrote a while ago that implements an efficient, batched LSTM forward and backward pass. You can also have a look at my numpy-based NeuralTalk which uses an RNN/LSTM to caption images, or maybe this Caffe implementation by Jeff Donahue.
Conclusion
We’ve learned about RNNs, how they work, why they have become a big deal, we’ve trained an RNN character-level language model on several fun datasets, and we’ve seen where RNNs are going. You can confidently expect a large amount of innovation in the space of RNNs, and I believe they will become a pervasive and critical component to intelligent systems.
Lastly, to add some meta to this post, I trained an RNN on the source file of this blog post. Unfortunately, at about 46K characters I haven’t written enough data to properly feed the RNN, but the returned sample (generated with low temperature to get a more typical sample) is:
  I've the RNN with and works, but the computed with program of the 
RNN with and the computed of the RNN with with and the code
  
Yes, the post was about RNN and how well it works, so clearly this works :). See you next time!
 EDIT (extra links): 
Videos:
 I gave a talk on this work at the London Deep Learning meetup (video).
Discussions:
  HN discussion 
 Reddit discussion on r/machinelearning 
 Reddit discussion on r/programming 
Replies:
  Yoav Goldberg compared these RNN results to n-gram maximum likelihood (counting) baseline 
  @nylk trained char-rnn on cooking recipes. They look great!
  @MrChrisJohnson trained char-rnn on Eminem lyrics and then synthesized a rap song with robotic voice reading it out. Hilarious :)
  @samim trained char-rnn on Obama Speeches. They look fun!
  João Felipe trained char-rnn irish folk music and sampled music 
  Bob Sturm also trained char-rnn on music in ABC notation 
  RNN Bible bot by Maximilien 
  Learning Holiness learning the Bible
  Terminal.com snapshot that has char-rnn set up and ready to go in a browser-based virtual machine (thanks @samim)
The Unreasonable Effectiveness of Recurrent Neural Networks
http://karpathy.github.io/2015/05/21/rnn-effectiveness/
https://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-man.html
Elon Musk: The World's Raddest Man — Wait But Why
2015-05-07
Tim Urban
<div><div><p><em>This is Part 1 of a four-part series on Elon Musk’s companies.</em></p>
<p>__________</p>
<p><strong>PDF and ebook options: </strong>We made a fancy PDF of this post for printing and offline viewing (see a preview <a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Musk-Raddest-Man-Final-PDF-Preview.pdf">here</a>), and an ebook containing the whole four-part Elon Musk series:</p>
<p><a href="https://gum.co/wbw-musk"></a><a href="http://amzn.to/1RkVeEm"></a></p>
<p>___________</p>
<p>Last month, I got a surprising phone call.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Call-0.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Call-14.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Call-2b.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Call-31.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Call-4.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Call-51.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Call-6.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Call-7.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Call-83.png"></a></p>
<p>Elon Musk, for those unfamiliar, is the world’s raddest man.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Untitled-4.jpg"></a></p>
<p>I’ll use this post to explore how he became a self-made billionaire and the real-life inspiration for <i>Iron Man’s </i>Tony Stark, but for the moment, I’ll let Richard Branson explain things briefly:<a href="#footnote2-1-3623">1</a>
</p><p><em>Whatever skeptics have said can’t be done, Elon has gone out and made real. Remember in the 1990s, when we would call strangers and give them our credit-card numbers? Elon dreamed up a little thing called PayPal. His Tesla Motors and SolarCity companies are making a clean, renewable-energy future a reality…his SpaceX [is] reopening space for exploration…it’s a paradox that Elon is working to improve our planet at the same time he’s building spacecraft to help us leave it.</em></p>
<p>So no, that was not a phone call I had been expecting.</p>
<p>A few days later, I found myself in pajama pants, pacing frantically around my apartment, on the phone with Elon Musk. We had a discussion about Tesla, SpaceX, the automotive and aerospace and solar power industries, and he told me what he thought confused people about each of these things. He suggested that if these were topics I’d be interested in writing about, and it might be helpful, I could come out to California and sit down with him in person for a longer discussion.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/2Call1.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/2Call21.png"></a></p>
<p>For me, this project was one of the biggest no-brainers in history. Not just because Elon Musk is Elon Musk, but because here are two <em>separate </em>items that have been sitting for a while in my “Future Post Topics” document, verbatim:</p>
<p>– “electric vs hybrid vs gas cars, deal with tesla, sustainable energy”</p>
<p>– “spacex, musk, mars?? how learn to do rockets??”</p>
<p>I already wanted to write about these topics, for the same reason I <a href="https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html">wrote about</a> Artificial Intelligence—I knew they would be hugely important in the future but that I also didn’t understand them well enough. And Musk is leading a revolution in both of these worlds.</p>
<p>It would be like if you had plans to write about the process of throwing lightning bolts and then one day out of the blue Zeus called and asked if you wanted to question him about a lot of stuff.</p>
<p>So it was on. The plan was that I’d come out to California, see the Tesla and SpaceX factories, meet with some of the engineers at each company, and have an extended sit down with Musk. <em>Exciting.</em></p>
<p>The first order of business was to have a full panic. I needed to not sit down with these people—these world-class engineers and rocket scientists—and know almost nothing about anything. I had a lot of quick learning to do.</p>
<p>The problem with Elon Fucking Musk, though, is that he happens to be involved in <em>all </em>of the following industries:</p>
<ul>
<li>Automotive</li>
<li>Aerospace</li>
<li>Solar Energy</li>
<li>Energy Storage</li>
<li>Satellite</li>
<li>High-Speed Ground Transportation</li>
<li>And, um, Multi-Planetary Expansion</li>
</ul>
<p>Zeus would have been less stressful.</p>
<p>So I spent the two weeks leading up to the West Coast visit reading and reading and reading, and it became quickly clear that this was gonna need to be a multi-post series. There’s a <em>lot </em>to get into.</p>
<p>We’ll dive deep into Musk’s companies and the industries surrounding them in the coming posts, but today, let’s start by going over exactly who this dude is and why he’s such a big deal.<a href="#footnote-1-3623">1</a><a href="#footnote2-2-3623">2</a><sup>← click these</sup></p>
<p><strong>The Making of Elon Musk</strong></p>
<p><em>Note: There’s a great biography on Musk coming out May 19th, written by tech writer Ashlee Vance. I was able to get an advance copy, and it’s been a key source in putting together these posts. I’m going to keep to a brief overview of his life here—if you want the full story, <a href="http://www.amazon.com/gp/product/0062301233/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0062301233&amp;linkCode=as2&amp;tag=wabuwh00-20&amp;linkId=UMV6CN256IWSDQDR">get the bio</a>.</em></p>
<p>Musk was born in 1971 in South Africa. Childhood wasn’t a great time for him—he had a tough family life and never fit in well at school.<a href="#footnote-2-3623">2</a> But, like you often read in the bios of extraordinary people, he was an avid self-learner early on. His brother Kimbal has said Elon would often read for 10 hours a day—a lot of science fiction and eventually, a lot of non-fiction too. By fourth grade, he was constantly buried in the <i>Encyclopedia Britannica. </i></p>
<p>One thing you’ll learn about Musk as you read these posts is that he thinks of humans as computers, which, in their most literal sense, they are. A human’s hardware is his physical body and brain. His software is the way he learns to think, his value system, his habits, his personality. And learning, for Musk, is simply the process of “downloading data and algorithms into your brain.”<a href="#footnote2-3-3623">3</a> Among his many frustrations with formal classroom learning is the “ridiculously slow download speed” of sitting in a classroom while a teacher explains something, and to this day, most of what he knows he’s learned through reading.</p>
<p>He became consumed with a second fixation at the age of nine when he got his hands on his first computer, the <a href="http://en.wikipedia.org/wiki/Commodore_VIC-20">Commodore VIC-20</a>. It came with five kilobytes of memory and a “how to program” guide that was intended to take the user six months to complete. Nine-year-old Elon finished it in three days. At 12, he used his skills to create a video game called Blastar, which he told me was “a trivial game…but better than Flappy Bird.” But in 1983, it was good enough to be sold to a computer magazine for $500 ($1,200 in today’s money)—not bad for a 12-year-old.<a href="#footnote-3-3623">3</a>
</p><p>Musk never felt much of a connection to South Africa—he didn’t fit in with the jockish, white Afrikaner culture, and it was a nightmare country for a potential entrepreneur. He saw Silicon Valley as the Promised Land, and at the age of 17, he left South Africa forever. He started out in Canada, which was an easier place to immigrate to because his mom is a Canadian citizen, and a few years later, used a college transfer to the University of Pennsylvania as a way into the US.<a href="#footnote-4-3623">4</a>
</p><p>In college, he thought about what he wanted to do with his life, using as his starting point the question, “What will most affect the future of humanity?” The answer he came up with was a list of five things: “the internet; sustainable energy; space exploration, in particular the permanent extension of life beyond Earth; artificial intelligence; and reprogramming the human genetic code.”<a href="#footnote2-4-3623">4</a>
</p><p>He was iffy about how positive the impact of the latter two would be, and though he was optimistic about each of the first three, he never considered at the time that he’d ever be involved in space exploration. That left the internet and sustainable energy as his options.</p>
<p>He decided to go with sustainable energy. After finishing college, he enrolled in a Stanford PhD program to study high energy density capacitors, a technology aimed at coming up with a more efficient way than traditional batteries to store energy—which he knew could be key to a sustainable energy future and help accelerate the advent of an electric car industry.</p>
<p>But two days into the program, he got massive FOMO because it was 1995 and he “couldn’t stand to just watch the internet go by—[he] wanted to jump in and make it better.”<a href="#footnote2-5-3623">5</a> So he dropped out and decided to try the internet instead.</p>
<p>His first move was to go try to get a job at the monster of the 1995 internet, Netscape. The tactic he came up with was to walk into the lobby, uninvited, stand there awkwardly, be too shy to talk to anyone, and walk out.</p>
<p>Musk bounced back from the unimpressive career beginning by teaming up with his brother Kimbal (who had followed Elon to the US) to start their own company—Zip2. Zip2 was like a primitive combination of Yelp and Google Maps, far before anything like either of those existed. The goal was to get businesses to realize that being in the Yellow Pages would become outdated at some point and that it was a good idea to get themselves into an online directory. The brothers had no money, slept in the office and showered at the YMCA, and Elon, their lead programmer, sat obsessively at his computer working around the clock. In 1995, it was hard to convince businesses that the internet was important—many told them that advertising on the internet sounded like “the dumbest thing they had ever heard of”<a href="#footnote2-6-3623">6</a>—but eventually, they began to rack up customers and the company grew. It was the heat of the 90s internet boom, startup companies were being snatched up left and right, and in 1999, Compaq snatched up Zip2 for $307 million. Musk, who was 27, made off with $22 million.</p>
<p>In what would become a recurring theme for Musk, he finished one venture and immediately dove into a new, harder, more complex one. If he were following the dot-com millionaire rulebook, he’d have known that what you’re supposed to do after hitting it big during the 90s boom is either retire off into the sunset of leisure and angel investing, or if you still have ambition, start a new company with someone else’s money. But Musk doesn’t tend to follow normal rulebooks, and he plunged three quarters of his net worth into his new idea, an outrageously bold plan to build essentially an online bank—replete with checking, savings, and brokerage accounts—called X.com. This seems less insane now, but in 1999, an internet startup trying to compete with the large banks was unheard of.</p>
<p>In the same building that X.com worked out of was another internet finance company called Confinity, founded by Peter Thiel and Max Levchin. One of X.com’s many features was an easy money-transfer service, and later, Confinity would develop a similar service. Both companies began to notice a strong demand for their money-transfer service, which put the two companies in sudden furious competition with each other, and they finally decided to just merge into what we know today as PayPal.</p>
<p>This brought together a lot of egos and conflicting opinions—Musk was now joined by Peter Thiel and a bunch of other now-super-successful internet guys—and despite the company growing rapidly, things inside the office did not go smoothly. The conflicts boiled over in late 2000, and when Musk was on a half fundraising trip / half honeymoon (with his first wife Justine), the anti-Musk crowd staged a coup and replaced him as CEO with Thiel. Musk handled this surprisingly well, and to this day, he says he doesn’t agree with that decision but he understands why they did it. He stayed on the team in a senior role, continued investing in the company, and played an instrumental role in selling the company to eBay in 2002, for $1.5 billion. Musk, the company’s largest shareholder, walked away with $180 million (after taxes).<a href="#footnote-5-3623">5</a>
</p><p>If there was ever a semblance of the normal life rulebook in Musk’s decision-making, it was at this point in his life—as a beyond-wealthy 31-year-old in 2002—that he dropped the rulebook into the fire for good.</p>
<p>The subject of what he did over the next 13 years leading up to today is what we’ll thoroughly explore over the rest of this series. For now, here’s the short story:</p>
<p>In 2002, before the sale of PayPal even went through, Musk started voraciously reading about rocket technology, and later that year, with $100 million, he started one of the most unthinkable and ill-advised ventures of all time: a rocket company called SpaceX, whose stated purpose was to revolutionize the cost of space travel in order to make humans a multi-planetary species by colonizing Mars with at least a million people over the next century.</p>
<p>Mm hm.</p>
<p>Then, in 2004, as that “project” was just getting going, Musk decided to multi-task by launching the second-most unthinkable and ill-advised venture of all time: an electric car company called Tesla, whose stated purpose was to revolutionize the worldwide car industry by significantly accelerating the advent of a mostly-electric-car world—in order to bring humanity on a huge leap toward a sustainable energy future. Musk funded this one personally as well, pouring in $70 million, despite the tiny fact that the last time a US car startup succeeded was Chrysler in 1925, and the last time someone<em> </em>started a successful <em>electric</em> car startup was never.</p>
<p>And since why the fuck not, a couple years later, in 2006, he threw in $10 million to found, with his cousins, <em>another </em>company, called SolarCity, whose goal was to revolutionize energy production by creating a large, distributed utility that would install solar panel systems on millions of people’s homes, dramatically reducing their consumption of fossil fuel-generated electricity and ultimately “accelerating mass adoption of sustainable energy.”<a href="#footnote2-7-3623">7</a>
</p><p>If you were observing all of this in those four years following the PayPal sale, you’d think it was a sad story. A delusional internet millionaire, comically in over his head with a slew of impossible projects, doing everything he could to squander his fortune.</p>
<p>By 2008, this seemed to be playing out, to the letter. SpaceX had figured out how to build rockets, just not rockets that actually worked—it had attempted three launches so far and all three had blown up before reaching orbit. In order to bring in any serious outside investment or payload contracts, SpaceX had to show that they could successfully launch a rocket—but Musk said he had funds left for one and only one more launch. If the fourth launch also failed, SpaceX would be done.</p>
<p>Meanwhile, up in the Bay Area, Tesla was also in the shit. They had yet to deliver their first car—the Tesla Roadster—to the market, which didn’t look good to the outside world. Silicon Valley gossip blog Valleywag made the Tesla Roadster its #1 tech company fail of 2007. This would have been more okay if the global economy hadn’t suddenly crashed, hitting the automotive industry the absolute hardest and sucking dry any flow of investments into car companies, especially new and unproven ones. And Tesla was running out of money fast.</p>
<p>During this double implosion of his career, the one thing that held stable and strong in Musk’s life was his marriage of eight years, if by stable and strong you mean falling apart entirely in a soul-crushing, messy divorce.</p>
<p>Darkness.</p>
<p>But here’s the thing—Musk is not a fool, and he hadn’t built bad companies. He had built very, <em>very </em>good companies. It’s just that creating a reliable rocket is unfathomably difficult, as is launching a startup car company, and because no one wanted to invest in what seemed to the outside world like overambitious and probably-doomed ventures—especially during a recession—Musk had to rely on his own personal funds. PayPal made him rich, but not rich enough to keep these companies afloat for very long on his own. Without outside money, both SpaceX and Tesla had a short runway. So it’s not that SpaceX and Tesla were bad—it’s that they needed more time to succeed, and they were out of time.</p>
<p>And then, in the most dire hour, everything turned around.</p>
<p>First, in September of 2008, SpaceX launched their fourth rocket—and their last one if it didn’t successfully put a payload into orbit—and it succeeded. Perfectly.</p>
<p>That was enough for NASA to say “fuck it, let’s give this Musk guy a try,” and it took a gamble, offering SpaceX a $1.6 billion contract to carry out 12 launches for the agency. Runway extended. SpaceX saved.</p>
<p><em>The next day,</em> on Christmas Eve 2008, when Musk scrounged up the last money he could manage to keep Tesla going, Tesla’s investors reluctantly agreed to match his investment. Runway extended. Five months later, things began looking up, and another critical investment came in—$50 million from Daimler. Tesla saved.</p>
<p>While 2008 hardly marked the end of the bumps in the road for Musk, the overarching story of the next seven years would be the soaring, earthshaking success of Elon Musk and his companies.</p>
<p>Since their first three failed launches, SpaceX has launched 20 times—all successes. NASA is now a regular client, and one of many, since the innovations at SpaceX have allowed companies to launch things to space for the lowest cost in history. Within those 20 launches have been all kinds of “firsts” for a commercial rocket company—to this day, the four entities in history who have managed to launch a spacecraft into orbit and successfully return it to Earth are the US, Russia, China—and SpaceX. SpaceX is currently testing their new spacecraft, which will bring humans to space, and they’re busy at work on the much larger rocket that will be able to bring 100 people to Mars at once. A recent investment by Google and Fidelity has valued the company at $12 billion.</p>
<p>Tesla’s Model S has become a smashing success, blowing away the automotive industry with the highest ever Consumer Reports rating of a 99/100, and the highest safety rating in history from the National Highway Safety Administration, a 5.4/5. Now they’re getting closer and closer to releasing their true disruptor—the much more affordable Model 3—and the company’s market cap is just under $30 billion. They’re also becoming the world’s most formidable battery company, currently working on their giant Nevada “Gigafactory,” which will more than double the world’s <em>total </em>annual production of lithium-ion batteries.</p>
<p>SolarCity, which went public in 2012, now has a market cap of just under $6 billion and has become the largest installer of solar panels in the US. They’re now building the country’s largest solar panel-manufacturing factory in Buffalo, and they’ll likely be entering into a partnership with Tesla to package their product with Tesla’s new home battery, the <a href="https://www.youtube.com/watch?v=ZVjW9XdPlhg">Powerwall</a>.</p>
<p>And since that’s not enough, in his spare time, Musk is pushing the development a whole new mode of transport—the <a href="https://waitbutwhy.com/2015/06/hyperloop.html">Hyperloop</a>.</p>
<p>In a couple of years, when their newest factories are complete, Musk’s three companies will employ over 30,000 people. After nearly going broke in 2008 and telling a friend that he and his wife may have to “move into his wife’s parents’ basement,”<a href="#footnote2-8-3623">8</a> Musk’s current net worth <a href="http://www.forbes.com/profile/elon-musk/">clocks in</a> at $12.9 billion.</p>
<p>All of this has made Musk somewhat of a living legend. In building a successful automotive startup and its worldwide network of Supercharger stations, Musk has been compared to visionary industrialists like Henry Ford and John D. Rockefeller. The pioneering work of SpaceX on rocket technology has led to comparisons to Howard Hughes, and many have drawn parallels between Musk and Thomas Edison because of the advancements in engineering Musk has been able to achieve across industries. Perhaps most often, he’s compared to Steve Jobs, for his remarkable ability to disrupt giant, long-stagnant industries with things customers didn’t even know they wanted. Some believe he’ll be remembered in a class of his own. Tech writer and Musk biographer Ashlee Vance has suggested that what Musk is building “has the potential to be much grander than anything Hughes or Jobs produced. Musk has taken industries like aerospace and automotive that America seemed to have given up on and recast them as something new and fantastic.”<a href="#footnote2-9-3623">9</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/F.jpg"></a>Chris Anderson, who runs TED Talks, <a href="https://www.youtube.com/watch?v=tUMZTtQU10o">calls</a> Musk “the world’s most remarkable living entrepreneur.” Others know him as “the real life Iron Man,” and not for no reason—Jon Favreau actually sent Robert Downey, Jr. to spend time with Musk in the SpaceX factory prior to filming the first <em>Iron Man</em> movie so he could model his character off of Musk.<a href="#footnote2-10-3623">10</a> He’s even <a href="http://simpsons.wikia.com/wiki/The_Musk_Who_Fell_to_Earth">been on</a> The Simpsons.</p>
<p>And this is the man I was somehow on the phone with as I frantically paced back and forth in my apartment, in pajama pants.</p>
<p>On the call, he made it clear that he wasn’t looking for me to advertise his companies—he only wanted me to help explain what’s going on in the worlds surrounding those companies and why the things happening with electric cars, sustainable energy production, and aerospace matter so much.</p>
<p>He seemed particularly bored with people spending time writing about <em>him—</em>he feels there are so many things of critical importance going on in the industries he’s involved in, and every time someone writes about him, he wishes they were writing about fossil fuel supply or battery advancements or the importance of making humanity multi-planetary (this is especially clear in the intro to the upcoming biography on him, when the author explains how <em>not</em> interested Musk was in having a bio written about him).</p>
<p>So I’m sure this first post, whose title is “Elon Musk: The World’s Raddest Man,” will annoy him.</p>
<p>But I have reasons. To me, there are two worthy areas of exploration in this post series:</p>
<p><strong>1) To understand why Musk is doing what he’s doing.</strong> He deeply believes that he’s taken on the most pressing possible causes to give humanity the best chance of a good future. I want to explore those causes in depth and the reasons he’s so concerned about them.</p>
<p><strong>2) To understand why Musk is <em>able to </em>do what he’s doing. </strong>There are a few people in each generation who dramatically change the world, and those people are worth studying. They do things differently from everyone else—and I think there’s a lot to learn from them.</p>
<p>So on my visit to California, I had two goals in mind: to understand as best I could what Musk and his teams were working on so feverishly and why it mattered so much, and to try to gain insight into what it is that makes him so capable of changing the world.</p>
<p>___________</p>
<p><strong>Visiting the Factories</strong></p>
<p>The Tesla Factory (in Northern CA) and the SpaceX Factory (in Southern CA), in addition to both being huge, and rad, have a lot in common.</p>
<p>Both factories are bright and clean, shiny and painted white, with super high ceilings. Both feel more like laboratories than traditional factories. And in both places, the engineers doing white collar jobs and the technicians doing blue collar jobs are deliberately placed in the same working quarters so they’ll work closely together and give each other feedback—and Musk believes it’s crucial for those designing the machines to be around those machines as they’re being manufactured. And while a traditional factory environment wouldn’t be ideal for an engineer on a computer and a traditional office environment wouldn’t be a good workplace for a technician, a clean, futuristic laboratory feels right for both professions. There are almost no closed offices in either factory—everyone is out in the open, exposed to everyone else.</p>
<p>When I pulled up to the Tesla factory (joined by Andrew), I was first taken by its size—and when I looked it up, I wasn’t surprised <a href="http://en.wikipedia.org/wiki/List_of_largest_buildings_in_the_world">to see</a> that it has the second-largest building footprint (aka base area) in the U.S.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/IMG_6427.jpg"></a></p>
<p>The factory was formerly jointly owned by GM and Toyota, who sold it to Tesla in 2010. We started off the day with a full tour of the factory—a sea of red robots making cars and being silly:<a href="#footnote-6-3623">6</a>
</p><p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/IMG_6434.jpg"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/IMG_6439.jpg"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/IMG_6444.jpg"></a></p>
<p>And other cool things, like a vast section of the factory that just makes the car battery, and another that houses the 20,000 pound rolls of aluminum they slice and press and weld into Teslas.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/130.jpg"></a></p>
<p>And this giant press, which costs $50 million and presses metal with <em>4,500 tons </em>of pressure (the same pressure you’d get if you stacked 2,500 cars on top of something).</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/142.jpg"></a></p>
<p>The Tesla factory is working on upping its output from 30,000 cars/year to 50,000, or about 1,000 per week. They seemed to be pumping out cars incredibly quickly, so I was blown away to learn that Toyota had been on a 1,000 cars per <em>day </em>clip when they inhabited the factory.</p>
<p>I had a chance to visit the Tesla design studio (no pictures allowed), where there were designers sketching car designs on computer screens and, on the other side of the room, full-size car models made of clay. An actual-size clay version of the upcoming Model 3 was surrounded by specialists sculpting it with tiny instruments and blades, shaving off fractions of a millimeter to examine the way light bounced off the curves. There was also a 3D printer that could quickly “print” out a shoe-sized 3D model of a sketched Tesla design so a designer could actually hold their design and look at it from different angles. Deliciously futuristic.</p>
<p>The next day was the SpaceX factory, which might be even cooler, but the building contains advanced rocket technology, which according to the government is “weapons technology,” and apparently random bloggers aren’t allowed to take pictures of weapons technology.</p>
<p>Anyway, after the tours, I had a chance to sit down with several senior engineers and designers at both companies. They’d explain that they were a foremost expert in their field, I’d explain that I had recently figured out how big the building would be that could hold all humans, and we’d begin our discussion. I’d ask them about their work, their thoughts on the company as a whole and the broader industry, and then I’d ask them about their relationship with Elon and what it was like to work for him. Without exception, they were really nice-seeming, friendly people, who all came off as ridiculously smart but in a non-pretentious way. Musk has said he has a strict “no assholes” hiring policy, and I could see that at work in these meetings.</p>
<p>So what’s Musk like as a boss?</p>
<p>Let’s start by seeing what the internet says—there’s a <a href="http://www.quora.com/What-is-it-like-to-work-with-Elon-Musk">Quora thread</a> that poses the question: “What is it like to work with Elon Musk?”</p>
<p>The first answer is from a longtime SpaceX employee who no longer works there, who describes the day that their 3rd launch failed, a devastating blow for the company and for all the people who had worked for years to try to make it work.</p>
<p>She describes Elon emerging from mission command to address the company and delivering a rousing speech. She refers to Elon’s “infinite wisdom” and says, “I think most of us would have followed him into the gates of hell carrying suntan oil after that. It was the most impressive display of leadership that I have ever witnessed.”</p>
<p>Right below that answer is another answer, from an anonymous SpaceX engineer, who describes working for Musk like this:</p>
<p>“You can always tell when someone’s left an Elon meeting: they’re defeated…nothing you ever do will be good enough so you have to find your own value, not depending on praise to get you through your obviously insufficient 80 hour work weeks.”</p>
<p>Reading about Musk online and in Vance’s book, I was struck by how representative both of these Quora comments were of whole camps of opinion on working for Musk. Doing so seems to bring out a tremendous amount of adoration and a tremendous amount of exasperation, sometimes with a tone of bitterness—and even more oddly, much of the time, you hear both sides of this story expressed by the same person. For example, later in the comment of the effusive Quora commenter comes “Working with him isn’t a comfortable experience, he is never satisfied with himself so he is never really satisfied with anyone around him…the challenge is that he is a machine and the rest of us aren’t.” And the frustrated anonymous commenter later concedes that the way Elon is “is understandable” given the enormity of the task at hand, and that “it is a great company and I do love it.”</p>
<p>My own talks with Musk’s engineers and designers told a similar story. I was told: “Elon always wants to know, ‘Why are we not going faster?’ He always wants bigger, better, faster” by the same person who a few minutes later was emphasizing how fair and thoughtful Musk tends to be in handling the terms for a recently fired employee.</p>
<p>The same person who told me he has “lots of sleepless nights” said in the adjacent sentence how happy he is to be at the company and that he hopes to “never leave.”</p>
<p>One senior executive described interacting with Musk like this: “Any conversation’s fairly high stakes because he’ll be very opinionated, and he can go deeper than you expect or are prepared for or deeper than your knowledge goes on a given topic, and it does feel like a high wire act interacting with him, especially when you find yourself in a [gulp] technical disagreement.”<a href="#footnote-7-3623">7</a> The same executive, who had previously worked at a huge tech company, also called Musk “the most grounded billionaire I’ve ever worked with.”</p>
<p>What I began to understand is that the explanation for both sides of the story—the cult-like adulation right alongside the grudging willingness to endure what sounds like blatant hell—comes down to respect. The people who work for Musk, no matter how they feel about his management style, feel an immense amount of respect—for his intelligence, for his work ethic, for his guts, and for the gravity of the missions he’s undertaken, missions that make all other potential jobs seem trivial and pointless.</p>
<p>Many of the people I talked to also alluded to their respect for his integrity. One way this integrity comes through is in his consistency. He’s been saying the same things in interviews for a decade, often using the same exact phrasing many years apart. He says what he really means, no matter the situation—one employee close to Musk told me that after a press conference or a business negotiation, once in private he’d ask Musk what his real angle was and what he <em>really </em>thinks. Musk’s response would always be boring: “I think exactly what I said.”</p>
<p>A few people I spoke with referenced Musk’s obsession with truth and accuracy. He’s fine with and even welcoming of negative criticism about him when he believes it’s accurate, but when the press gets something wrong about him or his companies, he usually can’t help himself and will engage them and correct their error. He detests vague spin-doctor phrases like “studies say” and “scientists disagree,” and he refuses to advertise for Tesla, something most startup car companies wouldn’t think twice about—because he sees advertising as manipulative and dishonest.</p>
<p>There’s even an undertone of integrity in Musk’s tyrannical demands of workers, because while he may be a tyrant, he’s not a hypocrite. Employees pressured to work 80 hours a week tend to be less bitter about it when at least the CEO is in there working 100.</p>
<p>Speaking of the CEO, let’s go have a hamburger with him.</p>
<p><strong>My Lunch With Elon</strong></p>
<p>It started like this:</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Lunch-1.png"></a><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Lunch-2.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Lunch-3.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Lunch-4.png"><br />
</a><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Lunch-5.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Lunch-6.png"></a></p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/Lunch-2.png"></a></p>
<p>After about seven minutes of this, I was able to get out my first question, a smalltalk-y question about how he thought the recent launch had gone (they had attempted an extremely difficult rocket-landing maneuver—more on that in the SpaceX post). His response included the following words: hypersonic, rarefied, densifying, supersonic, Mach 1, Mach 3, Mach 4, Mach 5, vacuum, regimes, thrusters, nitrogen, helium, mass, momentum, ballistic, and boost-back. While this was happening, I was still mostly blacked out from the surreality of the situation, and when I started to come to, I was scared to ask any questions about what he was saying in case he had already explained it while I was unconscious.</p>
<p>I eventually regained the ability to have adult human conversation, and we began what turned into a highly interesting and engaging two-hour discussion.<a href="#footnote-8-3623">8</a> This guy has a lot on his mind across a <em>lot </em>of topics. In this one lunch alone, we covered electric cars, climate change, artificial intelligence, the Fermi Paradox, consciousness, reusable rockets, colonizing Mars, creating an atmosphere on Mars, voting on Mars, genetic programming, his kids, population decline, physics vs. engineering, Edison vs. Tesla, solar power, a carbon tax, the definition of a company, warping spacetime and how this isn’t actually something you can do, nanobots in your bloodstream and how this isn’t actually something you can do, Galileo, Shakespeare, the American forefathers, Henry Ford, Isaac Newton, satellites, and ice ages.</p>
<p>I’ll get into the specifics of what he had to say about many of these things in later posts, but some notes for now:</p>
<p>— He’s a pretty tall and burly dude. Doesn’t really come through on camera.</p>
<p>— He ordered a burger and ate it in either two or three bites over a span of about 15 seconds. I’ve never seen anything like it.</p>
<p>— <strong>He is very, <em>very </em>concerned about AI.</strong> I quoted him in my <a href="https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html">posts</a> on AI saying that he fears that by working to bring about Superintelligent AI (ASI), we’re “summoning the demon,” but I didn’t know how <em>much</em> he thought about the topic. He cited AI safety as one of the <em>three </em>things he thinks about most—the other two being sustainable energy and becoming a multi-planet species, i.e. Tesla and SpaceX. Musk is a smart motherfucker, and he knows a ton about AI, and his sincere concern about this makes me scared.</p>
<p>— <strong>The Fermi Paradox also worries him.</strong> In <a href="https://waitbutwhy.com/2014/05/fermi-paradox.html">my post</a> on that, I divided Fermi thinkers into two camps—those who think there’s no other highly intelligent life out there at all because of some Great Filter, and those who believe there must be plenty of intelligent life and that we don’t see signs of any for some other reason. Musk wasn’t sure which camp seemed more likely, but he suspects that there may be an upsetting Great Filter situation going on. He thinks the paradox “just doesn’t make sense” and that it “gets more and more worrying” the more time that goes by. Considering the possibility that maybe we’re a rare civilization who made it past the Great Filter through a freak occurrence makes him feel even more conviction about SpaceX’s mission: “If we are very rare, we better get to the multi-planet situation <em>fast</em>, because if civilization is tenuous, then we must do whatever we can to ensure that our already-weak probability of surviving is improved dramatically.” Again, his fear here makes me feel not great.</p>
<p>— <strong>One topic I disagreed with him on is the nature of consciousness.</strong> I think of consciousness as a smooth <a href="https://waitbutwhy.com/2014/10/religion-for-the-nonreligious.html">spectrum</a>. To me, what we experience as consciousness is just what it feels like to be human-level intelligent. We’re smarter, and “more conscious” than an ape, who is more conscious than a chicken, etc. And an alien much smarter than us would be to us as we are to an ape (or an ant) in every way. We talked about this, and Musk seemed convinced that human-level consciousness is a black-and-white thing—that it’s like a switch that flips on at some point in the evolutionary process and that no other animals share. He doesn’t buy the “ants : humans :: humans : [a much smarter extra-terrestrial]” thing, believing that humans are weak computers and that something smarter than humans would just be a stronger computer, not something so beyond us we couldn’t even fathom its existence.</p>
<p>— <strong>I talked to him for a while about genetic reprogramming.</strong> He doesn’t buy the efficacy of typical anti-aging technology efforts, because he believes humans have general expiration dates, and no one fix can help that. He explained: “The whole system is collapsing. You don’t see someone who’s 90 years old and it’s like, they can run super fast but their eyesight is bad. The <em>whole system</em> is shutting down. In order to change that in a serious way, you need to reprogram the genetics or replace every cell in the body.” Now with anyone else—literally <em>anyone</em> else—I would shrug and agree, since he made a good point. But this was Elon Musk, and Elon Musk <em>fixes shit for humanity.</em> So what did I do?</p>
<p><strong>Me: </strong>Well…but isn’t this important enough to try? Is this something you’d ever turn your attention to?</p>
<p><strong>Elon: </strong>The thing is that all the geneticists have agreed not to reprogram human DNA. So you have to fight not a technical battle but a moral battle.</p>
<p><strong>Me: </strong>You’re fighting a lot of battles. You could set up your own thing. The geneticists who are interested—you bring them here. You create a laboratory, and you could change everything.</p>
<p><strong>Elon:</strong> You know, I call it the Hitler Problem. Hitler was all about creating the Übermensch and genetic purity, and it’s like—how do you avoid the Hitler Problem? I don’t know.</p>
<p><strong>Me: </strong>I think there’s a way. You’ve said before about Henry Ford that he always just found a way around any obstacle, and you do the same thing, you always find a way. And I just think that that’s as important and ambitious a mission as your other things, and I think it’s worth fighting for a way, somehow, around moral issues, around other things.</p>
<p><strong>Elon: </strong>I mean I do think there’s…in order to fundamentally solve a lot of these issues, we are going to have to reprogram our DNA. That’s the only way to do it.</p>
<p><strong>Me: </strong>And deep down, DNA is just a physical material.</p>
<p><strong>Elon: </strong>[Nods, then pauses as he looks over my shoulder in a daze] It’s software.</p>
<p>Comments:</p>
<p>1) It’s really funny to brashly pressure Elon Musk to take on yet another seemingly-insurmountable task and to act a little disappointed in him that he’s not currently doing it, when he’s already doing more for humanity than literally anyone on the planet.</p>
<p>2) It’s also super fun to casually brush off the moral issues around genetic programming with “I think there’s a way” and to refer to <em>DNA—</em>literally the smallest and most complex substance ever—as “just a physical material deep down” when I have absolutely no idea what I’m talking about. Because those things will be his problem to figure out, not mine.</p>
<p>3) I think I’ve successfully planted the seed. If Musk takes on human genetics 15 years from now and we all end up living to 250 because of it, you all owe me a drink.</p>
<address>___________</address>
<p>Watching interviews with Musk, you see a lot of people ask him some variation of this question Chris Anderson <a href="https://www.ted.com/talks/elon_musk_the_mind_behind_tesla_spacex_solarcity#t-1088953">asked him</a> on stage at the 2013 TED conference:</p>
<p><em>How have you done this? These projects—PayPal, SolarCity, Tesla, SpaceX—they’re so spectacularly different. They’re such ambitious projects, at scale. How on Earth has one person been able to innovate in this way—what is it about you? Can we have some of that secret sauce?</em></p>
<p>There are a lot of things about Musk that make him so successful, but I do think there’s a “secret sauce” that puts Musk in a different league from even the other renowned billionaires of our time. I have a theory about what that is, which has to do with the way Musk thinks, the way that he reasons through problems, and the way he views the world. As this series continues, think about this, and we’ll discuss a lot more in the last post.</p>
<p>For now, I’ll leave you with Elon Musk holding a <a href="https://waitbutwhy.com/2013/10/why-procrastinators-procrastinate.html">Panic Monster</a>.</p>
<p><a href="https://waitbutwhy.com/wp-content/uploads/2015/05/IMG_6470.jpg"></a></p>
<p>___________</p>
<p>If you’re into Wait But Why, sign up for the <strong><a href="#">Wait But Why email list</a></strong> and we’ll send you the new posts right when they come out. Better than having to check the site!</p>
<p>If you’re interested in supporting Wait But Why, <strong><a href="https://www.patreon.com/waitbutwhy">here’s our Patreon</a></strong>.</p>
<p>Next up in this series:<strong> Part 2: <a href="https://waitbutwhy.com/2015/06/how-tesla-will-change-your-life.html">How Tesla Will Change the World</a></strong></p>
<p><strong>Other posts in the series:</strong></p>
<p><strong>Part 3:</strong> <strong><a href="https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html">How (and Why) SpaceX Will Colonize Mars<br />
</a>Part 4: <a href="https://waitbutwhy.com/2015/11/the-cook-and-the-chef-musks-secret-sauce.html">The Cook and the Chef: Musk’s Secret Sauce</a></strong></p>
<p><strong>Extra Post #1: <a href="https://waitbutwhy.com/2015/06/the-deal-with-solar.html">The Deal With Solar City</a></strong><br />
<strong> Extra Post #2: <a href="https://waitbutwhy.com/2015/06/hyperloop.html">The Deal With the Hyperloop<br />
</a>Extra Post #3: <a href="https://waitbutwhy.com/2016/09/spacexs-big-fking-rocket-the-full-story.html">SpaceX’s Big Fucking Rocket — The Full Story</a></strong></p>
<p><strong>And a new one, written in 2017, about a whole new Elon company: <a href="https://waitbutwhy.com/neuralink">Neuralink and the Brain’s Magical Future</a></strong></p>
<p><strong>Some Musk-y Wait But Why Posts:</strong></p>
<p><a href="https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html">The AI Revolution: The Road to Superintelligence</a></p>
<p><a href="https://waitbutwhy.com/2014/05/fermi-paradox.html">The Fermi Paradox</a></p>
<p><a href="https://waitbutwhy.com/2014/12/what-makes-you-you.html">What Makes You You?</a></p>
<p><a href="https://gum.co/wbw-musk"></a></p>
<p><strong>Sources</strong></p>
<p>A large part of what I learned for this post came from my own conversations with Musk and his staff. As I mentioned above, Ashlee Vance’s upcoming biography, <em><a href="http://www.amazon.com/gp/product/0062301233/ref=as_li_tl?ie=UTF8&amp;amp;camp=1789&amp;amp;creative=390957&amp;amp;creativeASIN=0062301233&amp;amp;linkCode=as2&amp;amp;tag=wabuwh00-20&amp;amp;linkId=UMV6CN256IWSDQDR">Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future</a>, </em>is excellent and helped me fill in a bunch of gaps. Further info came from the sources below:</p>
<p>Documentary: <a href="http://en.wikipedia.org/wiki/Revenge_of_the_Electric_Car"><em>Revenge of the Electric Car</em></a><br />
TED Talks: <a href="https://www.ted.com/talks/elon_musk_the_mind_behind_tesla_spacex_solarcity#t-1160586"><em>Elon Musk: The mind behind Tesla, SpaceX, SolarCity</em></a><br />
Khan Academy: <a href="https://www.khanacademy.org/talks-and-interviews/khan-academy-living-room-chats/v/elon-musk">Interview With Elon Musk</a><br />
Quora: <em><a href="http://www.quora.com/What-is-it-like-to-work-with-Elon-Musk">What is it like to work with Elon Musk?</a></em><br />
SXSW: <a href="https://www.youtube.com/watch?v=LeQMWdOMa-A">Interview with Elon Musk</a><br />
Consumer Reports: <a href="http://www.consumerreports.org/cro/news/2012/11/tesla-model-s-the-electric-car-that-shatters-every-myth/index.htm"><em>Tesla Model S: The Electric Car that Shatters Every Myth</em></a><br />
Wired: <a href="https://www.youtube.com/watch?v=8_lfxPI5ObM"><em>How the Tesla Model S is Made</em></a><br />
Interview: <a href="https://www.youtube.com/watch?v=MJHTY0gWOGw">Elon Musk says he’s a bigger fan of Edison than Tesla</a><br />
Interview: <a href="https://www.youtube.com/watch?v=I00quUJmM9M&amp;feature=youtu.be">Elon Musk gets introspective</a><br />
Business Insider: <a href="http://www.businessinsider.com/how-elon-musk-learned-rocket-science-for-spacex-2014-10"><em>Former SpaceX Exec Explains How Elon Musk Taught Himself Rocket Science</em></a><br />
Esquire: <a href="http://www.esquire.com/news-politics/a16681/elon-musk-interview-1212/"><em>Elon Musk: The Triumph of His Will</em></a><br />
Oxford Martin School: <a href="https://www.youtube.com/watch?v=c1HZIQliuoA">Elon Musk on The Future of Energy and Transport</a><br />
MIT Interview: <a href="https://www.youtube.com/watch?v=JfJjx12wkVQ">Elon Musk compares AI efforts to “Summoning the Demon”</a><br />
Documentary: <em><a href="https://www.youtube.com/watch?v=FkMuc8pXiTI">Billionaire Elon Musk : How I Became The Real ‘Iron Man’</a></em><br />
Reddit: <a href="https://www.reddit.com/r/IAmA/comments/2rgsan/i_am_elon_musk_ceocto_of_a_rocket_company_ama">Elon Musk AMA</a><br />
Chris Anderson: <a href="https://www.youtube.com/watch?v=tUMZTtQU10o">Chris Anderson on Elon Musk, the World’s Most Remarkable Entrepreneur</a><br />
Engineering.com: <a href="https://www.youtube.com/watch?v=g9FD1UE6E1g">Who’s Better? Engineers or Scientists?</a><br />
Forbes: <a href="http://www.forbes.com/sites/hannahelliott/2012/10/10/elon-musk-to-maye-musk-mom-i-havent-started-yet/"><em>Big Day For SpaceX As Elon Musk Tells His Mom ‘I Haven’t Started Yet’</em></a></p>
</div>||||I|||| * Home

 Menu
 * Homepage
 * about
 + wait but who
 + faq
 + contact
 * archive
 * minis
 * the shed
 * dinner table
 * store
 + store home
 + new releases
 + posters
 + phone cases
 + cards & wrapping paper
 + squishy things
 + men’s tees
 + women’s tees
 + coffee mugs
 + store support
 * support wbw
 * #7246 (no title)
 * Finn’s Cave
 * 

 Elon Musk: The World’s Raddest Man

 May 7, 2015 By Tim Urban


 This is Part 1 of a four-part series on Elon Musk’s companies.

 __________

 PDF and ebook options: We made a fancy PDF of this post for printing and offline viewing (see a preview here), and an ebook containing the whole four-part Elon Musk series:

 ___________

 Last month, I got a surprising phone call.

 Elon Musk, for those unfamiliar, is the world’s raddest man.

 I’ll use this post to explore how he became a self-made billionaire and the real-life inspiration for Iron Man’s Tony Stark, but for the moment, I’ll let Richard Branson explain things briefly:1

 Whatever skeptics have said can’t be done, Elon has gone out and made real. Remember in the 1990s, when we would call strangers and give them our credit-card numbers? Elon dreamed up a little thing called PayPal. His Tesla Motors and SolarCity companies are making a clean, renewable-energy future a reality…his SpaceX [is] reopening space for exploration…it’s a paradox that Elon is working to improve our planet at the same time he’s building spacecraft to help us leave it.

 So no, that was not a phone call I had been expecting.

 A few days later, I found myself in pajama pants, pacing frantically around my apartment, on the phone with Elon Musk. We had a discussion about Tesla, SpaceX, the automotive and aerospace and solar power industries, and he told me what he thought confused people about each of these things. He suggested that if these were topics I’d be interested in writing about, and it might be helpful, I could come out to California and sit down with him in person for a longer discussion.

 For me, this project was one of the biggest no-brainers in history. Not just because Elon Musk is Elon Musk, but because here are two separate items that have been sitting for a while in my “Future Post Topics” document, verbatim:

 – “electric vs hybrid vs gas cars, deal with tesla, sustainable energy”

 – “spacex, musk, mars?? how learn to do rockets??”

 I already wanted to write about these topics, for the same reason I wrote about Artificial Intelligence—I knew they would be hugely important in the future but that I also didn’t understand them well enough. And Musk is leading a revolution in both of these worlds.

 It would be like if you had plans to write about the process of throwing lightning bolts and then one day out of the blue Zeus called and asked if you wanted to question him about a lot of stuff.

 So it was on. The plan was that I’d come out to California, see the Tesla and SpaceX factories, meet with some of the engineers at each company, and have an extended sit down with Musk. Exciting.

 The first order of business was to have a full panic. I needed to not sit down with these people—these world-class engineers and rocket scientists—and know almost nothing about anything. I had a lot of quick learning to do.

 The problem with Elon Fucking Musk, though, is that he happens to be involved in all of the following industries:

 * Automotive
 * Aerospace
 * Solar Energy
 * Energy Storage
 * Satellite
 * High-Speed Ground Transportation
 * And, um, Multi-Planetary Expansion

 Zeus would have been less stressful.

 So I spent the two weeks leading up to the West Coast visit reading and reading and reading, and it became quickly clear that this was gonna need to be a multi-post series. There’s a lot to get into.

 We’ll dive deep into Musk’s companies and the industries surrounding them in the coming posts, but today, let’s start by going over exactly who this dude is and why he’s such a big deal.12← click these

 The Making of Elon Musk

 Note: There’s a great biography on Musk coming out May 19th, written by tech writer Ashlee Vance. I was able to get an advance copy, and it’s been a key source in putting together these posts. I’m going to keep to a brief overview of his life here—if you want the full story, get the bio.

 Musk was born in 1971 in South Africa. Childhood wasn’t a great time for him—he had a tough family life and never fit in well at school.2 But, like you often read in the bios of extraordinary people, he was an avid self-learner early on. His brother Kimbal has said Elon would often read for 10 hours a day—a lot of science fiction and eventually, a lot of non-fiction too. By fourth grade, he was constantly buried in the Encyclopedia Britannica.

 One thing you’ll learn about Musk as you read these posts is that he thinks of humans as computers, which, in their most literal sense, they are. A human’s hardware is his physical body and brain. His software is the way he learns to think, his value system, his habits, his personality. And learning, for Musk, is simply the process of “downloading data and algorithms into your brain.”3 Among his many frustrations with formal classroom learning is the “ridiculously slow download speed” of sitting in a classroom while a teacher explains something, and to this day, most of what he knows he’s learned through reading.

 He became consumed with a second fixation at the age of nine when he got his hands on his first computer, the Commodore VIC-20. It came with five kilobytes of memory and a “how to program” guide that was intended to take the user six months to complete. Nine-year-old Elon finished it in three days. At 12, he used his skills to create a video game called Blastar, which he told me was “a trivial game…but better than Flappy Bird.” But in 1983, it was good enough to be sold to a computer magazine for $500 ($1,200 in today’s money)—not bad for a 12-year-old.3

 Musk never felt much of a connection to South Africa—he didn’t fit in with the jockish, white Afrikaner culture, and it was a nightmare country for a potential entrepreneur. He saw Silicon Valley as the Promised Land, and at the age of 17, he left South Africa forever. He started out in Canada, which was an easier place to immigrate to because his mom is a Canadian citizen, and a few years later, used a college transfer to the University of Pennsylvania as a way into the US.4

 In college, he thought about what he wanted to do with his life, using as his starting point the question, “What will most affect the future of humanity?” The answer he came up with was a list of five things: “the internet; sustainable energy; space exploration, in particular the permanent extension of life beyond Earth; artificial intelligence; and reprogramming the human genetic code.”4

 He was iffy about how positive the impact of the latter two would be, and though he was optimistic about each of the first three, he never considered at the time that he’d ever be involved in space exploration. That left the internet and sustainable energy as his options.

 He decided to go with sustainable energy. After finishing college, he enrolled in a Stanford PhD program to study high energy density capacitors, a technology aimed at coming up with a more efficient way than traditional batteries to store energy—which he knew could be key to a sustainable energy future and help accelerate the advent of an electric car industry.

 But two days into the program, he got massive FOMO because it was 1995 and he “couldn’t stand to just watch the internet go by—[he] wanted to jump in and make it better.”5 So he dropped out and decided to try the internet instead.

 His first move was to go try to get a job at the monster of the 1995 internet, Netscape. The tactic he came up with was to walk into the lobby, uninvited, stand there awkwardly, be too shy to talk to anyone, and walk out.

 Musk bounced back from the unimpressive career beginning by teaming up with his brother Kimbal (who had followed Elon to the US) to start their own company—Zip2. Zip2 was like a primitive combination of Yelp and Google Maps, far before anything like either of those existed. The goal was to get businesses to realize that being in the Yellow Pages would become outdated at some point and that it was a good idea to get themselves into an online directory. The brothers had no money, slept in the office and showered at the YMCA, and Elon, their lead programmer, sat obsessively at his computer working around the clock. In 1995, it was hard to convince businesses that the internet was important—many told them that advertising on the internet sounded like “the dumbest thing they had ever heard of”6—but eventually, they began to rack up customers and the company grew. It was the heat of the 90s internet boom, startup companies were being snatched up left and right, and in 1999, Compaq snatched up Zip2 for $307 million. Musk, who was 27, made off with $22 million.

 In what would become a recurring theme for Musk, he finished one venture and immediately dove into a new, harder, more complex one. If he were following the dot-com millionaire rulebook, he’d have known that what you’re supposed to do after hitting it big during the 90s boom is either retire off into the sunset of leisure and angel investing, or if you still have ambition, start a new company with someone else’s money. But Musk doesn’t tend to follow normal rulebooks, and he plunged three quarters of his net worth into his new idea, an outrageously bold plan to build essentially an online bank—replete with checking, savings, and brokerage accounts—called X.com. This seems less insane now, but in 1999, an internet startup trying to compete with the large banks was unheard of.

 In the same building that X.com worked out of was another internet finance company called Confinity, founded by Peter Thiel and Max Levchin. One of X.com’s many features was an easy money-transfer service, and later, Confinity would develop a similar service. Both companies began to notice a strong demand for their money-transfer service, which put the two companies in sudden furious competition with each other, and they finally decided to just merge into what we know today as PayPal.

 This brought together a lot of egos and conflicting opinions—Musk was now joined by Peter Thiel and a bunch of other now-super-successful internet guys—and despite the company growing rapidly, things inside the office did not go smoothly. The conflicts boiled over in late 2000, and when Musk was on a half fundraising trip / half honeymoon (with his first wife Justine), the anti-Musk crowd staged a coup and replaced him as CEO with Thiel. Musk handled this surprisingly well, and to this day, he says he doesn’t agree with that decision but he understands why they did it. He stayed on the team in a senior role, continued investing in the company, and played an instrumental role in selling the company to eBay in 2002, for $1.5 billion. Musk, the company’s largest shareholder, walked away with $180 million (after taxes).5

 If there was ever a semblance of the normal life rulebook in Musk’s decision-making, it was at this point in his life—as a beyond-wealthy 31-year-old in 2002—that he dropped the rulebook into the fire for good.

 The subject of what he did over the next 13 years leading up to today is what we’ll thoroughly explore over the rest of this series. For now, here’s the short story:

 In 2002, before the sale of PayPal even went through, Musk started voraciously reading about rocket technology, and later that year, with $100 million, he started one of the most unthinkable and ill-advised ventures of all time: a rocket company called SpaceX, whose stated purpose was to revolutionize the cost of space travel in order to make humans a multi-planetary species by colonizing Mars with at least a million people over the next century.

 Mm hm.

 Then, in 2004, as that “project” was just getting going, Musk decided to multi-task by launching the second-most unthinkable and ill-advised venture of all time: an electric car company called Tesla, whose stated purpose was to revolutionize the worldwide car industry by significantly accelerating the advent of a mostly-electric-car world—in order to bring humanity on a huge leap toward a sustainable energy future. Musk funded this one personally as well, pouring in $70 million, despite the tiny fact that the last time a US car startup succeeded was Chrysler in 1925, and the last time someone started a successful electric car startup was never.

 And since why the fuck not, a couple years later, in 2006, he threw in $10 million to found, with his cousins, another company, called SolarCity, whose goal was to revolutionize energy production by creating a large, distributed utility that would install solar panel systems on millions of people’s homes, dramatically reducing their consumption of fossil fuel-generated electricity and ultimately “accelerating mass adoption of sustainable energy.”7

 If you were observing all of this in those four years following the PayPal sale, you’d think it was a sad story. A delusional internet millionaire, comically in over his head with a slew of impossible projects, doing everything he could to squander his fortune.

 By 2008, this seemed to be playing out, to the letter. SpaceX had figured out how to build rockets, just not rockets that actually worked—it had attempted three launches so far and all three had blown up before reaching orbit. In order to bring in any serious outside investment or payload contracts, SpaceX had to show that they could successfully launch a rocket—but Musk said he had funds left for one and only one more launch. If the fourth launch also failed, SpaceX would be done.

 Meanwhile, up in the Bay Area, Tesla was also in the shit. They had yet to deliver their first car—the Tesla Roadster—to the market, which didn’t look good to the outside world. Silicon Valley gossip blog Valleywag made the Tesla Roadster its #1 tech company fail of 2007. This would have been more okay if the global economy hadn’t suddenly crashed, hitting the automotive industry the absolute hardest and sucking dry any flow of investments into car companies, especially new and unproven ones. And Tesla was running out of money fast.

 During this double implosion of his career, the one thing that held stable and strong in Musk’s life was his marriage of eight years, if by stable and strong you mean falling apart entirely in a soul-crushing, messy divorce.

 Darkness.

 But here’s the thing—Musk is not a fool, and he hadn’t built bad companies. He had built very, very good companies. It’s just that creating a reliable rocket is unfathomably difficult, as is launching a startup car company, and because no one wanted to invest in what seemed to the outside world like overambitious and probably-doomed ventures—especially during a recession—Musk had to rely on his own personal funds. PayPal made him rich, but not rich enough to keep these companies afloat for very long on his own. Without outside money, both SpaceX and Tesla had a short runway. So it’s not that SpaceX and Tesla were bad—it’s that they needed more time to succeed, and they were out of time.

 And then, in the most dire hour, everything turned around.

 First, in September of 2008, SpaceX launched their fourth rocket—and their last one if it didn’t successfully put a payload into orbit—and it succeeded. Perfectly.

 That was enough for NASA to say “fuck it, let’s give this Musk guy a try,” and it took a gamble, offering SpaceX a $1.6 billion contract to carry out 12 launches for the agency. Runway extended. SpaceX saved.

 The next day, on Christmas Eve 2008, when Musk scrounged up the last money he could manage to keep Tesla going, Tesla’s investors reluctantly agreed to match his investment. Runway extended. Five months later, things began looking up, and another critical investment came in—$50 million from Daimler. Tesla saved.

 While 2008 hardly marked the end of the bumps in the road for Musk, the overarching story of the next seven years would be the soaring, earthshaking success of Elon Musk and his companies.

 Since their first three failed launches, SpaceX has launched 20 times—all successes. NASA is now a regular client, and one of many, since the innovations at SpaceX have allowed companies to launch things to space for the lowest cost in history. Within those 20 launches have been all kinds of “firsts” for a commercial rocket company—to this day, the four entities in history who have managed to launch a spacecraft into orbit and successfully return it to Earth are the US, Russia, China—and SpaceX. SpaceX is currently testing their new spacecraft, which will bring humans to space, and they’re busy at work on the much larger rocket that will be able to bring 100 people to Mars at once. A recent investment by Google and Fidelity has valued the company at $12 billion.

 Tesla’s Model S has become a smashing success, blowing away the automotive industry with the highest ever Consumer Reports rating of a 99/100, and the highest safety rating in history from the National Highway Safety Administration, a 5.4/5. Now they’re getting closer and closer to releasing their true disruptor—the much more affordable Model 3—and the company’s market cap is just under $30 billion. They’re also becoming the world’s most formidable battery company, currently working on their giant Nevada “Gigafactory,” which will more than double the world’s total annual production of lithium-ion batteries.

 SolarCity, which went public in 2012, now has a market cap of just under $6 billion and has become the largest installer of solar panels in the US. They’re now building the country’s largest solar panel-manufacturing factory in Buffalo, and they’ll likely be entering into a partnership with Tesla to package their product with Tesla’s new home battery, the Powerwall.

 And since that’s not enough, in his spare time, Musk is pushing the development a whole new mode of transport—the Hyperloop.

 In a couple of years, when their newest factories are complete, Musk’s three companies will employ over 30,000 people. After nearly going broke in 2008 and telling a friend that he and his wife may have to “move into his wife’s parents’ basement,”8 Musk’s current net worth clocks in at $12.9 billion.

 All of this has made Musk somewhat of a living legend. In building a successful automotive startup and its worldwide network of Supercharger stations, Musk has been compared to visionary industrialists like Henry Ford and John D. Rockefeller. The pioneering work of SpaceX on rocket technology has led to comparisons to Howard Hughes, and many have drawn parallels between Musk and Thomas Edison because of the advancements in engineering Musk has been able to achieve across industries. Perhaps most often, he’s compared to Steve Jobs, for his remarkable ability to disrupt giant, long-stagnant industries with things customers didn’t even know they wanted. Some believe he’ll be remembered in a class of his own. Tech writer and Musk biographer Ashlee Vance has suggested that what Musk is building “has the potential to be much grander than anything Hughes or Jobs produced. Musk has taken industries like aerospace and automotive that America seemed to have given up on and recast them as something new and fantastic.”9

 Chris Anderson, who runs TED Talks, calls Musk “the world’s most remarkable living entrepreneur.” Others know him as “the real life Iron Man,” and not for no reason—Jon Favreau actually sent Robert Downey, Jr. to spend time with Musk in the SpaceX factory prior to filming the first Iron Man movie so he could model his character off of Musk.10 He’s even been on The Simpsons.

 And this is the man I was somehow on the phone with as I frantically paced back and forth in my apartment, in pajama pants.

 On the call, he made it clear that he wasn’t looking for me to advertise his companies—he only wanted me to help explain what’s going on in the worlds surrounding those companies and why the things happening with electric cars, sustainable energy production, and aerospace matter so much.

 He seemed particularly bored with people spending time writing about him—he feels there are so many things of critical importance going on in the industries he’s involved in, and every time someone writes about him, he wishes they were writing about fossil fuel supply or battery advancements or the importance of making humanity multi-planetary (this is especially clear in the intro to the upcoming biography on him, when the author explains how not interested Musk was in having a bio written about him).

 So I’m sure this first post, whose title is “Elon Musk: The World’s Raddest Man,” will annoy him.

 But I have reasons. To me, there are two worthy areas of exploration in this post series:

 1) To understand why Musk is doing what he’s doing. He deeply believes that he’s taken on the most pressing possible causes to give humanity the best chance of a good future. I want to explore those causes in depth and the reasons he’s so concerned about them.

 2) To understand why Musk is able to do what he’s doing. There are a few people in each generation who dramatically change the world, and those people are worth studying. They do things differently from everyone else—and I think there’s a lot to learn from them.

 So on my visit to California, I had two goals in mind: to understand as best I could what Musk and his teams were working on so feverishly and why it mattered so much, and to try to gain insight into what it is that makes him so capable of changing the world.

 ___________

 Visiting the Factories

 The Tesla Factory (in Northern CA) and the SpaceX Factory (in Southern CA), in addition to both being huge, and rad, have a lot in common.

 Both factories are bright and clean, shiny and painted white, with super high ceilings. Both feel more like laboratories than traditional factories. And in both places, the engineers doing white collar jobs and the technicians doing blue collar jobs are deliberately placed in the same working quarters so they’ll work closely together and give each other feedback—and Musk believes it’s crucial for those designing the machines to be around those machines as they’re being manufactured. And while a traditional factory environment wouldn’t be ideal for an engineer on a computer and a traditional office environment wouldn’t be a good workplace for a technician, a clean, futuristic laboratory feels right for both professions. There are almost no closed offices in either factory—everyone is out in the open, exposed to everyone else.

 When I pulled up to the Tesla factory (joined by Andrew), I was first taken by its size—and when I looked it up, I wasn’t surprised to see that it has the second-largest building footprint (aka base area) in the U.S.

 The factory was formerly jointly owned by GM and Toyota, who sold it to Tesla in 2010. We started off the day with a full tour of the factory—a sea of red robots making cars and being silly:6

 And other cool things, like a vast section of the factory that just makes the car battery, and another that houses the 20,000 pound rolls of aluminum they slice and press and weld into Teslas.

 And this giant press, which costs $50 million and presses metal with 4,500 tons of pressure (the same pressure you’d get if you stacked 2,500 cars on top of something).

 The Tesla factory is working on upping its output from 30,000 cars/year to 50,000, or about 1,000 per week. They seemed to be pumping out cars incredibly quickly, so I was blown away to learn that Toyota had been on a 1,000 cars per day clip when they inhabited the factory.

 I had a chance to visit the Tesla design studio (no pictures allowed), where there were designers sketching car designs on computer screens and, on the other side of the room, full-size car models made of clay. An actual-size clay version of the upcoming Model 3 was surrounded by specialists sculpting it with tiny instruments and blades, shaving off fractions of a millimeter to examine the way light bounced off the curves. There was also a 3D printer that could quickly “print” out a shoe-sized 3D model of a sketched Tesla design so a designer could actually hold their design and look at it from different angles. Deliciously futuristic.

 The next day was the SpaceX factory, which might be even cooler, but the building contains advanced rocket technology, which according to the government is “weapons technology,” and apparently random bloggers aren’t allowed to take pictures of weapons technology.

 Anyway, after the tours, I had a chance to sit down with several senior engineers and designers at both companies. They’d explain that they were a foremost expert in their field, I’d explain that I had recently figured out how big the building would be that could hold all humans, and we’d begin our discussion. I’d ask them about their work, their thoughts on the company as a whole and the broader industry, and then I’d ask them about their relationship with Elon and what it was like to work for him. Without exception, they were really nice-seeming, friendly people, who all came off as ridiculously smart but in a non-pretentious way. Musk has said he has a strict “no assholes” hiring policy, and I could see that at work in these meetings.

 So what’s Musk like as a boss?

 Let’s start by seeing what the internet says—there’s a Quora thread that poses the question: “What is it like to work with Elon Musk?”

 The first answer is from a longtime SpaceX employee who no longer works there, who describes the day that their 3rd launch failed, a devastating blow for the company and for all the people who had worked for years to try to make it work.

 She describes Elon emerging from mission command to address the company and delivering a rousing speech. She refers to Elon’s “infinite wisdom” and says, “I think most of us would have followed him into the gates of hell carrying suntan oil after that. It was the most impressive display of leadership that I have ever witnessed.”

 Right below that answer is another answer, from an anonymous SpaceX engineer, who describes working for Musk like this:

 “You can always tell when someone’s left an Elon meeting: they’re defeated…nothing you ever do will be good enough so you have to find your own value, not depending on praise to get you through your obviously insufficient 80 hour work weeks.”

 Reading about Musk online and in Vance’s book, I was struck by how representative both of these Quora comments were of whole camps of opinion on working for Musk. Doing so seems to bring out a tremendous amount of adoration and a tremendous amount of exasperation, sometimes with a tone of bitterness—and even more oddly, much of the time, you hear both sides of this story expressed by the same person. For example, later in the comment of the effusive Quora commenter comes “Working with him isn’t a comfortable experience, he is never satisfied with himself so he is never really satisfied with anyone around him…the challenge is that he is a machine and the rest of us aren’t.” And the frustrated anonymous commenter later concedes that the way Elon is “is understandable” given the enormity of the task at hand, and that “it is a great company and I do love it.”

 My own talks with Musk’s engineers and designers told a similar story. I was told: “Elon always wants to know, ‘Why are we not going faster?’ He always wants bigger, better, faster” by the same person who a few minutes later was emphasizing how fair and thoughtful Musk tends to be in handling the terms for a recently fired employee.

 The same person who told me he has “lots of sleepless nights” said in the adjacent sentence how happy he is to be at the company and that he hopes to “never leave.”

 One senior executive described interacting with Musk like this: “Any conversation’s fairly high stakes because he’ll be very opinionated, and he can go deeper than you expect or are prepared for or deeper than your knowledge goes on a given topic, and it does feel like a high wire act interacting with him, especially when you find yourself in a [gulp] technical disagreement.”7 The same executive, who had previously worked at a huge tech company, also called Musk “the most grounded billionaire I’ve ever worked with.”

 What I began to understand is that the explanation for both sides of the story—the cult-like adulation right alongside the grudging willingness to endure what sounds like blatant hell—comes down to respect. The people who work for Musk, no matter how they feel about his management style, feel an immense amount of respect—for his intelligence, for his work ethic, for his guts, and for the gravity of the missions he’s undertaken, missions that make all other potential jobs seem trivial and pointless.

 Many of the people I talked to also alluded to their respect for his integrity. One way this integrity comes through is in his consistency. He’s been saying the same things in interviews for a decade, often using the same exact phrasing many years apart. He says what he really means, no matter the situation—one employee close to Musk told me that after a press conference or a business negotiation, once in private he’d ask Musk what his real angle was and what he really thinks. Musk’s response would always be boring: “I think exactly what I said.”

 A few people I spoke with referenced Musk’s obsession with truth and accuracy. He’s fine with and even welcoming of negative criticism about him when he believes it’s accurate, but when the press gets something wrong about him or his companies, he usually can’t help himself and will engage them and correct their error. He detests vague spin-doctor phrases like “studies say” and “scientists disagree,” and he refuses to advertise for Tesla, something most startup car companies wouldn’t think twice about—because he sees advertising as manipulative and dishonest.

 There’s even an undertone of integrity in Musk’s tyrannical demands of workers, because while he may be a tyrant, he’s not a hypocrite. Employees pressured to work 80 hours a week tend to be less bitter about it when at least the CEO is in there working 100.

 Speaking of the CEO, let’s go have a hamburger with him.

 My Lunch With Elon

 It started like this:


 After about seven minutes of this, I was able to get out my first question, a smalltalk-y question about how he thought the recent launch had gone (they had attempted an extremely difficult rocket-landing maneuver—more on that in the SpaceX post). His response included the following words: hypersonic, rarefied, densifying, supersonic, Mach 1, Mach 3, Mach 4, Mach 5, vacuum, regimes, thrusters, nitrogen, helium, mass, momentum, ballistic, and boost-back. While this was happening, I was still mostly blacked out from the surreality of the situation, and when I started to come to, I was scared to ask any questions about what he was saying in case he had already explained it while I was unconscious.

 I eventually regained the ability to have adult human conversation, and we began what turned into a highly interesting and engaging two-hour discussion.8 This guy has a lot on his mind across a lot of topics. In this one lunch alone, we covered electric cars, climate change, artificial intelligence, the Fermi Paradox, consciousness, reusable rockets, colonizing Mars, creating an atmosphere on Mars, voting on Mars, genetic programming, his kids, population decline, physics vs. engineering, Edison vs. Tesla, solar power, a carbon tax, the definition of a company, warping spacetime and how this isn’t actually something you can do, nanobots in your bloodstream and how this isn’t actually something you can do, Galileo, Shakespeare, the American forefathers, Henry Ford, Isaac Newton, satellites, and ice ages.

 I’ll get into the specifics of what he had to say about many of these things in later posts, but some notes for now:

 — He’s a pretty tall and burly dude. Doesn’t really come through on camera.

 — He ordered a burger and ate it in either two or three bites over a span of about 15 seconds. I’ve never seen anything like it.

 — He is very, very concerned about AI. I quoted him in my posts on AI saying that he fears that by working to bring about Superintelligent AI (ASI), we’re “summoning the demon,” but I didn’t know how much he thought about the topic. He cited AI safety as one of the three things he thinks about most—the other two being sustainable energy and becoming a multi-planet species, i.e. Tesla and SpaceX. Musk is a smart motherfucker, and he knows a ton about AI, and his sincere concern about this makes me scared.

 — The Fermi Paradox also worries him. In my post on that, I divided Fermi thinkers into two camps—those who think there’s no other highly intelligent life out there at all because of some Great Filter, and those who believe there must be plenty of intelligent life and that we don’t see signs of any for some other reason. Musk wasn’t sure which camp seemed more likely, but he suspects that there may be an upsetting Great Filter situation going on. He thinks the paradox “just doesn’t make sense” and that it “gets more and more worrying” the more time that goes by. Considering the possibility that maybe we’re a rare civilization who made it past the Great Filter through a freak occurrence makes him feel even more conviction about SpaceX’s mission: “If we are very rare, we better get to the multi-planet situation fast, because if civilization is tenuous, then we must do whatever we can to ensure that our already-weak probability of surviving is improved dramatically.” Again, his fear here makes me feel not great.

 — One topic I disagreed with him on is the nature of consciousness. I think of consciousness as a smooth spectrum. To me, what we experience as consciousness is just what it feels like to be human-level intelligent. We’re smarter, and “more conscious” than an ape, who is more conscious than a chicken, etc. And an alien much smarter than us would be to us as we are to an ape (or an ant) in every way. We talked about this, and Musk seemed convinced that human-level consciousness is a black-and-white thing—that it’s like a switch that flips on at some point in the evolutionary process and that no other animals share. He doesn’t buy the “ants : humans :: humans : [a much smarter extra-terrestrial]” thing, believing that humans are weak computers and that something smarter than humans would just be a stronger computer, not something so beyond us we couldn’t even fathom its existence.

 — I talked to him for a while about genetic reprogramming. He doesn’t buy the efficacy of typical anti-aging technology efforts, because he believes humans have general expiration dates, and no one fix can help that. He explained: “The whole system is collapsing. You don’t see someone who’s 90 years old and it’s like, they can run super fast but their eyesight is bad. The whole system is shutting down. In order to change that in a serious way, you need to reprogram the genetics or replace every cell in the body.” Now with anyone else—literally anyone else—I would shrug and agree, since he made a good point. But this was Elon Musk, and Elon Musk fixes shit for humanity. So what did I do?

 Me: Well…but isn’t this important enough to try? Is this something you’d ever turn your attention to?

 Elon: The thing is that all the geneticists have agreed not to reprogram human DNA. So you have to fight not a technical battle but a moral battle.

 Me: You’re fighting a lot of battles. You could set up your own thing. The geneticists who are interested—you bring them here. You create a laboratory, and you could change everything.

 Elon: You know, I call it the Hitler Problem. Hitler was all about creating the Übermensch and genetic purity, and it’s like—how do you avoid the Hitler Problem? I don’t know.

 Me: I think there’s a way. You’ve said before about Henry Ford that he always just found a way around any obstacle, and you do the same thing, you always find a way. And I just think that that’s as important and ambitious a mission as your other things, and I think it’s worth fighting for a way, somehow, around moral issues, around other things.

 Elon: I mean I do think there’s…in order to fundamentally solve a lot of these issues, we are going to have to reprogram our DNA. That’s the only way to do it.

 Me: And deep down, DNA is just a physical material.

 Elon: [Nods, then pauses as he looks over my shoulder in a daze] It’s software.

 Comments:

 1) It’s really funny to brashly pressure Elon Musk to take on yet another seemingly-insurmountable task and to act a little disappointed in him that he’s not currently doing it, when he’s already doing more for humanity than literally anyone on the planet.

 2) It’s also super fun to casually brush off the moral issues around genetic programming with “I think there’s a way” and to refer to DNA—literally the smallest and most complex substance ever—as “just a physical material deep down” when I have absolutely no idea what I’m talking about. Because those things will be his problem to figure out, not mine.

 3) I think I’ve successfully planted the seed. If Musk takes on human genetics 15 years from now and we all end up living to 250 because of it, you all owe me a drink.

 ___________

 Watching interviews with Musk, you see a lot of people ask him some variation of this question Chris Anderson asked him on stage at the 2013 TED conference:

 How have you done this? These projects—PayPal, SolarCity, Tesla, SpaceX—they’re so spectacularly different. They’re such ambitious projects, at scale. How on Earth has one person been able to innovate in this way—what is it about you? Can we have some of that secret sauce?

 There are a lot of things about Musk that make him so successful, but I do think there’s a “secret sauce” that puts Musk in a different league from even the other renowned billionaires of our time. I have a theory about what that is, which has to do with the way Musk thinks, the way that he reasons through problems, and the way he views the world. As this series continues, think about this, and we’ll discuss a lot more in the last post.

 For now, I’ll leave you with Elon Musk holding a Panic Monster.

 ___________

 If you’re into Wait But Why, sign up for the Wait But Why email list and we’ll send you the new posts right when they come out. Better than having to check the site!

 If you’re interested in supporting Wait But Why, here’s our Patreon.

 Next up in this series: Part 2: How Tesla Will Change the World

 Other posts in the series:

 Part 3: How (and Why) SpaceX Will Colonize Mars
 Part 4: The Cook and the Chef: Musk’s Secret Sauce

 Extra Post #1: The Deal With Solar City
 Extra Post #2: The Deal With the Hyperloop
 Extra Post #3: SpaceX’s Big Fucking Rocket — The Full Story

 And a new one, written in 2017, about a whole new Elon company: Neuralink and the Brain’s Magical Future

 Some Musk-y Wait But Why Posts:

 The AI Revolution: The Road to Superintelligence

 The Fermi Paradox

 What Makes You You?

 Sources

 A large part of what I learned for this post came from my own conversations with Musk and his staff. As I mentioned above, Ashlee Vance’s upcoming biography, Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future, is excellent and helped me fill in a bunch of gaps. Further info came from the sources below:

 Documentary: Revenge of the Electric Car
 TED Talks: Elon Musk: The mind behind Tesla, SpaceX, SolarCity
 Khan Academy: Interview With Elon Musk
 Quora: What is it like to work with Elon Musk?
 SXSW: Interview with Elon Musk
 Consumer Reports: Tesla Model S: The Electric Car that Shatters Every Myth
 Wired: How the Tesla Model S is Made
 Interview: Elon Musk says he’s a bigger fan of Edison than Tesla
 Interview: Elon Musk gets introspective
 Business Insider: Former SpaceX Exec Explains How Elon Musk Taught Himself Rocket Science
 Esquire: Elon Musk: The Triumph of His Will
 Oxford Martin School: Elon Musk on The Future of Energy and Transport
 MIT Interview: Elon Musk compares AI efforts to “Summoning the Demon”
 Documentary: Billionaire Elon Musk : How I Became The Real ‘Iron Man’
 Reddit: Elon Musk AMA
 Chris Anderson: Chris Anderson on Elon Musk, the World’s Most Remarkable Entrepreneur
 Engineering.com: Who’s Better? Engineers or Scientists?
 Forbes: Big Day For SpaceX As Elon Musk Tells His Mom ‘I Haven’t Started Yet’

 1. Thank you for following instructions. I came across much more in my research than I have room to fit in these posts, so I’ll tuck extra tidbits and related thoughts into these blue circle footnotes throughout the post. Click these if you have time.↩

 2. He was badly bullied in his early teens, including one particularly traumatic incident in which a group of guys who constantly picked on Elon attacked him in full force one day, pushing him down a flight of stairs and then beating him unconscious. He has breathing problems to this day because of the injuries.↩

 3. He first became enamored with computers and video games during a trip to the US he accompanied his father on when he was a little kid and all the hotels they stayed in had arcades—this was also when he first became enamored with America.↩

 4. As an experiment, he lived for a while on $1/day during college, eating mostly hot dogs.↩

 5. Musk and the PayPal team stayed on good terms, for the most part, and a number of them have since invested in Musk’s later companies.↩

 6. Here’s a cool video of the robots in action.↩

 7. He didn’t actually gulp.↩

 8. I did an odd but kind of a hilarious thing and fucked with him at the very beginning. I knew from watching interviews with him the certain things he absolutely hates being asked about because he thinks the topics are impossibly stupid and impractical. I picked the three that seemed to bother him most, and right in the beginning of the interview, said: “So by the way, since we spoke on the phone, I’ve altered the plan a bit, and I’m going to focus on three main things in these posts: hydrogen fuel cells, solar panels in space, and the space elevator.” He looked at me with horrified disappointment and after a pause, said, “Really??” Then I told him I was just messing with him and he exhaled hugely and said, “Oh thank god.” Fun.↩

 1. http://time100.time.com/2013/04/18/time-100/slide/elon-musk/↩

 2. Gray square footnotes are just for sources. Only bother with these if you want to see the source of a quote or fact or read more about the topic.↩

 3. https://www.youtube.com/watch?v=vDwzmJpI4io&feature=youtu.be&t=35m49s↩

 4. http://www.startalkradio.net/show/the-future-of-humanity-with-elon-musk/↩

 5. https://www.khanacademy.org/talks-and-interviews/khan-academy-living-room-chats/v/elon-musk↩

 6. Ashlee Vance, Elon Musk, 64.↩

 7. http://www.greenoptimistic.com/solarcity-acquires-silevo-plans-one-largest-solar-panel-factories-world-20140618/#.VUmgnNNViko↩

 8. Ashlee Vance, Elon Musk, 189 ↩

 9. Ashlee Vance, Elon Musk, 21.↩

 10. Ashlee Vance, Elon Musk, 181.↩

 Tweet

 About Tim Urban

 View all posts by Tim Urban →
 Previous Post
 Next Post

 RECOMMENDED POSTS

 * The Big and the Small

 September 22, 2020

 * A Sick Giant

 January 8, 2020

 * Idea Labs and Echo Chambers

 October 8, 2019
 <!­­-- END TAG ­­-->
 Home Archive

 Join 608,367 other humans and have new posts emailed to you.

 Follow these special men
 412,637
 731,774
 130,240

 * Popular Posts

 * The AI Revolution

 January 22, 2015 1,107

 * Why Procrastinators Procrastinate

 October 30, 2013 566

 * The Fermi Paradox

 May 21, 2014 1,391

 * The Elon Musk Series

 May 7, 2015 511

 * How to Pick a Life Partner

 February 12, 2014 341

 * Why Gen Y Yuppies Are Unhappy

 September 9, 2013 1,197

 * 7 Ways to be Insufferable on Facebook

 July 8, 2013 973

 * Putting Time In Perspective

 August 22, 2013 559

 * How to Name a Baby

 December 11, 2013 327

 * Why You Should Stop Caring What Other People Think

 June 13, 2014 351

 * Religion for the Nonreligious

 October 19, 2014 830

 * 10 Types of 30-Year-Old Single Guys

 October 9, 2013 661

 * From Muhammad to ISIS: Iraq's Full Story

 September 12, 2014 324

 Follow these special men
 412,637
 731,774
 130,240

 Popular Posts

 * Hi.
 * Why Going to the Doctor Sucks
 * Mailbag #2
 * The Trump-Biden Debate
 * The Big and the Small
 * You Won’t Believe My Morning
 * A Sick Giant
 * It’s 2020 and you’re in the future
 * Political Disney World
 * Idea Labs and Echo Chambers

 Links

 * Homepage
 * about
 + wait but who
 + faq
 + contact
 * archive
 * minis
 * the shed
 * dinner table
 * store
 + store home
 + new releases
 + posters
 + phone cases
 + cards & wrapping paper
 + squishy things
 + men’s tees
 + women’s tees
 + coffee mugs
 + store support
 * support wbw
 * #7246 (no title)
 * Finn’s Cave
 © WaitButWhy 2023 ContactPartner With Us
 Privacy Policy
 * 
 * 
This is Part 1 of a four-part series on Elon Musk’s companies. 
__________
 PDF and ebook options: We made a fancy PDF of this post for printing and offline viewing (see a preview here), and an ebook containing the whole four-part Elon Musk series:
     
___________
Last month, I got a surprising phone call.
   
   
   
   
   
   
   
   
   
Elon Musk, for those unfamiliar, is the world’s raddest man.
   
I’ll use this post to explore how he became a self-made billionaire and the real-life inspiration for Iron Man’s Tony Stark, but for the moment, I’ll let Richard Branson explain things briefly:1
  Whatever skeptics have said can’t be done, Elon has gone out and made real. Remember in the 1990s, when we would call strangers and give them our credit-card numbers? Elon dreamed up a little thing called PayPal. His Tesla Motors and SolarCity companies are making a clean, renewable-energy future a reality…his SpaceX [is] reopening space for exploration…it’s a paradox that Elon is working to improve our planet at the same time he’s building spacecraft to help us leave it. 
So no, that was not a phone call I had been expecting.
A few days later, I found myself in pajama pants, pacing frantically around my apartment, on the phone with Elon Musk. We had a discussion about Tesla, SpaceX, the automotive and aerospace and solar power industries, and he told me what he thought confused people about each of these things. He suggested that if these were topics I’d be interested in writing about, and it might be helpful, I could come out to California and sit down with him in person for a longer discussion.
   
   
For me, this project was one of the biggest no-brainers in history. Not just because Elon Musk is Elon Musk, but because here are two separate items that have been sitting for a while in my “Future Post Topics” document, verbatim:
– “electric vs hybrid vs gas cars, deal with tesla, sustainable energy”
– “spacex, musk, mars?? how learn to do rockets??”
I already wanted to write about these topics, for the same reason I wrote about Artificial Intelligence—I knew they would be hugely important in the future but that I also didn’t understand them well enough. And Musk is leading a revolution in both of these worlds.
It would be like if you had plans to write about the process of throwing lightning bolts and then one day out of the blue Zeus called and asked if you wanted to question him about a lot of stuff.
So it was on. The plan was that I’d come out to California, see the Tesla and SpaceX factories, meet with some of the engineers at each company, and have an extended sit down with Musk. Exciting. 
The first order of business was to have a full panic. I needed to not sit down with these people—these world-class engineers and rocket scientists—and know almost nothing about anything. I had a lot of quick learning to do.
The problem with Elon Fucking Musk, though, is that he happens to be involved in all of the following industries:
Automotive
Aerospace
Solar Energy
Energy Storage
Satellite
High-Speed Ground Transportation
And, um, Multi-Planetary Expansion
Zeus would have been less stressful.
So I spent the two weeks leading up to the West Coast visit reading and reading and reading, and it became quickly clear that this was gonna need to be a multi-post series. There’s a lot to get into.
We’ll dive deep into Musk’s companies and the industries surrounding them in the coming posts, but today, let’s start by going over exactly who this dude is and why he’s such a big deal.1 2 ← click these 
 The Making of Elon Musk 
 Note: There’s a great biography on Musk coming out May 19th, written by tech writer Ashlee Vance. I was able to get an advance copy, and it’s been a key source in putting together these posts. I’m going to keep to a brief overview of his life here—if you want the full story, get the bio. 
Musk was born in 1971 in South Africa. Childhood wasn’t a great time for him—he had a tough family life and never fit in well at school.2 But, like you often read in the bios of extraordinary people, he was an avid self-learner early on. His brother Kimbal has said Elon would often read for 10 hours a day—a lot of science fiction and eventually, a lot of non-fiction too. By fourth grade, he was constantly buried in the Encyclopedia Britannica.  
One thing you’ll learn about Musk as you read these posts is that he thinks of humans as computers, which, in their most literal sense, they are. A human’s hardware is his physical body and brain. His software is the way he learns to think, his value system, his habits, his personality. And learning, for Musk, is simply the process of “downloading data and algorithms into your brain.”3 Among his many frustrations with formal classroom learning is the “ridiculously slow download speed” of sitting in a classroom while a teacher explains something, and to this day, most of what he knows he’s learned through reading.
He became consumed with a second fixation at the age of nine when he got his hands on his first computer, the Commodore VIC-20. It came with five kilobytes of memory and a “how to program” guide that was intended to take the user six months to complete. Nine-year-old Elon finished it in three days. At 12, he used his skills to create a video game called Blastar, which he told me was “a trivial game…but better than Flappy Bird.” But in 1983, it was good enough to be sold to a computer magazine for $500 ($1,200 in today’s money)—not bad for a 12-year-old.3
 Musk never felt much of a connection to South Africa—he didn’t fit in with the jockish, white Afrikaner culture, and it was a nightmare country for a potential entrepreneur. He saw Silicon Valley as the Promised Land, and at the age of 17, he left South Africa forever. He started out in Canada, which was an easier place to immigrate to because his mom is a Canadian citizen, and a few years later, used a college transfer to the University of Pennsylvania as a way into the US.4
 In college, he thought about what he wanted to do with his life, using as his starting point the question, “What will most affect the future of humanity?” The answer he came up with was a list of five things: “the internet; sustainable energy; space exploration, in particular the permanent extension of life beyond Earth; artificial intelligence; and reprogramming the human genetic code.”4
 He was iffy about how positive the impact of the latter two would be, and though he was optimistic about each of the first three, he never considered at the time that he’d ever be involved in space exploration. That left the internet and sustainable energy as his options.
He decided to go with sustainable energy. After finishing college, he enrolled in a Stanford PhD program to study high energy density capacitors, a technology aimed at coming up with a more efficient way than traditional batteries to store energy—which he knew could be key to a sustainable energy future and help accelerate the advent of an electric car industry.
But two days into the program, he got massive FOMO because it was 1995 and he “couldn’t stand to just watch the internet go by—[he] wanted to jump in and make it better.”5 So he dropped out and decided to try the internet instead.
His first move was to go try to get a job at the monster of the 1995 internet, Netscape. The tactic he came up with was to walk into the lobby, uninvited, stand there awkwardly, be too shy to talk to anyone, and walk out.
Musk bounced back from the unimpressive career beginning by teaming up with his brother Kimbal (who had followed Elon to the US) to start their own company—Zip2. Zip2 was like a primitive combination of Yelp and Google Maps, far before anything like either of those existed. The goal was to get businesses to realize that being in the Yellow Pages would become outdated at some point and that it was a good idea to get themselves into an online directory. The brothers had no money, slept in the office and showered at the YMCA, and Elon, their lead programmer, sat obsessively at his computer working around the clock. In 1995, it was hard to convince businesses that the internet was important—many told them that advertising on the internet sounded like “the dumbest thing they had ever heard of”6—but eventually, they began to rack up customers and the company grew. It was the heat of the 90s internet boom, startup companies were being snatched up left and right, and in 1999, Compaq snatched up Zip2 for $307 million. Musk, who was 27, made off with $22 million.
In what would become a recurring theme for Musk, he finished one venture and immediately dove into a new, harder, more complex one. If he were following the dot-com millionaire rulebook, he’d have known that what you’re supposed to do after hitting it big during the 90s boom is either retire off into the sunset of leisure and angel investing, or if you still have ambition, start a new company with someone else’s money. But Musk doesn’t tend to follow normal rulebooks, and he plunged three quarters of his net worth into his new idea, an outrageously bold plan to build essentially an online bank—replete with checking, savings, and brokerage accounts—called X.com. This seems less insane now, but in 1999, an internet startup trying to compete with the large banks was unheard of.
In the same building that X.com worked out of was another internet finance company called Confinity, founded by Peter Thiel and Max Levchin. One of X.com’s many features was an easy money-transfer service, and later, Confinity would develop a similar service. Both companies began to notice a strong demand for their money-transfer service, which put the two companies in sudden furious competition with each other, and they finally decided to just merge into what we know today as PayPal.
This brought together a lot of egos and conflicting opinions—Musk was now joined by Peter Thiel and a bunch of other now-super-successful internet guys—and despite the company growing rapidly, things inside the office did not go smoothly. The conflicts boiled over in late 2000, and when Musk was on a half fundraising trip / half honeymoon (with his first wife Justine), the anti-Musk crowd staged a coup and replaced him as CEO with Thiel. Musk handled this surprisingly well, and to this day, he says he doesn’t agree with that decision but he understands why they did it. He stayed on the team in a senior role, continued investing in the company, and played an instrumental role in selling the company to eBay in 2002, for $1.5 billion. Musk, the company’s largest shareholder, walked away with $180 million (after taxes).5
 If there was ever a semblance of the normal life rulebook in Musk’s decision-making, it was at this point in his life—as a beyond-wealthy 31-year-old in 2002—that he dropped the rulebook into the fire for good.
The subject of what he did over the next 13 years leading up to today is what we’ll thoroughly explore over the rest of this series. For now, here’s the short story:
In 2002, before the sale of PayPal even went through, Musk started voraciously reading about rocket technology, and later that year, with $100 million, he started one of the most unthinkable and ill-advised ventures of all time: a rocket company called SpaceX, whose stated purpose was to revolutionize the cost of space travel in order to make humans a multi-planetary species by colonizing Mars with at least a million people over the next century.
Mm hm.
Then, in 2004, as that “project” was just getting going, Musk decided to multi-task by launching the second-most unthinkable and ill-advised venture of all time: an electric car company called Tesla, whose stated purpose was to revolutionize the worldwide car industry by significantly accelerating the advent of a mostly-electric-car world—in order to bring humanity on a huge leap toward a sustainable energy future. Musk funded this one personally as well, pouring in $70 million, despite the tiny fact that the last time a US car startup succeeded was Chrysler in 1925, and the last time someone started a successful electric car startup was never.
And since why the fuck not, a couple years later, in 2006, he threw in $10 million to found, with his cousins, another company, called SolarCity, whose goal was to revolutionize energy production by creating a large, distributed utility that would install solar panel systems on millions of people’s homes, dramatically reducing their consumption of fossil fuel-generated electricity and ultimately “accelerating mass adoption of sustainable energy.”7
 If you were observing all of this in those four years following the PayPal sale, you’d think it was a sad story. A delusional internet millionaire, comically in over his head with a slew of impossible projects, doing everything he could to squander his fortune.
By 2008, this seemed to be playing out, to the letter. SpaceX had figured out how to build rockets, just not rockets that actually worked—it had attempted three launches so far and all three had blown up before reaching orbit. In order to bring in any serious outside investment or payload contracts, SpaceX had to show that they could successfully launch a rocket—but Musk said he had funds left for one and only one more launch. If the fourth launch also failed, SpaceX would be done.
Meanwhile, up in the Bay Area, Tesla was also in the shit. They had yet to deliver their first car—the Tesla Roadster—to the market, which didn’t look good to the outside world. Silicon Valley gossip blog Valleywag made the Tesla Roadster its #1 tech company fail of 2007. This would have been more okay if the global economy hadn’t suddenly crashed, hitting the automotive industry the absolute hardest and sucking dry any flow of investments into car companies, especially new and unproven ones. And Tesla was running out of money fast.
During this double implosion of his career, the one thing that held stable and strong in Musk’s life was his marriage of eight years, if by stable and strong you mean falling apart entirely in a soul-crushing, messy divorce.
Darkness.
But here’s the thing—Musk is not a fool, and he hadn’t built bad companies. He had built very, very good companies. It’s just that creating a reliable rocket is unfathomably difficult, as is launching a startup car company, and because no one wanted to invest in what seemed to the outside world like overambitious and probably-doomed ventures—especially during a recession—Musk had to rely on his own personal funds. PayPal made him rich, but not rich enough to keep these companies afloat for very long on his own. Without outside money, both SpaceX and Tesla had a short runway. So it’s not that SpaceX and Tesla were bad—it’s that they needed more time to succeed, and they were out of time.
And then, in the most dire hour, everything turned around.
First, in September of 2008, SpaceX launched their fourth rocket—and their last one if it didn’t successfully put a payload into orbit—and it succeeded. Perfectly.
That was enough for NASA to say “fuck it, let’s give this Musk guy a try,” and it took a gamble, offering SpaceX a $1.6 billion contract to carry out 12 launches for the agency. Runway extended. SpaceX saved.
 The next day, on Christmas Eve 2008, when Musk scrounged up the last money he could manage to keep Tesla going, Tesla’s investors reluctantly agreed to match his investment. Runway extended. Five months later, things began looking up, and another critical investment came in—$50 million from Daimler. Tesla saved.
While 2008 hardly marked the end of the bumps in the road for Musk, the overarching story of the next seven years would be the soaring, earthshaking success of Elon Musk and his companies.
Since their first three failed launches, SpaceX has launched 20 times—all successes. NASA is now a regular client, and one of many, since the innovations at SpaceX have allowed companies to launch things to space for the lowest cost in history. Within those 20 launches have been all kinds of “firsts” for a commercial rocket company—to this day, the four entities in history who have managed to launch a spacecraft into orbit and successfully return it to Earth are the US, Russia, China—and SpaceX. SpaceX is currently testing their new spacecraft, which will bring humans to space, and they’re busy at work on the much larger rocket that will be able to bring 100 people to Mars at once. A recent investment by Google and Fidelity has valued the company at $12 billion.
Tesla’s Model S has become a smashing success, blowing away the automotive industry with the highest ever Consumer Reports rating of a 99/100, and the highest safety rating in history from the National Highway Safety Administration, a 5.4/5. Now they’re getting closer and closer to releasing their true disruptor—the much more affordable Model 3—and the company’s market cap is just under $30 billion. They’re also becoming the world’s most formidable battery company, currently working on their giant Nevada “Gigafactory,” which will more than double the world’s total annual production of lithium-ion batteries.
SolarCity, which went public in 2012, now has a market cap of just under $6 billion and has become the largest installer of solar panels in the US. They’re now building the country’s largest solar panel-manufacturing factory in Buffalo, and they’ll likely be entering into a partnership with Tesla to package their product with Tesla’s new home battery, the Powerwall.
And since that’s not enough, in his spare time, Musk is pushing the development a whole new mode of transport—the Hyperloop.
In a couple of years, when their newest factories are complete, Musk’s three companies will employ over 30,000 people. After nearly going broke in 2008 and telling a friend that he and his wife may have to “move into his wife’s parents’ basement,”8 Musk’s current net worth clocks in at $12.9 billion.
All of this has made Musk somewhat of a living legend. In building a successful automotive startup and its worldwide network of Supercharger stations, Musk has been compared to visionary industrialists like Henry Ford and John D. Rockefeller. The pioneering work of SpaceX on rocket technology has led to comparisons to Howard Hughes, and many have drawn parallels between Musk and Thomas Edison because of the advancements in engineering Musk has been able to achieve across industries. Perhaps most often, he’s compared to Steve Jobs, for his remarkable ability to disrupt giant, long-stagnant industries with things customers didn’t even know they wanted. Some believe he’ll be remembered in a class of his own. Tech writer and Musk biographer Ashlee Vance has suggested that what Musk is building “has the potential to be much grander than anything Hughes or Jobs produced. Musk has taken industries like aerospace and automotive that America seemed to have given up on and recast them as something new and fantastic.”9
   Chris Anderson, who runs TED Talks, calls Musk “the world’s most remarkable living entrepreneur.” Others know him as “the real life Iron Man,” and not for no reason—Jon Favreau actually sent Robert Downey, Jr. to spend time with Musk in the SpaceX factory prior to filming the first Iron Man movie so he could model his character off of Musk.10 He’s even been on The Simpsons.
And this is the man I was somehow on the phone with as I frantically paced back and forth in my apartment, in pajama pants.
On the call, he made it clear that he wasn’t looking for me to advertise his companies—he only wanted me to help explain what’s going on in the worlds surrounding those companies and why the things happening with electric cars, sustainable energy production, and aerospace matter so much.
He seemed particularly bored with people spending time writing about him—he feels there are so many things of critical importance going on in the industries he’s involved in, and every time someone writes about him, he wishes they were writing about fossil fuel supply or battery advancements or the importance of making humanity multi-planetary (this is especially clear in the intro to the upcoming biography on him, when the author explains how not interested Musk was in having a bio written about him).
So I’m sure this first post, whose title is “Elon Musk: The World’s Raddest Man,” will annoy him.
But I have reasons. To me, there are two worthy areas of exploration in this post series:
 1) To understand why Musk is doing what he’s doing. He deeply believes that he’s taken on the most pressing possible causes to give humanity the best chance of a good future. I want to explore those causes in depth and the reasons he’s so concerned about them.
 2) To understand why Musk is able to do what he’s doing. There are a few people in each generation who dramatically change the world, and those people are worth studying. They do things differently from everyone else—and I think there’s a lot to learn from them.
So on my visit to California, I had two goals in mind: to understand as best I could what Musk and his teams were working on so feverishly and why it mattered so much, and to try to gain insight into what it is that makes him so capable of changing the world.
___________
 Visiting the Factories 
The Tesla Factory (in Northern CA) and the SpaceX Factory (in Southern CA), in addition to both being huge, and rad, have a lot in common.
Both factories are bright and clean, shiny and painted white, with super high ceilings. Both feel more like laboratories than traditional factories. And in both places, the engineers doing white collar jobs and the technicians doing blue collar jobs are deliberately placed in the same working quarters so they’ll work closely together and give each other feedback—and Musk believes it’s crucial for those designing the machines to be around those machines as they’re being manufactured. And while a traditional factory environment wouldn’t be ideal for an engineer on a computer and a traditional office environment wouldn’t be a good workplace for a technician, a clean, futuristic laboratory feels right for both professions. There are almost no closed offices in either factory—everyone is out in the open, exposed to everyone else.
When I pulled up to the Tesla factory (joined by Andrew), I was first taken by its size—and when I looked it up, I wasn’t surprised to see that it has the second-largest building footprint (aka base area) in the U.S.
   
The factory was formerly jointly owned by GM and Toyota, who sold it to Tesla in 2010. We started off the day with a full tour of the factory—a sea of red robots making cars and being silly:6
    
   
   
And other cool things, like a vast section of the factory that just makes the car battery, and another that houses the 20,000 pound rolls of aluminum they slice and press and weld into Teslas.
   
And this giant press, which costs $50 million and presses metal with 4,500 tons of pressure (the same pressure you’d get if you stacked 2,500 cars on top of something).
   
The Tesla factory is working on upping its output from 30,000 cars/year to 50,000, or about 1,000 per week. They seemed to be pumping out cars incredibly quickly, so I was blown away to learn that Toyota had been on a 1,000 cars per day clip when they inhabited the factory.
I had a chance to visit the Tesla design studio (no pictures allowed), where there were designers sketching car designs on computer screens and, on the other side of the room, full-size car models made of clay. An actual-size clay version of the upcoming Model 3 was surrounded by specialists sculpting it with tiny instruments and blades, shaving off fractions of a millimeter to examine the way light bounced off the curves. There was also a 3D printer that could quickly “print” out a shoe-sized 3D model of a sketched Tesla design so a designer could actually hold their design and look at it from different angles. Deliciously futuristic.
The next day was the SpaceX factory, which might be even cooler, but the building contains advanced rocket technology, which according to the government is “weapons technology,” and apparently random bloggers aren’t allowed to take pictures of weapons technology.
Anyway, after the tours, I had a chance to sit down with several senior engineers and designers at both companies. They’d explain that they were a foremost expert in their field, I’d explain that I had recently figured out how big the building would be that could hold all humans, and we’d begin our discussion. I’d ask them about their work, their thoughts on the company as a whole and the broader industry, and then I’d ask them about their relationship with Elon and what it was like to work for him. Without exception, they were really nice-seeming, friendly people, who all came off as ridiculously smart but in a non-pretentious way. Musk has said he has a strict “no assholes” hiring policy, and I could see that at work in these meetings.
So what’s Musk like as a boss?
Let’s start by seeing what the internet says—there’s a Quora thread that poses the question: “What is it like to work with Elon Musk?”
The first answer is from a longtime SpaceX employee who no longer works there, who describes the day that their 3rd launch failed, a devastating blow for the company and for all the people who had worked for years to try to make it work.
She describes Elon emerging from mission command to address the company and delivering a rousing speech. She refers to Elon’s “infinite wisdom” and says, “I think most of us would have followed him into the gates of hell carrying suntan oil after that. It was the most impressive display of leadership that I have ever witnessed.”
Right below that answer is another answer, from an anonymous SpaceX engineer, who describes working for Musk like this:
“You can always tell when someone’s left an Elon meeting: they’re defeated…nothing you ever do will be good enough so you have to find your own value, not depending on praise to get you through your obviously insufficient 80 hour work weeks.”
Reading about Musk online and in Vance’s book, I was struck by how representative both of these Quora comments were of whole camps of opinion on working for Musk. Doing so seems to bring out a tremendous amount of adoration and a tremendous amount of exasperation, sometimes with a tone of bitterness—and even more oddly, much of the time, you hear both sides of this story expressed by the same person. For example, later in the comment of the effusive Quora commenter comes “Working with him isn’t a comfortable experience, he is never satisfied with himself so he is never really satisfied with anyone around him…the challenge is that he is a machine and the rest of us aren’t.” And the frustrated anonymous commenter later concedes that the way Elon is “is understandable” given the enormity of the task at hand, and that “it is a great company and I do love it.”
My own talks with Musk’s engineers and designers told a similar story. I was told: “Elon always wants to know, ‘Why are we not going faster?’ He always wants bigger, better, faster” by the same person who a few minutes later was emphasizing how fair and thoughtful Musk tends to be in handling the terms for a recently fired employee.
The same person who told me he has “lots of sleepless nights” said in the adjacent sentence how happy he is to be at the company and that he hopes to “never leave.”
One senior executive described interacting with Musk like this: “Any conversation’s fairly high stakes because he’ll be very opinionated, and he can go deeper than you expect or are prepared for or deeper than your knowledge goes on a given topic, and it does feel like a high wire act interacting with him, especially when you find yourself in a [gulp] technical disagreement.”7 The same executive, who had previously worked at a huge tech company, also called Musk “the most grounded billionaire I’ve ever worked with.”
What I began to understand is that the explanation for both sides of the story—the cult-like adulation right alongside the grudging willingness to endure what sounds like blatant hell—comes down to respect. The people who work for Musk, no matter how they feel about his management style, feel an immense amount of respect—for his intelligence, for his work ethic, for his guts, and for the gravity of the missions he’s undertaken, missions that make all other potential jobs seem trivial and pointless.
Many of the people I talked to also alluded to their respect for his integrity. One way this integrity comes through is in his consistency. He’s been saying the same things in interviews for a decade, often using the same exact phrasing many years apart. He says what he really means, no matter the situation—one employee close to Musk told me that after a press conference or a business negotiation, once in private he’d ask Musk what his real angle was and what he really thinks. Musk’s response would always be boring: “I think exactly what I said.”
A few people I spoke with referenced Musk’s obsession with truth and accuracy. He’s fine with and even welcoming of negative criticism about him when he believes it’s accurate, but when the press gets something wrong about him or his companies, he usually can’t help himself and will engage them and correct their error. He detests vague spin-doctor phrases like “studies say” and “scientists disagree,” and he refuses to advertise for Tesla, something most startup car companies wouldn’t think twice about—because he sees advertising as manipulative and dishonest.
There’s even an undertone of integrity in Musk’s tyrannical demands of workers, because while he may be a tyrant, he’s not a hypocrite. Employees pressured to work 80 hours a week tend to be less bitter about it when at least the CEO is in there working 100.
Speaking of the CEO, let’s go have a hamburger with him.
 My Lunch With Elon 
It started like this:
     
   
  
   
   
   
After about seven minutes of this, I was able to get out my first question, a smalltalk-y question about how he thought the recent launch had gone (they had attempted an extremely difficult rocket-landing maneuver—more on that in the SpaceX post). His response included the following words: hypersonic, rarefied, densifying, supersonic, Mach 1, Mach 3, Mach 4, Mach 5, vacuum, regimes, thrusters, nitrogen, helium, mass, momentum, ballistic, and boost-back. While this was happening, I was still mostly blacked out from the surreality of the situation, and when I started to come to, I was scared to ask any questions about what he was saying in case he had already explained it while I was unconscious.
I eventually regained the ability to have adult human conversation, and we began what turned into a highly interesting and engaging two-hour discussion.8 This guy has a lot on his mind across a lot of topics. In this one lunch alone, we covered electric cars, climate change, artificial intelligence, the Fermi Paradox, consciousness, reusable rockets, colonizing Mars, creating an atmosphere on Mars, voting on Mars, genetic programming, his kids, population decline, physics vs. engineering, Edison vs. Tesla, solar power, a carbon tax, the definition of a company, warping spacetime and how this isn’t actually something you can do, nanobots in your bloodstream and how this isn’t actually something you can do, Galileo, Shakespeare, the American forefathers, Henry Ford, Isaac Newton, satellites, and ice ages.
I’ll get into the specifics of what he had to say about many of these things in later posts, but some notes for now:
— He’s a pretty tall and burly dude. Doesn’t really come through on camera.
— He ordered a burger and ate it in either two or three bites over a span of about 15 seconds. I’ve never seen anything like it.
— He is very, very concerned about AI. I quoted him in my posts on AI saying that he fears that by working to bring about Superintelligent AI (ASI), we’re “summoning the demon,” but I didn’t know how much he thought about the topic. He cited AI safety as one of the three things he thinks about most—the other two being sustainable energy and becoming a multi-planet species, i.e. Tesla and SpaceX. Musk is a smart motherfucker, and he knows a ton about AI, and his sincere concern about this makes me scared.
— The Fermi Paradox also worries him. In my post on that, I divided Fermi thinkers into two camps—those who think there’s no other highly intelligent life out there at all because of some Great Filter, and those who believe there must be plenty of intelligent life and that we don’t see signs of any for some other reason. Musk wasn’t sure which camp seemed more likely, but he suspects that there may be an upsetting Great Filter situation going on. He thinks the paradox “just doesn’t make sense” and that it “gets more and more worrying” the more time that goes by. Considering the possibility that maybe we’re a rare civilization who made it past the Great Filter through a freak occurrence makes him feel even more conviction about SpaceX’s mission: “If we are very rare, we better get to the multi-planet situation fast, because if civilization is tenuous, then we must do whatever we can to ensure that our already-weak probability of surviving is improved dramatically.” Again, his fear here makes me feel not great.
— One topic I disagreed with him on is the nature of consciousness. I think of consciousness as a smooth spectrum. To me, what we experience as consciousness is just what it feels like to be human-level intelligent. We’re smarter, and “more conscious” than an ape, who is more conscious than a chicken, etc. And an alien much smarter than us would be to us as we are to an ape (or an ant) in every way. We talked about this, and Musk seemed convinced that human-level consciousness is a black-and-white thing—that it’s like a switch that flips on at some point in the evolutionary process and that no other animals share. He doesn’t buy the “ants : humans :: humans : [a much smarter extra-terrestrial]” thing, believing that humans are weak computers and that something smarter than humans would just be a stronger computer, not something so beyond us we couldn’t even fathom its existence.
— I talked to him for a while about genetic reprogramming. He doesn’t buy the efficacy of typical anti-aging technology efforts, because he believes humans have general expiration dates, and no one fix can help that. He explained: “The whole system is collapsing. You don’t see someone who’s 90 years old and it’s like, they can run super fast but their eyesight is bad. The whole system is shutting down. In order to change that in a serious way, you need to reprogram the genetics or replace every cell in the body.” Now with anyone else—literally anyone else—I would shrug and agree, since he made a good point. But this was Elon Musk, and Elon Musk fixes shit for humanity. So what did I do?
 Me: Well…but isn’t this important enough to try? Is this something you’d ever turn your attention to?
 Elon: The thing is that all the geneticists have agreed not to reprogram human DNA. So you have to fight not a technical battle but a moral battle.
 Me: You’re fighting a lot of battles. You could set up your own thing. The geneticists who are interested—you bring them here. You create a laboratory, and you could change everything.
 Elon: You know, I call it the Hitler Problem. Hitler was all about creating the Übermensch and genetic purity, and it’s like—how do you avoid the Hitler Problem? I don’t know.
 Me: I think there’s a way. You’ve said before about Henry Ford that he always just found a way around any obstacle, and you do the same thing, you always find a way. And I just think that that’s as important and ambitious a mission as your other things, and I think it’s worth fighting for a way, somehow, around moral issues, around other things.
 Elon: I mean I do think there’s…in order to fundamentally solve a lot of these issues, we are going to have to reprogram our DNA. That’s the only way to do it.
 Me: And deep down, DNA is just a physical material.
 Elon: [Nods, then pauses as he looks over my shoulder in a daze] It’s software.
Comments:
1) It’s really funny to brashly pressure Elon Musk to take on yet another seemingly-insurmountable task and to act a little disappointed in him that he’s not currently doing it, when he’s already doing more for humanity than literally anyone on the planet.
2) It’s also super fun to casually brush off the moral issues around genetic programming with “I think there’s a way” and to refer to DNA—literally the smallest and most complex substance ever—as “just a physical material deep down” when I have absolutely no idea what I’m talking about. Because those things will be his problem to figure out, not mine.
3) I think I’ve successfully planted the seed. If Musk takes on human genetics 15 years from now and we all end up living to 250 because of it, you all owe me a drink.
___________
Watching interviews with Musk, you see a lot of people ask him some variation of this question Chris Anderson asked him on stage at the 2013 TED conference:
 How have you done this? These projects—PayPal, SolarCity, Tesla, SpaceX—they’re so spectacularly different. They’re such ambitious projects, at scale. How on Earth has one person been able to innovate in this way—what is it about you? Can we have some of that secret sauce? 
There are a lot of things about Musk that make him so successful, but I do think there’s a “secret sauce” that puts Musk in a different league from even the other renowned billionaires of our time. I have a theory about what that is, which has to do with the way Musk thinks, the way that he reasons through problems, and the way he views the world. As this series continues, think about this, and we’ll discuss a lot more in the last post.
For now, I’ll leave you with Elon Musk holding a Panic Monster.
   
___________
If you’re into Wait But Why, sign up for the  Wait But Why email list  and we’ll send you the new posts right when they come out. Better than having to check the site!
If you’re interested in supporting Wait But Why,  here’s our Patreon .
Next up in this series: Part 2: How Tesla Will Change the World  
 Other posts in the series: 
 Part 3:  How (and Why) SpaceX Will Colonize Mars
Part 4: The Cook and the Chef: Musk’s Secret Sauce  
 Extra Post #1: The Deal With Solar City  
 Extra Post #2: The Deal With the Hyperloop
Extra Post #3: SpaceX’s Big Fucking Rocket — The Full Story  
 And a new one, written in 2017, about a whole new Elon company: Neuralink and the Brain’s Magical Future  
 Some Musk-y Wait But Why Posts: 
 The AI Revolution: The Road to Superintelligence 
 The Fermi Paradox 
 What Makes You You? 
   
 Sources 
A large part of what I learned for this post came from my own conversations with Musk and his staff. As I mentioned above, Ashlee Vance’s upcoming biography,  Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future, is excellent and helped me fill in a bunch of gaps. Further info came from the sources below:
Documentary:  Revenge of the Electric Car  
TED Talks:  Elon Musk: The mind behind Tesla, SpaceX, SolarCity  
Khan Academy: Interview With Elon Musk 
Quora:  What is it like to work with Elon Musk?  
SXSW: Interview with Elon Musk 
Consumer Reports:  Tesla Model S: The Electric Car that Shatters Every Myth  
Wired:  How the Tesla Model S is Made  
Interview: Elon Musk says he’s a bigger fan of Edison than Tesla 
Interview: Elon Musk gets introspective 
Business Insider:  Former SpaceX Exec Explains How Elon Musk Taught Himself Rocket Science  
Esquire:  Elon Musk: The Triumph of His Will  
Oxford Martin School: Elon Musk on The Future of Energy and Transport 
MIT Interview: Elon Musk compares AI efforts to “Summoning the Demon” 
Documentary:  Billionaire Elon Musk : How I Became The Real ‘Iron Man’  
Reddit: Elon Musk AMA 
Chris Anderson: Chris Anderson on Elon Musk, the World’s Most Remarkable Entrepreneur 
Engineering.com: Who’s Better? Engineers or Scientists? 
Forbes:  Big Day For SpaceX As Elon Musk Tells His Mom ‘I Haven’t Started Yet’
Elon Musk: The World's Raddest Man — Wait But Why
https://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-man.html
https://numinous.productions/ttft/
How can we develop transformative tools for thought?
2016-10-08
Matuschak; Andy; Nielsen; Michael
<div><div>
 <p>
	Part of the origin myth of modern computing is the story of a
	golden age in the 1960s and 1970s. In this story, visionary
	pioneers pursued a dream in which computers enabled powerful
	tools for thought, that is, tools to augment human
	intelligence E.g., Douglas
	Engelbart, <a href="assets/Engelbart1962.pdf">Augmenting Human
	Intellect: A Conceptual Framework</a> (1962).. One of
	those pioneers, Alan Kay, summed up the optimism of this dream
	when he wrote of the potential of the personal computer:
	“the very use of it would actually change the thought
	patterns of an entire
	civilization” Alan
	Kay, <a href="assets/Kay1989.pdf">User Interface: A Personal
	View</a> (1989)..
 </p>
 <p>
	It's an inspiring dream, which helped lead to modern
	interactive graphics, windowing interfaces, word processors,
	and much else. But retrospectively it's difficult not to be
	disappointed, to feel that computers have not yet been nearly
	as transformative as far older tools for thought, such as
	language and writing. Today, it's common in technology circles
	to pay lip service to the pioneering dreams of the past. But
	nostalgia aside there is little determined effort to pursue
	the vision of transformative new tools for thought.
 </p>
 <p>
	We believe now is a good time to work hard on this vision
	again. In this essay we sketch out a set of ideas we believe
	can be used to help develop transformative new tools for
	thought. In the first part of the essay we describe an
	experimental prototype system that we've built, a kind
	of <em>mnemonic medium</em> intended to augment human memory.
	This is a snapshot of an ongoing project, detailing both
	encouraging progress as well as many challenges and
	opportunities. In the second part of the essay, we broaden
	the focus. We sketch several other prototype systems. And we
	address the question: why is it that the technology industry
	has made comparatively little effort developing this vision of
	transformative tools for thought?
 </p>
 <p>
	In the opening we mentioned some visionaries of the past. To
	those could be added many others – Ivan Sutherland,
	Seymour Papert, Vannevar Bush, and more. Online there is much
	well-deserved veneration for these people. But such veneration
	can veer into an unhealthy reverence for the good old days, a
	belief that giants once roamed the earth, and today's work is
	lesser. Yes, those pioneers did amazing things, and arguably
	had ways of working that modern technologists, in both
	industry and academia, are poorly equipped to carry on. But
	they also made mistakes, and were ignorant of powerful ideas
	that are available today. And so a theme through both parts of
	the essay is to identify powerful ideas that weren't formerly
	known or weren't acted upon. Out of this understanding arises
	a conviction that a remarkable set of opportunities is open
	today.
 </p>
 <p>
	A word on nomenclature: the term “tools for
	thought” rolls off neither the tongue nor the
	keyboard. What's more, the term “tool” implies a
	certain narrowness. Alan Kay has
	argued Again, in Alan
	Kay, <a href="assets/Kay1989.pdf">User Interface: A Personal
	View</a> (1989), among other places. that a more
	powerful aim is to develop a new <em>medium for
	thought</em>. A medium such as, say,
	Adobe <em>Illustrator</em> is essentially different from any
	of the individual tools <em>Illustrator</em> contains. Such a
	medium creates a powerful immersive context, a context in
	which the user can have new kinds of thought, thoughts that
	were formerly impossible for them. Speaking loosely, the range
	of expressive thoughts possible in such a medium is an
	emergent property of the elementary objects and actions in
	that medium. If those are well chosen, the medium expands the
	possible range of human thought.
 </p>
 <p>
	With that said, the term “tools for thought” has
	been widely used since Iverson's 1950s and 1960s
	work An account may be found in
	Iverson's Turing Award
	lecture, <a href="assets/Iverson1979.pdf">Notation as a Tool
	of Thought</a> (1979). Incidentally, even Iverson is really
	describing a medium for thought, the APL programming language,
	not a narrow tool. introducing the term. And so we
	shall use “tools for thought” as our catch all
	phrase, while giving ourselves license to explore a broader
	range, and also occasionally preferring the term
	“medium” when it is apt.
 </p>
 <p>
	Let's come back to that phrase from the opening, about
	changing “the thought patterns of an entire
	civilization”. It sounds ludicrous, a kind of tech
	soothsaying. Except, of course, such changes have happened
	multiple times during human history: the development of
	language, of writing, and our other most powerful tools for
	thought. And, for better and worse, computers really have
	affected the thought patterns of our civilization over the
	past 60 years, and those changes seem like just the
	beginning. This essay is a small contribution to understanding
	how such changes happen, and what is still possible.
 </p>
 <p>
	The musician and comedian Martin Mull has observed that
	“writing about music is like dancing about
	architecture”. In a similar way, there's an inherent
	inadequacy in writing about tools for thought. To the extent
	that such a tool succeeds, it expands your thinking beyond
	what can be achieved using existing tools, including
	writing. The more transformative the tool, the larger the gap
	that is opened. Conversely, the larger the gap, the more
	difficult the new tool is to evoke in writing. But what
	writing can do, and the reason we wrote this essay, is act as
	a bootstrap. It's a way of identifying points of leverage that
	may help develop new tools for thought. So let's get on with
	it.
 </p>
 <h2>Part I: Memory systems</h2>
 <h2>Introducing the mnemonic medium</h2>
 
 <p>
	Few subjects are more widely regarded as difficult than
	quantum computing and quantum mechanics. Indeed, popular media
	accounts often regale (and intimidate) readers with quotes
	from famous physicists in the vein of: “anyone who
	thinks they’ve understood quantum mechanics has not understood
	quantum mechanics”.
 </p>
 
 <p>
	What makes these subjects difficult? In fact, individually
	many of the underlying ideas are not too complicated for
	people with a technical background. But the ideas come in an
	overwhelming number, a tsunami of unfamiliar concepts and
	notation. People must learn in rapid succession of qubits, the
	bra-ket notation, Hadamard gates, controlled-not gates, and
	many, many other abstract, unfamiliar notions. They're
	imbibing an entire new language. Even if they can follow at
	first, understanding later ideas requires fluency with all the
	earlier ideas. It's overwhelming and eventually disheartening.
 </p>
 <p>
	As an experiment, we have developed a
	website, <a href="https://quantum.country/"><em>Quantum
	Country</em></a>, which explores a new approach to explaining
	quantum computing and quantum
	mechanics. Ostensibly, <em>Quantum Country</em> appears to be
	a conventional essay introduction to these subjects. There is
	text, explanations, and equations, much as in any other
	technical essay. Here's an excerpt:
 </p>
	
 <p></p><p>
	But it's not a conventional essay. Rather, <em>Quantum
	Country</em> is a prototype for a new type of <em>mnemonic
	medium</em>. Aspirationally, the mnemonic medium makes it
	almost effortless for users to remember what they read. That
	may sound like an impossible aspiration. What makes it
	plausible is that cognitive scientists know a considerable
	amount about how human beings store long-term
	memories. Indeed, what they know can almost be distilled to an
	actionable recipe: follow these steps, and you can remember
	whatever you choose.
 </p>
 <p>
	Unfortunately, those steps are poorly supported by existing
	media.For more on this argument, see
	Andy Matuschak, <a href="https://andymatuschak.org/books/">Why
	books don’t work</a> (2019). Is it possible to
	design a new medium which much more actively supports
	memorization? That is, the medium would build in (and, ideally,
	make almost effortless) the key steps involved in memory. If we
	could do this, then instead of memory being a haphazard event,
	subject to chance, the mnemonic medium would make memory into a
	choice. Of course, on its own this wouldn't make it trivial to
	learn subjects such as quantum mechanics and quantum computing
	– learning those subjects is about much more than memory.
	But it would help in addressing one core difficulty: the
	overwhelming number of new concepts and notation.
 </p>
 
 <p>
	In fact, there are many ways of redesigning the essay medium
	to do that. Before showing you our prototype, please pause for
	a moment and consider the following questions: how could you
	build a medium to better support a person's memory of what
	they read? What interactions could easily and enjoyably help
	people consolidate memories? And, more broadly: is it possible
	to 2x what people remember? 10x? And would that make any
	long-term difference to their effectiveness?
 </p>
 <p>
	Let's sketch the user experience of <em>Quantum
	Country</em>. At the time of this writing the site contains
	three mnemonic essays (i.e., particular instances of the
	mnemonic medium). We'll focus on the introductory essay,
	<a href="https://quantum.country/qcvc">“Quantum
	 Computing for the Very Curious”</a>. Embedded within
	 the text of the essay are 112 questions about that
	 text. Users are asked to create an account, and quizzed as
	 they read on whether they remember the answers to those
	 questions. Here's what the interaction looks like, as a user
	 answers three questions.
 </p>
	
 <p>
	Note that this interaction occurs within the text of the essay
	itself. Here's a zoomed-out view, so you can see how such
	questions are surrounded by essay text both above and below:
 </p>
	<p></p><p>
	We use the term <em>cards</em> for these interface elements
	pairing questions and answers.
 </p>
 <p>
	Of course, for long-term memory it's not enough for users to
	be tested just once on their recall. Instead, a few days after
	first reading the essay, the user receives an email asking
	them to sign into a review session. In that review session
	they're tested again, in a manner similar to what was shown
	above. Then, through repeated review sessions in the days and
	weeks ahead, people consolidate the answers to those questions
	into their long-term memory.
 </p>
 <p>
	So far, this looks like no more than an essay which integrates
	old-fashioned flashcards. But notice the intervals indicated
	at the bottom of the cards:
 </p>
	
 
 <p>
	The highlighted time interval is the duration until the user
	is tested again on the question. Questions start out with the
	time interval “in-text”, meaning the user is being
	tested as they read the essay. That rises to five days, if the
	user remembers the answer to the question. The interval then
	continues to rise upon each successful review, from five days
	to two weeks, then a month, and so on. After just five
	successful reviews the interval is at four months. If the user
	doesn't remember at any point, the time interval drops down
	one level, e.g., from two weeks to five days.
 </p>
 <p>
	This takes advantage of a fundamental fact about human memory:
	as we are repeatedly tested on a question, our memory of the
	answer gets stronger, and we are likely to retain it for
	longer The literature on this effect
	is vast. A useful entrée is: Gwern
	Branwen, <a href="https://www.gwern.net/Spaced-repetition">Spaced
	Repetition for Efficient Learning</a>.. This
	exponential rise perhaps seems innocuous, but it's
	transformative. It means that a relatively small number of
	reviews will enable a user to remember for years. With the
	time taken to review a typical question being just a few
	seconds, that means a user can achieve long-term recall with
	no more than a few minutes' work. By contrast, with
	conventional flashcards it takes hours of review to achieve
	the same durability. Exponential scheduling is far more
	efficient.
 </p>
 <h2>The early impact of the prototype mnemonic medium</h2>
 <p>
	Although it's early days for <em>Quantum Country</em> we can
	begin to see some of the impact of the mnemonic
	medium. Plotted below is the demonstrated retention of answers
	for each user, versus the number of times each question in the
	mnemonic essay has been reviewed:
 </p>
	<p></p><p>
	The graph takes a little unpacking to explain. By a card's
	“demonstrated retention” we mean the maximum time
	between a successful review of that card, and the prior review
	of that card. A little more concretely, consider repetition
	number 6, say (on the horizontal axis). At the point, a user
	has reviewed all 112 questions in the essay 6 times. And the
	vertical axis shows the total demonstrated retention, summed
	across all cards, with each blue dot representing a single
	user who has reached repetition 6.
 </p>
 <p>
	So, for instance, after 6 repetitions, we see from the graph
	that most users are up around 6,000 days of demonstrated
	retention. That means an average of about 6,000 / 112 ~ 54
	days per question in the essay. Intuitively, that seems
	pretty good – if you're anything like us, a couple of
	months after reading something you have only a hazy memory. By
	contrast, these users have, at low time cost to themselves (of
	which more below), achieved nearly two months of demonstrated
	retention across 112 detailed questions.
 </p>
 <p>
	Furthermore, you can see the exponential rise in retention
	with the number of times cards have been reviewed. After the
	first review, users typically have an average of just over 2
	days of demonstrated retention, per
	cardParticularly careful readers may
	wonder how this is possible, given that we stated earlier that
	the first review interval is 5 days. The explanation is that
	we recently modified the review schedule so the first review
	is after 5 days. For most of <em>Quantum Country</em>'s
	history the review schedule was more conservative, and this is
	the reason for the difference.. But by the sixth review
	that rises to an average of 54 days of demonstrated
	retention. That typically takes about 95 minutes of total
	review time to achieve. Given that the essay takes about 4 or
	so hours to read, this suggests that a less than 50% overhead
	in time commitment can provide many months or years of
	retention for almost all the important details in the essay.
 </p>
 <p>
	This is the big, counterintuitive advantage of spaced
	repetition: you get exponential returns for increased
	effort. On average, every extra minute of effort spent in
	review provides more and more benefit. This is in sharp
	contrast with most experiences in life, where we run into
	diminishing returns. For instance, ordinarily if you increase
	the amount of time you spend reading by 50%, you expect to get
	no more than 50% extra out of it, and possibly much less. But
	with the mnemonic medium when you increase the amount of time
	you spend reading by 50%, you may get 10x as much out of
	it. Of course, we don't quite mean those numbers
	literally. But it does convey the key idea of getting a
	strongly non-linear return. It's a change in the quality of
	the medium.
 </p>
 <p>
	This delayed benefit makes the mnemonic medium unusual in
	multiple ways. Another is this: most online media use
	short-term engagement models, using variations on operant
	conditioning to drive user behavior. This is done by Twitter,
	Facebook, Instagram, and many other popular media forms. The
	mnemonic medium is much more like meditation – in some
	ways, the anti-product, since it violates so much conventional
	Silicon Valley wisdom – in that the benefits are
	delayed, and hard to have any immediate sense of. Indeed, with
	the mnemonic medium, the greater the delay, the more the
	benefit.
 </p>
 <p>
	These are preliminary results, and need more
	investigation. One naturally wonders what would happen if we'd
	been much more aggressive with the review schedule, setting
	the initial interval between reviews to (say) 2 months? If
	users reliably retained information up to that point, then the
	graph would start very high, and we wouldn't see the
	exponential. We need to investigate these and many similar
	questions to better understand what's going on with user's
	memories.
 </p>
 <p>
	Early feedback from users makes us cautiously optimistic that
	they're finding the mnemonic medium useful. In May 2019, one
	of us posted to Twitter a short thread explaining the
	technical details of how quantum teleportation works. One user
	of <em>Quantum Country</em>
	<a href="https://twitter.com/KeithMansfield/status/1132031824513966080">replied
	 to the thread</a> with:
 </p>
 <blockquote>I've only done your first quantum country
 course (so far) but I find it remarkable that I can view
 the proof and follow it, knowing what everything means.
 It's almost like Neo in The Matrix telling Morpheus,
 ‘I know quantum computing’ In the movie <em>The Matrix</em> one of
 the characters (Neo) uses a computer to very rapidly upload
 martial arts skills into his mind. As he opens his eyes
 after completing the upload he tells another character
 (Morpheus): “I know Kung Fu”..
 </blockquote>
 <p>
	A user with significantly more prior experience of quantum
	computing <a href="https://news.ycombinator.com/item?id=19429035">wrote</a>:
 </p>
 <blockquote>
	I have a PhD in quantum information/computing and I knew
	everything in the essay before reading it, but the additional
	understanding I got from doing the given spaced repetition
	flashcards significantly improved my understanding of the
	material. Everyone who is reading this essay, should sign up
	and give spaced repetition a try.
 </blockquote>
 <p>
	Another user, new to quantum computing, told us
	that <em>Quantum Country</em> “is by far the best way
	that I could imagine being introduced to this
	material”. When we asked how he'd used what he'd
	learned, he explained that when a visitor to his company gave
	a technical seminar about quantum computing, he expected to
	get lost after about 10 minutes. Instead:
 </p>
 <blockquote>
	Wow, I actually followed that for 40 or 45 minutes because the
	matrices looked familiar… [the medium means] you run
	into concepts over and over again… It affords
	interactions at a more effective level of abstraction.
 </blockquote>
 <p>
	Site analytics show a constant flow of people steadily working
	through the review sessions in the manner we intended. Six
	months after release of the prototype, 195 users had
	demonstrated one full month of retention on at least 80% of
	cards in the essay, demonstrating an extraordinary level of
	commitment to the process We've made
	no attempt at all to scale this out. It's interesting to
	ponder doing so.. We don't yet have a good model of
	exactly what those people are learning, but it seems plausible
	they are taking away considerably more than from a
	conventional essay, or perhaps even from a conventional class.
 </p>
	
 <p>
	Of course, this kind of feedback and these kinds of results
	should be taken with a grain of salt. The mnemonic medium is
	in its early days, has many deficiencies, and needs
	improvement in many ways (of which more soon). It is,
	however, encouraging to hear that some users already find the
	medium exceptionally helpful, and suggests developing and
	testing the medium further. At a minimum, it seems likely the
	mnemonic medium is genuinely helping people remember. And
	furthermore it has the exponentially increasing efficiency
	described above: the more people study, the more benefit they
	get per minute studied.
 </p>
 
 
 <p>
	In another informal experiment, we tried to figure out how
	much it affected user's memories when they <em>weren't</em>
	asked to review cards. To do this, we introduced a deliberate
	short (two-week) delay on reviews for a small subset of 8
	cards. That is, some users would review those 8 cards upon an
	initial read, and then would be prevented from reviewing them
	again for at least two weeks. Other users would continue to
	study as normal on the 8 cards. By comparing the two groups we
	could estimate the effect that reviewing the cards had on
	user's memories.
 </p>
 <p>
	What happened? Well, for those users whose reviews were
	delayed, accuracy dropped from 91% (upon the initial read) to
	87% (after two weeks). This may seem a small drop, but keep in
	mind that users continued to review other cards, which almost
	certainly boosted their final performance, since those other
	cards had some overlap in content with the delayed cards. It's
	difficult to avoid this kind of overlap without delaying
	reviews on all cards, a more drastic change in user experience
	than we wanted to impose. For users who were asked to review
	the cards as normal, accuracy improved from 89% to 96%. The
	short summary is: when users didn't review the cards, accuracy
	dropped by 4%; when they did review the cards, accuracy
	increased by 7% In more detail: there
	were 16 users in the group that did the reviews, per usual,
	and 25 users in the group where reviews were delayed. The 95%
	confidence intervals were: 91 ± 4%, 87 ± 5%, 89
	± 5%, 96 ± 3%, assuming each variable is
	binomial, independent and identically distributed. This latter
	assumption is approximate, since we'd expect some user- and
	question-dependent effects. Note also that this was done in an
	earlier version of <em>Quantum Country</em>, where the review
	schedule had intervals of one day, three days, one week, and
	and two weeks.. 
 </p>
 <p>
	Another way of looking at the data from this informal
	experiment is to ask which users saw improved or unchanged
	performance, and which saw their performance get worse. In
	fact, every single user (100%) who reviewed cards on the
	regular schedule saw their performance either stay the same or
	improve. By contrast, 40% of the users whose reviews were
	delayed saw their performance get worse, while 60% saw it stay
	the same or improve.
 </p>
 <p>
	These are small-but-promising results. Of course, our
	experiment was only done over two weeks, and we'd expect
	larger effects in experiments done over longer periods. And,
	as already mentioned, the effect is likely diminished by
	overlaps between the cards. Nonetheless, this informal
	experiment again suggests the mnemonic medium is helping
	people's memory, and suggests more comprehensive studies.
 </p>
 
 <p>
	Despite these suggestive preliminary results, it's still
	tempting to be dismissive. Isn't this “just” an
	essay with flashcards embedded? At some level, of course,
	that's correct. In the same way, wikis are just editable web
	pages; Twitter is just a way of sharing very short form
	writing; and Facebook is just a way of sharing writing and
	pictures with friends. Indeed, writing itself is just a clever
	way of ordering a small number of symbols on a page. While a
	medium may be simple, that doesn't mean it's not profound. As
	we shall see, the mnemonic medium has many surprising
	properties. It turns out that flashcards are dramatically
	under-appreciated, and it's possible to go much, much further
	in developing the mnemonic medium than is <em>a priori</em>
	obvious.
 </p>
 <p>
	Before we delve deeper into the mnemonic medium, let's mention
	one challenge in the discussion: the inherent difficulty in
	achieving a good balance between conveying enthusiasm and the
	kind of arm's-length skepticism appropriate for evaluation. On
	the one hand, we would not have built the mnemonic medium if
	we weren't excited about the underlying ideas, and wanted to
	develop those enthusiasms. To explain the mnemonic medium
	well, we need to bring you, the reader, inside that
	thinking. But having done that, we also need to step back and
	think more skeptically about questions such as: is this medium
	really working? What effect is it actually having on people?
	Can it be made 10x better? 100x better? Or, contrariwise, are
	there blockers that make this an irredeemably bad or at best
	mediocre idea? How important a role does memory play in
	cognition, anyway? So far, we've focused on the enthusiastic
	case for the medium, why one might consider this design at
	all. But later in this essay we'll gradually step back and
	reflect in a more skeptical frame.
 </p>
 <h2>Expanding the scope of memory systems: what types of
 understanding can they be used for?</h2>
 <p>
	<em>Quantum Country</em> is an example of a <em>memory
	system</em>. That is, it's a system designed to help users
	easily consolidate what they've learned into long-term
	memory. It's part of a long history of memory systems, going
	back to ancient times, when the orator Cicero and the
	rhetorician Quintilian described mnemonic techniques that
	could be used to memorize long texts.
 </p>
 <p>
	In modern times, many memory systems have been
	developed. Among the better known are Anki, SuperMemo,
	Quizlet, Duolingo, and Memrise. Like <em>Quantum Country</em>,
	each of these systems uses increasing time intervals between
	reviews of particular questions. Such systems are sometimes
	known as <em>spaced-repetition memory systems</em>
	(or <em>SRM</em> systems) Strictly
	speaking, Quizlet's basic product doesn't use spaced
	repetition. There is, however, a paid version using spaced
	repetition, and it's otherwise quite similar to many of these
	systems.. They're usually justified in a manner similar
	to our explanation for <em>Quantum Country</em>: some notion
	of each review gradually increasing the consolidation strength
	for a memory.
 </p>
 <p>
	SRM systems are most widely used in language
	learning. Duolingo, for instance, claims
	<a href="https://techcrunch.com/2018/08/01/duolingo-hires-its-first-chief-marketing-officer-as-active-user-numbers-stagnate/">25
	 million monthly active users</a>. Reports are mixed on
	 success. Some serious users
	 are <a href="https://www.thrillist.com/travel/nation/duolingo-app-review-foreign-language">enthusiastic
	 about their success</a> with
	 Duolingo. But <a href="https://www.thecut.com/2019/01/does-duolingo-even-work.html">others
	 find it of limited utility</a>. The company, of
	 course, <a href="https://support.duolingo.com/hc/en-us/articles/115000035183-Are-there-official-studies-about-Duolingo-">touts
	 research</a> showing that it's incredibly successful. It
	 seems likely to us that Duolingo and similar systems are
	 useful for many users as part of (but only part of) a
	serious language learning program.
 </p>
 <p>
	What about memory systems for uses beyond language? Quizlet is
	popular,
	with <a href="https://techcrunch.com/2018/10/25/quizlet-hits-50m-monthly-users/">50
	million monthly active users</a>. It's widely used in
	classrooms, especially for simple declarative knowledge
	– lists of American Presidents, capitals of countries,
	and so on. Anki and SuperMemo seem to most often be used for
	similar simple declarative knowledge, but have much smaller
	active user bases than Quizlet They
	are, however, widely used within some interesting niche
	audiences. For instance, there is
	a <a href="https://www.reddit.com/r/medicalschoolanki/">thriving
	population</a> of medical students using Anki..
 </p>
 <p>
	One of the ideas motivating <em>Quantum Country</em> is that
	memory systems aren't just useful for simple declarative
	knowledge, such as vocabulary words and lists of capitals. In
	fact, memory systems can be extraordinarily helpful for
	mastering abstract, conceptual knowledge, the kind of
	knowledge required to learn subjects such as quantum mechanics
	and quantum computing. This is achieved in part through many
	detailed strategies for constructing cards capable of encoding
	this kind of understanding. But, more importantly, it's
	possible because of the way the mnemonic medium embeds spaced
	repetition inside a narrative. That narrative embedding makes
	it possible for context and understanding to build in ways
	difficult in other memory systems.
 </p>
 <p>
	Other people have also developed ways of using memory systems
	for abstract, conceptual knowledge. Perhaps most prominently,
	the creator of the SuperMemo system, Piotr Wozniak,
	has <a href="http://supermemopedia.com/wiki/Main_Page">written
	extensively</a> about the many ingenious ways he uses memory
	systemsMore generally, Wozniak is,
	along with Sebastian Leitner, the principal pioneer of
	spaced-repetition memory systems. Much of Wozniak's thinking
	is available online at (or linked from) the
	remarkable <a href="http://supermemopedia.com/wiki/Main_Page">SuperMemopedia</a>.. And
	several other expert users of memory systems have also
	developed similar strategies. However, employing those
	strategies requires considerable skill. In practice, that
	skill barrier has meant these strategies are used by no more
	than a tiny handful of people.
 </p>
 <p>
	By contrast, in <em>Quantum Country</em> an expert writes the
	cards, an expert who is skilled not only in the subject matter
	of the essay, but also in strategies which can be used to
	encode abstract, conceptual knowledge. And so <em>Quantum
	Country</em> provides a much more scalable approach to using
	memory systems to do abstract, conceptual learning. In some
	sense, <em>Quantum Country</em> aims to expand the range of
	subjects users can comprehend at all. In that, it has very
	different aspirations to all prior memory systems.
 </p>
 <p>
	More generally, we believe memory systems are a far richer
	space than has previously been realized. Existing memory
	systems barely scratch the surface of what is possible. We've
	taken to thinking of <em>Quantum Country</em> as a <em>memory
	laboratory</em>. That is, it's a system which can be used both
	to better understand how memory works, and also to develop new
	kinds of memory system. We'd like to answer questions such as:
 </p>
 <ul>
	<li>What are new ways memory systems can be applied, beyond
	 the simple, declarative knowledge of past systems?</li>
	<li>How deep can the understanding developed through a memory
	 system be? What patterns will help users deepen their
	 understanding as much as possible?</li>
	<li>How far can we raise the human capacity for memory? And
	 with how much ease? What are the benefits and drawbacks?</li>
	<li>Might it be that one day most human beings will have a
	regular <em>memory practice</em>, as part of their everyday
	lives? Can we make it so memory becomes a choice; is it
	possible to in some sense solve the problem of memory?</li>
	</ul>
 
 <p>
	Over the next few sections we sketch out some of our thinking
	about how memory systems may be developed. We'll see that
	memory systems are a small part of a much bigger picture. Not
	only is seriously developing memory systems likely to lead to
	one or more transformative tools for thought, we also believe
	it will teach us much about the general problem of developing
	such tools.
 </p>
 <h2>Improving the mnemonic medium: making better cards</h2>
 <p>
	In writing mnemonic essays, it's tempting to treat the content
	of the cards rather casually. After all, a card is just a
	question and an answer, each containing a little text, perhaps
	a figure. Surely they ought to be easy to write?
 </p>
 <p>
	While thinking in this way is tempting, it's a mistake. In
	fact, cards are fundamental building blocks of the mnemonic
	medium, and card-writing is better thought of as an open-ended
	skill. Do it poorly, and the mnemonic medium works poorly. Do
	it superbly well, and the mnemonic medium can work very well
	indeed. By developing the card-writing skill it's possible to
	expand the possibilities of the medium.
 </p>
 <p>
	A helpful comparison is to the sentence in written prose. For
	the beginning writer it's tempting to treat sentences
	casually. But in the hands of a great writer – say, a
	Nabokov – sentences can be developed into a virtuoso
	artform. What would it take to achieve virtuoso skill in
	writing the cards of the mnemonic medium?
 </p>
 <p>
	It's not obvious <em>a priori</em> that writing cards is such
	a rich activity. One of us wrote 17,000- and 6,000-word
	essaysMichael
	Nielsen, <a href="https://augmentingcognition.com/ltm.html">Augmenting
	Long-term Memory</a> (2018), and Michael
	Nielsen, <a href="http://cognitivemedium.com/srs-mathematics">Using
	spaced repetition systems to see through a piece of
	mathematics</a> (2019). whose subject was in large part
	understanding how to write good cards. He didn't realize that
	was going to be the subject when he began writing; it only
	became clear in retrospect how rich card writing is. It turns
	out that answering the question “how to write good
	cards?” requires thinking hard about your theory of
	knowledge and how to represent it, and your theory of
	learning. The better those theories, the better your cards
	will be. Small wonder it's a rich, open-ended problem!
 </p>
 <p>
	All that said, let's make a few concrete observations about
	good card-writing. While the specific examples that follow are
	relatively banal, they should give you some feeling for the
	profound issues that arise in improving the mnemonic
	medium. We'll begin with three principles we used when writing
	the cards in <em>Quantum Country</em>. Note that these are
	just three of many more principles – a more detailed
	discussion of good principles of card construction may be
	found
	in <a href="http://augmentingcognition.com/ltm.html">Augmenting
	Long-term Memory</a>.
 </p>
 <ul>
	<li><strong>Most questions and answers should be
	atomic:</strong> Early in his own personal memory practice,
	one of us was learning the Unix command to create links in the
	filesystem. He entered the following question into his memory
	system: “How to create a soft link from linkname to
	filename”. Together with the corresponding answer
	“ln -s filename linkname”. This looks like a good
	question, but he routinely forgot the answer. To address this,
	he refactored the card into two more atomic cards. One card:
	“What’s the basic command and option to create a soft
	link?” (A: “ln -s”). Second card:
	“When creating a soft link, in what order do linkname
	and filename go?” (A: “filename
	linkname”). Breaking the card into more atomic pieces
	turned a question he routinely got wrong into two questions he
	routinely got right. It seemed that the more atomic questions
	brought more sharply into focus what he was forgetting, and so
	provided a better tool for improving memory. And what of the
	original card? Initially, he deleted it. But he eventually
	added the card back, with the same question and answer, since
	it served to integrate the understanding in the more atomic
	cards.
	</li>
	<li><strong>Make sure the early questions in a mnemonic essay
	 are trivial: it helps many users realize they aren't
	 paying enough attention as they
	 read:Note added December 9, 2019:
	 This claim appears to be based on an error in our data
	 analysis, and is now retracted. We've left the text in for
	 historic reasons, but we no longer believe the
	 claim. </strong> This was a discovery made when we
	 released the first <em>Quantum Country</em>
	 essay. Anticipating that users would be struggling with a
	 new interface, we deliberately made the first few
	 questions in the essay utterly trivial – sort of a
	 quantum equivalent to “2+2 = ?” – so
	 they could focus on the interface. To our surprise, users
	 performed poorly on these questions, worse than they did
	 on the (much harder) later questions. Our current
	 hypothesis to explain this is that when users failed to
	 answer the first few questions correctly it served as a
	 wakeup call. The questions were so transparently simple
	 that they realized they hadn't really been paying
	 attention as they read, and so were subsequently more
	 careful.
	</li>
	<li><strong>Avoid orphan cards:</strong> These are cards which
	don't connect closely to anything else. Suppose, for the sake
	of illustration, that you're trying to learn about African
	geography, and have a question: “What's the territory in
	Africa that Morocco disputes?” (A: “The Western
	Sahara”) If you don't know anything about the Western
	Sahara or Morocco or why there's a dispute, that question will
	be an orphan, disconnected from everything else. Ideally,
	you'll have a densely interconnected web of questions and
	answers, everything interwoven in striking ways.
	</li>
 </ul>
 <p>
	Ultimately, we'd like to distill out a set of useful practical
	principles and idioms to help write good cards and, more
	generally, good mnemonic essays. Aspirationally, such a set of
	principles and idioms would work much like <em>The Elements of
	Style</em> (or some similar book of prose advice), and would
	help other people learn to write high-quality mnemonic essays.
 </p>
 <p>
	When we first described <em>Quantum Country</em> above we
	explained it using a simple model of spaced repetition:
	increased consolidation strength for memories leading to
	increased time intervals between reviews. This is a helpful
	simple model, but risks creating the misleading impression
	that it's all that's going on in the system. In fact, for the
	mnemonic medium to work effectively, spaced repetition must be
	deployed in concert with many other ideas. The three ideas we
	just described – atomicity of questions and answers,
	making early questions trivial, avoiding orphan cards –
	are just three of dozens of important ideas used in the
	mnemonic medium. We won't enumerate all those other ideas here
	– that's not the purpose of this essay. But we want to
	emphasize this point, since it's common for people to have the
	simplistic model “good memory system = spaced
	repetition”. That's false, and an actively unhelpful way
	of thinking.
 </p>
 <p>
	Indeed, thinking in this way is one reason spaced-repetition
	memory systems often fail for individuals. We often meet
	people who say “Oh, I thought spaced repetition sounded
	great, and I tried Anki [<em>etc</em>], but it doesn't work
	for me”. Dig down a little, and it turns out the person
	is using their memory system in a way guaranteed to
	fail. They'll be writing terrible questions, or using it to
	learn a subject they don't care about, or making some other
	error. They're a little like a person who thinks
	“learning the guitar sounds great”, picks it up
	for half an hour, and then puts it down, saying that they
	sound terrible and therefore it's a bad instrument. Of course,
	what's really going on is that the guitar and memory systems
	are both skills that take time to develop. But, with that
	said, we want to build as much support as possible into the
	medium. Ideally, even novices would benefit tremendously from
	the mnemonic medium. That means building in many ideas that go
	beyond the simplistic model of spaced repetition.
 </p>
 <p>
	One of us has previously
	assertedMichael
	Nielsen, <a href="https://augmentingcognition.com/ltm.html">Augmenting
	Long-Term Memory</a> (2018). that in spaced-repetition
	memory systems, users need to make their own cards. The
	reasoning is informal: users often report dissatisfaction and
	poor results when working with cards made by others. The
	reason seems to be that making the cards is itself an
	important act of understanding, and helps with committing
	material to memory. When users work with cards made by others,
	they lose those benefits.
 </p>
 <p>
	<em>Quantum Country</em> violates this principle, since users
	are not making the cards. This violation was a major concern
	when we began working on <em>Quantum Country</em>. However,
	preliminary user feedback suggests it has worked out
	adequately. A possible explanation is that, as noted above,
	making good cards is a difficult skill to master, and so what
	users lose by not making their own cards is made up by using
	what are likely to be much higher-quality cards than they
	could have made on their own. In future, it's worth digging
	deeper into this issue, both to understand it beyond informal
	models, and to explore ways of getting the benefits of active
	card making.
 </p>
 <p>
	Above we discussed three principles of good
	question-and-answer construction. Of course, it's also
	possible to make more structural modifications to the nature
	of the cards themselves. Here's three questions suggesting
	experiments in this vein:
 </p>
 <ul>
	 <li><strong>How can we ensure users don't just learn surface
	 features of questions?</strong> One question in <em>Quantum
	 Country</em> asks: “Who has made progress on using
	 quantum computers to simulate quantum field theory?”
	 with the answer: “John Preskill and his
	 collaborators”. This is the only
	 “Who…?” question in the entire essay, and
	 many users quickly learn to recognize it from just the
	 “Who…?” pattern, and parrot the answer
	 without engaging deeply with the question. This is a common
	 failure mode in memory systems, and it's deadly to
	 understanding. One response, which we plan to trial soon, is
	 to present the question in multiple different-but-equivalent
	 forms. So the user first sees the question as “Who has
	 made progress [<em>etc</em>]?”; but then the second
	 time the question is presented as a fill-in-the-blanks:
	 “___ and his collaborators have made progress on using
	 quantum computers to simulate quantum field theory.”
	 And so on, multiple different forms of the question,
	 designed so the user must always engage deeply with the
	 meaning of the question, not its superficial
	 appearance. Ultimately, we'd like to develop a library of
	 techniques for identifying when this
	 learning-the-surface-feature pattern is occurring, and for
	 remedying it.
	 </li>
	 <li><strong>How to best help users when they forget the
	 answer to a question?</strong> Suppose a user can't remember
	 the answer to the question: “Who was the second
	 President of the United States?” Perhaps they think
	 it's Thomas Jefferson, and are surprised to learn it's John
	 Adams. In a typical spaced-repetition memory system this
	 would be dealt with by decreasing the time interval until
	 the question is reviewed again. But it may be more effective
	 to follow up with questions designed to help the user
	 understand some of the surrounding context. E.g.: “Who
	 was George Washington's Vice President?” (A:
	 “John Adams”). Indeed, there could be a whole
	 series of followup questions, all designed to help better
	 encode the answer to the initial question in memory.
	 </li>
	 <li><strong>How to encode stories in the mnemonic
	 medium?</strong> People often find certain ideas most
	 compelling in story form. Here's a short, fun example: did
	 you know that Steve Jobs actively opposed the development of
	 the App Store in the early days of the iPhone? It was
	 instead championed by another executive at Apple, Scott
	 Forstall. Such a story carries a force not carried by
	 declarative facts alone. It's one thing to know in the
	 abstract that even the visionaries behind new technologies
	 often fail to see many of their uses. It's quite another to
	 hear of Steve Jobs arguing with Scott Forstall against what
	 is today a major use of a technology Jobs is credited with
	 inventing. Can the mnemonic medium be used to help people
	 internalize such stories? To do so would likely violate the
	 principle of atomicity, since good stories are rarely atomic
	 (though this particular example comes close). Nonetheless,
	 the benefits of such stories seem well worth violating
	 atomicity, if they can be encoded in the cards effectively.
	 </li>
	</ul>
 
 <p>
	It's easy to generate dozens more questions and ideas in a
	similar vein. The mnemonic medium is not a fixed form, but
	rather a platform for experimentation and continued
	improvement.
 </p>
 <p>
	One useful metaphor for thinking about how to improve the
	mnemonic medium is to think of each mnemonic essay as a
	conventional essay accompanied by a kind of “reflected
	essay” – the knowledge encoded by all the cards. A
	user can, with ease, choose to remember as much of that
	reflected essay as they wish. Of course, the reflection is
	imperfect. But by developing good card-making strategies we
	can make the reflected essay a nearly faithful reflection of
	all the important ideas, the ideas a reader would ideally like
	to retain.
 </p>
 <p>
	We said above that it's a mistake to use the simplistic model
	“good memory system = spaced repetition”. In fact,
	while spaced repetition is a helpful way to
	introduce <em>Quantum Country</em>, we certainly shouldn't
	pigeonhole the mnemonic medium inside the paradigm of existing
	SRM systems. Instead, it's better to go back to first
	principles, and to ask questions like: what would
	make <em>Quantum Country</em> a good memory system? Are there
	other powerful principles about memory which we could we build
	into the system, apart from spaced repetition?
 </p>
 <p>
	In fact, there are ideas about memory very different from
	spaced repetition, but of comparable power. One such idea is
	<em>elaborative encoding</em>. Roughly speaking, this is the
	idea that the richer the associations we have to a concept,
	the better we will remember it. As a consequence, we can
	improve our memory by enriching that network of associations.
 </p>
 <p>
	This is in some sense an obvious idea, according well with
	everyday experience. For instance, it's part of the reason
	it's so much easier to learn new facts in an area we're
	already expert in – we quickly form associations to our
	existing knowledge. But just because the idea is obvious, that
	doesn't mean it's particularly well supported by existing
	media forms. There's a lot of low-hanging fruit which we can
	actively support inside the mnemonic medium. Indeed, several
	of the suggestions above already implicitly build on the idea
	of elaborative encoding – principles like “avoid
	orphan cards” are based on this. Here's three more
	suggestions which build on elaborative encoding:
 </p>
 <ul>
	 <li><strong>Provide questions and answers in multiple
	 forms:</strong> In 1971, the psychologist Allan Paivio
	 proposed the dual-coding theory, namely, the assertion that
	 verbal and non-verbal information are stored separately in
	 long-term memory. Paivio and others investigated
	 the <em>picture superiority effect</em>, demonstrating that
	 pictures and words together are often recalled substantially
	 better than words alone. This suggests, for instance, that
	 the question “Who was George Washington's Vice
	 President?” may have a higher recall rate if
	 accompanied by a picture of Washington, or if the answer
	 (John Adams) is accompanied by a picture of Adams. For
	 memory systems the dual-coding theory and picture
	 superiority effect suggest many questions and ideas. How
	 much benefit is there in presenting questions and answer in
	 multiple forms? Perhaps even with multiple pictures, or in
	 audio or video (perhaps with multiple speakers of different
	 genders, different accents, <em>etc</em>), or in computer
	 code? Perhaps in a form that demands some form of
	 interaction? And in each case: what works best?
	 </li>
	 <li><strong>Vary the context:</strong> In 1978, the
	 psychologists Steven Smith, Arthur Glenberg, and Robert
	 Bjork Steven M. Smith, Arthur
	 Glenberg, and Robert
	 A. Bjork, <a href="assets/Smith1978.pdf">Environmental
	 context and human memory</a> (1978). reported several
	 experiments studying the effect of place on human memory. In
	 one of their experiments, they found that studying material
	 in two different places, instead of twice in the same place,
	 provided a 40% improvement in later recall. This is part of
	 a broader pattern of experiments showing that varying the
	 context of review promotes memory. We can use memory systems
	 to support things like: changing the location of review;
	 changing the time of day of review; changing the background
	 sound, or lack thereof, while reviewing. In each case,
	 experiments have been done suggesting an impact on
	 recall. It's not necessarily clear how robust the results
	 are, or how reproducible – it's possible some (or all)
	 are the results of other effects, uncontrolled in the
	 original experiment. Still, it seems worth building systems
	 to test and (if possible) improve on these results.
	 </li>
	 <li><strong>How do the cards interact with one another? What
	 is the ideal network structure of knowledge?</strong>
	 This is a very complicated and somewhat subtle set of
	 questions. Let's give a simple example to illustrate the
	 idea. We've presented the cards in the mnemonic medium
	 as though they are standalone entities. But there are
	 connections between the cards. Suppose you have cards:
	 “Who was George Washington’s Vice
	 President?” (Answer: “John Adams”,
	 with a picture of Adams); “What did John Adams
	 look like?” (Answer: a picture of Adams); perhaps
	 a question involving a sketch of Adams and Washington
	 together at some key moment; and so on. Now, this set of
	 cards forms a network of interrelated cards. And you can
	 use a memory system like <em>Quantum Country</em> to
	 study that network. What happens to people's observed
	 recall if you remove a card? Are there crucial lynchpin
	 cards? Are there particularly effective network
	 structures? Particularly effective types of
	 relationship between cards? Crucially: are there general
	 principles we can identify about finding the deepest,
	 most powerful ways of representing knowledge in this
	 system?
	 </li>
	</ul>
 <p>
	By now it's obvious that the prototype mnemonic medium we've
	developed is the tip of a much larger iceberg. What's more,
	the suggestions we've made and questions we've asked here are
	also merely a beginning, to give you the flavor of what is
	possible.
 </p>
 <h2>Two cheers for mnemonic techniques</h2>
 <p>
	When we discuss memory systems with people, many immediately
	respond that we should look into mnemonic techniques. This is
	an approach to memory systems very different to <em>Quantum
	Country</em>, Duolingo, Anki, and the other systems we've
	discussed. You're perhaps familiar with simple mnemonic
	techniques from school. One common form is tricks such as
	remembering the colors of the rainbow as the name Roy G. Biv
	(red, orange, yellow, green, <em>etc</em>). Or remembering the
	periodic table of elements using
	a <a href="https://www.youtube.com/watch?v=VgVQKCcfwnU">song</a>An
	enjoyable extended introduction to such techniques may be
	found in Joshua Foer's book “Moonwalking with
	Einstein” (2012)..
 </p>
 <p>
	A more complex variation is visualization techniques such as
	the <em>method of loci</em>. Suppose you want to remember your
	shopping list. To do so using the method of loci, you
	visualize yourself in some familiar location – say, your
	childhood home. And then you visualize yourself walking from
	room to room, placing an item from your shopping list
	prominently in each room. When you go shopping, you can recall
	the list by imagining yourself walking through the house
	– your so-called <em>memory palace</em> – and
	looking at the items in each room.
 </p>
 <p>
	If you've never used memory palaces this sounds like it
	couldn't possibly work. But even novices are often shocked by
	how well such techniques work, with just a small amount of
	practice. Experts who work hard developing these techniques
	can do remarkable things, like memorizing the order of a
	shuffled deck of cards, or lists of hundreds of digits. It's a
	way of using people's immensely powerful visual and spatial
	memories as a form of leverage for other types of
	memory A small minority of the
	population does not possess a mind's eye, and so cannot
	mentally visualize. This condition is known
	as <a href="https://en.wikipedia.org/wiki/Aphantasia">aphantasia</a>. One
	of
	us <a href="https://twitter.com/michael_nielsen/status/1144281873197096961">asked
	on Twitter</a> if any aphantasics had tried using the method
	of loci, and if so how well it worked for them. The replies
	were remarkably heterogeneous (and striking), but most said
	such mnemonic techniques did not work for them. This deserves
	further study..
 </p>
 <p>
	Given all this, it's perhaps not surprising that we often meet
	people who tell us that mnemonic techniques are a much more
	promising approach to memory than ideas such as spaced
	repetition.
 </p>
 <p>
	We're enthusiastic about such mnemonic techniques. But it's
	important to understand their limitations, and not be
	bedazzled by the impressiveness of someone who can rapidly
	memorize a deck of cards.
 </p>
 <p>
	One caution concerns the range of what can be memorized using
	mnemonic techniques. In practice they're often quite
	specialized. Mnemonic experts will, for instance, use somewhat
	different approaches to memorize lists of digits versus decks
	of cards. Those approaches must be mastered separately –
	a heavy time investment for two narrow kinds of
	memory. Furthermore, the mnemonic techniques tend to be much
	better suited for concrete objects than abstract conceptual
	knowledge – it's difficult to store, say, the main
	points in the Treaty of Versailles in your memory palace. This
	doesn't mean it can't be done – mnemonic experts have
	developed clever techniques for converting abstract conceptual
	knowledge into concrete objects which can be stored in a
	memory palace. But, in general, an advantage of spaced
	repetition is that it works across a far broader range of
	knowledge than do any of the mnemonic techniques.
 </p>
 <p>
	A second caution relates to elaborative encoding. The mnemonic
	techniques are, as you have likely realized, an example of
	elaborative encoding in action, connecting the things we want
	to memorize (say, our shopping list) to something which
	already has meaning for us (say, our memory palace). By
	contrast, when an expert learns new information in their
	field, they don't make up artificial connections to their
	memory palace. Instead, they find meaningful connections to
	what they already know. Those connections are themselves
	useful expertise; they're building out a dense network of
	understanding. It's a deeper and more desirable kind of
	expertise, connections native to the subject itself, not
	artificially constructed mnemonics.
 </p>
 <p>
	All this makes us seem negative about mnemonic techniques. In
	fact, we're enthusiastic, and have to date certainly underused
	them in the mnemonic medium. What we've written here is merely
	meant to temper the over-enthusiasm we sometimes
	encounter. We've had people go so far as to tell us that
	mnemonics make memory a solved problem. That is simply
	false. But with their limitations understood, they're a
	powerful tool. This is particularly true for knowledge which
	has an arbitrary, <em>ad hoc</em> structure. For example, it's
	difficult to remember the colors of the rainbow because those
	colors are not obviously connected to anything else, unless
	you happen to have
	the <a href="https://en.wikipedia.org/wiki/Visible_spectrum">spectrum
	of visible light</a> memorized for other reasons! That makes a
	mnemonic like Roy G. Biv extremely helpful. And so mnemonic
	techniques should be thought of as a useful tool to use in
	building powerful memory systems, especially when combined
	with ideas such as spaced repetition.
 </p>
 <h2>How important is memory, anyway?</h2>
 <p>
	People tend to fall into two buckets when told of the mnemonic
	medium. One group is fascinated by the idea, and wants to try
	it out. The second group is skeptical or even repulsed. In
	caricature, they say: “Why should I care about memory? I
	want deeper kinds of understanding! Can't I just look stuff up
	on the internet? I want creativity! I want conceptual
	understanding! I want to know how to solve important problems!
	Only dull, detail-obsessed grinds focus on rote memory.”
 </p>
 <p>
	It's worth thinking hard about such objections. To develop the
	best possible memory system we need to understand and address
	the underlying concerns. In part, this means digging down far
	enough to identify the mistaken or superficial parts of these
	concerns. It also means distilling as sharply as possible the
	truth in the concerns. Doing both will help us improve and go
	beyond the current prototype mnemonic medium.
 </p>
 <p>
	One response to such objections is the argument from lived
	experience. In the past, one of us (MN) has often helped
	students learn technical subjects such as quantum
	mechanics. He noticed that people often think they're getting
	stuck on esoteric, complex issues. But, as suggested in the
	introduction to this essay, often what's really going on is
	that they're having a hard time with basic notation and
	terminology. It's difficult to understand quantum mechanics
	when you're unclear about every third word or piece of
	notation. Every sentence is a struggle.
 </p>
 <p>
	It's like they're trying to compose a beautiful sonnet in
	French, but only know 200 words of French. They're frustrated
	and think the trouble is the difficulty of finding a good
	theme, striking sentiments and images, and so on. But really
	the issue is that they have only 200 words with which to
	compose.
 </p>
 <p>
	At the time, MN's somewhat self-satisfied belief was that if
	people only focused more on remembering the basics, and
	worried less about the “difficult” high-level
	issues, they'd find the high-level issues took care of
	themselves. What he didn't realize is that this also applied
	to him. When he began using the memory system Anki to read
	papers in new fields, he found it almost unsettling how much
	easier Anki made learning the basics of such subjects. And it
	made him start wondering if memory was often a binding
	constraint in learning new fields
	See <a href="http://augmentingcognition.com/ltm.html">here</a>
	and <a href="http://cognitivemedium.com/srs-mathematics">here</a>
	for more on learning new fields using Anki. The last four
	paragraphs are adapted from
	<a href="http://augmentingcognition.com/ltm.html">Augmenting
	Long-term Memory</a> (2018)..
 </p>
 <p>
	One particularly common negative response to the mnemonic
	medium is that people don't want to remember
	“unimportant details”, and are just looking for
	“a broad, conceptual understanding”. It's
	difficult to know what to make of this argument. Bluntly, it
	seems likely that such people are fooling themselves,
	confusing a sense of enjoyment with any sort of durable
	understanding.
 </p>
 <p>
	Imagine meeting a person who told you they “had a broad
	conceptual understanding” of how to speak French, but it
	turned out they didn’t know the meaning of
	“bonjour”, “au revoir”, or “tres
	bien”. You'd think their claim to have a broad
	conceptual understanding of French was hilarious. If you want
	to understand a subject in any real sense you need to know the
	details of the fundamentals. What's more, that means not just
	knowing them immediately after reading. It means internalizing
	them for the long termThe last two
	paragraphs are adapted from our forthcoming mnemonic essay:
	Andy Matuschak and Michael
	Nielsen, <a href="https://quantum.country/teleportation">How
	quantum teleportation works</a> (2019)..
 </p>
 <p>
	A better model is that conceptual mastery is actually enabled
	by a mastery of details. One user of <em>Quantum Country</em>
	told us that she found the experience of reading unexpectedly
	relaxing, because she “no longer had to worry”
	about whether she would remember the content. She simply
	trusted that the medium itself would ensure that she did. And
	she reported that she was instead able to spend more of her
	time on conceptual issues.
 </p>
 <p>
	When people respond to the mnemonic medium with “why do
	you focus on all that boring memory stuff?”, they are
	missing the point. By largely automating away the problem of
	memory, the mnemonic medium makes it easier for people to
	spend more time focusing on other parts of learning, such as
	conceptual issues.
 </p>
 <p>
	Another common argument against spaced repetition systems is
	that it's better to rely on natural repetition. For instance,
	if you're learning a programming language, the argument goes,
	you shouldn't memorize every detail of that language. Instead,
	as you use the language in real projects you'll naturally
	repeatedly use, and eventually commit to memory, those parts
	of the language most important to learn.
 </p>
 <p>
	There are important partial truths in this. It is good to use
	what you're learning as part of your creative
	projects. Indeed, an ideal memory system might help that
	happen, prompting you as you work, rather than in an
	artificial card-based environment. Furthermore, a common
	failure mode with memory systems is that people attempt to
	memorize things they're unlikely to ever have any use for. For
	instance, it's no good (but surprisingly common) for someone
	to memorize lots of details of a programming language they
	plan to use for just one small project. Or to memorize details
	“just in case” they ever need them. These patterns
	are mistakes.
 </p>
 <p>
	But the truths of the last paragraph also have limits. If
	you're learning French, but don't know any French speakers,
	then waiting for “natural opportunities” to speak
	just won't work. And even if you do have (or create)
	opportunities to speak, it's desirable to accelerate the
	awkward, uncomfortable early stages that form such a barrier
	to using the language.
 </p>
 <p>
	It's in this phase that memory systems shine. They can
	accelerate people through the awkward early stages of learning
	a subject. Ideally, they'll support and enable work on
	creative projects. For this to work well takes good heuristics
	for what any given person should commit to memory; what is
	good for one person to memorize may be bad for
	another. Working such heuristics out is an ongoing challenge
	in the design of memory systems.
 </p>
 <p>
	(Incidentally, a surprising number of people say they are
	“repulsed”, or some similarly strong word, by
	spaced-repetition memory systems. Their line of argument is
	usually some variant on: it is claimed that spaced-repetition
	systems help with memory; if that is true I <em>must</em> use
	the systems; but I hate using the systems. The response is to
	deny the first step of the argument. Of course, the mistake is
	elsewhere: there is absolutely no reason anyone
	“should” use such systems, even if they help with
	memory. Someone who hates using them should simply choose not
	to do so. Using memory systems is not a moral imperative!)
 </p>
 <p>
	An immense amount of research has been done on the
	relationship of memory to mastery. Much of this research is
	detailed and context specific. But at the level of broader
	conclusions, one especially interesting series of studies was
	done in the 1970s by Herbert Simon and his collaborators. They
	studied chess players, and discovered
	See, e.g., William G. Chase and Herbert
	A. Simon, <a href="assets/Chase197s.pdf">Perception in
	Chess</a> (1973). Some fascinating earlier work in a related
	vein was done by Adrian D. de Groot, and summarized in his
	book <em>Thought and Choice in Chess</em> (1965). that
	when master chess players look at a position in chess they
	don't see it in terms of the individual pieces, a rook here, a
	pawn there. Instead, over years of playing and analyzing games
	the players learn to recognize somewhere between 25,000 and
	100,000 patterns of chess pieces. These much more elaborate
	“chunks” are combinations of pieces that the
	players perceive as a unity, and are able to reason about at a
	higher level of abstraction than the individual pieces. At
	least in part it's the ability to recognize and reason about
	these chunks which made their gameplay so much better than
	novices. Furthermore, although Simon did this work in the
	context of chess, subsequent studies have found
	<a href="https://en.wikipedia.org/wiki/Chunking_(psychology)">similar
	 results</a> in other areas of
	 expertise We've met many
	 mathematicians and physicists who say that one reason they
	 went into mathematics or physics is because they hated the
	 rote memorization common in many subjects, and preferred
	 subjects where it is possible to derive everything from
	 scratch. But in conversation it quickly becomes evident that
	 they have memorized an enormous number of concepts,
	 connections, and facts in their discipline. It's fascinating
	 these people are so blind to the central role memory plays
	 in their own thinking.. It seems plausible, though
	 needs further study, that the mnemonic medium can help speed
	 up the acquisition of such chunks, and so the acquisition of
	 mastery.
 </p>
 
 <p>
	So, does all this mean we're fans of rote memory, the kind of
	forced memorization common schools?
 </p>
 <p>
	Of course not. What we do believe is that many people's
	dislike of rote memorization has led them to a generalized
	dislike of memory, and consequently to underrate the role it
	plays in cognition. Memory is, in fact, a central part of
	cognition. But the right response to this is not immense
	amounts of dreary rote memorization. Rather, it's to use good
	tools and good judgment to memorize what truly matters.
 </p>
 <p>
	We've identified some ways in which criticisms of memory
	systems are mistaken or miss the point. But what about the
	ways in which those criticisms are insightful? What are the
	shortcomings of memory systems? In what ways should we be wary
	of them?
 </p>
 <p>
	We've already implicitly mentioned a few points in this
	vein. Think about problems like the need to avoid orphan
	questions. Or to make sure that users don't merely learn
	surface features of questions. These are ways in which memory
	systems can fail, if used poorly. Here's a few more key
	concerns about memory systems:
 </p>
 <ul>
	 <li><strong>Memory systems don't make it easy to decide what
	 to memorize:</strong> Most obviously, we meet a lot of
	 people who use memory systems for poorly chosen
	 purposes. The following is surprisingly close to a
	 transcript of a conversation we've both had many times:
	 <blockquote>
	 “I don’t like [memory system]. I tried to memorize
	 the countries in Africa, and it was boring.”<br />
	 “Why were you trying to remember the countries in
	 Africa?”<br />
	 [blank look of confusion.]
	 </blockquote>
	 It's easy to poke fun at this kind of thing. But we've
	 both done the equivalent in our own memory practices. Even
	 some users of <em>Quantum Country</em> seem to be going
	 through the motions out of some misplaced sense of
	 duty. The question “what will be beneficial to
	 memorize” is fundamental, and answering that
	 question well is not trivial.
	 </li>
	 <li><strong>What's the real impact of the mnemonic medium on
	 people's cognition? How does it change people's
	 behavior?</strong> A famous boxer is supposed to have
	 said that everyone has a plan until they get punched in
	 the face. Regular users of memory systems sometimes
	 report that while they can remembers answers when being
	 tested by their system, that doesn't mean they can
	 recall them when they really need them. There can be a
	 tip-of-the-tongue feeling of “Oh, I know
	 this”, but not actual recall, much less the fluent
	 facility one ultimately wants for effective
	 action. Furthermore, the user may not even recognize
	 opportunities to use what they have learned. More
	 broadly: memory is not an end-goal in itself. It's
	 embedded in a larger context: things like creative
	 problem-solving, problem-finding, and all the many ways
	 there are of taking action in the world. We suspect the
	 impact of memory systems will vary a lot, depending on
	 their design. They may be used as crutches for people to
	 lean on. Or they may be used to greatly enable people to
	 develop other parts of their cognition. We don't yet
	 understand very well how to ensure they're enablers,
	 rather than crutches. But later in the essay we'll
	 describe some other tools for thought that, when
	 integrated with memory systems, may better enable this
	 transition to more effective action.
	 </li>
	</ul>
 
 
 
 <h2>How to invent Hindu-Arabic numerals?</h2>
 <p>
	Let's briefly get away from memory systems. Imagine you're a
	designer living in ancient Rome, working for MDC (Mathematical
	Designs Corporation). A client comes in one day, expressing a
	desire to improve on Roman numerals. Of course, that's not
	literally how they describe their problem to you – more
	likely it's a tax collector wanting to tabulate taxes more
	efficiently, and having some vague notion that MDC may be able
	to help. But to you, an experienced designer, it seems that an
	improved system of numerals may be what they need.
 </p>
 <p>
	How should you respond to this request? From our modern
	vantage point we know a vastly better system of numerals is
	possible, the Hindu-Arabic numerals. Hindu-Arabic numerals
	were, in fact, a great leap in the history of tools for
	thought. Could you, as a designer, have made that leap? What
	creative steps would be needed to invent Hindu-Arabic
	numerals, starting from the Roman numerals? Is there a
	creative practice in which such steps would be likely to
	occur?
 </p>
 <p>
	To be clear: this is a somewhat fanciful thought
	experiment. Many of the ideas needed to get to Hindu-Arabic
	numerals were, in fact, known earlier to the Babylonians, to
	the Greeks, and in other cultures. They were also inchoate in
	the abacus and similar devices. And so we're not asking a
	literal historical question. Rather, it's a question meant to
	stimulate thought: what <em>design process</em> could take you
	from Roman numerals to Hindu-Arabic numerals?
 </p>
 <p>
	We can't know the answer to this question for sure. But it's
	worth pointing out that the Hindu-Arabic numerals aren't just
	an extraordinary piece of design. They're also an
	extraordinary mathematical insight. They involve many
	non-obvious ideas, if all you know is Roman numerals. Perhaps
	most remarkably, the meaning of a numeral
	actually <em>changes</em>, depending on its position within a
	number. Also remarkable, consider that when we add the numbers
	72 and 83 we at some point will likely use 2+3=5; similarly,
	when we add 27 and 38 we will also use 2+3=5, despite the fact
	that the meaning of 2 and 3 in the second sum is completely
	different than in the first sum. In modern user interface
	terms, the numerals have the same affordances, despite their
	meaning being very different in the two cases. We take this
	for granted, but this similarity in behavior is a consequence
	of deep facts about the number system: commutativity,
	associativity, and distributivityThe
	same phenomenon occurs in the conventional grade-school
	algorithms for multiplication and division. One of us has spun
	a short piece
	of <a href="https://twitter.com/michael_nielsen/status/1174420006907473920">discovery
	fiction</a> discussing in more detail the way a hypothetical
	designer might have arrived at these ideas.. All these
	properties (and many more) point to the design and
	mathematical insights being inextricably entangled: the
	mathematical insights are, in some sense, design insights, and
	vice versa.
 </p>
 <p>
	Indeed, it seems fair to say that any person who could invent
	Hindu-Arabic numerals, starting from the Roman numerals, would
	be both one of the great mathematical geniuses who ever lived,
	and one of the great design geniuses who ever lived. They'd
	have to be extraordinarily capable in both domains, capable of
	an insight-through-making loop which used the evolving system
	of numerals to improve not just their own mathematical ideas,
	but to have original, world-class insights into mathematics;
	and also to use those mathematical insights to improve their
	evolving system of numerals.
 </p>
 <p>
	This is rather sobering if we compare to conventional modern
	design practice. In a typical practice, you'd interview domain
	experts (in this case, mathematicians), and read any relevant
	literature. You'd talk to users of existing systems, and
	analyze serious behavior, both individually and at scale. In
	short, you'd do what people in the design community refer to
	as immersing themselves in the target field.
 </p>
 <p>
	This is a powerful practice. At its best it causes systems to
	come into existence which would otherwise be inconceivable. If
	applied to Roman numerals (in hypothetical ancient Rome, not
	todayOf course, a designer who spoke
	to an expert on, say, Babylonian mathematics, might well have
	come across some of these ideas. We'll ignore that, since it
	depends on the oddity that many excellent prior ideas about
	numeral systems had been displaced in Roman culture.)
	this practice would likely improve them a great deal. But it
	would not provide anywhere near the level of mathematical
	insight needed to arrive at Hindu-Arabic numerals.
 </p>
 <p>
	Our story about Hindu-Arabic numerals and mathematics is
	fanciful. But it expresses a general truth: <em>the most
	powerful tools for thought express deep insights into the
	underlying subject matter.</em> In the case of memory
	systems, this means they're not just “applied cognitive
	science”, a collage of existing ideas from cognitive
	science pasted together using modern design practice. Rather,
	they will express deep original insights into memory, insights
	no-one else in the world has ever had. A truly great memory
	system will be cognitive science of the highest order.
 </p>
 <p>
	From this discussion, we take away a warning and an
	aspiration.
 </p>
 <p>
	The warning is this: conventional tech industry product
	practice will not produce deep enough subject matter insights
	to create transformative tools for thought. Indeed, that's
	part of the reason there's been so little progress from the
	tech industry on tools for thought. This sounds like a knock
	on conventional product practice, but it's not. That practice
	has been astoundingly successful at its purpose: creating
	great businesses. But it's also what Alan Kay has dubbed a pop
	culture, not a research culture. To build transformative tools
	for thought we need to go beyond that pop culture.
 </p>
 <p>
	The aspiration is for any team serious about making
	transformative tools for thought. It's to create a culture
	that combines the best parts of modern product practice with
	the best parts of the (very different) modern research
	culture. 
	You need the insight-through-making loop to operate, whereby
	deep, original insights about the subject feed back to change
	and improve the system, and changes to the system result in
	deep, original insights about the subject.
 </p>
 <p></p><p>
	Note that we are not making the common argument that making
	new tools can lead to new subject matter insights for the
	toolmaker, and vice versa. This is correct, but is much weaker
	than what we are saying. Rather: making new tools can lead to
	new subject matter insights <em>for humanity as a whole</em>
	(i.e., significant original research insights), and vice
	versa, and this would ideally be a rapidly-turning loop to
	develop the most transformative tools.
 </p>
 
 <p>
	Doing this is a cultural struggle. It seems to be
	extraordinarily rare to find the insight-through-making loop
	working at full throttle. People with expertise on one side of
	the loop often have trouble perceiving (much less
	understanding and participating in) the nature of the work
	that goes on on the other side of the loop. You have
	researchers, brilliant in their domain, who think of making as
	something essentially trivial, “just a matter of
	implementation”. And you have makers who don't
	understand research at all, who see it as merely a rather slow
	and dysfunctional (and unprofitable) making process. This is
	certainly true in Silicon Valley, where it's common to meet
	accomplished technical makers who, after reading a few stories
	from Richard Hamming and Richard Feynman, think they
	understand research well enough that they can “create
	the new Bell Labs”. Usually they're victims of
	Dunning-Krugeritis, so ignorant they're not even aware of
	their ignorance.
 </p>
 <p>
	Of course, we've got a long way to go with <em>Quantum
	Country</em>. It's not yet generating nearly deep enough ideas
	about memory and cognition; it's not yet one of the world's
	foremost memory laboratories. And considered as a product,
	it's also in the very earliest days; we're not yet iterating
	nearly fast enough, nor learning nearly fast enough from the
	system. Getting the insight-through-making loop to operate at
	full throttle will mean reinventing parts of both research
	culture and conventional product development culture; it will
	mean new norms and a new type of person involved in key
	decision making. But that's the aspiration, and what we
	believe is necessary to develop transformative tools for
	thought.
 </p>
 
 
 
 <p>
	We've examined the mnemonic medium in some depth. The intent
	was to show you the early stages in the development of a
	specific tool for thought, and some of the thinking enabled by
	that development. In this second part of the essay we explore
	more broadly, briefly sketching ideas for several other tools
	for thought. And we'll address some broader questions,
	especially around why there hasn't been more work on tools for
	thought.
 </p>
 <h2>Mnemonic video</h2>
 <p>
	In 2014, the digital artist Eric Wernquist released an
	extraordinary short video entitled
	“<a href="https://vimeo.com/108650530">Wanderers</a>”. The
	video provides a first-person glimpse of what it would be like
	to explore the Solar System:
 </p>
	<p>
 
	</p>
 <p>
	We hear the narrator's (Carl Sagan) wonder and awe, and cannot
	help but empathize with his deep belief in the value of
	exploration. We get a sense of how many mysteries and how much
	beauty there is in our own cosmic neighborhood. The music
	begins with a wistful nostalgia for those of our ancestors who
	dared to explore, and then changes to convey excitement and
	danger and the boldness of those members of our and future
	generations who continue that exploration.
 </p>
 <p>
	It's interesting to contrast the video to the video's script,
	a short text by Carl
	Sagan. The <a href="http://mentalfloss.com/article/60335/wanderers-glimpse-humanitys-possible-future">text</a>
	is beautiful, but reading it is a much more remote and
	cerebral experience, conveying a much less visceral emotional
	understanding.
 </p>
 <p>
	We have a friend, Grant Sanderson, who makes astonishing
	mathematics videos on his YouTube
	channel, <a href="https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw">3Blue1Brown</a>. Below
	is one of our favorites, a video sketching a proof of a
	relatively recent research result in geometry, using ideas
	from algebraic topology. This sounds fearsome, but the video
	is beautiful and accessible, and has been viewed more than 1.2
	million times.
 </p>
 <p>
	</p>
 <p>
	As with <em>Wanderers</em>, watching this video is a
	remarkable emotional experience. It's obvious the person
	narrating the video loves mathematics, and you cannot help but
	empathize. As you watch, you experience repeated
	“Ahah!” moments, moments of surprising insight, as
	connections that were formerly invisible become obvious. It
	shows mathematics as something beautiful, containing
	extraordinary ideas and intriguing mysteries, while at the same
	time showing that doing mathematics is not mysterious, that it
	is something anyone can understand and even do.
 </p>
 <p>
	It's tempting to overlook or undervalue this kind of emotional
	connection to a subject. But it's the foundation of all
	effective learning and of all effective action. And it is much
	easier to create such an emotional connection using video than
	using text.
 </p>
 <p>
	There's a flipside to this emotional connection,
	however. We've often heard people describe Sanderson's videos
	as about “teaching mathematics”. But in
	conversation he's told us he doesn't think more than a small
	fraction of viewers are taking away much detailed
	understanding of mathematics. We suspect this is generally
	true, that high affect videos usually do little to change
	people's detailed intellectual understanding. Rather, the
	extraordinary value of the videos lies in the emotional
	connection they create.
 </p>
 <p>
	Is it possible to create a medium which blends the best
	qualities of both video and text?
 </p>
 <p>
	In particular, is it possible to create a medium which has the
	emotional range possible in video – a range which can be
	used to convey awe and mystery and surprise and beauty? But
	which can also firmly ground that emotional connection in
	detailed understanding, the mastery of details which is the
	raison d'être of both conventional text and, perhaps even
	moreso, of the mnemonic medium?
 </p>
 <p>
	We believe this may be possible, and we plan to develop a
	<em>mnemonic video</em> form that provides both the emotional
	connection possible in video, and the mastery of details
	possible in the mnemonic medium.
 </p>
 <p>
	Creating such a form is challenging. Many MOOC platforms have
	attempted something a little in this vein. The typical approach
	is to have a low-affect talking head video, with the videos
	interrupted occasionally for brief quizzesThese images excerpted from <a href="https://www.coursera.org/learn/astro">Astronomy:
	Exploring Time and Space</a>.. Here's how it works on
	one MOOC platform, Coursera:
	</p>
	
 <p>
	Other MOOCs differ in the details. But the overall emotional
	experience may be summed up as:
 </p>
	
 <p>
	The very best parts of the video may be emotionally
	compelling, though it's rare that they achieve the emotional
	range and connection of the best videos from people like Grant
	Sanderson. And the overall emotional experience is
	disjointed, almost repellent. Is it possible to create an
	integrated medium, with a unified and carefully crafted
	emotional and intellectual experience? Ideally, is it possible
	to create something like the following?
 </p>
	
 <p>
	In MOOCs, questions are typically presented in a very dry
	form, detached from context. In the mnemonic video the
	narrator would explain why the questions are important, and
	why the user will benefit from participating, as a seamless
	part of the overall narration. Done right – perhaps with
	appropriate music, and a sense of urgency or play in the
	narration – it would create a real sense of the
	stakes. At the same time, the video player can be modified so
	the user can respond directly to questions, as part of the
	spaced-repetition experience. The result would be much softer
	transitions between the high-affect core narration and the
	moderate-affect questions.
 </p>
 <p>
	Here's a short sketch of one approach to doing
	thisThe narrator's script in this
	sketch is adapted from <em>Cosmos</em> (1984), episode
	4., showing how the narrator could ask questions aloud
	as an integrated part of the overall narration:
 </p>
	
 <p>
	It seems likely that the rhythm of mnemonic videos would be
	quite different to mnemonic essays. In particularly, the
	frequency and density of questions would be lower than in the
	mnemonic medium, and it would be necessary to test different
	beats and cadences to ensure a good balance of emotional and
	intellectual experience. Even high-affect video typically has
	quieter moments; it achieves the high affect in part by
	contrast to the lower-intensity moments. Think about the way a
	good action movie or thriller needs lulls; if it's too
	high-intensity all the time, eventually our emotional response
	is dulled. We could design mnemonic video so the questions
	help fill this lower-intensity emotional beat.
 </p>
 <p>
	Of course, this is merely a quick sketch of one approach to
	the design of mnemonic video. Ideally, there would also be a
	spaced-repetition component, perhaps with the questions asked
	in text instead of video. This sort of sketch seems to us a
	promising direction, but needs considerable development and
	intense testing. In particular, we need to do detailed,
	second-by-second user experience testing, to understand and
	shape users' emotional and intellectual experience. That would
	continue until we were confident that our target users were
	having the desired experience. Ideally, we'd also generate
	several more very different designs, and try to understand how
	each approach would impact the user's emotional and
	intellectual experience.
 </p>
 <p>
	The broader point here is about taking emotion
	seriously. Historically, a lot of work on tools for thought
	has either ignored emotion, or treated it as no more than a
	secondary concern. Instead, that work has focused on new
	skills acquired, on what the user “learns”.
	They've been designing for Spock, when emotional connection is
	a high-order bit. Do users feel disinterested? Afraid?
	Hostile? Anxious? Or do they internalize a sense of
	excitement, of beauty, perhaps even an expansion in their own
	goals, an expansion of their self?
 </p>
 <p>
	By contrast, media forms such as movies and music and (often)
	video games do take emotion seriously. The designers of such
	forms often have incredibly elaborate models of user's
	emotional responses. These models range from detailed,
	second-by-second understanding, to deep thinking about a
	user's overall emotional journey. We believe it's possible and
	desirable to use such approaches in the development of tools
	for thought.
 </p>
 <p>
	At the same time, a positive emotional experience alone is not
	enough. For tools for thought to attain enduring power, the
	user must experience a real growth in mastery, an expansion in
	their ability to act. And so we'd like to take both the
	emotional and intellectual experience of tools for thought
	seriously. Mnemonic video is a good venue for such
	exploration. To paraphrase Einstein, attaining a detailed
	understanding without forming an emotional connection is lame;
	while forming an emotional connection without detailed
	understanding has no enduring power.
 </p>
 
 <h2>Why isn't there more work on tools for thought today?</h2>
 <p>
	If tools for thought are so great, why isn't more work being
	done on them? Why aren't they a major industry?
 </p>
 <p>
	As noted in the introduction, there's certainly a lot of lip
	service paid. It is, for instance, common to hear
	technologists allude to Steve Jobs's metaphor of computers as
	“bicycles for the mind”. But in practice it's
	rarely more than lip service. Many pioneers of computing have
	been deeply disappointed in the limited use of computers as
	tools to improve human cognition. Douglas
	Engelbart <a href="http://dougengelbart.org/content/view/348/000/">disparaged</a>
	the “dangerous, disappointing, narrow, path that we seem
	to be stuck with following”. When asked in 2006 how much
	of his vision had been achieved, Engelbart replied
	facetiously <a href="https://www.smithsonianmag.com/innovation/douglas-engelbart-invented-future-180967498/">“about
	2.8 percent”</a>. Alan Kay gives talks
	asserting <a href="http://www.vpri.org/pdf/m2007007a_revolution.pdf">“The
	real computer revolution hasn't happened yet”</a> and in
	<a href="https://www.fastcompany.com/40435064/what-alan-kay-thinks-about-the-iphone-and-technology-now">an
	interview</a> has described the modern web as
	“reinventing the flat tire… at least give us what
	Engelbart did, for Christ's sake.”
 </p>
 <p>
	Our experience is that many of today's technology leaders
	genuinely venerate Engelbart, Kay, and their colleagues. Many
	even feel that computers have huge potential as tools for
	improving human thinking. But they don't see how to build good
	businesses around developing new tools for thought. And
	without such business opportunities, work languishes.
 </p>
 <p>
	What makes it difficult to build companies that develop tools
	for thought? To answer this, consider Adobe, one of the few
	large companies serious about developing new tools for
	thoughtAnother large company which
	takes tools for thought extremely seriously is AutoDesk. A
	similar story could be told about it.. It's poured
	money into developing new mediums The
	plural of medium is, of course, media. However, in this
	context media would usually mean many pieces of new
	content. That's not what we mean: we mean multiple different
	new mediums
	(<em>Illustrator</em>, <em>Photoshop</em> <em>etc</em>). We'll
	reserve the unusual pluralization for this somewhat unusual
	meaning. for designers and artists – programs
	such as <em>Illustrator</em>, <em>Photoshop</em>, and so
	on. These mediums are remarkable tools for thought.
 </p>
 <p>
	Unfortunately for Adobe, such mediums are extremely expensive
	to develop, and it's difficult to prevent other companies from
	cheaply copying the ideas or developing
	near-equivalents. Consider, for example, the way the
	program <em>Sketch</em> has eaten into Adobe's market share,
	after duplicating many of the best features from several of
	Adobe's products, perhaps most notably
	<em>Illustrator</em>. And consider the way <em>Figma</em> is
	now eating into both <em>Sketch</em>
	and <em>Illustrator</em>'s market share. Both <em>Sketch</em>
	and <em>Figma</em> have done this without needing to make an
	enormous investment in research. That's a big advantage they
	have over Adobe.
 </p>
 <p>
	As Marc Andreessen has observed In
	 Elad Gil's “High Growth Handbook”
	 (2018).:
 </p>
 <blockquote>
	true defensibility purely at the product level is really rare
	in [Silicon] Valley, because there are a lot of really good
	engineers… And then there's the issue of
	leap-frogging. The next team has the opportunity to learn from
	what you did and then build something better.
 </blockquote>
 <p>
	Put another way, many tools for thought are public goods. They
	often cost a lot to develop initially, but it's easy for
	others to duplicate and improve on them, free riding on the
	initial investment. While such duplication and improvement is
	good for our society as a whole, it's bad for the companies
	that make that initial investment. And so such tools for
	thought suffer the fate of many public goods: our society
	collectively underinvests in them, relative to the benefits
	they provideOf course, it does cost
	money for companies such as <em>Sketch</em> and <em>Figma</em>
	to duplicate features originating in <em>Illustrator</em>, and
	they have introduced some improvements. So our
	characterization as a public good is only
	approximate. .
 </p>
 <p>
	Earlier, we argued that modern design practice generally isn't
	up to the challenge of producing genuinely transformative
	tools for thought. On the surface, that process-level argument
	appears very different to the public goods argument we just
	made. In fact, the process-level explanation is a consequence
	of the public goods explanation: companies don't use the
	necessary processes because there's little value to them in
	doing so. By contrast, in “harder-tech” industries
	– say, chip design – companies have much more
	incentive to do deep research work. In those industries it's
	considerably harder for other companies to duplicate or
	capture the value of that research.
 </p>
 <p>
	It's illuminating to contrast with video games. Game companies
	develop many genuinely new interface ideas. This perhaps seems
	surprising, since you'd expect such interface ideas to also
	suffer from the public goods problem: game designers need to
	invest enormous effort to develop those interface ideas, and
	they are often immediately copied (and improved on) by other
	companies, at little cost. In that sense, they are public
	goods, and enrich the entire video game ecosystem.
 </p>
 <p>
	But there's a big difference between video game companies and
	companies such as Adobe. Many video games make most of their
	money from the first few months of sales. While other
	companies can (and do) come in and copy or riff on any new
	ideas, it often does little to affect revenue from the
	original game, which has already made most of its
	money In fact, cloning is a real
	issue in gaming, especially in very technically simple
	games. An example is the game <em>Threes</em>, which took the
	developers more than a year to make. Much of that time was
	spent developing beautiful new interface ideas. The resulting
	game was so simple that clones and near-clones began appearing
	within days. One near clone, a game called <em>2048</em>,
	sparked a mini-craze, and became far more successful
	than <em>Threes</em>. At the other extreme, some game
	companies prolong the revenue-generating lifetime of their
	games with re-releases, long-lived online versions, and so
	on. This is particularly common for capital-intensive AAA
	games, such as the <em>Grand Theft Auto</em> series. In such
	cases the business model relies less on clever new ideas, and
	more on improved artwork (for re-release), network effects
	(for online versions), and branding.
	. While this copying is no doubt irritating for the
	companies being copied, it's still worth it for them to make
	the up-front investment.
 </p>
 <p>
	The net result is that in gaming, clever new interface ideas
	can be distinguishing features which become a game's primary
	advantage in the marketplace. Indeed, new interface ideas may
	even help games become classics – consider the many
	original (at the time) ideas in games ranging from <em>Space
	Invaders</em> to <em>Wolfenstein 3D</em> to <em>Braid</em>
	to <em>Monument Valley</em>. As a result, rather than
	underinvesting, many companies make sizeable investments in
	developing new interface ideas, even though they then become
	public goods. In this way the video game industry has largely
	solved the public goods problems.
 </p>
 <p>
	By contrast, a company like Adobe builds their business around
	distribution and long-term lock in. They convince people
	– indeed, entire organizations – to make long-term
	commitments to their products. Schools offer classes so people
	can call themselves “<em>Photoshop</em> experts”
	or “<em>Illustrator</em> experts”. Companies
	designate their design departments as “Adobe
	shops”. So while Adobe does invest in developing clever
	new interface ideas (for them, unlike the video game
	companies, this genuinely means tools for thought), it's less
	central to their competitive advantage, and they invest less
	than they would if it was their central advantage. And Adobe
	does perhaps as much or more work developing tools for thought
	as any company.
 </p>
 <p>
	It's encouraging that the video game industry can make inroads
	on the public goods problem. Is there a solution for tools for
	thought? Unfortunately, the novelty-based short-term revenue
	approach of the game industry doesn't work. You want people to
	really master the best new tools for thought, developing
	virtuoso skill, not spend a few dozen hours (as with most
	games) getting pretty good, and then moving onto something
	new.
 </p>
 <p>
	Another plausible solution to the public goods problem is
	patents, granting a temporary monopoly over use of an
	invention. Many software companies, including Adobe, develop a
	large patent portfolio. However, the current patent system is
	not a solution for this problem. In 2017, Dana Rao, Adobe's
	Vice President for Intellectual Property and Litigation,
	posted
	a <a href="https://morningconsult.com/opinions/improving-patents-will-not-kill-innovation/">call</a>
	for major reforms to the patent system, stating that:
 </p>
 <blockquote>
	[the patent] system is broken… What happened? A patent
	gold rush built by patent profiteers… Their value lies
	not in the innovation behind the patent but in the vagueness
	of the patent's claims and the ability to enforce it in a
	plaintiff-friendly forum… Where did the material for
	these bad patents come from? The advent of software…
	This led to idea-only patents being granted with broad and
	often invalid claims, and eager patent profiteers were only
	too glad to take advantage.
 </blockquote>
 <p>
	Adobe shares in common with many other software companies that
	much of their patenting is defensive: they patent ideas so
	patent trolls cannot sue them for similar ideas. The situation
	is almost exactly the reverse of what you'd like. Innovative
	companies can easily be attacked by patent trolls who have
	made broad and often rather vague claims in a huge portfolio
	of patents, none of which they've worked out in much
	detail. But when the innovative companies develop (at much
	greater cost) and ship a genuinely good new idea, others can
	often copy the essential core of that idea, while varying it
	enough to plausibly evade any patent. The patent system is not
	protecting the right things.
 </p>
 <p>
	Switching away from the viewpoint of individual companies, and
	to the viewpoint of society as a whole, not only do we want to
	incentivize invention, we also want ideas to move reasonably
	rapidly into the public domain. Think about fundamental tools
	for thought such as writing and the number system. Obviously,
	it's good that those spread throughout society, unencumbered
	by IP concerns! More broadly, many tools of thought become
	more valuable for society as they become more
	ubiquitous. Again, here the modern patent system has numerous
	well-known problems, striking a poor balance between private
	and public interest. While a well-designed patent system might
	very well help solve the public goods problem, the patent
	system we actually have seems poorly adapted to the problem.
 </p>
 <p>
	Is it possible to avoid the public goods problem altogether?
	Here's three classes of tools for thought which do:
 </p>
 <ul>
	 <li>Search engines such as Google are tools for
	 thought. They avoid the public goods problem because their
	 value is in their brand and in hard-to-duplicate and
	 capital intensive backend elements (including their data
	 centers, proprietary algorithms, ad network, and
	 distribution), not in their interface ideas.
	 </li>
	 <li>A service such as Twitter can be considered a tool for
	 collective thought. While the interface is easily copied,
	 the company is hard to duplicate, due to network effects.
	 </li>
	 <li>
	 Novel hardware devices (e.g., for VR, or the Wii remote,
	 or for new musical instruments) can be used as the basis
	 for new tools for thought. While hardware can be
	 duplicated, it's often much more expensive than
	 duplicating software. And, in any case, the advantage for
	 such companies is often in distribution, marketing, and
	 relationships with vendors who make products for the
	 platform.
	 </li>
	</ul>
 
 
 <p>
	While these suggestions all avoid the public goods problem,
	they don't directly solve the public goods problem. And many
	promising directions – including ideas such as the
	mnemonic medium and mnemonic videos – involve a
	substantial public goods element. Is it possible to solve the
	public goods problem in such cases? The two most promising
	approaches seem to us to be:
 </p>
 <ul>
	 <li>Philanthropic funding for research. This approach was
	 used, for instance, by the field of computer animation and
	 animated movies. Decades of public research work on computer
	 animation resulted in a large number of powerful and (in
	 many cases) publicly available ideas. This, in turn, helped
	 prepare the way for companies such as Pixar and Dreamworks,
	 which developed many of the ideas further, and took them to
	 scale.
	 </li>
	 <li>The model used by Adobe and similar companies, in which
	 new tools for thought are a central part of the company's
	 operations, but not the core of their competitive moat. That
	 moat may instead be built around training, marketing,
	 documentation, and so on.
	 </li>
	</ul>
 
 <h2>Questioning our basic premises</h2>
 <p>
	There are three important premises we've taken for granted up
	to now. First is the assertion that we're still in the early
	days, that many more transformative tools for thought are yet
	to be discovered. Second is the assertion that work on tools
	for thought is stalled, that there's not lots of interesting
	work going on. And third, a kind of meta-premise, is the
	assertion that this kind of work is worth doing, relative to
	the current fashion for related ideas such as artificial
	general intelligence and brain-computer interfaces. In this
	section we discuss these premises.
 </p>
 <h3>What if the best tools for thought have already been discovered?</h3>
 <p>
	In other words, perhaps the 1960s and 1970s were an
	unrepeatable golden age, and all we can expect in the future
	is gradual incremental improvement, and perhaps the occasional
	major breakthrough, at a decreasing frequency?
 </p>
 <p>
	There's a plausible story suggesting this is true. Tech is an
	enormous industry, well funded, with many bright, ambitious,
	talented people. Surely if there were major ideas to discover,
	people would do so? This argument is reinforced by the fact
	that, at the individual level, we meet many brilliant people
	who are fascinated by (and often working on) tools for
	thought, but who nonetheless seem to be making slow progress.
 </p>
 <p>
	But while this story has a superficial appeal, it's
	misleading. Really difficult problems – problems like
	inventing Hindu-Arabic numerals – aren't solved by good
	intentions and interest alone. A major thing missing is
	foundational ideas powerful enough to make progress. In the
	earliest days of a discipline – the proto-disciplinary
	stage – a few extraordinary people – people like
	Ivan Sutherland, Doug Engelbart, Alan Kay, and Bret Victor
	– may be able to make progress. But it's a very bespoke,
	individual progress, difficult to help others become
	proficient in, or to scale out to a community. It's not yet
	really a discipline. What's needed is the development of a
	powerful praxis, a set of core ideas which are explicit and
	powerful enough that new people can rapidly assimilate them,
	and begin to develop their own practice. We're not yet at that
	stage with tools for thought. But we believe that we're not so
	far away either.
 </p>
 <p>
	While that argument is helpful context, it doesn't address the
	core point: it doesn't mean there are a lot of new
	transformative tools for thought waiting to be discovered.
	Again: maybe the most important tools for thought have already
	been discovered?
 </p>
 <p>
	We can't predict the future, so it's not possible to answer
	this question with certainty. But it seems to us that the
	human race just hasn't really tried very hard yet. When small
	groups of motivated people do – as in pioneering labs
	such as PARC, SRI, and other DARPA-inspired early efforts, as
	well as modern labs such as Dynamicland – they make
	rapid progress. It's extremely encouraging that those efforts
	– tiny efforts, in the scheme of humanity's overall
	research effort – make such rapid progress. To us, that
	suggests scaling them up, becoming much more ambitious.
 </p>
 <h3>Isn't this what the tech industry does? Isn't there a lot of ongoing progress on tools for thought?</h3>
 <p>
	In particular, aren't there already a lot of imaginative,
	determined, well-funded people working on this? Isn't tech in
	considerable part already about developing new tools for
	thought?
 </p>
 <p>
	Part of this question is caused by a confusion in
	terms. Obviously, many tech companies build special-purpose
	tools for solving specific problems. But while those may be
	valuable tools, they're certainly not “tools for
	thought” in the broad sense we're discussing – not
	like language or writing or, for that matter, Illustrator.
 </p>
 <p>
	Still, there are tech companies which really do develop tools
	for thought. We already discussed some examples where
	companies have partially or totally avoided the public goods
	problem, tools such as: Illustrator, Google Search, Twitter,
	Slack, Google Docs, programmer
	toolsProgrammer tools are a case
	where the insight-through-making loop operates quite well.
	The avoidance of the public goods problem is enabled by a
	complex set of factors, including the fact that this is a case
	where many highly skilled researchers are simultaneously also
	accomplished makers, and it's either their job to produce
	public goods (researchers) or an enjoyable hobby. To a lesser
	extent the same is true in the artistic and music
	communities., and so on. All really are significant
	tools for thought.
 </p>
 <p>
	But consider our most fundamental tools for thought –
	language, writing, music, <em>etc</em>. Those are public
	goods. No-one owns language; to the extent that it is owned
	(trademarks and so on) it may actually limit the utility of
	language. These tools are all about introducing fundamental
	new mental representations and mental operations. Those aren't
	owned by any company, they're patterns owned by humanity.
 </p>
 <p>
	This argument makes it seem likely that many of the most
	fundamental and powerful tools for thought do suffer the
	public goods problem. And that means tech companies focus
	elsewhere; it means many imaginative and ambitious people
	decide to focus elsewhere; it means we haven't developed the
	powerful practices needed to do work in the area, and a result
	the field is still in a pre-disciplinary stage. The result,
	ultimately, is that it means the most fundamental and powerful
	tools for thought are undersupplied.
 </p>
 <h3>Why not
	work on artificial general intelligence (AGI) or
	brain-computer interfaces (BCI) instead?</h3>
 
 <p>
	We're often asked: why don't you work on AGI
	or BCI instead of tools for thought? Aren't those more
	important and more exciting? And for AGI, in particular, many
	of the skills required seem related.
 </p>
 <p>
	They certainly are important and exciting subjects. What's
	more, at present AGI and BCI are far more fashionable (and
	better funded). As a reader, you may be rolling your eyes,
	supposing our thinking here is pre-determined: we wouldn't be
	writing this essay if we didn't favor work on tools for
	thought. But these are questions we've wrestled hard with in
	deciding how to spend our own lives. One of us wrote a book
	about artificial intelligence before deciding to focus
	primarily on tools for thought; it was not a decision made
	lightly, and it's one he revisits from time to time. Indeed,
	given the ongoing excitement about AGI and BCI, it would be
	surprising if people working on tools for thought didn't
	regularly have a little voice inside their head saying
	“hey, shouldn't you be over there instead?”
	Fashion is seductive.
 </p>
 <p>
	One striking difference is that AGI and BCI are based on
	relatively specific, well-defined goals. By contrast, work on
	tools for thought is much less clearly defined. For the most
	part we can't point to well-defined, long-range goals; rather,
	we have long-range visions and aspirations, almost
	evocations. The work is really about exploration of an
	open-ended question: how can we develop tools that change and
	expand the range of thoughts human beings can think?
 </p>
 <p>
	Culturally, tech is dominated by an engineering, goal-driven
	mindset. It's much easier to set KPIs, evaluate OKRs, and
	manage deliverables, when you have a very specific end-goal in
	mind. And so it's perhaps not surprising that tech culture is
	much more sympathetic to AGI and BCI as overall programs of
	work.
 </p>
 <p>
	But historically it's not the case that humanity's biggest
	breakthroughs have come about in this goal-driven way. The
	creation of language – the ur tool for thought –
	is perhaps the most important occurrence of humanity's
	existence. And although the origin of language is hotly
	debated and uncertain, it seems extremely unlikely to have
	been the result of a goal-driven process. It's amusing to try
	imagining some prehistoric quarterly OKRs leading to the
	development of language. What sort of goals could one possibly
	set? Perhaps a quota of new irregular verbs? It's
	inconceivable!
 </p>
 <p>
	Similarly, the invention of other tools for thought –
	writing, the printing press, and so on – are among our
	greatest ever breakthroughs. And, as far as we know, all
	emerged primarily out of open-ended exploration, not in a
	primarily goal-driven way. Even the computer itself came out
	of an exploration that would be regarded as ridiculously
	speculative and poorly-defined in tech today. Someone didn't
	sit down and think “I need to invent the
	computer”; that's not a thought they had any frame of
	reference for. Rather, pioneers such as Alan Turing and Alonzo
	Church were exploring extremely basic and fundamental (and
	seemingly esoteric) questions about logic, mathematics, and
	the nature of what is provable. Out of those explorations the
	idea of a computer emerged, after many years; it was a
	discovered concept, not a goal. Fundamental, open-ended
	questions seem to be at least as good a source of
	breakthroughs as goals, no matter how ambitious. This is
	difficult to imagine or convince others of in Silicon Valley's
	goal-driven culture. Indeed, we ourselves feel the attraction
	of a goal-driven culture. But empirically open-ended
	exploration can be just as, or more successful.
 </p>
 <p>
	“What will new tools for thought be like?” is a
	question we hear often. And yet, almost by definition, we
	cannot say. As we noted earlier, if we could communicate the
	experience in an essay, then the tools would be failing at
	their job; they would not be transforming a person's thinking,
	or even their consciousness. Concretely: to understand the
	mnemonic medium you must use it intensively over an extended
	period. And even then you may not be conscious of the effect;
	we've done interviews with users who are apparently unaware of
	the incredible level of recall they have of material in the
	essay they have read. One of the most famous papers in the
	philosophy of consciousness is entitled “What is it like
	to be a bat?” Each tool for thought poses a similar
	question, near impossible to answer without immersion in the
	tool: “What is it like to be a language user? A
	musician?” and so on.
 </p>
 <p>
	It seems plausible to us that work on tools for thought will
	be, over the next few decades, more important than work on AGI
	and BCI. And, given how fashionable and well-funded work on
	AGI and BCI currently is, it seems nearly certain that work on
	tools for thought offers vastly greater benefit, at the
	margin.
 </p>
 <p>
	What about the longer term? There, the situation is less
	clear. It seems likely that the three fields will merge, or at
	least feed strongly into one another. Together with Shan
	Carter, one of us has argued that one of the most promising
	applications for AI is as a way of discovering new tools for
	thought Shan Carter and Michael
	Nielsen, <a href="https://distill.pub/2017/aia/">Using
	Artificial Intelligence to Augment Human Intelligence</a>
	(2017)..
 </p>
 <p>
	BCI seems likely to be even more closely related. BCI is
	sometimes described using ideas like a memory chip for
	long-term memories, or some way of increasing short-term
	working memory. Such ideas may well become important. But it
	also seems possible that BCIs will be used to enable new
	mental operations, new mental representations, and new
	affordances for thought; in short, the same kind of things as
	are involved in developing non-BCI tools for thought. Perhaps
	we'll develop the capacity to directly imagine ourselves in 4
	or 5 or more dimensions; or traversing a Riemann manifold; or
	the ability to have multiple tracks of conscious
	attention. These are about changing the interface for thought,
	the basic abstractions and operations which are allowed. And
	so it seems plausible that work today on tools for thought
	will directly impact the way we use BCIs in the future.
 </p>
	
 <h2>Executable books</h2>
 <blockquote>
	The skill of writing is to create a context in which other
	people can think.<p>— Edwin Schlossberg</p>
 </blockquote>
 <p>
	The computer scientist Peter Norvig has written
	an <a href="https://github.com/norvig/pytudes/blob/master/ipynb/Economics.ipynb">interactive
	essay</a> discussing the distribution of wealth in
	society. Norvig's essay is a Jupyter notebook which expresses
	many of the ideas in running Python code. That code sets up a
	population of agents, with an initial distribution of
	wealth. Agents randomly (and repeatedly) meet one another in
	pairs, and engage in simple economic transactions. More
	concretely: a simple transaction model could be that when two
	people meet, their joint wealth is pooled, and then randomly
	divided between the two of them. That model is just to give
	you the gist – more complex transaction models are, of
	course, possible. The notebook simulates how the distribution
	of wealth evolves over time.
 </p>
 <p>
	Part of what makes Norvig's essay beautiful is that with just
	a few lines of Python code Norvig is able to show some
	surprising results about wealth inequality. For instance, his
	results suggest that the initial distribution of wealth in the
	economy doesn't much affect the long-run distribution of
	wealth. Rather, it's the nature of the transactions which
	determines the long-run distribution of wealth. This likely
	violates at least some user's intuitions. As another example,
	his results also suggest that constraining agents to trade
	only with people geographically near them makes little
	difference to the final distribution of wealth.
 </p>
 <p>
	Results like these will challenge the intuition of some
	users. But instead of those challenges being on the basis of
	easily-ignored abstract arguments, users can immediately
	engage with Norvig's model. Suppose someone doesn't like the
	idea that the initial distribution of wealth doesn't affect
	long-run wealth inequality. They're challenged to find a
	counterexample, an initial distribution of wealth which does
	affect long-run inequality. They can experiment easily, making
	simple modifications to just one or a few lines of Python
	code, trying to find instances where the initial distribution
	matters. No matter whether they succeed or fail, they will
	build a better understanding of the
	problemSufficiently motivated
	reasoners will, of course, ignore any conclusion they don't
	like. Such people are most likely a lost cause to serious
	thought..
 </p>
 <p>
	Suppose the content of Norvig's essay had instead been
	presented in a more conventional static form. For a reader to
	extend or interrogate the results would require total mastery
	of the material, and a high level of mathematical
	competence. But in the notebook format it's much easier for
	the reader to experiment. Their exploration is scaffolded,
	they can make small modifications and see the results, even
	the answers to questions Norvig did not anticipate. This kind
	of scaffolded exploration is a way to build up their own
	understanding, and perhaps even push the frontiers of
	knowledge.
 </p>
 
 <p>
	Norvig's essay is one of thousands (or perhaps even millions)
	of Jupyter notebooks that have been created. Of course, most
	such notebooks are hastily and poorly written. But in the
	hands of an excellent writer and thinker like Norvig,
	notebooks can become remarkable environments for thought, both
	individual and shared. It's tempting to regard them as merely
	a mashup of essay and code. But really they're a new media
	form, with different possibilities from either essays or code,
	and with striking opportunities to go much further. In this
	section we explore those
	opportunitiesOf course, systems like
	Jupyter go back decades. There are antecedents in Knuth's
	notion of literate programming, in Mathematica notebooks, in
	PARC's Learning Research Group, in the PLATO system (and, more
	broadly, computer-assisted instruction), to name but a few. In
	the discussion below we emphasize opportunities that seem to
	us undervalued in many of these prior systems..
 </p>
 <p>
	We described Norvig's essay as an “interactive
	essay”. It's useful to have a more specific term, to
	distinguish it from other interactive forms, like the mnemonic
	medium. In this essay, we'll use the term “executable
	book”This is, of course, a
	placeholder term. “Executable essay” would in many
	ways be more natural, though unfortunately that term is
	perhaps even more naturally interpreted as meaning “web
	page”, in the current context.. We won't define
	this precisely here; definition is not the point. Rather, the
	point is to try to better understand the potential of media
	forms which combine prose and code in something like this
	form.
 </p>
 <h3>Tools for thought must be developed while doing
	serious work. The aspiration to canonical content</h3>
 
 <p>
	Seymour Papert, one of the principal creators of the Logo
	programming language, had a remarkable aspiration for
	Logo. Logo is sometimes described as a “programming
	language for children”, and people sometimes think
	Papert was mostly interested in helping children learn how to
	program. But that wasn't Papert's principal intent. Rather,
	Papert wanted to create an immersive environment – a
	kind of “Mathland” – in which children could
	be immersed in mathematical ideas. In essence, children could
	learn differential geometry by going to Mathland.
 </p>
 <p>
	It's a beautiful aspiration, and Logo contains many striking
	ideas. But as far as we know, no professional differential
	geometer (or, more generally, mathematician) uses Logo
	seriously as a tool in their work. And upon reflection that
	seems troubling. If Logo genuinely expresses the ideas of
	differential geometry, why don't differential geometers use
	it? You start to wonder: might it be that Logo leaves out
	important ideas about differential geometry, maybe even the
	most important ideas about differential geometry? After all,
	while mathematically trained, Papert wasn't himself an
	accomplished differential geometer. How would he even have
	known what to include? And certainly most of the people
	interested in Logo aren't qualified to make that judgment.
 </p>
 <p>
	There's a standard retort to this, which we've heard from
	within the Logo community. It's to talk about the
	“floor” and “ceiling” of different
	environments for thought. In this account, Logo has a low
	floor (meaning anyone can use it) and a low ceiling (so it's
	not well adapted for the sort of advanced work a professional
	would want to do).
 </p>
 <p>
	At first this sounds plausible. But upon reflection it's
	difficult to make much sense of. How do the creators of Logo
	know that mastering Logo helps later with understanding real
	(forgive us!) differential geometry? What's the criterion for
	success? One of us (MN) worked for several years doing
	research in the closely related field of Riemannian
	geometry. While Logo is enjoyable to use and contains many fun
	ideas, MN has trouble seeing that learning Logo would help
	much in learning differential geometry.
 </p>
 <p>
	At the end of Norvig's economics essay is a short afterword
	explaining how he came to write the essay. Shortly before
	writing the essay he'd heard about the kinds of economic
	models discussed in the notebook, and he wanted to explore
	several questions about them. After talking it over with some
	colleagues they decided to each independently attack the
	problems, and to compare notes. Although Norvig's essay is, in
	some sense, “educational”, Norvig's intent was to
	explore a set of problems he himself was genuinely curious
	about. The educational aspect was a byproduct.
 </p>
 <p>
	And so what you have is a world-class research scientist who
	wanted to explore a set of questions. He used the Jupyter
	medium to do those explorations, and then to share that
	exploration with the world. And he shared it in a form where
	others could immediately build upon and extend his thinking.
 </p>
 <p>
	There's a lot of work on tools for thought that takes the form
	of toys, or “educational” environments. Tools for
	writing that aren't used by actual writers. Tools for
	mathematics that aren't used by actual mathematicians. And so
	on. Even though the creators of such tools have good
	intentions, it's difficult not to be suspicious of this
	pattern. It's very easy to slip into a cargo cult mode, doing
	work that seems (say) mathematical, but which actually avoids
	engagement with the heart of the subject. Often the creators
	of these toys have not ever done serious original work in the
	subjects for which they are supposedly building tools. How can
	they know what needs to be included?
 </p>
 <p>
	Concretely: suppose you want to build tools for subject X (say
	X = differential geometry). Unless you are deeply involved in
	practicing that subject, it's going to be extremely difficult
	to build good tools. It'll be much like trying to build new
	tools for carpentry without actually doing any carpentry
	yourself. This is perhaps part of why tools
	like <em>Mathematica</em> work quite well – the
	principal designer, Stephen Wolfram, has genuine research
	interests in mathematics and physics. Of course, not all parts
	of <em>Mathematica</em> work equally well; some parts feel
	like toys, and it seems likely those are the ones <em>not</em>
	being used seriously internal to the company.
 </p>
 <p>
	There's a general principle here: <em>good tools for thought
	 arise mostly as a byproduct of doing original work on
	 serious problems</em>. They tend either be created by the
	 people doing that work, or by people working very closely to
	 them, people who are genuinely bought
	 in A related argument has been made
	 in Eric von Hippel's book “Democratizing
	 Innovation” (2005), which identifies many instances
	 where what appears to be commercial product development is
	 based in large or considerable part on innovations from
	 users.. Furthermore, the problems themselves are
	 typically of intense personal interest to the
	 problem-solvers. They're not working on the problem for a
	 paycheck; they're working on it because they desperately
	 want to know the answer.
 </p>
 <p>
	Many people have asked why we wrote our first mnemonic essay
	about quantum computing. If we'd chosen an easier subject we
	could have attracted a much larger audience. But we also
	wanted the essay to be authentic, to be about problems we
	wanted to solve. One of us (MN) has done a lot of original
	research work on quantum computing. The essay reflects that
	thinking. Indeed, the framing of the essay is about answering
	a question MN personally wanted to answer: if humans ever
	discovered aliens, would they have computers, and if so, what
	types of computer would they have? This perhaps sounds like a
	contrived question, but it's quite serious, and turns out to
	be a deep question with a nontrivial answer: writing the essay
	helped MN substantially improve his understanding of the
	question.
 </p>
 <p>
	That said, answering that question wasn't the principal point
	of creating the essay: making the mnemonic medium was. And for
	future work on tools for thought, it'd be valuable to push
	much harder on questions we'd genuinely like to answer
	ourselves. That's a way of keeping yourself honest, ensuring
	you're not just building a flashy toy, but something genuinely
	useful for solving real problems that are of independent
	interest.
 </p>
 <p>
	In serious mediums, there's a notion of <em>canonical
	 media</em>. By this, we mean instances of the medium that
	 expand its range, and set a new standard widely known
	 amongst creators in that medium. For instance, <em>Citizen
	 Kane</em>, <em>The Godfather</em>, and <em>2001</em> all
	 expanded the range of film, and inspired later film
	 makers. It's also true in new media. YouTubers like Grant
	 Sanderson have created canonical videos: they expand the
	 range of what people think is possible in the video
	 form. And something like the <em>Feynman Lectures on
	 Physics</em> does it for textbooks. In each case one gets
	 the sense of people deeply committed to what they're
	 doing. In many of his lectures it's obvious that Feynman
	 isn't just educating: he's reporting the results of a
	 lifelong personal obsession with understanding how the world
	 works. It's thrilling, and it expands the form.
 </p>
 <p>
	We've been disappointed by how unambitious people are in this
	sense with <em>Jupyter</em> notebooks. They haven't pushed the
	medium all that hard; there is no <em>Citizen Kane</em> of
	Jupyter notebooks. Indeed, we're barely beyond the Lumière
	brothers. Examples like Norvig's notebook are fine work, but
	seem disappointing when evaluated as leading examples of the
	medium.
 </p>
 <p>
	Aspiring to canonicity, one fun project would be to take the
	most recent IPCC climate assessment report (perhaps starting
	with a small part), and develop a version which is
	executable. Instead of a report full of assertions and
	references, you'd have a live climate model – actually,
	many interrelated models – for people to explore. If it
	was good enough, people would teach classes from it; if it was
	really superb, not only would they teach classes from it, it
	could perhaps become the creative working environment for many
	climate scientists.
 </p>
 <p>
	One promising exploration in this direction
	is <em><a href="https://mitpress.mit.edu/sites/default/files/titles/content/sicm_edition_2/book.html">The
	Structure and Interpretation of Classical Mechanics</a></em>,
	a beautiful executable book building up classical
	mechanics. Many theorems of classical mechanics aren't just
	expressed in static form, on the page, but live, as code which
	can be modified by the user. Theorems become APIs, which can
	literally be applied to other objects, and chained
	together. It uses a much more powerful underlying model than
	Jupyter, developing a new symbolic language as part of the
	book. It has many flaws – among them, the book doesn't
	run live in the browser, making it difficult for users to
	experiment. And while the book is well written, the authors
	do not understand classical mechanics as deeply as the authors
	of some other books. But it's nonetheless an inspiring
	evocation of what is possible. And it hints at what is
	possible when authors use executable books for a serious
	purpose, and aspire toward canonical media.
 </p>
 <h3>Stronger emotional connection through an inverted writing structure</h3>
 
 <p>
	Consider an author writing a popular book about quantum
	mechanics. Such an author is in a strong position: they can
	begin their book with astonishing phenomena such as black hole
	evaporation, quantum teleportation, and the role of quantum
	fluctuations in the early universe. Or, if they wish, they can
	start with some of the deepest mysteries known to humanity:
	the relationship between quantum mechanics and gravity, or the
	quantum measurement problem. There is no shortage of
	extraordinary phenomena and beautiful mysteries. These are the
	kind of things which touch a chord inside many, perhaps most
	people. And so it's relatively easy to draw readers in, to get
	them engaged, and keep them connected.
 </p>
 <p>
	By contrast, consider a typical technical book about quantum
	mechanics. It's very unlikely to start with black hole
	evaporation or quantum teleportation – and if it does,
	such a discussion will be perfunctory. Instead, it will start
	out drily, with technical minutiae. Complex
	numbers. Wavefunctions. Many different types of differential
	equations, and how to solve them. Hermitian and unitary
	operators. And so on, piece by piece slowly building up all
	the machinery needed to solve quantum mechanical problems. It
	may be tens or even hundreds of pages before the book begins
	to connect to the exciting problems which form the
	bread-and-butter of popular accounts.
 </p>
 <p>
	People with little experience doing good technical writing
	often complain about this dry, bottom-up approach. They will
	complain that writers should instead stay closer to the fun
	material, and use less technical notation and
	nomenclature. But when competent writers attempt to follow
	this prescription, invariably it works poorly.
 </p>
 <p>
	One problem is that a person can spend years reading analogies
	about black hole evaporation, quantum teleportation, and so
	on. And at the end of all that reading they typically
	have… not much genuine understanding to show for it.
	The analogies and heuristic reasoning simply don't go
	far. They may be entertaining and produce some feeling of
	understanding. But the reasoning won't scale out; it can't be
	applied to other phenomena, at least not without lots of
	caveats, caveats the reader is in no position to understand or
	apply. As a result, good technical writers instead mostly
	build things up from first principles, with occasional
	digressions to the broader motivating picture. And that means
	starting with a lot of detailed, technical minutiae.
 </p>
 <p>
	It's striking to contrast conventional technical books with
	the possibilities enabled by executable books. You can imagine
	starting an executable book with, say, quantum teleportation,
	right on the first page. You'd provide an interface –
	perhaps a library is imported – that would let users
	teleport quantum systems immediately. They could experiment
	with different parts of the quantum teleportation protocol,
	illustrating immediately the most striking ideas about it. The
	user wouldn't necessarily understand all that was going
	on. But they'd begin to internalize an accurate picture of the
	meaning of teleportation. And over time, at leisure, the
	author could unpack some of what might <em>a priori</em> seem
	to be the drier details. Except by that point the reader will
	be bought into those details, and they won't be so
	dryA similar argument has been made
	by Rachel
	Thomas, <a href="https://www.fast.ai/2016/10/08/teaching-philosophy/">Providing
	a Good Education in Deep Learning</a> (2016).
 </p>
 <p>
	In other words, you could begin an executable book with
	material the users already care about, can connect to easily,
	and find motivating. For instance, you could begin by
	exploring teleportation or the Big Bang. But such an opening
	won't suffer the drawback of popular science, of being vague
	and imprecise. Rather, the interface would be completely well
	specified. And, with some care, the interface could be scaled
	out, applied in ever-expanding contexts. The understanding
	would be transferable. Even a user who has understood only a
	tiny part of the material could begin tinkering, building up
	an understanding based on play and exploration. It's common to
	dismiss such an approach as leading to a toy understanding; we
	believe, on the contrary, that with well enough designed
	scaffolding it can lead to a deep
	understandingThe masters of this are
	video game designers. See, for example, Dan
	Cook, <a href="assets/Cook2008.pdf">Building a Princess Saving
	App</a> (2008), and Jonathan Blow and Marc ten
	Bosch, <a href="https://www.youtube.com/watch?v=OGSeLSmOALU">Designing
	to Reveal the Nature of the Universe</a>
	(2011). . Developed in enough depth, such an
	environment may even be used to explore novel research
	ideas. To our knowledge this kind of project has never been
	seriously pursued. But it'd be fun to try.
 </p>
 
 <h2>Summary and Conclusion</h2>
 <p>
	We've covered a lot, and it's helpful to distill the main
	takeaways – general principles, questions, beliefs, and
	aspirations. Let's begin with memory systems, particularly
	the mnemonic medium:
 </p>
 
 <ul>
	<li><strong>Memory systems make memory into a choice, rather
	 than an event left up to chance:</strong> This changes the
	 relationship to what we're learning, reduces worry, and
	 frees up attention to focus on other kinds of learning,
	 including conceptual, problem-solving, and creative.
	</li>
	<li><strong>Memory systems are in their infancy:</strong> it
	 is possible to increase effective human memory by an order
	 of magnitude, even beyond what existing memory systems do;
	 and systems such as the mnemonic medium may help expand the
	 range of subjects users can comprehend at all.
	</li>
	<li><strong>What would a virtuoso use of the mnemonic medium
	 look like?</strong> There's some sense in which the
	 mnemonic medium is “just” flash cards. The
	 right conclusion isn't that it's therefore trivial; it's
	 that flash cards are greatly underrated. In
	 writing <em>Quantum Country</em> we treated the writing of
	 the cards with reverence; ideally, authors would take card
	 writing as seriously as Nabokov took sentence writing. Of
	 course, we didn't reach that level, but the aspiration
	 expands the reach of the medium. What would virtuoso or
	 even canonical uses of the mnemonic medium look like?
	</li>
	<li><strong>Memory systems can be used to build genuine
 	 conceptual understanding, not just learn facts:</strong>
 	 In <em>Quantum Country</em> we achieve this in part
 	 through the aspiration to virtuoso card writing, and in
 	 part through a narrative embedding of spaced repetition
 	 that gradually builds context and understanding.
	</li>
	<li><strong>Mnemonic techniques such as memory palaces are
	 great, but not versatile enough to build genuine
	 conceptual understanding:</strong> Such techniques are
	 very specialized, and emphasize artificial connections,
	 not the inherent connections present in much conceptual
	 knowledge. The mnemonic techniques are, however, useful
	 for bootstrapping knowledge with an <em>ad hoc</em>
	 structure.
	</li>
	<li><strong>Memory is far more important than people tend to
	 think:</strong> It plays a role in nearly every part of
	 cognition, including problem-solving, creative work, and
	 meta-cognition. The flip side is that memory systems
	 themselves want to grow into other types of tools –
	 tools for reading, tools for problem-solving, tools for
	 creating, tools for attention management. That said, we
	 don't yet know what memory systems want to be. To
	 reiterate: memory systems are in their infancy.
	</li>
 </ul>
 
 <p>
	The mnemonic medium is merely one prototype tool for
	thought. We also discussed several other ideas, including
	mnemonic video and executable books. Here are some key
	takeaways:
 </p>
 
 <ul>
	<li><strong>What practices would lead to tools for thought as
	 transformative as Hindu-Arabic numerals? And in what ways does
	 modern design practice and tech industry product practice
	 fall short?</strong> To be successful, you need
	 an <em>insight-through-making loop</em> to be operating at
	 full throttle, combining the best of deep research
	 culture with the best of Silicon Valley product culture.
	</li>
	
 	<li><strong>Tools for thought are (mostly) public goods, and
	 as a result are undersupplied:</strong> That said, there
	 are closely-related models of production which have
	 succeeded (the games industry, Adobe, AutoDesk,
	 Pixar). These models should be studied, emulated where
	 possible, and used as inspiration to find more such
	 models.
	</li>
	<li><strong>Take emotion seriously:</strong> Historically,
	work on tools for thought has focused principally on
	cognition; much of the work has been stuck in Spock-space. But
	it should take emotion as seriously as the best musicians,
	movie directors, and video game designers. Mnemonic video is a
	promising vehicle for such explorations, possibly combining
	both deep emotional connection with the detailed intellectual
	mastery the mnemonic medium aspires toward.
	</li>
	<li><strong>Tools for thought must be developed in tandem with
	deep, original creative work:</strong> Much work on tools for
	thought focuses on toy problems and toy environments. This is
	useful when prototyping, but to be successful such tools must
	ultimately be used to do serious, original creative
	work. That's a baseline litmus test for whether the tools are
	genuinely working, or merely telling a good story. Ideally,
	for any such tool there will be a stream of canonical media
	expanding the form, and entering the consciousness of other
	creators.
	</li>
 </ul>
 <p>
	Let's return to the question that began the essay: how to
	build transformative tools for thought? Of course, we haven't
	even precisely defined what such transformative tools are! But
	they're the kind of tools where relatively low cost changes in
	practice produce transformative changes in outcome –
	non-linear returns and qualitative shifts in thinking. This is
	in contrast with the usual situation, where a small change in
	practice causes a small change in results.
 </p>
 <p>
	Historically, humans have invented many such transformative
	tools for thought. Writing and music are ancient examples; in
	modern times, tools such as <em>Photoshop</em>
	and <em>AutoCAD</em> qualify. Although it's very early days,
	we believe the mnemonic medium shows much promise. It needs
	to be developed much further, along the lines we've described,
	and likely requires additional powerful ideas. But we believe
	it's possible for humanity to have a widespread memory
	practice that radically changes the way we think.
 </p>
 <p>
	More broadly, we hope the principles in this essay will help
	support the creation of more transformative tools for
	thought. Historically, most invention of tools for thought has
	been done bespoke, by inspired individuals and groups. But we
	believe that in the future there will be an established
	community that routinely does this kind of invention.
 </p>
 
	</div>||||I|||| How can we develop transformative tools for thought?

 Andy Matuschak and Michael Nielsen

 October 2019

 Part of the origin myth of modern computing is the story of a golden age in the 1960s and 1970s. In this story, visionary pioneers pursued a dream in which computers enabled powerful tools for thought, that is, tools to augment human intelligence E.g., Douglas Engelbart, Augmenting Human Intellect: A Conceptual Framework (1962). . One of those pioneers, Alan Kay, summed up the optimism of this dream when he wrote of the potential of the personal computer: “the very use of it would actually change the thought patterns of an entire civilization” Alan Kay, User Interface: A Personal View (1989). .

 It's an inspiring dream, which helped lead to modern interactive graphics, windowing interfaces, word processors, and much else. But retrospectively it's difficult not to be disappointed, to feel that computers have not yet been nearly as transformative as far older tools for thought, such as language and writing. Today, it's common in technology circles to pay lip service to the pioneering dreams of the past. But nostalgia aside there is little determined effort to pursue the vision of transformative new tools for thought.

 We believe now is a good time to work hard on this vision again. In this essay we sketch out a set of ideas we believe can be used to help develop transformative new tools for thought. In the first part of the essay we describe an experimental prototype system that we've built, a kind of mnemonic medium intended to augment human memory. This is a snapshot of an ongoing project, detailing both encouraging progress as well as many challenges and opportunities. In the second part of the essay, we broaden the focus. We sketch several other prototype systems. And we address the question: why is it that the technology industry has made comparatively little effort developing this vision of transformative tools for thought?

 In the opening we mentioned some visionaries of the past. To those could be added many others – Ivan Sutherland, Seymour Papert, Vannevar Bush, and more. Online there is much well-deserved veneration for these people. But such veneration can veer into an unhealthy reverence for the good old days, a belief that giants once roamed the earth, and today's work is lesser. Yes, those pioneers did amazing things, and arguably had ways of working that modern technologists, in both industry and academia, are poorly equipped to carry on. But they also made mistakes, and were ignorant of powerful ideas that are available today. And so a theme through both parts of the essay is to identify powerful ideas that weren't formerly known or weren't acted upon. Out of this understanding arises a conviction that a remarkable set of opportunities is open today.

 A word on nomenclature: the term “tools for thought” rolls off neither the tongue nor the keyboard. What's more, the term “tool” implies a certain narrowness. Alan Kay has argued Again, in Alan Kay, User Interface: A Personal View (1989), among other places. that a more powerful aim is to develop a new medium for thought. A medium such as, say, Adobe Illustrator is essentially different from any of the individual tools Illustrator contains. Such a medium creates a powerful immersive context, a context in which the user can have new kinds of thought, thoughts that were formerly impossible for them. Speaking loosely, the range of expressive thoughts possible in such a medium is an emergent property of the elementary objects and actions in that medium. If those are well chosen, the medium expands the possible range of human thought.

 With that said, the term “tools for thought” has been widely used since Iverson's 1950s and 1960s work An account may be found in Iverson's Turing Award lecture, Notation as a Tool of Thought (1979). Incidentally, even Iverson is really describing a medium for thought, the APL programming language, not a narrow tool. introducing the term. And so we shall use “tools for thought” as our catch all phrase, while giving ourselves license to explore a broader range, and also occasionally preferring the term “medium” when it is apt.

 Let's come back to that phrase from the opening, about changing “the thought patterns of an entire civilization”. It sounds ludicrous, a kind of tech soothsaying. Except, of course, such changes have happened multiple times during human history: the development of language, of writing, and our other most powerful tools for thought. And, for better and worse, computers really have affected the thought patterns of our civilization over the past 60 years, and those changes seem like just the beginning. This essay is a small contribution to understanding how such changes happen, and what is still possible.

 The musician and comedian Martin Mull has observed that “writing about music is like dancing about architecture”. In a similar way, there's an inherent inadequacy in writing about tools for thought. To the extent that such a tool succeeds, it expands your thinking beyond what can be achieved using existing tools, including writing. The more transformative the tool, the larger the gap that is opened. Conversely, the larger the gap, the more difficult the new tool is to evoke in writing. But what writing can do, and the reason we wrote this essay, is act as a bootstrap. It's a way of identifying points of leverage that may help develop new tools for thought. So let's get on with it.

 Part I: Memory systems

 Introducing the mnemonic medium

 Few subjects are more widely regarded as difficult than quantum computing and quantum mechanics. Indeed, popular media accounts often regale (and intimidate) readers with quotes from famous physicists in the vein of: “anyone who thinks they’ve understood quantum mechanics has not understood quantum mechanics”.

 What makes these subjects difficult? In fact, individually many of the underlying ideas are not too complicated for people with a technical background. But the ideas come in an overwhelming number, a tsunami of unfamiliar concepts and notation. People must learn in rapid succession of qubits, the bra-ket notation, Hadamard gates, controlled-not gates, and many, many other abstract, unfamiliar notions. They're imbibing an entire new language. Even if they can follow at first, understanding later ideas requires fluency with all the earlier ideas. It's overwhelming and eventually disheartening.

 As an experiment, we have developed a website, Quantum Country, which explores a new approach to explaining quantum computing and quantum mechanics. Ostensibly, Quantum Country appears to be a conventional essay introduction to these subjects. There is text, explanations, and equations, much as in any other technical essay. Here's an excerpt:

 But it's not a conventional essay. Rather, Quantum Country is a prototype for a new type of mnemonic medium. Aspirationally, the mnemonic medium makes it almost effortless for users to remember what they read. That may sound like an impossible aspiration. What makes it plausible is that cognitive scientists know a considerable amount about how human beings store long-term memories. Indeed, what they know can almost be distilled to an actionable recipe: follow these steps, and you can remember whatever you choose.

 Unfortunately, those steps are poorly supported by existing media. For more on this argument, see Andy Matuschak, Why books don’t work (2019). Is it possible to design a new medium which much more actively supports memorization? That is, the medium would build in (and, ideally, make almost effortless) the key steps involved in memory. If we could do this, then instead of memory being a haphazard event, subject to chance, the mnemonic medium would make memory into a choice. Of course, on its own this wouldn't make it trivial to learn subjects such as quantum mechanics and quantum computing – learning those subjects is about much more than memory. But it would help in addressing one core difficulty: the overwhelming number of new concepts and notation.

 In fact, there are many ways of redesigning the essay medium to do that. Before showing you our prototype, please pause for a moment and consider the following questions: how could you build a medium to better support a person's memory of what they read? What interactions could easily and enjoyably help people consolidate memories? And, more broadly: is it possible to 2x what people remember? 10x? And would that make any long-term difference to their effectiveness?

 Let's sketch the user experience of Quantum Country. At the time of this writing the site contains three mnemonic essays (i.e., particular instances of the mnemonic medium). We'll focus on the introductory essay, “Quantum Computing for the Very Curious”. Embedded within the text of the essay are 112 questions about that text. Users are asked to create an account, and quizzed as they read on whether they remember the answers to those questions. Here's what the interaction looks like, as a user answers three questions.

 Note that this interaction occurs within the text of the essay itself. Here's a zoomed-out view, so you can see how such questions are surrounded by essay text both above and below:

 We use the term cards for these interface elements pairing questions and answers.

 Of course, for long-term memory it's not enough for users to be tested just once on their recall. Instead, a few days after first reading the essay, the user receives an email asking them to sign into a review session. In that review session they're tested again, in a manner similar to what was shown above. Then, through repeated review sessions in the days and weeks ahead, people consolidate the answers to those questions into their long-term memory.

 So far, this looks like no more than an essay which integrates old-fashioned flashcards. But notice the intervals indicated at the bottom of the cards:

 The highlighted time interval is the duration until the user is tested again on the question. Questions start out with the time interval “in-text”, meaning the user is being tested as they read the essay. That rises to five days, if the user remembers the answer to the question. The interval then continues to rise upon each successful review, from five days to two weeks, then a month, and so on. After just five successful reviews the interval is at four months. If the user doesn't remember at any point, the time interval drops down one level, e.g., from two weeks to five days.

 This takes advantage of a fundamental fact about human memory: as we are repeatedly tested on a question, our memory of the answer gets stronger, and we are likely to retain it for longer The literature on this effect is vast. A useful entrée is: Gwern Branwen, Spaced Repetition for Efficient Learning . . This exponential rise perhaps seems innocuous, but it's transformative. It means that a relatively small number of reviews will enable a user to remember for years. With the time taken to review a typical question being just a few seconds, that means a user can achieve long-term recall with no more than a few minutes' work. By contrast, with conventional flashcards it takes hours of review to achieve the same durability. Exponential scheduling is far more efficient.

 The early impact of the prototype mnemonic medium

 Although it's early days for Quantum Country we can begin to see some of the impact of the mnemonic medium. Plotted below is the demonstrated retention of answers for each user, versus the number of times each question in the mnemonic essay has been reviewed:

 The graph takes a little unpacking to explain. By a card's “demonstrated retention” we mean the maximum time between a successful review of that card, and the prior review of that card. A little more concretely, consider repetition number 6, say (on the horizontal axis). At the point, a user has reviewed all 112 questions in the essay 6 times. And the vertical axis shows the total demonstrated retention, summed across all cards, with each blue dot representing a single user who has reached repetition 6.

 So, for instance, after 6 repetitions, we see from the graph that most users are up around 6,000 days of demonstrated retention. That means an average of about 6,000 / 112 ~ 54 days per question in the essay. Intuitively, that seems pretty good – if you're anything like us, a couple of months after reading something you have only a hazy memory. By contrast, these users have, at low time cost to themselves (of which more below), achieved nearly two months of demonstrated retention across 112 detailed questions.

 Furthermore, you can see the exponential rise in retention with the number of times cards have been reviewed. After the first review, users typically have an average of just over 2 days of demonstrated retention, per card Particularly careful readers may wonder how this is possible, given that we stated earlier that the first review interval is 5 days. The explanation is that we recently modified the review schedule so the first review is after 5 days. For most of Quantum Country 's history the review schedule was more conservative, and this is the reason for the difference. . But by the sixth review that rises to an average of 54 days of demonstrated retention. That typically takes about 95 minutes of total review time to achieve. Given that the essay takes about 4 or so hours to read, this suggests that a less than 50% overhead in time commitment can provide many months or years of retention for almost all the important details in the essay.

 This is the big, counterintuitive advantage of spaced repetition: you get exponential returns for increased effort. On average, every extra minute of effort spent in review provides more and more benefit. This is in sharp contrast with most experiences in life, where we run into diminishing returns. For instance, ordinarily if you increase the amount of time you spend reading by 50%, you expect to get no more than 50% extra out of it, and possibly much less. But with the mnemonic medium when you increase the amount of time you spend reading by 50%, you may get 10x as much out of it. Of course, we don't quite mean those numbers literally. But it does convey the key idea of getting a strongly non-linear return. It's a change in the quality of the medium.

 This delayed benefit makes the mnemonic medium unusual in multiple ways. Another is this: most online media use short-term engagement models, using variations on operant conditioning to drive user behavior. This is done by Twitter, Facebook, Instagram, and many other popular media forms. The mnemonic medium is much more like meditation – in some ways, the anti-product, since it violates so much conventional Silicon Valley wisdom – in that the benefits are delayed, and hard to have any immediate sense of. Indeed, with the mnemonic medium, the greater the delay, the more the benefit.

 These are preliminary results, and need more investigation. One naturally wonders what would happen if we'd been much more aggressive with the review schedule, setting the initial interval between reviews to (say) 2 months? If users reliably retained information up to that point, then the graph would start very high, and we wouldn't see the exponential. We need to investigate these and many similar questions to better understand what's going on with user's memories.

 Early feedback from users makes us cautiously optimistic that they're finding the mnemonic medium useful. In May 2019, one of us posted to Twitter a short thread explaining the technical details of how quantum teleportation works. One user of Quantum Country replied to the thread with:

 I've only done your first quantum country course (so far) but I find it remarkable that I can view the proof and follow it, knowing what everything means. It's almost like Neo in The Matrix telling Morpheus, ‘I know quantum computing’ In the movie The Matrix one of the characters (Neo) uses a computer to very rapidly upload martial arts skills into his mind. As he opens his eyes after completing the upload he tells another character (Morpheus): “I know Kung Fu”. .

 A user with significantly more prior experience of quantum computing wrote:

 I have a PhD in quantum information/computing and I knew everything in the essay before reading it, but the additional understanding I got from doing the given spaced repetition flashcards significantly improved my understanding of the material. Everyone who is reading this essay, should sign up and give spaced repetition a try.

 Another user, new to quantum computing, told us that Quantum Country “is by far the best way that I could imagine being introduced to this material”. When we asked how he'd used what he'd learned, he explained that when a visitor to his company gave a technical seminar about quantum computing, he expected to get lost after about 10 minutes. Instead:

 Wow, I actually followed that for 40 or 45 minutes because the matrices looked familiar… [the medium means] you run into concepts over and over again… It affords interactions at a more effective level of abstraction.

 Site analytics show a constant flow of people steadily working through the review sessions in the manner we intended. Six months after release of the prototype, 195 users had demonstrated one full month of retention on at least 80% of cards in the essay, demonstrating an extraordinary level of commitment to the process We've made no attempt at all to scale this out. It's interesting to ponder doing so. . We don't yet have a good model of exactly what those people are learning, but it seems plausible they are taking away considerably more than from a conventional essay, or perhaps even from a conventional class.

 Of course, this kind of feedback and these kinds of results should be taken with a grain of salt. The mnemonic medium is in its early days, has many deficiencies, and needs improvement in many ways (of which more soon). It is, however, encouraging to hear that some users already find the medium exceptionally helpful, and suggests developing and testing the medium further. At a minimum, it seems likely the mnemonic medium is genuinely helping people remember. And furthermore it has the exponentially increasing efficiency described above: the more people study, the more benefit they get per minute studied.

 In another informal experiment, we tried to figure out how much it affected user's memories when they weren't asked to review cards. To do this, we introduced a deliberate short (two-week) delay on reviews for a small subset of 8 cards. That is, some users would review those 8 cards upon an initial read, and then would be prevented from reviewing them again for at least two weeks. Other users would continue to study as normal on the 8 cards. By comparing the two groups we could estimate the effect that reviewing the cards had on user's memories.

 What happened? Well, for those users whose reviews were delayed, accuracy dropped from 91% (upon the initial read) to 87% (after two weeks). This may seem a small drop, but keep in mind that users continued to review other cards, which almost certainly boosted their final performance, since those other cards had some overlap in content with the delayed cards. It's difficult to avoid this kind of overlap without delaying reviews on all cards, a more drastic change in user experience than we wanted to impose. For users who were asked to review the cards as normal, accuracy improved from 89% to 96%. The short summary is: when users didn't review the cards, accuracy dropped by 4%; when they did review the cards, accuracy increased by 7% In more detail: there were 16 users in the group that did the reviews, per usual, and 25 users in the group where reviews were delayed. The 95% confidence intervals were: 91 ± 4%, 87 ± 5%, 89 ± 5%, 96 ± 3%, assuming each variable is binomial, independent and identically distributed. This latter assumption is approximate, since we'd expect some user- and question-dependent effects. Note also that this was done in an earlier version of Quantum Country , where the review schedule had intervals of one day, three days, one week, and and two weeks. .

 Another way of looking at the data from this informal experiment is to ask which users saw improved or unchanged performance, and which saw their performance get worse. In fact, every single user (100%) who reviewed cards on the regular schedule saw their performance either stay the same or improve. By contrast, 40% of the users whose reviews were delayed saw their performance get worse, while 60% saw it stay the same or improve.

 These are small-but-promising results. Of course, our experiment was only done over two weeks, and we'd expect larger effects in experiments done over longer periods. And, as already mentioned, the effect is likely diminished by overlaps between the cards. Nonetheless, this informal experiment again suggests the mnemonic medium is helping people's memory, and suggests more comprehensive studies.

 Despite these suggestive preliminary results, it's still tempting to be dismissive. Isn't this “just” an essay with flashcards embedded? At some level, of course, that's correct. In the same way, wikis are just editable web pages; Twitter is just a way of sharing very short form writing; and Facebook is just a way of sharing writing and pictures with friends. Indeed, writing itself is just a clever way of ordering a small number of symbols on a page. While a medium may be simple, that doesn't mean it's not profound. As we shall see, the mnemonic medium has many surprising properties. It turns out that flashcards are dramatically under-appreciated, and it's possible to go much, much further in developing the mnemonic medium than is a priori obvious.

 Before we delve deeper into the mnemonic medium, let's mention one challenge in the discussion: the inherent difficulty in achieving a good balance between conveying enthusiasm and the kind of arm's-length skepticism appropriate for evaluation. On the one hand, we would not have built the mnemonic medium if we weren't excited about the underlying ideas, and wanted to develop those enthusiasms. To explain the mnemonic medium well, we need to bring you, the reader, inside that thinking. But having done that, we also need to step back and think more skeptically about questions such as: is this medium really working? What effect is it actually having on people? Can it be made 10x better? 100x better? Or, contrariwise, are there blockers that make this an irredeemably bad or at best mediocre idea? How important a role does memory play in cognition, anyway? So far, we've focused on the enthusiastic case for the medium, why one might consider this design at all. But later in this essay we'll gradually step back and reflect in a more skeptical frame.

 Expanding the scope of memory systems: what types of understanding can they be used for?

 Quantum Country is an example of a memory system. That is, it's a system designed to help users easily consolidate what they've learned into long-term memory. It's part of a long history of memory systems, going back to ancient times, when the orator Cicero and the rhetorician Quintilian described mnemonic techniques that could be used to memorize long texts.

 In modern times, many memory systems have been developed. Among the better known are Anki, SuperMemo, Quizlet, Duolingo, and Memrise. Like Quantum Country, each of these systems uses increasing time intervals between reviews of particular questions. Such systems are sometimes known as spaced-repetition memory systems (or SRM systems) Strictly speaking, Quizlet's basic product doesn't use spaced repetition. There is, however, a paid version using spaced repetition, and it's otherwise quite similar to many of these systems. . They're usually justified in a manner similar to our explanation for Quantum Country: some notion of each review gradually increasing the consolidation strength for a memory.

 SRM systems are most widely used in language learning. Duolingo, for instance, claims 25 million monthly active users. Reports are mixed on success. Some serious users are enthusiastic about their success with Duolingo. But others find it of limited utility. The company, of course, touts research showing that it's incredibly successful. It seems likely to us that Duolingo and similar systems are useful for many users as part of (but only part of) a serious language learning program.

 What about memory systems for uses beyond language? Quizlet is popular, with 50 million monthly active users. It's widely used in classrooms, especially for simple declarative knowledge – lists of American Presidents, capitals of countries, and so on. Anki and SuperMemo seem to most often be used for similar simple declarative knowledge, but have much smaller active user bases than Quizlet They are, however, widely used within some interesting niche audiences. For instance, there is a thriving population of medical students using Anki. .

 One of the ideas motivating Quantum Country is that memory systems aren't just useful for simple declarative knowledge, such as vocabulary words and lists of capitals. In fact, memory systems can be extraordinarily helpful for mastering abstract, conceptual knowledge, the kind of knowledge required to learn subjects such as quantum mechanics and quantum computing. This is achieved in part through many detailed strategies for constructing cards capable of encoding this kind of understanding. But, more importantly, it's possible because of the way the mnemonic medium embeds spaced repetition inside a narrative. That narrative embedding makes it possible for context and understanding to build in ways difficult in other memory systems.

 Other people have also developed ways of using memory systems for abstract, conceptual knowledge. Perhaps most prominently, the creator of the SuperMemo system, Piotr Wozniak, has written extensively about the many ingenious ways he uses memory systems More generally, Wozniak is, along with Sebastian Leitner, the principal pioneer of spaced-repetition memory systems. Much of Wozniak's thinking is available online at (or linked from) the remarkable SuperMemopedia . . And several other expert users of memory systems have also developed similar strategies. However, employing those strategies requires considerable skill. In practice, that skill barrier has meant these strategies are used by no more than a tiny handful of people.

 By contrast, in Quantum Country an expert writes the cards, an expert who is skilled not only in the subject matter of the essay, but also in strategies which can be used to encode abstract, conceptual knowledge. And so Quantum Country provides a much more scalable approach to using memory systems to do abstract, conceptual learning. In some sense, Quantum Country aims to expand the range of subjects users can comprehend at all. In that, it has very different aspirations to all prior memory systems.

 More generally, we believe memory systems are a far richer space than has previously been realized. Existing memory systems barely scratch the surface of what is possible. We've taken to thinking of Quantum Country as a memory laboratory. That is, it's a system which can be used both to better understand how memory works, and also to develop new kinds of memory system. We'd like to answer questions such as:

 * What are new ways memory systems can be applied, beyond the simple, declarative knowledge of past systems?
 * How deep can the understanding developed through a memory system be? What patterns will help users deepen their understanding as much as possible?
 * How far can we raise the human capacity for memory? And with how much ease? What are the benefits and drawbacks?
 * Might it be that one day most human beings will have a regular memory practice, as part of their everyday lives? Can we make it so memory becomes a choice; is it possible to in some sense solve the problem of memory?

 Over the next few sections we sketch out some of our thinking about how memory systems may be developed. We'll see that memory systems are a small part of a much bigger picture. Not only is seriously developing memory systems likely to lead to one or more transformative tools for thought, we also believe it will teach us much about the general problem of developing such tools.

 Improving the mnemonic medium: making better cards

 In writing mnemonic essays, it's tempting to treat the content of the cards rather casually. After all, a card is just a question and an answer, each containing a little text, perhaps a figure. Surely they ought to be easy to write?

 While thinking in this way is tempting, it's a mistake. In fact, cards are fundamental building blocks of the mnemonic medium, and card-writing is better thought of as an open-ended skill. Do it poorly, and the mnemonic medium works poorly. Do it superbly well, and the mnemonic medium can work very well indeed. By developing the card-writing skill it's possible to expand the possibilities of the medium.

 A helpful comparison is to the sentence in written prose. For the beginning writer it's tempting to treat sentences casually. But in the hands of a great writer – say, a Nabokov – sentences can be developed into a virtuoso artform. What would it take to achieve virtuoso skill in writing the cards of the mnemonic medium?

 It's not obvious a priori that writing cards is such a rich activity. One of us wrote 17,000- and 6,000-word essays Michael Nielsen, Augmenting Long-term Memory (2018), and Michael Nielsen, Using spaced repetition systems to see through a piece of mathematics (2019). whose subject was in large part understanding how to write good cards. He didn't realize that was going to be the subject when he began writing; it only became clear in retrospect how rich card writing is. It turns out that answering the question “how to write good cards?” requires thinking hard about your theory of knowledge and how to represent it, and your theory of learning. The better those theories, the better your cards will be. Small wonder it's a rich, open-ended problem!

 All that said, let's make a few concrete observations about good card-writing. While the specific examples that follow are relatively banal, they should give you some feeling for the profound issues that arise in improving the mnemonic medium. We'll begin with three principles we used when writing the cards in Quantum Country. Note that these are just three of many more principles – a more detailed discussion of good principles of card construction may be found in Augmenting Long-term Memory.

 * Most questions and answers should be atomic: Early in his own personal memory practice, one of us was learning the Unix command to create links in the filesystem. He entered the following question into his memory system: “How to create a soft link from linkname to filename”. Together with the corresponding answer “ln -s filename linkname”. This looks like a good question, but he routinely forgot the answer. To address this, he refactored the card into two more atomic cards. One card: “What’s the basic command and option to create a soft link?” (A: “ln -s”). Second card: “When creating a soft link, in what order do linkname and filename go?” (A: “filename linkname”). Breaking the card into more atomic pieces turned a question he routinely got wrong into two questions he routinely got right. It seemed that the more atomic questions brought more sharply into focus what he was forgetting, and so provided a better tool for improving memory. And what of the original card? Initially, he deleted it. But he eventually added the card back, with the same question and answer, since it served to integrate the understanding in the more atomic cards.
 * Make sure the early questions in a mnemonic essay are trivial: it helps many users realize they aren't paying enough attention as they read: Note added December 9, 2019: This claim appears to be based on an error in our data analysis, and is now retracted. We've left the text in for historic reasons, but we no longer believe the claim. This was a discovery made when we released the first Quantum Country essay. Anticipating that users would be struggling with a new interface, we deliberately made the first few questions in the essay utterly trivial – sort of a quantum equivalent to “2+2 = ?” – so they could focus on the interface. To our surprise, users performed poorly on these questions, worse than they did on the (much harder) later questions. Our current hypothesis to explain this is that when users failed to answer the first few questions correctly it served as a wakeup call. The questions were so transparently simple that they realized they hadn't really been paying attention as they read, and so were subsequently more careful.
 * Avoid orphan cards: These are cards which don't connect closely to anything else. Suppose, for the sake of illustration, that you're trying to learn about African geography, and have a question: “What's the territory in Africa that Morocco disputes?” (A: “The Western Sahara”) If you don't know anything about the Western Sahara or Morocco or why there's a dispute, that question will be an orphan, disconnected from everything else. Ideally, you'll have a densely interconnected web of questions and answers, everything interwoven in striking ways.

 Ultimately, we'd like to distill out a set of useful practical principles and idioms to help write good cards and, more generally, good mnemonic essays. Aspirationally, such a set of principles and idioms would work much like The Elements of Style (or some similar book of prose advice), and would help other people learn to write high-quality mnemonic essays.

 When we first described Quantum Country above we explained it using a simple model of spaced repetition: increased consolidation strength for memories leading to increased time intervals between reviews. This is a helpful simple model, but risks creating the misleading impression that it's all that's going on in the system. In fact, for the mnemonic medium to work effectively, spaced repetition must be deployed in concert with many other ideas. The three ideas we just described – atomicity of questions and answers, making early questions trivial, avoiding orphan cards – are just three of dozens of important ideas used in the mnemonic medium. We won't enumerate all those other ideas here – that's not the purpose of this essay. But we want to emphasize this point, since it's common for people to have the simplistic model “good memory system = spaced repetition”. That's false, and an actively unhelpful way of thinking.

 Indeed, thinking in this way is one reason spaced-repetition memory systems often fail for individuals. We often meet people who say “Oh, I thought spaced repetition sounded great, and I tried Anki [etc], but it doesn't work for me”. Dig down a little, and it turns out the person is using their memory system in a way guaranteed to fail. They'll be writing terrible questions, or using it to learn a subject they don't care about, or making some other error. They're a little like a person who thinks “learning the guitar sounds great”, picks it up for half an hour, and then puts it down, saying that they sound terrible and therefore it's a bad instrument. Of course, what's really going on is that the guitar and memory systems are both skills that take time to develop. But, with that said, we want to build as much support as possible into the medium. Ideally, even novices would benefit tremendously from the mnemonic medium. That means building in many ideas that go beyond the simplistic model of spaced repetition.

 One of us has previously asserted Michael Nielsen, Augmenting Long-Term Memory (2018). that in spaced-repetition memory systems, users need to make their own cards. The reasoning is informal: users often report dissatisfaction and poor results when working with cards made by others. The reason seems to be that making the cards is itself an important act of understanding, and helps with committing material to memory. When users work with cards made by others, they lose those benefits.

 Quantum Country violates this principle, since users are not making the cards. This violation was a major concern when we began working on Quantum Country. However, preliminary user feedback suggests it has worked out adequately. A possible explanation is that, as noted above, making good cards is a difficult skill to master, and so what users lose by not making their own cards is made up by using what are likely to be much higher-quality cards than they could have made on their own. In future, it's worth digging deeper into this issue, both to understand it beyond informal models, and to explore ways of getting the benefits of active card making.

 Above we discussed three principles of good question-and-answer construction. Of course, it's also possible to make more structural modifications to the nature of the cards themselves. Here's three questions suggesting experiments in this vein:

 * How can we ensure users don't just learn surface features of questions? One question in Quantum Country asks: “Who has made progress on using quantum computers to simulate quantum field theory?” with the answer: “John Preskill and his collaborators”. This is the only “Who…?” question in the entire essay, and many users quickly learn to recognize it from just the “Who…?” pattern, and parrot the answer without engaging deeply with the question. This is a common failure mode in memory systems, and it's deadly to understanding. One response, which we plan to trial soon, is to present the question in multiple different-but-equivalent forms. So the user first sees the question as “Who has made progress [etc]?”; but then the second time the question is presented as a fill-in-the-blanks: “___ and his collaborators have made progress on using quantum computers to simulate quantum field theory.” And so on, multiple different forms of the question, designed so the user must always engage deeply with the meaning of the question, not its superficial appearance. Ultimately, we'd like to develop a library of techniques for identifying when this learning-the-surface-feature pattern is occurring, and for remedying it.
 * How to best help users when they forget the answer to a question? Suppose a user can't remember the answer to the question: “Who was the second President of the United States?” Perhaps they think it's Thomas Jefferson, and are surprised to learn it's John Adams. In a typical spaced-repetition memory system this would be dealt with by decreasing the time interval until the question is reviewed again. But it may be more effective to follow up with questions designed to help the user understand some of the surrounding context. E.g.: “Who was George Washington's Vice President?” (A: “John Adams”). Indeed, there could be a whole series of followup questions, all designed to help better encode the answer to the initial question in memory.
 * How to encode stories in the mnemonic medium? People often find certain ideas most compelling in story form. Here's a short, fun example: did you know that Steve Jobs actively opposed the development of the App Store in the early days of the iPhone? It was instead championed by another executive at Apple, Scott Forstall. Such a story carries a force not carried by declarative facts alone. It's one thing to know in the abstract that even the visionaries behind new technologies often fail to see many of their uses. It's quite another to hear of Steve Jobs arguing with Scott Forstall against what is today a major use of a technology Jobs is credited with inventing. Can the mnemonic medium be used to help people internalize such stories? To do so would likely violate the principle of atomicity, since good stories are rarely atomic (though this particular example comes close). Nonetheless, the benefits of such stories seem well worth violating atomicity, if they can be encoded in the cards effectively.

 It's easy to generate dozens more questions and ideas in a similar vein. The mnemonic medium is not a fixed form, but rather a platform for experimentation and continued improvement.

 One useful metaphor for thinking about how to improve the mnemonic medium is to think of each mnemonic essay as a conventional essay accompanied by a kind of “reflected essay” – the knowledge encoded by all the cards. A user can, with ease, choose to remember as much of that reflected essay as they wish. Of course, the reflection is imperfect. But by developing good card-making strategies we can make the reflected essay a nearly faithful reflection of all the important ideas, the ideas a reader would ideally like to retain.

 We said above that it's a mistake to use the simplistic model “good memory system = spaced repetition”. In fact, while spaced repetition is a helpful way to introduce Quantum Country, we certainly shouldn't pigeonhole the mnemonic medium inside the paradigm of existing SRM systems. Instead, it's better to go back to first principles, and to ask questions like: what would make Quantum Country a good memory system? Are there other powerful principles about memory which we could we build into the system, apart from spaced repetition?

 In fact, there are ideas about memory very different from spaced repetition, but of comparable power. One such idea is elaborative encoding. Roughly speaking, this is the idea that the richer the associations we have to a concept, the better we will remember it. As a consequence, we can improve our memory by enriching that network of associations.

 This is in some sense an obvious idea, according well with everyday experience. For instance, it's part of the reason it's so much easier to learn new facts in an area we're already expert in – we quickly form associations to our existing knowledge. But just because the idea is obvious, that doesn't mean it's particularly well supported by existing media forms. There's a lot of low-hanging fruit which we can actively support inside the mnemonic medium. Indeed, several of the suggestions above already implicitly build on the idea of elaborative encoding – principles like “avoid orphan cards” are based on this. Here's three more suggestions which build on elaborative encoding:

 * Provide questions and answers in multiple forms: In 1971, the psychologist Allan Paivio proposed the dual-coding theory, namely, the assertion that verbal and non-verbal information are stored separately in long-term memory. Paivio and others investigated the picture superiority effect, demonstrating that pictures and words together are often recalled substantially better than words alone. This suggests, for instance, that the question “Who was George Washington's Vice President?” may have a higher recall rate if accompanied by a picture of Washington, or if the answer (John Adams) is accompanied by a picture of Adams. For memory systems the dual-coding theory and picture superiority effect suggest many questions and ideas. How much benefit is there in presenting questions and answer in multiple forms? Perhaps even with multiple pictures, or in audio or video (perhaps with multiple speakers of different genders, different accents, etc), or in computer code? Perhaps in a form that demands some form of interaction? And in each case: what works best?
 * Vary the context: In 1978, the psychologists Steven Smith, Arthur Glenberg, and Robert Bjork Steven M. Smith, Arthur Glenberg, and Robert A. Bjork, Environmental context and human memory (1978). reported several experiments studying the effect of place on human memory. In one of their experiments, they found that studying material in two different places, instead of twice in the same place, provided a 40% improvement in later recall. This is part of a broader pattern of experiments showing that varying the context of review promotes memory. We can use memory systems to support things like: changing the location of review; changing the time of day of review; changing the background sound, or lack thereof, while reviewing. In each case, experiments have been done suggesting an impact on recall. It's not necessarily clear how robust the results are, or how reproducible – it's possible some (or all) are the results of other effects, uncontrolled in the original experiment. Still, it seems worth building systems to test and (if possible) improve on these results.
 * How do the cards interact with one another? What is the ideal network structure of knowledge? This is a very complicated and somewhat subtle set of questions. Let's give a simple example to illustrate the idea. We've presented the cards in the mnemonic medium as though they are standalone entities. But there are connections between the cards. Suppose you have cards: “Who was George Washington’s Vice President?” (Answer: “John Adams”, with a picture of Adams); “What did John Adams look like?” (Answer: a picture of Adams); perhaps a question involving a sketch of Adams and Washington together at some key moment; and so on. Now, this set of cards forms a network of interrelated cards. And you can use a memory system like Quantum Country to study that network. What happens to people's observed recall if you remove a card? Are there crucial lynchpin cards? Are there particularly effective network structures? Particularly effective types of relationship between cards? Crucially: are there general principles we can identify about finding the deepest, most powerful ways of representing knowledge in this system?

 By now it's obvious that the prototype mnemonic medium we've developed is the tip of a much larger iceberg. What's more, the suggestions we've made and questions we've asked here are also merely a beginning, to give you the flavor of what is possible.

 Two cheers for mnemonic techniques

 When we discuss memory systems with people, many immediately respond that we should look into mnemonic techniques. This is an approach to memory systems very different to Quantum Country, Duolingo, Anki, and the other systems we've discussed. You're perhaps familiar with simple mnemonic techniques from school. One common form is tricks such as remembering the colors of the rainbow as the name Roy G. Biv (red, orange, yellow, green, etc). Or remembering the periodic table of elements using a song An enjoyable extended introduction to such techniques may be found in Joshua Foer's book “Moonwalking with Einstein” (2012). .

 A more complex variation is visualization techniques such as the method of loci. Suppose you want to remember your shopping list. To do so using the method of loci, you visualize yourself in some familiar location – say, your childhood home. And then you visualize yourself walking from room to room, placing an item from your shopping list prominently in each room. When you go shopping, you can recall the list by imagining yourself walking through the house – your so-called memory palace – and looking at the items in each room.

 If you've never used memory palaces this sounds like it couldn't possibly work. But even novices are often shocked by how well such techniques work, with just a small amount of practice. Experts who work hard developing these techniques can do remarkable things, like memorizing the order of a shuffled deck of cards, or lists of hundreds of digits. It's a way of using people's immensely powerful visual and spatial memories as a form of leverage for other types of memory A small minority of the population does not possess a mind's eye, and so cannot mentally visualize. This condition is known as aphantasia . One of us asked on Twitter if any aphantasics had tried using the method of loci, and if so how well it worked for them. The replies were remarkably heterogeneous (and striking), but most said such mnemonic techniques did not work for them. This deserves further study. .

 Given all this, it's perhaps not surprising that we often meet people who tell us that mnemonic techniques are a much more promising approach to memory than ideas such as spaced repetition.

 We're enthusiastic about such mnemonic techniques. But it's important to understand their limitations, and not be bedazzled by the impressiveness of someone who can rapidly memorize a deck of cards.

 One caution concerns the range of what can be memorized using mnemonic techniques. In practice they're often quite specialized. Mnemonic experts will, for instance, use somewhat different approaches to memorize lists of digits versus decks of cards. Those approaches must be mastered separately – a heavy time investment for two narrow kinds of memory. Furthermore, the mnemonic techniques tend to be much better suited for concrete objects than abstract conceptual knowledge – it's difficult to store, say, the main points in the Treaty of Versailles in your memory palace. This doesn't mean it can't be done – mnemonic experts have developed clever techniques for converting abstract conceptual knowledge into concrete objects which can be stored in a memory palace. But, in general, an advantage of spaced repetition is that it works across a far broader range of knowledge than do any of the mnemonic techniques.

 A second caution relates to elaborative encoding. The mnemonic techniques are, as you have likely realized, an example of elaborative encoding in action, connecting the things we want to memorize (say, our shopping list) to something which already has meaning for us (say, our memory palace). By contrast, when an expert learns new information in their field, they don't make up artificial connections to their memory palace. Instead, they find meaningful connections to what they already know. Those connections are themselves useful expertise; they're building out a dense network of understanding. It's a deeper and more desirable kind of expertise, connections native to the subject itself, not artificially constructed mnemonics.

 All this makes us seem negative about mnemonic techniques. In fact, we're enthusiastic, and have to date certainly underused them in the mnemonic medium. What we've written here is merely meant to temper the over-enthusiasm we sometimes encounter. We've had people go so far as to tell us that mnemonics make memory a solved problem. That is simply false. But with their limitations understood, they're a powerful tool. This is particularly true for knowledge which has an arbitrary, ad hoc structure. For example, it's difficult to remember the colors of the rainbow because those colors are not obviously connected to anything else, unless you happen to have the spectrum of visible light memorized for other reasons! That makes a mnemonic like Roy G. Biv extremely helpful. And so mnemonic techniques should be thought of as a useful tool to use in building powerful memory systems, especially when combined with ideas such as spaced repetition.

 How important is memory, anyway?

 People tend to fall into two buckets when told of the mnemonic medium. One group is fascinated by the idea, and wants to try it out. The second group is skeptical or even repulsed. In caricature, they say: “Why should I care about memory? I want deeper kinds of understanding! Can't I just look stuff up on the internet? I want creativity! I want conceptual understanding! I want to know how to solve important problems! Only dull, detail-obsessed grinds focus on rote memory.”

 It's worth thinking hard about such objections. To develop the best possible memory system we need to understand and address the underlying concerns. In part, this means digging down far enough to identify the mistaken or superficial parts of these concerns. It also means distilling as sharply as possible the truth in the concerns. Doing both will help us improve and go beyond the current prototype mnemonic medium.

 One response to such objections is the argument from lived experience. In the past, one of us (MN) has often helped students learn technical subjects such as quantum mechanics. He noticed that people often think they're getting stuck on esoteric, complex issues. But, as suggested in the introduction to this essay, often what's really going on is that they're having a hard time with basic notation and terminology. It's difficult to understand quantum mechanics when you're unclear about every third word or piece of notation. Every sentence is a struggle.

 It's like they're trying to compose a beautiful sonnet in French, but only know 200 words of French. They're frustrated and think the trouble is the difficulty of finding a good theme, striking sentiments and images, and so on. But really the issue is that they have only 200 words with which to compose.

 At the time, MN's somewhat self-satisfied belief was that if people only focused more on remembering the basics, and worried less about the “difficult” high-level issues, they'd find the high-level issues took care of themselves. What he didn't realize is that this also applied to him. When he began using the memory system Anki to read papers in new fields, he found it almost unsettling how much easier Anki made learning the basics of such subjects. And it made him start wondering if memory was often a binding constraint in learning new fields See here and here for more on learning new fields using Anki. The last four paragraphs are adapted from Augmenting Long-term Memory (2018). .

 One particularly common negative response to the mnemonic medium is that people don't want to remember “unimportant details”, and are just looking for “a broad, conceptual understanding”. It's difficult to know what to make of this argument. Bluntly, it seems likely that such people are fooling themselves, confusing a sense of enjoyment with any sort of durable understanding.

 Imagine meeting a person who told you they “had a broad conceptual understanding” of how to speak French, but it turned out they didn’t know the meaning of “bonjour”, “au revoir”, or “tres bien”. You'd think their claim to have a broad conceptual understanding of French was hilarious. If you want to understand a subject in any real sense you need to know the details of the fundamentals. What's more, that means not just knowing them immediately after reading. It means internalizing them for the long term The last two paragraphs are adapted from our forthcoming mnemonic essay: Andy Matuschak and Michael Nielsen, How quantum teleportation works (2019). .

 A better model is that conceptual mastery is actually enabled by a mastery of details. One user of Quantum Country told us that she found the experience of reading unexpectedly relaxing, because she “no longer had to worry” about whether she would remember the content. She simply trusted that the medium itself would ensure that she did. And she reported that she was instead able to spend more of her time on conceptual issues.

 When people respond to the mnemonic medium with “why do you focus on all that boring memory stuff?”, they are missing the point. By largely automating away the problem of memory, the mnemonic medium makes it easier for people to spend more time focusing on other parts of learning, such as conceptual issues.

 Another common argument against spaced repetition systems is that it's better to rely on natural repetition. For instance, if you're learning a programming language, the argument goes, you shouldn't memorize every detail of that language. Instead, as you use the language in real projects you'll naturally repeatedly use, and eventually commit to memory, those parts of the language most important to learn.

 There are important partial truths in this. It is good to use what you're learning as part of your creative projects. Indeed, an ideal memory system might help that happen, prompting you as you work, rather than in an artificial card-based environment. Furthermore, a common failure mode with memory systems is that people attempt to memorize things they're unlikely to ever have any use for. For instance, it's no good (but surprisingly common) for someone to memorize lots of details of a programming language they plan to use for just one small project. Or to memorize details “just in case” they ever need them. These patterns are mistakes.

 But the truths of the last paragraph also have limits. If you're learning French, but don't know any French speakers, then waiting for “natural opportunities” to speak just won't work. And even if you do have (or create) opportunities to speak, it's desirable to accelerate the awkward, uncomfortable early stages that form such a barrier to using the language.

 It's in this phase that memory systems shine. They can accelerate people through the awkward early stages of learning a subject. Ideally, they'll support and enable work on creative projects. For this to work well takes good heuristics for what any given person should commit to memory; what is good for one person to memorize may be bad for another. Working such heuristics out is an ongoing challenge in the design of memory systems.

 (Incidentally, a surprising number of people say they are “repulsed”, or some similarly strong word, by spaced-repetition memory systems. Their line of argument is usually some variant on: it is claimed that spaced-repetition systems help with memory; if that is true I must use the systems; but I hate using the systems. The response is to deny the first step of the argument. Of course, the mistake is elsewhere: there is absolutely no reason anyone “should” use such systems, even if they help with memory. Someone who hates using them should simply choose not to do so. Using memory systems is not a moral imperative!)

 An immense amount of research has been done on the relationship of memory to mastery. Much of this research is detailed and context specific. But at the level of broader conclusions, one especially interesting series of studies was done in the 1970s by Herbert Simon and his collaborators. They studied chess players, and discovered See, e.g., William G. Chase and Herbert A. Simon, Perception in Chess (1973). Some fascinating earlier work in a related vein was done by Adrian D. de Groot, and summarized in his book Thought and Choice in Chess (1965). that when master chess players look at a position in chess they don't see it in terms of the individual pieces, a rook here, a pawn there. Instead, over years of playing and analyzing games the players learn to recognize somewhere between 25,000 and 100,000 patterns of chess pieces. These much more elaborate “chunks” are combinations of pieces that the players perceive as a unity, and are able to reason about at a higher level of abstraction than the individual pieces. At least in part it's the ability to recognize and reason about these chunks which made their gameplay so much better than novices. Furthermore, although Simon did this work in the context of chess, subsequent studies have found similar results in other areas of expertise We've met many mathematicians and physicists who say that one reason they went into mathematics or physics is because they hated the rote memorization common in many subjects, and preferred subjects where it is possible to derive everything from scratch. But in conversation it quickly becomes evident that they have memorized an enormous number of concepts, connections, and facts in their discipline. It's fascinating these people are so blind to the central role memory plays in their own thinking. . It seems plausible, though needs further study, that the mnemonic medium can help speed up the acquisition of such chunks, and so the acquisition of mastery.

 So, does all this mean we're fans of rote memory, the kind of forced memorization common schools?

 Of course not. What we do believe is that many people's dislike of rote memorization has led them to a generalized dislike of memory, and consequently to underrate the role it plays in cognition. Memory is, in fact, a central part of cognition. But the right response to this is not immense amounts of dreary rote memorization. Rather, it's to use good tools and good judgment to memorize what truly matters.

 We've identified some ways in which criticisms of memory systems are mistaken or miss the point. But what about the ways in which those criticisms are insightful? What are the shortcomings of memory systems? In what ways should we be wary of them?

 We've already implicitly mentioned a few points in this vein. Think about problems like the need to avoid orphan questions. Or to make sure that users don't merely learn surface features of questions. These are ways in which memory systems can fail, if used poorly. Here's a few more key concerns about memory systems:

 * Memory systems don't make it easy to decide what to memorize: Most obviously, we meet a lot of people who use memory systems for poorly chosen purposes. The following is surprisingly close to a transcript of a conversation we've both had many times:
 “I don’t like [memory system]. I tried to memorize the countries in Africa, and it was boring.”
 “Why were you trying to remember the countries in Africa?”
 [blank look of confusion.]
 It's easy to poke fun at this kind of thing. But we've both done the equivalent in our own memory practices. Even some users of Quantum Country seem to be going through the motions out of some misplaced sense of duty. The question “what will be beneficial to memorize” is fundamental, and answering that question well is not trivial.
 * What's the real impact of the mnemonic medium on people's cognition? How does it change people's behavior? A famous boxer is supposed to have said that everyone has a plan until they get punched in the face. Regular users of memory systems sometimes report that while they can remembers answers when being tested by their system, that doesn't mean they can recall them when they really need them. There can be a tip-of-the-tongue feeling of “Oh, I know this”, but not actual recall, much less the fluent facility one ultimately wants for effective action. Furthermore, the user may not even recognize opportunities to use what they have learned. More broadly: memory is not an end-goal in itself. It's embedded in a larger context: things like creative problem-solving, problem-finding, and all the many ways there are of taking action in the world. We suspect the impact of memory systems will vary a lot, depending on their design. They may be used as crutches for people to lean on. Or they may be used to greatly enable people to develop other parts of their cognition. We don't yet understand very well how to ensure they're enablers, rather than crutches. But later in the essay we'll describe some other tools for thought that, when integrated with memory systems, may better enable this transition to more effective action.

 How to invent Hindu-Arabic numerals?

 Let's briefly get away from memory systems. Imagine you're a designer living in ancient Rome, working for MDC (Mathematical Designs Corporation). A client comes in one day, expressing a desire to improve on Roman numerals. Of course, that's not literally how they describe their problem to you – more likely it's a tax collector wanting to tabulate taxes more efficiently, and having some vague notion that MDC may be able to help. But to you, an experienced designer, it seems that an improved system of numerals may be what they need.

 How should you respond to this request? From our modern vantage point we know a vastly better system of numerals is possible, the Hindu-Arabic numerals. Hindu-Arabic numerals were, in fact, a great leap in the history of tools for thought. Could you, as a designer, have made that leap? What creative steps would be needed to invent Hindu-Arabic numerals, starting from the Roman numerals? Is there a creative practice in which such steps would be likely to occur?

 To be clear: this is a somewhat fanciful thought experiment. Many of the ideas needed to get to Hindu-Arabic numerals were, in fact, known earlier to the Babylonians, to the Greeks, and in other cultures. They were also inchoate in the abacus and similar devices. And so we're not asking a literal historical question. Rather, it's a question meant to stimulate thought: what design process could take you from Roman numerals to Hindu-Arabic numerals?

 We can't know the answer to this question for sure. But it's worth pointing out that the Hindu-Arabic numerals aren't just an extraordinary piece of design. They're also an extraordinary mathematical insight. They involve many non-obvious ideas, if all you know is Roman numerals. Perhaps most remarkably, the meaning of a numeral actually changes, depending on its position within a number. Also remarkable, consider that when we add the numbers 72 and 83 we at some point will likely use 2+3=5; similarly, when we add 27 and 38 we will also use 2+3=5, despite the fact that the meaning of 2 and 3 in the second sum is completely different than in the first sum. In modern user interface terms, the numerals have the same affordances, despite their meaning being very different in the two cases. We take this for granted, but this similarity in behavior is a consequence of deep facts about the number system: commutativity, associativity, and distributivity The same phenomenon occurs in the conventional grade-school algorithms for multiplication and division. One of us has spun a short piece of discovery fiction discussing in more detail the way a hypothetical designer might have arrived at these ideas. . All these properties (and many more) point to the design and mathematical insights being inextricably entangled: the mathematical insights are, in some sense, design insights, and vice versa.

 Indeed, it seems fair to say that any person who could invent Hindu-Arabic numerals, starting from the Roman numerals, would be both one of the great mathematical geniuses who ever lived, and one of the great design geniuses who ever lived. They'd have to be extraordinarily capable in both domains, capable of an insight-through-making loop which used the evolving system of numerals to improve not just their own mathematical ideas, but to have original, world-class insights into mathematics; and also to use those mathematical insights to improve their evolving system of numerals.

 This is rather sobering if we compare to conventional modern design practice. In a typical practice, you'd interview domain experts (in this case, mathematicians), and read any relevant literature. You'd talk to users of existing systems, and analyze serious behavior, both individually and at scale. In short, you'd do what people in the design community refer to as immersing themselves in the target field.

 This is a powerful practice. At its best it causes systems to come into existence which would otherwise be inconceivable. If applied to Roman numerals (in hypothetical ancient Rome, not today Of course, a designer who spoke to an expert on, say, Babylonian mathematics, might well have come across some of these ideas. We'll ignore that, since it depends on the oddity that many excellent prior ideas about numeral systems had been displaced in Roman culture. ) this practice would likely improve them a great deal. But it would not provide anywhere near the level of mathematical insight needed to arrive at Hindu-Arabic numerals.

 Our story about Hindu-Arabic numerals and mathematics is fanciful. But it expresses a general truth: the most powerful tools for thought express deep insights into the underlying subject matter. In the case of memory systems, this means they're not just “applied cognitive science”, a collage of existing ideas from cognitive science pasted together using modern design practice. Rather, they will express deep original insights into memory, insights no-one else in the world has ever had. A truly great memory system will be cognitive science of the highest order.

 From this discussion, we take away a warning and an aspiration.

 The warning is this: conventional tech industry product practice will not produce deep enough subject matter insights to create transformative tools for thought. Indeed, that's part of the reason there's been so little progress from the tech industry on tools for thought. This sounds like a knock on conventional product practice, but it's not. That practice has been astoundingly successful at its purpose: creating great businesses. But it's also what Alan Kay has dubbed a pop culture, not a research culture. To build transformative tools for thought we need to go beyond that pop culture.

 The aspiration is for any team serious about making transformative tools for thought. It's to create a culture that combines the best parts of modern product practice with the best parts of the (very different) modern research culture. You need the insight-through-making loop to operate, whereby deep, original insights about the subject feed back to change and improve the system, and changes to the system result in deep, original insights about the subject.

 Note that we are not making the common argument that making new tools can lead to new subject matter insights for the toolmaker, and vice versa. This is correct, but is much weaker than what we are saying. Rather: making new tools can lead to new subject matter insights for humanity as a whole (i.e., significant original research insights), and vice versa, and this would ideally be a rapidly-turning loop to develop the most transformative tools.

 Doing this is a cultural struggle. It seems to be extraordinarily rare to find the insight-through-making loop working at full throttle. People with expertise on one side of the loop often have trouble perceiving (much less understanding and participating in) the nature of the work that goes on on the other side of the loop. You have researchers, brilliant in their domain, who think of making as something essentially trivial, “just a matter of implementation”. And you have makers who don't understand research at all, who see it as merely a rather slow and dysfunctional (and unprofitable) making process. This is certainly true in Silicon Valley, where it's common to meet accomplished technical makers who, after reading a few stories from Richard Hamming and Richard Feynman, think they understand research well enough that they can “create the new Bell Labs”. Usually they're victims of Dunning-Krugeritis, so ignorant they're not even aware of their ignorance.

 Of course, we've got a long way to go with Quantum Country. It's not yet generating nearly deep enough ideas about memory and cognition; it's not yet one of the world's foremost memory laboratories. And considered as a product, it's also in the very earliest days; we're not yet iterating nearly fast enough, nor learning nearly fast enough from the system. Getting the insight-through-making loop to operate at full throttle will mean reinventing parts of both research culture and conventional product development culture; it will mean new norms and a new type of person involved in key decision making. But that's the aspiration, and what we believe is necessary to develop transformative tools for thought.

 Part II: Exploring tools for thought more broadly

 We've examined the mnemonic medium in some depth. The intent was to show you the early stages in the development of a specific tool for thought, and some of the thinking enabled by that development. In this second part of the essay we explore more broadly, briefly sketching ideas for several other tools for thought. And we'll address some broader questions, especially around why there hasn't been more work on tools for thought.

 Mnemonic video

 In 2014, the digital artist Eric Wernquist released an extraordinary short video entitled “Wanderers”. The video provides a first-person glimpse of what it would be like to explore the Solar System:

 We hear the narrator's (Carl Sagan) wonder and awe, and cannot help but empathize with his deep belief in the value of exploration. We get a sense of how many mysteries and how much beauty there is in our own cosmic neighborhood. The music begins with a wistful nostalgia for those of our ancestors who dared to explore, and then changes to convey excitement and danger and the boldness of those members of our and future generations who continue that exploration.

 It's interesting to contrast the video to the video's script, a short text by Carl Sagan. The text is beautiful, but reading it is a much more remote and cerebral experience, conveying a much less visceral emotional understanding.

 We have a friend, Grant Sanderson, who makes astonishing mathematics videos on his YouTube channel, 3Blue1Brown. Below is one of our favorites, a video sketching a proof of a relatively recent research result in geometry, using ideas from algebraic topology. This sounds fearsome, but the video is beautiful and accessible, and has been viewed more than 1.2 million times.

 As with Wanderers, watching this video is a remarkable emotional experience. It's obvious the person narrating the video loves mathematics, and you cannot help but empathize. As you watch, you experience repeated “Ahah!” moments, moments of surprising insight, as connections that were formerly invisible become obvious. It shows mathematics as something beautiful, containing extraordinary ideas and intriguing mysteries, while at the same time showing that doing mathematics is not mysterious, that it is something anyone can understand and even do.

 It's tempting to overlook or undervalue this kind of emotional connection to a subject. But it's the foundation of all effective learning and of all effective action. And it is much easier to create such an emotional connection using video than using text.

 There's a flipside to this emotional connection, however. We've often heard people describe Sanderson's videos as about “teaching mathematics”. But in conversation he's told us he doesn't think more than a small fraction of viewers are taking away much detailed understanding of mathematics. We suspect this is generally true, that high affect videos usually do little to change people's detailed intellectual understanding. Rather, the extraordinary value of the videos lies in the emotional connection they create.

 Is it possible to create a medium which blends the best qualities of both video and text?

 In particular, is it possible to create a medium which has the emotional range possible in video – a range which can be used to convey awe and mystery and surprise and beauty? But which can also firmly ground that emotional connection in detailed understanding, the mastery of details which is the raison d'être of both conventional text and, perhaps even moreso, of the mnemonic medium?

 We believe this may be possible, and we plan to develop a mnemonic video form that provides both the emotional connection possible in video, and the mastery of details possible in the mnemonic medium.

 Creating such a form is challenging. Many MOOC platforms have attempted something a little in this vein. The typical approach is to have a low-affect talking head video, with the videos interrupted occasionally for brief quizzes These images excerpted from Astronomy: Exploring Time and Space . . Here's how it works on one MOOC platform, Coursera:

 Other MOOCs differ in the details. But the overall emotional experience may be summed up as:

 The very best parts of the video may be emotionally compelling, though it's rare that they achieve the emotional range and connection of the best videos from people like Grant Sanderson. And the overall emotional experience is disjointed, almost repellent. Is it possible to create an integrated medium, with a unified and carefully crafted emotional and intellectual experience? Ideally, is it possible to create something like the following?

 In MOOCs, questions are typically presented in a very dry form, detached from context. In the mnemonic video the narrator would explain why the questions are important, and why the user will benefit from participating, as a seamless part of the overall narration. Done right – perhaps with appropriate music, and a sense of urgency or play in the narration – it would create a real sense of the stakes. At the same time, the video player can be modified so the user can respond directly to questions, as part of the spaced-repetition experience. The result would be much softer transitions between the high-affect core narration and the moderate-affect questions.

 Here's a short sketch of one approach to doing this The narrator's script in this sketch is adapted from Cosmos (1984), episode 4. , showing how the narrator could ask questions aloud as an integrated part of the overall narration:

 It seems likely that the rhythm of mnemonic videos would be quite different to mnemonic essays. In particularly, the frequency and density of questions would be lower than in the mnemonic medium, and it would be necessary to test different beats and cadences to ensure a good balance of emotional and intellectual experience. Even high-affect video typically has quieter moments; it achieves the high affect in part by contrast to the lower-intensity moments. Think about the way a good action movie or thriller needs lulls; if it's too high-intensity all the time, eventually our emotional response is dulled. We could design mnemonic video so the questions help fill this lower-intensity emotional beat.

 Of course, this is merely a quick sketch of one approach to the design of mnemonic video. Ideally, there would also be a spaced-repetition component, perhaps with the questions asked in text instead of video. This sort of sketch seems to us a promising direction, but needs considerable development and intense testing. In particular, we need to do detailed, second-by-second user experience testing, to understand and shape users' emotional and intellectual experience. That would continue until we were confident that our target users were having the desired experience. Ideally, we'd also generate several more very different designs, and try to understand how each approach would impact the user's emotional and intellectual experience.

 The broader point here is about taking emotion seriously. Historically, a lot of work on tools for thought has either ignored emotion, or treated it as no more than a secondary concern. Instead, that work has focused on new skills acquired, on what the user “learns”. They've been designing for Spock, when emotional connection is a high-order bit. Do users feel disinterested? Afraid? Hostile? Anxious? Or do they internalize a sense of excitement, of beauty, perhaps even an expansion in their own goals, an expansion of their self?

 By contrast, media forms such as movies and music and (often) video games do take emotion seriously. The designers of such forms often have incredibly elaborate models of user's emotional responses. These models range from detailed, second-by-second understanding, to deep thinking about a user's overall emotional journey. We believe it's possible and desirable to use such approaches in the development of tools for thought.

 At the same time, a positive emotional experience alone is not enough. For tools for thought to attain enduring power, the user must experience a real growth in mastery, an expansion in their ability to act. And so we'd like to take both the emotional and intellectual experience of tools for thought seriously. Mnemonic video is a good venue for such exploration. To paraphrase Einstein, attaining a detailed understanding without forming an emotional connection is lame; while forming an emotional connection without detailed understanding has no enduring power.

 Why isn't there more work on tools for thought today?

 If tools for thought are so great, why isn't more work being done on them? Why aren't they a major industry?

 As noted in the introduction, there's certainly a lot of lip service paid. It is, for instance, common to hear technologists allude to Steve Jobs's metaphor of computers as “bicycles for the mind”. But in practice it's rarely more than lip service. Many pioneers of computing have been deeply disappointed in the limited use of computers as tools to improve human cognition. Douglas Engelbart disparaged the “dangerous, disappointing, narrow, path that we seem to be stuck with following”. When asked in 2006 how much of his vision had been achieved, Engelbart replied facetiously “about 2.8 percent”. Alan Kay gives talks asserting “The real computer revolution hasn't happened yet” and in an interview has described the modern web as “reinventing the flat tire… at least give us what Engelbart did, for Christ's sake.”

 Our experience is that many of today's technology leaders genuinely venerate Engelbart, Kay, and their colleagues. Many even feel that computers have huge potential as tools for improving human thinking. But they don't see how to build good businesses around developing new tools for thought. And without such business opportunities, work languishes.

 What makes it difficult to build companies that develop tools for thought? To answer this, consider Adobe, one of the few large companies serious about developing new tools for thought Another large company which takes tools for thought extremely seriously is AutoDesk. A similar story could be told about it. . It's poured money into developing new mediums The plural of medium is, of course, media. However, in this context media would usually mean many pieces of new content. That's not what we mean: we mean multiple different new mediums ( Illustrator , Photoshop etc ). We'll reserve the unusual pluralization for this somewhat unusual meaning. for designers and artists – programs such as Illustrator, Photoshop, and so on. These mediums are remarkable tools for thought.

 Unfortunately for Adobe, such mediums are extremely expensive to develop, and it's difficult to prevent other companies from cheaply copying the ideas or developing near-equivalents. Consider, for example, the way the program Sketch has eaten into Adobe's market share, after duplicating many of the best features from several of Adobe's products, perhaps most notably Illustrator. And consider the way Figma is now eating into both Sketch and Illustrator's market share. Both Sketch and Figma have done this without needing to make an enormous investment in research. That's a big advantage they have over Adobe.

 As Marc Andreessen has observed In Elad Gil's “High Growth Handbook” (2018). :

 true defensibility purely at the product level is really rare in [Silicon] Valley, because there are a lot of really good engineers… And then there's the issue of leap-frogging. The next team has the opportunity to learn from what you did and then build something better.

 Put another way, many tools for thought are public goods. They often cost a lot to develop initially, but it's easy for others to duplicate and improve on them, free riding on the initial investment. While such duplication and improvement is good for our society as a whole, it's bad for the companies that make that initial investment. And so such tools for thought suffer the fate of many public goods: our society collectively underinvests in them, relative to the benefits they provide Of course, it does cost money for companies such as Sketch and Figma to duplicate features originating in Illustrator , and they have introduced some improvements. So our characterization as a public good is only approximate. .

 Earlier, we argued that modern design practice generally isn't up to the challenge of producing genuinely transformative tools for thought. On the surface, that process-level argument appears very different to the public goods argument we just made. In fact, the process-level explanation is a consequence of the public goods explanation: companies don't use the necessary processes because there's little value to them in doing so. By contrast, in “harder-tech” industries – say, chip design – companies have much more incentive to do deep research work. In those industries it's considerably harder for other companies to duplicate or capture the value of that research.

 It's illuminating to contrast with video games. Game companies develop many genuinely new interface ideas. This perhaps seems surprising, since you'd expect such interface ideas to also suffer from the public goods problem: game designers need to invest enormous effort to develop those interface ideas, and they are often immediately copied (and improved on) by other companies, at little cost. In that sense, they are public goods, and enrich the entire video game ecosystem.

 But there's a big difference between video game companies and companies such as Adobe. Many video games make most of their money from the first few months of sales. While other companies can (and do) come in and copy or riff on any new ideas, it often does little to affect revenue from the original game, which has already made most of its money In fact, cloning is a real issue in gaming, especially in very technically simple games. An example is the game Threes , which took the developers more than a year to make. Much of that time was spent developing beautiful new interface ideas. The resulting game was so simple that clones and near-clones began appearing within days. One near clone, a game called 2048 , sparked a mini-craze, and became far more successful than Threes . At the other extreme, some game companies prolong the revenue-generating lifetime of their games with re-releases, long-lived online versions, and so on. This is particularly common for capital-intensive AAA games, such as the Grand Theft Auto series. In such cases the business model relies less on clever new ideas, and more on improved artwork (for re-release), network effects (for online versions), and branding. . While this copying is no doubt irritating for the companies being copied, it's still worth it for them to make the up-front investment.

 The net result is that in gaming, clever new interface ideas can be distinguishing features which become a game's primary advantage in the marketplace. Indeed, new interface ideas may even help games become classics – consider the many original (at the time) ideas in games ranging from Space Invaders to Wolfenstein 3D to Braid to Monument Valley. As a result, rather than underinvesting, many companies make sizeable investments in developing new interface ideas, even though they then become public goods. In this way the video game industry has largely solved the public goods problems.

 By contrast, a company like Adobe builds their business around distribution and long-term lock in. They convince people – indeed, entire organizations – to make long-term commitments to their products. Schools offer classes so people can call themselves “Photoshop experts” or “Illustrator experts”. Companies designate their design departments as “Adobe shops”. So while Adobe does invest in developing clever new interface ideas (for them, unlike the video game companies, this genuinely means tools for thought), it's less central to their competitive advantage, and they invest less than they would if it was their central advantage. And Adobe does perhaps as much or more work developing tools for thought as any company.

 It's encouraging that the video game industry can make inroads on the public goods problem. Is there a solution for tools for thought? Unfortunately, the novelty-based short-term revenue approach of the game industry doesn't work. You want people to really master the best new tools for thought, developing virtuoso skill, not spend a few dozen hours (as with most games) getting pretty good, and then moving onto something new.

 Another plausible solution to the public goods problem is patents, granting a temporary monopoly over use of an invention. Many software companies, including Adobe, develop a large patent portfolio. However, the current patent system is not a solution for this problem. In 2017, Dana Rao, Adobe's Vice President for Intellectual Property and Litigation, posted a call for major reforms to the patent system, stating that:

 [the patent] system is broken… What happened? A patent gold rush built by patent profiteers… Their value lies not in the innovation behind the patent but in the vagueness of the patent's claims and the ability to enforce it in a plaintiff-friendly forum… Where did the material for these bad patents come from? The advent of software… This led to idea-only patents being granted with broad and often invalid claims, and eager patent profiteers were only too glad to take advantage.

 Adobe shares in common with many other software companies that much of their patenting is defensive: they patent ideas so patent trolls cannot sue them for similar ideas. The situation is almost exactly the reverse of what you'd like. Innovative companies can easily be attacked by patent trolls who have made broad and often rather vague claims in a huge portfolio of patents, none of which they've worked out in much detail. But when the innovative companies develop (at much greater cost) and ship a genuinely good new idea, others can often copy the essential core of that idea, while varying it enough to plausibly evade any patent. The patent system is not protecting the right things.

 Switching away from the viewpoint of individual companies, and to the viewpoint of society as a whole, not only do we want to incentivize invention, we also want ideas to move reasonably rapidly into the public domain. Think about fundamental tools for thought such as writing and the number system. Obviously, it's good that those spread throughout society, unencumbered by IP concerns! More broadly, many tools of thought become more valuable for society as they become more ubiquitous. Again, here the modern patent system has numerous well-known problems, striking a poor balance between private and public interest. While a well-designed patent system might very well help solve the public goods problem, the patent system we actually have seems poorly adapted to the problem.

 Is it possible to avoid the public goods problem altogether? Here's three classes of tools for thought which do:

 * Search engines such as Google are tools for thought. They avoid the public goods problem because their value is in their brand and in hard-to-duplicate and capital intensive backend elements (including their data centers, proprietary algorithms, ad network, and distribution), not in their interface ideas.
 * A service such as Twitter can be considered a tool for collective thought. While the interface is easily copied, the company is hard to duplicate, due to network effects.
 * Novel hardware devices (e.g., for VR, or the Wii remote, or for new musical instruments) can be used as the basis for new tools for thought. While hardware can be duplicated, it's often much more expensive than duplicating software. And, in any case, the advantage for such companies is often in distribution, marketing, and relationships with vendors who make products for the platform.

 While these suggestions all avoid the public goods problem, they don't directly solve the public goods problem. And many promising directions – including ideas such as the mnemonic medium and mnemonic videos – involve a substantial public goods element. Is it possible to solve the public goods problem in such cases? The two most promising approaches seem to us to be:

 * Philanthropic funding for research. This approach was used, for instance, by the field of computer animation and animated movies. Decades of public research work on computer animation resulted in a large number of powerful and (in many cases) publicly available ideas. This, in turn, helped prepare the way for companies such as Pixar and Dreamworks, which developed many of the ideas further, and took them to scale.
 * The model used by Adobe and similar companies, in which new tools for thought are a central part of the company's operations, but not the core of their competitive moat. That moat may instead be built around training, marketing, documentation, and so on.

 Questioning our basic premises

 There are three important premises we've taken for granted up to now. First is the assertion that we're still in the early days, that many more transformative tools for thought are yet to be discovered. Second is the assertion that work on tools for thought is stalled, that there's not lots of interesting work going on. And third, a kind of meta-premise, is the assertion that this kind of work is worth doing, relative to the current fashion for related ideas such as artificial general intelligence and brain-computer interfaces. In this section we discuss these premises.

 What if the best tools for thought have already been discovered?

 In other words, perhaps the 1960s and 1970s were an unrepeatable golden age, and all we can expect in the future is gradual incremental improvement, and perhaps the occasional major breakthrough, at a decreasing frequency?

 There's a plausible story suggesting this is true. Tech is an enormous industry, well funded, with many bright, ambitious, talented people. Surely if there were major ideas to discover, people would do so? This argument is reinforced by the fact that, at the individual level, we meet many brilliant people who are fascinated by (and often working on) tools for thought, but who nonetheless seem to be making slow progress.

 But while this story has a superficial appeal, it's misleading. Really difficult problems – problems like inventing Hindu-Arabic numerals – aren't solved by good intentions and interest alone. A major thing missing is foundational ideas powerful enough to make progress. In the earliest days of a discipline – the proto-disciplinary stage – a few extraordinary people – people like Ivan Sutherland, Doug Engelbart, Alan Kay, and Bret Victor – may be able to make progress. But it's a very bespoke, individual progress, difficult to help others become proficient in, or to scale out to a community. It's not yet really a discipline. What's needed is the development of a powerful praxis, a set of core ideas which are explicit and powerful enough that new people can rapidly assimilate them, and begin to develop their own practice. We're not yet at that stage with tools for thought. But we believe that we're not so far away either.

 While that argument is helpful context, it doesn't address the core point: it doesn't mean there are a lot of new transformative tools for thought waiting to be discovered. Again: maybe the most important tools for thought have already been discovered?

 We can't predict the future, so it's not possible to answer this question with certainty. But it seems to us that the human race just hasn't really tried very hard yet. When small groups of motivated people do – as in pioneering labs such as PARC, SRI, and other DARPA-inspired early efforts, as well as modern labs such as Dynamicland – they make rapid progress. It's extremely encouraging that those efforts – tiny efforts, in the scheme of humanity's overall research effort – make such rapid progress. To us, that suggests scaling them up, becoming much more ambitious.

 Isn't this what the tech industry does? Isn't there a lot of ongoing progress on tools for thought?

 In particular, aren't there already a lot of imaginative, determined, well-funded people working on this? Isn't tech in considerable part already about developing new tools for thought?

 Part of this question is caused by a confusion in terms. Obviously, many tech companies build special-purpose tools for solving specific problems. But while those may be valuable tools, they're certainly not “tools for thought” in the broad sense we're discussing – not like language or writing or, for that matter, Illustrator.

 Still, there are tech companies which really do develop tools for thought. We already discussed some examples where companies have partially or totally avoided the public goods problem, tools such as: Illustrator, Google Search, Twitter, Slack, Google Docs, programmer tools Programmer tools are a case where the insight-through-making loop operates quite well. The avoidance of the public goods problem is enabled by a complex set of factors, including the fact that this is a case where many highly skilled researchers are simultaneously also accomplished makers, and it's either their job to produce public goods (researchers) or an enjoyable hobby. To a lesser extent the same is true in the artistic and music communities. , and so on. All really are significant tools for thought.

 But consider our most fundamental tools for thought – language, writing, music, etc. Those are public goods. No-one owns language; to the extent that it is owned (trademarks and so on) it may actually limit the utility of language. These tools are all about introducing fundamental new mental representations and mental operations. Those aren't owned by any company, they're patterns owned by humanity.

 This argument makes it seem likely that many of the most fundamental and powerful tools for thought do suffer the public goods problem. And that means tech companies focus elsewhere; it means many imaginative and ambitious people decide to focus elsewhere; it means we haven't developed the powerful practices needed to do work in the area, and a result the field is still in a pre-disciplinary stage. The result, ultimately, is that it means the most fundamental and powerful tools for thought are undersupplied.

 Why not work on artificial general intelligence (AGI) or brain-computer interfaces (BCI) instead?

 We're often asked: why don't you work on AGI or BCI instead of tools for thought? Aren't those more important and more exciting? And for AGI, in particular, many of the skills required seem related.

 They certainly are important and exciting subjects. What's more, at present AGI and BCI are far more fashionable (and better funded). As a reader, you may be rolling your eyes, supposing our thinking here is pre-determined: we wouldn't be writing this essay if we didn't favor work on tools for thought. But these are questions we've wrestled hard with in deciding how to spend our own lives. One of us wrote a book about artificial intelligence before deciding to focus primarily on tools for thought; it was not a decision made lightly, and it's one he revisits from time to time. Indeed, given the ongoing excitement about AGI and BCI, it would be surprising if people working on tools for thought didn't regularly have a little voice inside their head saying “hey, shouldn't you be over there instead?” Fashion is seductive.

 One striking difference is that AGI and BCI are based on relatively specific, well-defined goals. By contrast, work on tools for thought is much less clearly defined. For the most part we can't point to well-defined, long-range goals; rather, we have long-range visions and aspirations, almost evocations. The work is really about exploration of an open-ended question: how can we develop tools that change and expand the range of thoughts human beings can think?

 Culturally, tech is dominated by an engineering, goal-driven mindset. It's much easier to set KPIs, evaluate OKRs, and manage deliverables, when you have a very specific end-goal in mind. And so it's perhaps not surprising that tech culture is much more sympathetic to AGI and BCI as overall programs of work.

 But historically it's not the case that humanity's biggest breakthroughs have come about in this goal-driven way. The creation of language – the ur tool for thought – is perhaps the most important occurrence of humanity's existence. And although the origin of language is hotly debated and uncertain, it seems extremely unlikely to have been the result of a goal-driven process. It's amusing to try imagining some prehistoric quarterly OKRs leading to the development of language. What sort of goals could one possibly set? Perhaps a quota of new irregular verbs? It's inconceivable!

 Similarly, the invention of other tools for thought – writing, the printing press, and so on – are among our greatest ever breakthroughs. And, as far as we know, all emerged primarily out of open-ended exploration, not in a primarily goal-driven way. Even the computer itself came out of an exploration that would be regarded as ridiculously speculative and poorly-defined in tech today. Someone didn't sit down and think “I need to invent the computer”; that's not a thought they had any frame of reference for. Rather, pioneers such as Alan Turing and Alonzo Church were exploring extremely basic and fundamental (and seemingly esoteric) questions about logic, mathematics, and the nature of what is provable. Out of those explorations the idea of a computer emerged, after many years; it was a discovered concept, not a goal. Fundamental, open-ended questions seem to be at least as good a source of breakthroughs as goals, no matter how ambitious. This is difficult to imagine or convince others of in Silicon Valley's goal-driven culture. Indeed, we ourselves feel the attraction of a goal-driven culture. But empirically open-ended exploration can be just as, or more successful.

 “What will new tools for thought be like?” is a question we hear often. And yet, almost by definition, we cannot say. As we noted earlier, if we could communicate the experience in an essay, then the tools would be failing at their job; they would not be transforming a person's thinking, or even their consciousness. Concretely: to understand the mnemonic medium you must use it intensively over an extended period. And even then you may not be conscious of the effect; we've done interviews with users who are apparently unaware of the incredible level of recall they have of material in the essay they have read. One of the most famous papers in the philosophy of consciousness is entitled “What is it like to be a bat?” Each tool for thought poses a similar question, near impossible to answer without immersion in the tool: “What is it like to be a language user? A musician?” and so on.

 It seems plausible to us that work on tools for thought will be, over the next few decades, more important than work on AGI and BCI. And, given how fashionable and well-funded work on AGI and BCI currently is, it seems nearly certain that work on tools for thought offers vastly greater benefit, at the margin.

 What about the longer term? There, the situation is less clear. It seems likely that the three fields will merge, or at least feed strongly into one another. Together with Shan Carter, one of us has argued that one of the most promising applications for AI is as a way of discovering new tools for thought Shan Carter and Michael Nielsen, Using Artificial Intelligence to Augment Human Intelligence (2017). .

 BCI seems likely to be even more closely related. BCI is sometimes described using ideas like a memory chip for long-term memories, or some way of increasing short-term working memory. Such ideas may well become important. But it also seems possible that BCIs will be used to enable new mental operations, new mental representations, and new affordances for thought; in short, the same kind of things as are involved in developing non-BCI tools for thought. Perhaps we'll develop the capacity to directly imagine ourselves in 4 or 5 or more dimensions; or traversing a Riemann manifold; or the ability to have multiple tracks of conscious attention. These are about changing the interface for thought, the basic abstractions and operations which are allowed. And so it seems plausible that work today on tools for thought will directly impact the way we use BCIs in the future.

 Executable books

 The skill of writing is to create a context in which other people can think.

 — Edwin Schlossberg

 The computer scientist Peter Norvig has written an interactive essay discussing the distribution of wealth in society. Norvig's essay is a Jupyter notebook which expresses many of the ideas in running Python code. That code sets up a population of agents, with an initial distribution of wealth. Agents randomly (and repeatedly) meet one another in pairs, and engage in simple economic transactions. More concretely: a simple transaction model could be that when two people meet, their joint wealth is pooled, and then randomly divided between the two of them. That model is just to give you the gist – more complex transaction models are, of course, possible. The notebook simulates how the distribution of wealth evolves over time.

 Part of what makes Norvig's essay beautiful is that with just a few lines of Python code Norvig is able to show some surprising results about wealth inequality. For instance, his results suggest that the initial distribution of wealth in the economy doesn't much affect the long-run distribution of wealth. Rather, it's the nature of the transactions which determines the long-run distribution of wealth. This likely violates at least some user's intuitions. As another example, his results also suggest that constraining agents to trade only with people geographically near them makes little difference to the final distribution of wealth.

 Results like these will challenge the intuition of some users. But instead of those challenges being on the basis of easily-ignored abstract arguments, users can immediately engage with Norvig's model. Suppose someone doesn't like the idea that the initial distribution of wealth doesn't affect long-run wealth inequality. They're challenged to find a counterexample, an initial distribution of wealth which does affect long-run inequality. They can experiment easily, making simple modifications to just one or a few lines of Python code, trying to find instances where the initial distribution matters. No matter whether they succeed or fail, they will build a better understanding of the problem Sufficiently motivated reasoners will, of course, ignore any conclusion they don't like. Such people are most likely a lost cause to serious thought. .

 Suppose the content of Norvig's essay had instead been presented in a more conventional static form. For a reader to extend or interrogate the results would require total mastery of the material, and a high level of mathematical competence. But in the notebook format it's much easier for the reader to experiment. Their exploration is scaffolded, they can make small modifications and see the results, even the answers to questions Norvig did not anticipate. This kind of scaffolded exploration is a way to build up their own understanding, and perhaps even push the frontiers of knowledge.

 Norvig's essay is one of thousands (or perhaps even millions) of Jupyter notebooks that have been created. Of course, most such notebooks are hastily and poorly written. But in the hands of an excellent writer and thinker like Norvig, notebooks can become remarkable environments for thought, both individual and shared. It's tempting to regard them as merely a mashup of essay and code. But really they're a new media form, with different possibilities from either essays or code, and with striking opportunities to go much further. In this section we explore those opportunities Of course, systems like Jupyter go back decades. There are antecedents in Knuth's notion of literate programming, in Mathematica notebooks, in PARC's Learning Research Group, in the PLATO system (and, more broadly, computer-assisted instruction), to name but a few. In the discussion below we emphasize opportunities that seem to us undervalued in many of these prior systems. .

 We described Norvig's essay as an “interactive essay”. It's useful to have a more specific term, to distinguish it from other interactive forms, like the mnemonic medium. In this essay, we'll use the term “executable book” This is, of course, a placeholder term. “Executable essay” would in many ways be more natural, though unfortunately that term is perhaps even more naturally interpreted as meaning “web page”, in the current context. . We won't define this precisely here; definition is not the point. Rather, the point is to try to better understand the potential of media forms which combine prose and code in something like this form.

 Tools for thought must be developed while doing serious work. The aspiration to canonical content

 Seymour Papert, one of the principal creators of the Logo programming language, had a remarkable aspiration for Logo. Logo is sometimes described as a “programming language for children”, and people sometimes think Papert was mostly interested in helping children learn how to program. But that wasn't Papert's principal intent. Rather, Papert wanted to create an immersive environment – a kind of “Mathland” – in which children could be immersed in mathematical ideas. In essence, children could learn differential geometry by going to Mathland.

 It's a beautiful aspiration, and Logo contains many striking ideas. But as far as we know, no professional differential geometer (or, more generally, mathematician) uses Logo seriously as a tool in their work. And upon reflection that seems troubling. If Logo genuinely expresses the ideas of differential geometry, why don't differential geometers use it? You start to wonder: might it be that Logo leaves out important ideas about differential geometry, maybe even the most important ideas about differential geometry? After all, while mathematically trained, Papert wasn't himself an accomplished differential geometer. How would he even have known what to include? And certainly most of the people interested in Logo aren't qualified to make that judgment.

 There's a standard retort to this, which we've heard from within the Logo community. It's to talk about the “floor” and “ceiling” of different environments for thought. In this account, Logo has a low floor (meaning anyone can use it) and a low ceiling (so it's not well adapted for the sort of advanced work a professional would want to do).

 At first this sounds plausible. But upon reflection it's difficult to make much sense of. How do the creators of Logo know that mastering Logo helps later with understanding real (forgive us!) differential geometry? What's the criterion for success? One of us (MN) worked for several years doing research in the closely related field of Riemannian geometry. While Logo is enjoyable to use and contains many fun ideas, MN has trouble seeing that learning Logo would help much in learning differential geometry.

 At the end of Norvig's economics essay is a short afterword explaining how he came to write the essay. Shortly before writing the essay he'd heard about the kinds of economic models discussed in the notebook, and he wanted to explore several questions about them. After talking it over with some colleagues they decided to each independently attack the problems, and to compare notes. Although Norvig's essay is, in some sense, “educational”, Norvig's intent was to explore a set of problems he himself was genuinely curious about. The educational aspect was a byproduct.

 And so what you have is a world-class research scientist who wanted to explore a set of questions. He used the Jupyter medium to do those explorations, and then to share that exploration with the world. And he shared it in a form where others could immediately build upon and extend his thinking.

 There's a lot of work on tools for thought that takes the form of toys, or “educational” environments. Tools for writing that aren't used by actual writers. Tools for mathematics that aren't used by actual mathematicians. And so on. Even though the creators of such tools have good intentions, it's difficult not to be suspicious of this pattern. It's very easy to slip into a cargo cult mode, doing work that seems (say) mathematical, but which actually avoids engagement with the heart of the subject. Often the creators of these toys have not ever done serious original work in the subjects for which they are supposedly building tools. How can they know what needs to be included?

 Concretely: suppose you want to build tools for subject X (say X = differential geometry). Unless you are deeply involved in practicing that subject, it's going to be extremely difficult to build good tools. It'll be much like trying to build new tools for carpentry without actually doing any carpentry yourself. This is perhaps part of why tools like Mathematica work quite well – the principal designer, Stephen Wolfram, has genuine research interests in mathematics and physics. Of course, not all parts of Mathematica work equally well; some parts feel like toys, and it seems likely those are the ones not being used seriously internal to the company.

 There's a general principle here: good tools for thought arise mostly as a byproduct of doing original work on serious problems. They tend either be created by the people doing that work, or by people working very closely to them, people who are genuinely bought in A related argument has been made in Eric von Hippel's book “Democratizing Innovation” (2005), which identifies many instances where what appears to be commercial product development is based in large or considerable part on innovations from users. . Furthermore, the problems themselves are typically of intense personal interest to the problem-solvers. They're not working on the problem for a paycheck; they're working on it because they desperately want to know the answer.

 Many people have asked why we wrote our first mnemonic essay about quantum computing. If we'd chosen an easier subject we could have attracted a much larger audience. But we also wanted the essay to be authentic, to be about problems we wanted to solve. One of us (MN) has done a lot of original research work on quantum computing. The essay reflects that thinking. Indeed, the framing of the essay is about answering a question MN personally wanted to answer: if humans ever discovered aliens, would they have computers, and if so, what types of computer would they have? This perhaps sounds like a contrived question, but it's quite serious, and turns out to be a deep question with a nontrivial answer: writing the essay helped MN substantially improve his understanding of the question.

 That said, answering that question wasn't the principal point of creating the essay: making the mnemonic medium was. And for future work on tools for thought, it'd be valuable to push much harder on questions we'd genuinely like to answer ourselves. That's a way of keeping yourself honest, ensuring you're not just building a flashy toy, but something genuinely useful for solving real problems that are of independent interest.

 In serious mediums, there's a notion of canonical media. By this, we mean instances of the medium that expand its range, and set a new standard widely known amongst creators in that medium. For instance, Citizen Kane, The Godfather, and 2001 all expanded the range of film, and inspired later film makers. It's also true in new media. YouTubers like Grant Sanderson have created canonical videos: they expand the range of what people think is possible in the video form. And something like the Feynman Lectures on Physics does it for textbooks. In each case one gets the sense of people deeply committed to what they're doing. In many of his lectures it's obvious that Feynman isn't just educating: he's reporting the results of a lifelong personal obsession with understanding how the world works. It's thrilling, and it expands the form.

 We've been disappointed by how unambitious people are in this sense with Jupyter notebooks. They haven't pushed the medium all that hard; there is no Citizen Kane of Jupyter notebooks. Indeed, we're barely beyond the Lumière brothers. Examples like Norvig's notebook are fine work, but seem disappointing when evaluated as leading examples of the medium.

 Aspiring to canonicity, one fun project would be to take the most recent IPCC climate assessment report (perhaps starting with a small part), and develop a version which is executable. Instead of a report full of assertions and references, you'd have a live climate model – actually, many interrelated models – for people to explore. If it was good enough, people would teach classes from it; if it was really superb, not only would they teach classes from it, it could perhaps become the creative working environment for many climate scientists.

 One promising exploration in this direction is The Structure and Interpretation of Classical Mechanics, a beautiful executable book building up classical mechanics. Many theorems of classical mechanics aren't just expressed in static form, on the page, but live, as code which can be modified by the user. Theorems become APIs, which can literally be applied to other objects, and chained together. It uses a much more powerful underlying model than Jupyter, developing a new symbolic language as part of the book. It has many flaws – among them, the book doesn't run live in the browser, making it difficult for users to experiment. And while the book is well written, the authors do not understand classical mechanics as deeply as the authors of some other books. But it's nonetheless an inspiring evocation of what is possible. And it hints at what is possible when authors use executable books for a serious purpose, and aspire toward canonical media.

 Stronger emotional connection through an inverted writing structure

 Consider an author writing a popular book about quantum mechanics. Such an author is in a strong position: they can begin their book with astonishing phenomena such as black hole evaporation, quantum teleportation, and the role of quantum fluctuations in the early universe. Or, if they wish, they can start with some of the deepest mysteries known to humanity: the relationship between quantum mechanics and gravity, or the quantum measurement problem. There is no shortage of extraordinary phenomena and beautiful mysteries. These are the kind of things which touch a chord inside many, perhaps most people. And so it's relatively easy to draw readers in, to get them engaged, and keep them connected.

 By contrast, consider a typical technical book about quantum mechanics. It's very unlikely to start with black hole evaporation or quantum teleportation – and if it does, such a discussion will be perfunctory. Instead, it will start out drily, with technical minutiae. Complex numbers. Wavefunctions. Many different types of differential equations, and how to solve them. Hermitian and unitary operators. And so on, piece by piece slowly building up all the machinery needed to solve quantum mechanical problems. It may be tens or even hundreds of pages before the book begins to connect to the exciting problems which form the bread-and-butter of popular accounts.

 People with little experience doing good technical writing often complain about this dry, bottom-up approach. They will complain that writers should instead stay closer to the fun material, and use less technical notation and nomenclature. But when competent writers attempt to follow this prescription, invariably it works poorly.

 One problem is that a person can spend years reading analogies about black hole evaporation, quantum teleportation, and so on. And at the end of all that reading they typically have… not much genuine understanding to show for it. The analogies and heuristic reasoning simply don't go far. They may be entertaining and produce some feeling of understanding. But the reasoning won't scale out; it can't be applied to other phenomena, at least not without lots of caveats, caveats the reader is in no position to understand or apply. As a result, good technical writers instead mostly build things up from first principles, with occasional digressions to the broader motivating picture. And that means starting with a lot of detailed, technical minutiae.

 It's striking to contrast conventional technical books with the possibilities enabled by executable books. You can imagine starting an executable book with, say, quantum teleportation, right on the first page. You'd provide an interface – perhaps a library is imported – that would let users teleport quantum systems immediately. They could experiment with different parts of the quantum teleportation protocol, illustrating immediately the most striking ideas about it. The user wouldn't necessarily understand all that was going on. But they'd begin to internalize an accurate picture of the meaning of teleportation. And over time, at leisure, the author could unpack some of what might a priori seem to be the drier details. Except by that point the reader will be bought into those details, and they won't be so dry A similar argument has been made by Rachel Thomas, Providing a Good Education in Deep Learning (2016).

 In other words, you could begin an executable book with material the users already care about, can connect to easily, and find motivating. For instance, you could begin by exploring teleportation or the Big Bang. But such an opening won't suffer the drawback of popular science, of being vague and imprecise. Rather, the interface would be completely well specified. And, with some care, the interface could be scaled out, applied in ever-expanding contexts. The understanding would be transferable. Even a user who has understood only a tiny part of the material could begin tinkering, building up an understanding based on play and exploration. It's common to dismiss such an approach as leading to a toy understanding; we believe, on the contrary, that with well enough designed scaffolding it can lead to a deep understanding The masters of this are video game designers. See, for example, Dan Cook, Building a Princess Saving App (2008), and Jonathan Blow and Marc ten Bosch, Designing to Reveal the Nature of the Universe (2011). . Developed in enough depth, such an environment may even be used to explore novel research ideas. To our knowledge this kind of project has never been seriously pursued. But it'd be fun to try.

 Summary and Conclusion

 We've covered a lot, and it's helpful to distill the main takeaways – general principles, questions, beliefs, and aspirations. Let's begin with memory systems, particularly the mnemonic medium:

 * Memory systems make memory into a choice, rather than an event left up to chance: This changes the relationship to what we're learning, reduces worry, and frees up attention to focus on other kinds of learning, including conceptual, problem-solving, and creative.
 * Memory systems are in their infancy: it is possible to increase effective human memory by an order of magnitude, even beyond what existing memory systems do; and systems such as the mnemonic medium may help expand the range of subjects users can comprehend at all.
 * What would a virtuoso use of the mnemonic medium look like? There's some sense in which the mnemonic medium is “just” flash cards. The right conclusion isn't that it's therefore trivial; it's that flash cards are greatly underrated. In writing Quantum Country we treated the writing of the cards with reverence; ideally, authors would take card writing as seriously as Nabokov took sentence writing. Of course, we didn't reach that level, but the aspiration expands the reach of the medium. What would virtuoso or even canonical uses of the mnemonic medium look like?
 * Memory systems can be used to build genuine conceptual understanding, not just learn facts: In Quantum Country we achieve this in part through the aspiration to virtuoso card writing, and in part through a narrative embedding of spaced repetition that gradually builds context and understanding.
 * Mnemonic techniques such as memory palaces are great, but not versatile enough to build genuine conceptual understanding: Such techniques are very specialized, and emphasize artificial connections, not the inherent connections present in much conceptual knowledge. The mnemonic techniques are, however, useful for bootstrapping knowledge with an ad hoc structure.
 * Memory is far more important than people tend to think: It plays a role in nearly every part of cognition, including problem-solving, creative work, and meta-cognition. The flip side is that memory systems themselves want to grow into other types of tools – tools for reading, tools for problem-solving, tools for creating, tools for attention management. That said, we don't yet know what memory systems want to be. To reiterate: memory systems are in their infancy.

 The mnemonic medium is merely one prototype tool for thought. We also discussed several other ideas, including mnemonic video and executable books. Here are some key takeaways:

 * What practices would lead to tools for thought as transformative as Hindu-Arabic numerals? And in what ways does modern design practice and tech industry product practice fall short? To be successful, you need an insight-through-making loop to be operating at full throttle, combining the best of deep research culture with the best of Silicon Valley product culture.
 * Tools for thought are (mostly) public goods, and as a result are undersupplied: That said, there are closely-related models of production which have succeeded (the games industry, Adobe, AutoDesk, Pixar). These models should be studied, emulated where possible, and used as inspiration to find more such models.
 * Take emotion seriously: Historically, work on tools for thought has focused principally on cognition; much of the work has been stuck in Spock-space. But it should take emotion as seriously as the best musicians, movie directors, and video game designers. Mnemonic video is a promising vehicle for such explorations, possibly combining both deep emotional connection with the detailed intellectual mastery the mnemonic medium aspires toward.
 * Tools for thought must be developed in tandem with deep, original creative work: Much work on tools for thought focuses on toy problems and toy environments. This is useful when prototyping, but to be successful such tools must ultimately be used to do serious, original creative work. That's a baseline litmus test for whether the tools are genuinely working, or merely telling a good story. Ideally, for any such tool there will be a stream of canonical media expanding the form, and entering the consciousness of other creators.

 Let's return to the question that began the essay: how to build transformative tools for thought? Of course, we haven't even precisely defined what such transformative tools are! But they're the kind of tools where relatively low cost changes in practice produce transformative changes in outcome – non-linear returns and qualitative shifts in thinking. This is in contrast with the usual situation, where a small change in practice causes a small change in results.

 Historically, humans have invented many such transformative tools for thought. Writing and music are ancient examples; in modern times, tools such as Photoshop and AutoCAD qualify. Although it's very early days, we believe the mnemonic medium shows much promise. It needs to be developed much further, along the lines we've described, and likely requires additional powerful ideas. But we believe it's possible for humanity to have a widespread memory practice that radically changes the way we think.

 More broadly, we hope the principles in this essay will help support the creation of more transformative tools for thought. Historically, most invention of tools for thought has been done bespoke, by inspired individuals and groups. But we believe that in the future there will be an established community that routinely does this kind of invention.

 Acknowledgments

 This essay is based on conversations with many people. Particular thanks to David Albert, Shan Carter, May-Li Khoe, Robert Ochshorn, Grant Sanderson, Caitlin Sikora, and Bret Victor. Thanks also to Nicky Case for feedback on a draft of the essay. This work was supported by our Patreons. AM is supported in part by a grant from Emergent Ventures. MN's work was supported by YC Research.

 Citation

 For attribution in academic contexts, please cite this work as:

 Andy Matuschak and Michael Nielsen, “How can we develop transformative tools for thought?”, https://numinous.productions/ttft, San Francisco (2019).
 Authors are listed alphabetically.

 License

 This work is licensed under a Creative Commons Attribution 4.0 International License. This means you're free to copy, share, and build on the work, provided you attribute it appropriately. Please click on the following license link for details:

Part of the origin myth of modern computing is the story of a
	golden age in the 1960s and 1970s. In this story, visionary
	pioneers pursued a dream in which computers enabled powerful
	tools for thought, that is, tools to augment human
	intelligence E.g., Douglas
	Engelbart, Augmenting Human
	Intellect: A Conceptual Framework (1962).. One of
	those pioneers, Alan Kay, summed up the optimism of this dream
	when he wrote of the potential of the personal computer:
	“the very use of it would actually change the thought
	patterns of an entire
	civilization” Alan
	Kay, User Interface: A Personal
	View (1989)..
 
 
	It's an inspiring dream, which helped lead to modern
	interactive graphics, windowing interfaces, word processors,
	and much else. But retrospectively it's difficult not to be
	disappointed, to feel that computers have not yet been nearly
	as transformative as far older tools for thought, such as
	language and writing. Today, it's common in technology circles
	to pay lip service to the pioneering dreams of the past. But
	nostalgia aside there is little determined effort to pursue
	the vision of transformative new tools for thought.
 
 
	We believe now is a good time to work hard on this vision
	again. In this essay we sketch out a set of ideas we believe
	can be used to help develop transformative new tools for
	thought. In the first part of the essay we describe an
	experimental prototype system that we've built, a kind
	of mnemonic medium intended to augment human memory.
	This is a snapshot of an ongoing project, detailing both
	encouraging progress as well as many challenges and
	opportunities. In the second part of the essay, we broaden
	the focus. We sketch several other prototype systems. And we
	address the question: why is it that the technology industry
	has made comparatively little effort developing this vision of
	transformative tools for thought?
 
 
	In the opening we mentioned some visionaries of the past. To
	those could be added many others – Ivan Sutherland,
	Seymour Papert, Vannevar Bush, and more. Online there is much
	well-deserved veneration for these people. But such veneration
	can veer into an unhealthy reverence for the good old days, a
	belief that giants once roamed the earth, and today's work is
	lesser. Yes, those pioneers did amazing things, and arguably
	had ways of working that modern technologists, in both
	industry and academia, are poorly equipped to carry on. But
	they also made mistakes, and were ignorant of powerful ideas
	that are available today. And so a theme through both parts of
	the essay is to identify powerful ideas that weren't formerly
	known or weren't acted upon. Out of this understanding arises
	a conviction that a remarkable set of opportunities is open
	today.
 
 
	A word on nomenclature: the term “tools for
	thought” rolls off neither the tongue nor the
	keyboard. What's more, the term “tool” implies a
	certain narrowness. Alan Kay has
	argued Again, in Alan
	Kay, User Interface: A Personal
	View (1989), among other places. that a more
	powerful aim is to develop a new medium for
	thought. A medium such as, say,
	Adobe Illustrator is essentially different from any
	of the individual tools Illustrator contains. Such a
	medium creates a powerful immersive context, a context in
	which the user can have new kinds of thought, thoughts that
	were formerly impossible for them. Speaking loosely, the range
	of expressive thoughts possible in such a medium is an
	emergent property of the elementary objects and actions in
	that medium. If those are well chosen, the medium expands the
	possible range of human thought.
 
 
	With that said, the term “tools for thought” has
	been widely used since Iverson's 1950s and 1960s
	work An account may be found in
	Iverson's Turing Award
	lecture, Notation as a Tool
	of Thought (1979). Incidentally, even Iverson is really
	describing a medium for thought, the APL programming language,
	not a narrow tool. introducing the term. And so we
	shall use “tools for thought” as our catch all
	phrase, while giving ourselves license to explore a broader
	range, and also occasionally preferring the term
	“medium” when it is apt.
 
 
	Let's come back to that phrase from the opening, about
	changing “the thought patterns of an entire
	civilization”. It sounds ludicrous, a kind of tech
	soothsaying. Except, of course, such changes have happened
	multiple times during human history: the development of
	language, of writing, and our other most powerful tools for
	thought. And, for better and worse, computers really have
	affected the thought patterns of our civilization over the
	past 60 years, and those changes seem like just the
	beginning. This essay is a small contribution to understanding
	how such changes happen, and what is still possible.
 
 
	The musician and comedian Martin Mull has observed that
	“writing about music is like dancing about
	architecture”. In a similar way, there's an inherent
	inadequacy in writing about tools for thought. To the extent
	that such a tool succeeds, it expands your thinking beyond
	what can be achieved using existing tools, including
	writing. The more transformative the tool, the larger the gap
	that is opened. Conversely, the larger the gap, the more
	difficult the new tool is to evoke in writing. But what
	writing can do, and the reason we wrote this essay, is act as
	a bootstrap. It's a way of identifying points of leverage that
	may help develop new tools for thought. So let's get on with
	it.
 
 Part I: Memory systems
 Introducing the mnemonic medium
 
 
	Few subjects are more widely regarded as difficult than
	quantum computing and quantum mechanics. Indeed, popular media
	accounts often regale (and intimidate) readers with quotes
	from famous physicists in the vein of: “anyone who
	thinks they’ve understood quantum mechanics has not understood
	quantum mechanics”.
 
 
 
	What makes these subjects difficult? In fact, individually
	many of the underlying ideas are not too complicated for
	people with a technical background. But the ideas come in an
	overwhelming number, a tsunami of unfamiliar concepts and
	notation. People must learn in rapid succession of qubits, the
	bra-ket notation, Hadamard gates, controlled-not gates, and
	many, many other abstract, unfamiliar notions. They're
	imbibing an entire new language. Even if they can follow at
	first, understanding later ideas requires fluency with all the
	earlier ideas. It's overwhelming and eventually disheartening.
 
 
	As an experiment, we have developed a
	website,  Quantum
	Country , which explores a new approach to explaining
	quantum computing and quantum
	mechanics. Ostensibly, Quantum Country appears to be
	a conventional essay introduction to these subjects. There is
	text, explanations, and equations, much as in any other
	technical essay. Here's an excerpt:
 
	
   
	But it's not a conventional essay. Rather, Quantum
	Country is a prototype for a new type of mnemonic
	medium. Aspirationally, the mnemonic medium makes it
	almost effortless for users to remember what they read. That
	may sound like an impossible aspiration. What makes it
	plausible is that cognitive scientists know a considerable
	amount about how human beings store long-term
	memories. Indeed, what they know can almost be distilled to an
	actionable recipe: follow these steps, and you can remember
	whatever you choose.
 
 
	Unfortunately, those steps are poorly supported by existing
	media.For more on this argument, see
	Andy Matuschak, Why
	books don’t work (2019). Is it possible to
	design a new medium which much more actively supports
	memorization? That is, the medium would build in (and, ideally,
	make almost effortless) the key steps involved in memory. If we
	could do this, then instead of memory being a haphazard event,
	subject to chance, the mnemonic medium would make memory into a
	choice. Of course, on its own this wouldn't make it trivial to
	learn subjects such as quantum mechanics and quantum computing
	– learning those subjects is about much more than memory.
	But it would help in addressing one core difficulty: the
	overwhelming number of new concepts and notation.
 
 
 
	In fact, there are many ways of redesigning the essay medium
	to do that. Before showing you our prototype, please pause for
	a moment and consider the following questions: how could you
	build a medium to better support a person's memory of what
	they read? What interactions could easily and enjoyably help
	people consolidate memories? And, more broadly: is it possible
	to 2x what people remember? 10x? And would that make any
	long-term difference to their effectiveness?
 
 
	Let's sketch the user experience of Quantum
	Country. At the time of this writing the site contains
	three mnemonic essays (i.e., particular instances of the
	mnemonic medium). We'll focus on the introductory essay,
	“Quantum
	 Computing for the Very Curious”. Embedded within
	 the text of the essay are 112 questions about that
	 text. Users are asked to create an account, and quizzed as
	 they read on whether they remember the answers to those
	 questions. Here's what the interaction looks like, as a user
	 answers three questions.
 
	
 
	Note that this interaction occurs within the text of the essay
	itself. Here's a zoomed-out view, so you can see how such
	questions are surrounded by essay text both above and below:
 
	  
	We use the term cards for these interface elements
	pairing questions and answers.
 
 
	Of course, for long-term memory it's not enough for users to
	be tested just once on their recall. Instead, a few days after
	first reading the essay, the user receives an email asking
	them to sign into a review session. In that review session
	they're tested again, in a manner similar to what was shown
	above. Then, through repeated review sessions in the days and
	weeks ahead, people consolidate the answers to those questions
	into their long-term memory.
 
 
	So far, this looks like no more than an essay which integrates
	old-fashioned flashcards. But notice the intervals indicated
	at the bottom of the cards:
 
	
 
 
	The highlighted time interval is the duration until the user
	is tested again on the question. Questions start out with the
	time interval “in-text”, meaning the user is being
	tested as they read the essay. That rises to five days, if the
	user remembers the answer to the question. The interval then
	continues to rise upon each successful review, from five days
	to two weeks, then a month, and so on. After just five
	successful reviews the interval is at four months. If the user
	doesn't remember at any point, the time interval drops down
	one level, e.g., from two weeks to five days.
 
 
	This takes advantage of a fundamental fact about human memory:
	as we are repeatedly tested on a question, our memory of the
	answer gets stronger, and we are likely to retain it for
	longer The literature on this effect
	is vast. A useful entrée is: Gwern
	Branwen, Spaced
	Repetition for Efficient Learning.. This
	exponential rise perhaps seems innocuous, but it's
	transformative. It means that a relatively small number of
	reviews will enable a user to remember for years. With the
	time taken to review a typical question being just a few
	seconds, that means a user can achieve long-term recall with
	no more than a few minutes' work. By contrast, with
	conventional flashcards it takes hours of review to achieve
	the same durability. Exponential scheduling is far more
	efficient.
 
 The early impact of the prototype mnemonic medium
 
	Although it's early days for Quantum Country we can
	begin to see some of the impact of the mnemonic
	medium. Plotted below is the demonstrated retention of answers
	for each user, versus the number of times each question in the
	mnemonic essay has been reviewed:
 
	  
	The graph takes a little unpacking to explain. By a card's
	“demonstrated retention” we mean the maximum time
	between a successful review of that card, and the prior review
	of that card. A little more concretely, consider repetition
	number 6, say (on the horizontal axis). At the point, a user
	has reviewed all 112 questions in the essay 6 times. And the
	vertical axis shows the total demonstrated retention, summed
	across all cards, with each blue dot representing a single
	user who has reached repetition 6.
 
 
	So, for instance, after 6 repetitions, we see from the graph
	that most users are up around 6,000 days of demonstrated
	retention. That means an average of about 6,000 / 112 ~ 54
	days per question in the essay. Intuitively, that seems
	pretty good – if you're anything like us, a couple of
	months after reading something you have only a hazy memory. By
	contrast, these users have, at low time cost to themselves (of
	which more below), achieved nearly two months of demonstrated
	retention across 112 detailed questions.
 
 
	Furthermore, you can see the exponential rise in retention
	with the number of times cards have been reviewed. After the
	first review, users typically have an average of just over 2
	days of demonstrated retention, per
	cardParticularly careful readers may
	wonder how this is possible, given that we stated earlier that
	the first review interval is 5 days. The explanation is that
	we recently modified the review schedule so the first review
	is after 5 days. For most of Quantum Country's
	history the review schedule was more conservative, and this is
	the reason for the difference.. But by the sixth review
	that rises to an average of 54 days of demonstrated
	retention. That typically takes about 95 minutes of total
	review time to achieve. Given that the essay takes about 4 or
	so hours to read, this suggests that a less than 50% overhead
	in time commitment can provide many months or years of
	retention for almost all the important details in the essay.
 
 
	This is the big, counterintuitive advantage of spaced
	repetition: you get exponential returns for increased
	effort. On average, every extra minute of effort spent in
	review provides more and more benefit. This is in sharp
	contrast with most experiences in life, where we run into
	diminishing returns. For instance, ordinarily if you increase
	the amount of time you spend reading by 50%, you expect to get
	no more than 50% extra out of it, and possibly much less. But
	with the mnemonic medium when you increase the amount of time
	you spend reading by 50%, you may get 10x as much out of
	it. Of course, we don't quite mean those numbers
	literally. But it does convey the key idea of getting a
	strongly non-linear return. It's a change in the quality of
	the medium.
 
 
	This delayed benefit makes the mnemonic medium unusual in
	multiple ways. Another is this: most online media use
	short-term engagement models, using variations on operant
	conditioning to drive user behavior. This is done by Twitter,
	Facebook, Instagram, and many other popular media forms. The
	mnemonic medium is much more like meditation – in some
	ways, the anti-product, since it violates so much conventional
	Silicon Valley wisdom – in that the benefits are
	delayed, and hard to have any immediate sense of. Indeed, with
	the mnemonic medium, the greater the delay, the more the
	benefit.
 
 
	These are preliminary results, and need more
	investigation. One naturally wonders what would happen if we'd
	been much more aggressive with the review schedule, setting
	the initial interval between reviews to (say) 2 months? If
	users reliably retained information up to that point, then the
	graph would start very high, and we wouldn't see the
	exponential. We need to investigate these and many similar
	questions to better understand what's going on with user's
	memories.
 
 
	Early feedback from users makes us cautiously optimistic that
	they're finding the mnemonic medium useful. In May 2019, one
	of us posted to Twitter a short thread explaining the
	technical details of how quantum teleportation works. One user
	of Quantum Country
	replied
	 to the thread with:
 
 I've only done your first quantum country
 course (so far) but I find it remarkable that I can view
 the proof and follow it, knowing what everything means.
 It's almost like Neo in The Matrix telling Morpheus,
 ‘I know quantum computing’ In the movie The Matrix one of
 the characters (Neo) uses a computer to very rapidly upload
 martial arts skills into his mind. As he opens his eyes
 after completing the upload he tells another character
 (Morpheus): “I know Kung Fu”..
 
 
	A user with significantly more prior experience of quantum
	computing wrote:
 
 
	I have a PhD in quantum information/computing and I knew
	everything in the essay before reading it, but the additional
	understanding I got from doing the given spaced repetition
	flashcards significantly improved my understanding of the
	material. Everyone who is reading this essay, should sign up
	and give spaced repetition a try.
 
 
	Another user, new to quantum computing, told us
	that Quantum Country “is by far the best way
	that I could imagine being introduced to this
	material”. When we asked how he'd used what he'd
	learned, he explained that when a visitor to his company gave
	a technical seminar about quantum computing, he expected to
	get lost after about 10 minutes. Instead:
 
 
	Wow, I actually followed that for 40 or 45 minutes because the
	matrices looked familiar… [the medium means] you run
	into concepts over and over again… It affords
	interactions at a more effective level of abstraction.
 
 
	Site analytics show a constant flow of people steadily working
	through the review sessions in the manner we intended. Six
	months after release of the prototype, 195 users had
	demonstrated one full month of retention on at least 80% of
	cards in the essay, demonstrating an extraordinary level of
	commitment to the process We've made
	no attempt at all to scale this out. It's interesting to
	ponder doing so.. We don't yet have a good model of
	exactly what those people are learning, but it seems plausible
	they are taking away considerably more than from a
	conventional essay, or perhaps even from a conventional class.
 
	
 
	Of course, this kind of feedback and these kinds of results
	should be taken with a grain of salt. The mnemonic medium is
	in its early days, has many deficiencies, and needs
	improvement in many ways (of which more soon). It is,
	however, encouraging to hear that some users already find the
	medium exceptionally helpful, and suggests developing and
	testing the medium further. At a minimum, it seems likely the
	mnemonic medium is genuinely helping people remember. And
	furthermore it has the exponentially increasing efficiency
	described above: the more people study, the more benefit they
	get per minute studied.
 
 
 
 
	In another informal experiment, we tried to figure out how
	much it affected user's memories when they weren't
	asked to review cards. To do this, we introduced a deliberate
	short (two-week) delay on reviews for a small subset of 8
	cards. That is, some users would review those 8 cards upon an
	initial read, and then would be prevented from reviewing them
	again for at least two weeks. Other users would continue to
	study as normal on the 8 cards. By comparing the two groups we
	could estimate the effect that reviewing the cards had on
	user's memories.
 
 
	What happened? Well, for those users whose reviews were
	delayed, accuracy dropped from 91% (upon the initial read) to
	87% (after two weeks). This may seem a small drop, but keep in
	mind that users continued to review other cards, which almost
	certainly boosted their final performance, since those other
	cards had some overlap in content with the delayed cards. It's
	difficult to avoid this kind of overlap without delaying
	reviews on all cards, a more drastic change in user experience
	than we wanted to impose. For users who were asked to review
	the cards as normal, accuracy improved from 89% to 96%. The
	short summary is: when users didn't review the cards, accuracy
	dropped by 4%; when they did review the cards, accuracy
	increased by 7% In more detail: there
	were 16 users in the group that did the reviews, per usual,
	and 25 users in the group where reviews were delayed. The 95%
	confidence intervals were: 91 ± 4%, 87 ± 5%, 89
	± 5%, 96 ± 3%, assuming each variable is
	binomial, independent and identically distributed. This latter
	assumption is approximate, since we'd expect some user- and
	question-dependent effects. Note also that this was done in an
	earlier version of Quantum Country, where the review
	schedule had intervals of one day, three days, one week, and
	and two weeks.. 
 
 
	Another way of looking at the data from this informal
	experiment is to ask which users saw improved or unchanged
	performance, and which saw their performance get worse. In
	fact, every single user (100%) who reviewed cards on the
	regular schedule saw their performance either stay the same or
	improve. By contrast, 40% of the users whose reviews were
	delayed saw their performance get worse, while 60% saw it stay
	the same or improve.
 
 
	These are small-but-promising results. Of course, our
	experiment was only done over two weeks, and we'd expect
	larger effects in experiments done over longer periods. And,
	as already mentioned, the effect is likely diminished by
	overlaps between the cards. Nonetheless, this informal
	experiment again suggests the mnemonic medium is helping
	people's memory, and suggests more comprehensive studies.
 
 
 
	Despite these suggestive preliminary results, it's still
	tempting to be dismissive. Isn't this “just” an
	essay with flashcards embedded? At some level, of course,
	that's correct. In the same way, wikis are just editable web
	pages; Twitter is just a way of sharing very short form
	writing; and Facebook is just a way of sharing writing and
	pictures with friends. Indeed, writing itself is just a clever
	way of ordering a small number of symbols on a page. While a
	medium may be simple, that doesn't mean it's not profound. As
	we shall see, the mnemonic medium has many surprising
	properties. It turns out that flashcards are dramatically
	under-appreciated, and it's possible to go much, much further
	in developing the mnemonic medium than is a priori
	obvious.
 
 
	Before we delve deeper into the mnemonic medium, let's mention
	one challenge in the discussion: the inherent difficulty in
	achieving a good balance between conveying enthusiasm and the
	kind of arm's-length skepticism appropriate for evaluation. On
	the one hand, we would not have built the mnemonic medium if
	we weren't excited about the underlying ideas, and wanted to
	develop those enthusiasms. To explain the mnemonic medium
	well, we need to bring you, the reader, inside that
	thinking. But having done that, we also need to step back and
	think more skeptically about questions such as: is this medium
	really working? What effect is it actually having on people?
	Can it be made 10x better? 100x better? Or, contrariwise, are
	there blockers that make this an irredeemably bad or at best
	mediocre idea? How important a role does memory play in
	cognition, anyway? So far, we've focused on the enthusiastic
	case for the medium, why one might consider this design at
	all. But later in this essay we'll gradually step back and
	reflect in a more skeptical frame.
 
 Expanding the scope of memory systems: what types of
 understanding can they be used for?
 
	Quantum Country is an example of a memory
	system. That is, it's a system designed to help users
	easily consolidate what they've learned into long-term
	memory. It's part of a long history of memory systems, going
	back to ancient times, when the orator Cicero and the
	rhetorician Quintilian described mnemonic techniques that
	could be used to memorize long texts.
 
 
	In modern times, many memory systems have been
	developed. Among the better known are Anki, SuperMemo,
	Quizlet, Duolingo, and Memrise. Like Quantum Country,
	each of these systems uses increasing time intervals between
	reviews of particular questions. Such systems are sometimes
	known as spaced-repetition memory systems
	(or SRM systems) Strictly
	speaking, Quizlet's basic product doesn't use spaced
	repetition. There is, however, a paid version using spaced
	repetition, and it's otherwise quite similar to many of these
	systems.. They're usually justified in a manner similar
	to our explanation for Quantum Country: some notion
	of each review gradually increasing the consolidation strength
	for a memory.
 
 
	SRM systems are most widely used in language
	learning. Duolingo, for instance, claims
	25
	 million monthly active users. Reports are mixed on
	 success. Some serious users
	 are enthusiastic
	 about their success with
	 Duolingo. But others
	 find it of limited utility. The company, of
	 course, touts
	 research showing that it's incredibly successful. It
	 seems likely to us that Duolingo and similar systems are
	 useful for many users as part of (but only part of) a
	serious language learning program.
 
 
	What about memory systems for uses beyond language? Quizlet is
	popular,
	with 50
	million monthly active users. It's widely used in
	classrooms, especially for simple declarative knowledge
	– lists of American Presidents, capitals of countries,
	and so on. Anki and SuperMemo seem to most often be used for
	similar simple declarative knowledge, but have much smaller
	active user bases than Quizlet They
	are, however, widely used within some interesting niche
	audiences. For instance, there is
	a thriving
	population of medical students using Anki..
 
 
	One of the ideas motivating Quantum Country is that
	memory systems aren't just useful for simple declarative
	knowledge, such as vocabulary words and lists of capitals. In
	fact, memory systems can be extraordinarily helpful for
	mastering abstract, conceptual knowledge, the kind of
	knowledge required to learn subjects such as quantum mechanics
	and quantum computing. This is achieved in part through many
	detailed strategies for constructing cards capable of encoding
	this kind of understanding. But, more importantly, it's
	possible because of the way the mnemonic medium embeds spaced
	repetition inside a narrative. That narrative embedding makes
	it possible for context and understanding to build in ways
	difficult in other memory systems.
 
 
	Other people have also developed ways of using memory systems
	for abstract, conceptual knowledge. Perhaps most prominently,
	the creator of the SuperMemo system, Piotr Wozniak,
	has written
	extensively about the many ingenious ways he uses memory
	systemsMore generally, Wozniak is,
	along with Sebastian Leitner, the principal pioneer of
	spaced-repetition memory systems. Much of Wozniak's thinking
	is available online at (or linked from) the
	remarkable SuperMemopedia.. And
	several other expert users of memory systems have also
	developed similar strategies. However, employing those
	strategies requires considerable skill. In practice, that
	skill barrier has meant these strategies are used by no more
	than a tiny handful of people.
 
 
	By contrast, in Quantum Country an expert writes the
	cards, an expert who is skilled not only in the subject matter
	of the essay, but also in strategies which can be used to
	encode abstract, conceptual knowledge. And so Quantum
	Country provides a much more scalable approach to using
	memory systems to do abstract, conceptual learning. In some
	sense, Quantum Country aims to expand the range of
	subjects users can comprehend at all. In that, it has very
	different aspirations to all prior memory systems.
 
 
	More generally, we believe memory systems are a far richer
	space than has previously been realized. Existing memory
	systems barely scratch the surface of what is possible. We've
	taken to thinking of Quantum Country as a memory
	laboratory. That is, it's a system which can be used both
	to better understand how memory works, and also to develop new
	kinds of memory system. We'd like to answer questions such as:
 
 
	What are new ways memory systems can be applied, beyond
	 the simple, declarative knowledge of past systems?
	How deep can the understanding developed through a memory
	 system be? What patterns will help users deepen their
	 understanding as much as possible?
	How far can we raise the human capacity for memory? And
	 with how much ease? What are the benefits and drawbacks?
	Might it be that one day most human beings will have a
	regular memory practice, as part of their everyday
	lives? Can we make it so memory becomes a choice; is it
	possible to in some sense solve the problem of memory?
	
 
 
	Over the next few sections we sketch out some of our thinking
	about how memory systems may be developed. We'll see that
	memory systems are a small part of a much bigger picture. Not
	only is seriously developing memory systems likely to lead to
	one or more transformative tools for thought, we also believe
	it will teach us much about the general problem of developing
	such tools.
 
 Improving the mnemonic medium: making better cards
 
	In writing mnemonic essays, it's tempting to treat the content
	of the cards rather casually. After all, a card is just a
	question and an answer, each containing a little text, perhaps
	a figure. Surely they ought to be easy to write?
 
 
	While thinking in this way is tempting, it's a mistake. In
	fact, cards are fundamental building blocks of the mnemonic
	medium, and card-writing is better thought of as an open-ended
	skill. Do it poorly, and the mnemonic medium works poorly. Do
	it superbly well, and the mnemonic medium can work very well
	indeed. By developing the card-writing skill it's possible to
	expand the possibilities of the medium.
 
 
	A helpful comparison is to the sentence in written prose. For
	the beginning writer it's tempting to treat sentences
	casually. But in the hands of a great writer – say, a
	Nabokov – sentences can be developed into a virtuoso
	artform. What would it take to achieve virtuoso skill in
	writing the cards of the mnemonic medium?
 
 
	It's not obvious a priori that writing cards is such
	a rich activity. One of us wrote 17,000- and 6,000-word
	essaysMichael
	Nielsen, Augmenting
	Long-term Memory (2018), and Michael
	Nielsen, Using
	spaced repetition systems to see through a piece of
	mathematics (2019). whose subject was in large part
	understanding how to write good cards. He didn't realize that
	was going to be the subject when he began writing; it only
	became clear in retrospect how rich card writing is. It turns
	out that answering the question “how to write good
	cards?” requires thinking hard about your theory of
	knowledge and how to represent it, and your theory of
	learning. The better those theories, the better your cards
	will be. Small wonder it's a rich, open-ended problem!
 
 
	All that said, let's make a few concrete observations about
	good card-writing. While the specific examples that follow are
	relatively banal, they should give you some feeling for the
	profound issues that arise in improving the mnemonic
	medium. We'll begin with three principles we used when writing
	the cards in Quantum Country. Note that these are
	just three of many more principles – a more detailed
	discussion of good principles of card construction may be
	found
	in Augmenting
	Long-term Memory.
 
 
	 Most questions and answers should be
	atomic: Early in his own personal memory practice,
	one of us was learning the Unix command to create links in the
	filesystem. He entered the following question into his memory
	system: “How to create a soft link from linkname to
	filename”. Together with the corresponding answer
	“ln -s filename linkname”. This looks like a good
	question, but he routinely forgot the answer. To address this,
	he refactored the card into two more atomic cards. One card:
	“What’s the basic command and option to create a soft
	link?” (A: “ln -s”). Second card:
	“When creating a soft link, in what order do linkname
	and filename go?” (A: “filename
	linkname”). Breaking the card into more atomic pieces
	turned a question he routinely got wrong into two questions he
	routinely got right. It seemed that the more atomic questions
	brought more sharply into focus what he was forgetting, and so
	provided a better tool for improving memory. And what of the
	original card? Initially, he deleted it. But he eventually
	added the card back, with the same question and answer, since
	it served to integrate the understanding in the more atomic
	cards.
	
	 Make sure the early questions in a mnemonic essay
	 are trivial: it helps many users realize they aren't
	 paying enough attention as they
	 read:Note added December 9, 2019:
	 This claim appears to be based on an error in our data
	 analysis, and is now retracted. We've left the text in for
	 historic reasons, but we no longer believe the
	 claim.  This was a discovery made when we
	 released the first Quantum Country
	 essay. Anticipating that users would be struggling with a
	 new interface, we deliberately made the first few
	 questions in the essay utterly trivial – sort of a
	 quantum equivalent to “2+2 = ?” – so
	 they could focus on the interface. To our surprise, users
	 performed poorly on these questions, worse than they did
	 on the (much harder) later questions. Our current
	 hypothesis to explain this is that when users failed to
	 answer the first few questions correctly it served as a
	 wakeup call. The questions were so transparently simple
	 that they realized they hadn't really been paying
	 attention as they read, and so were subsequently more
	 careful.
	
	 Avoid orphan cards: These are cards which
	don't connect closely to anything else. Suppose, for the sake
	of illustration, that you're trying to learn about African
	geography, and have a question: “What's the territory in
	Africa that Morocco disputes?” (A: “The Western
	Sahara”) If you don't know anything about the Western
	Sahara or Morocco or why there's a dispute, that question will
	be an orphan, disconnected from everything else. Ideally,
	you'll have a densely interconnected web of questions and
	answers, everything interwoven in striking ways.
	
 
 
	Ultimately, we'd like to distill out a set of useful practical
	principles and idioms to help write good cards and, more
	generally, good mnemonic essays. Aspirationally, such a set of
	principles and idioms would work much like The Elements of
	Style (or some similar book of prose advice), and would
	help other people learn to write high-quality mnemonic essays.
 
 
	When we first described Quantum Country above we
	explained it using a simple model of spaced repetition:
	increased consolidation strength for memories leading to
	increased time intervals between reviews. This is a helpful
	simple model, but risks creating the misleading impression
	that it's all that's going on in the system. In fact, for the
	mnemonic medium to work effectively, spaced repetition must be
	deployed in concert with many other ideas. The three ideas we
	just described – atomicity of questions and answers,
	making early questions trivial, avoiding orphan cards –
	are just three of dozens of important ideas used in the
	mnemonic medium. We won't enumerate all those other ideas here
	– that's not the purpose of this essay. But we want to
	emphasize this point, since it's common for people to have the
	simplistic model “good memory system = spaced
	repetition”. That's false, and an actively unhelpful way
	of thinking.
 
 
	Indeed, thinking in this way is one reason spaced-repetition
	memory systems often fail for individuals. We often meet
	people who say “Oh, I thought spaced repetition sounded
	great, and I tried Anki [etc], but it doesn't work
	for me”. Dig down a little, and it turns out the person
	is using their memory system in a way guaranteed to
	fail. They'll be writing terrible questions, or using it to
	learn a subject they don't care about, or making some other
	error. They're a little like a person who thinks
	“learning the guitar sounds great”, picks it up
	for half an hour, and then puts it down, saying that they
	sound terrible and therefore it's a bad instrument. Of course,
	what's really going on is that the guitar and memory systems
	are both skills that take time to develop. But, with that
	said, we want to build as much support as possible into the
	medium. Ideally, even novices would benefit tremendously from
	the mnemonic medium. That means building in many ideas that go
	beyond the simplistic model of spaced repetition.
 
 
	One of us has previously
	assertedMichael
	Nielsen, Augmenting
	Long-Term Memory (2018). that in spaced-repetition
	memory systems, users need to make their own cards. The
	reasoning is informal: users often report dissatisfaction and
	poor results when working with cards made by others. The
	reason seems to be that making the cards is itself an
	important act of understanding, and helps with committing
	material to memory. When users work with cards made by others,
	they lose those benefits.
 
 
	Quantum Country violates this principle, since users
	are not making the cards. This violation was a major concern
	when we began working on Quantum Country. However,
	preliminary user feedback suggests it has worked out
	adequately. A possible explanation is that, as noted above,
	making good cards is a difficult skill to master, and so what
	users lose by not making their own cards is made up by using
	what are likely to be much higher-quality cards than they
	could have made on their own. In future, it's worth digging
	deeper into this issue, both to understand it beyond informal
	models, and to explore ways of getting the benefits of active
	card making.
 
 
	Above we discussed three principles of good
	question-and-answer construction. Of course, it's also
	possible to make more structural modifications to the nature
	of the cards themselves. Here's three questions suggesting
	experiments in this vein:
 
 
	  How can we ensure users don't just learn surface
	 features of questions? One question in Quantum
	 Country asks: “Who has made progress on using
	 quantum computers to simulate quantum field theory?”
	 with the answer: “John Preskill and his
	 collaborators”. This is the only
	 “Who…?” question in the entire essay, and
	 many users quickly learn to recognize it from just the
	 “Who…?” pattern, and parrot the answer
	 without engaging deeply with the question. This is a common
	 failure mode in memory systems, and it's deadly to
	 understanding. One response, which we plan to trial soon, is
	 to present the question in multiple different-but-equivalent
	 forms. So the user first sees the question as “Who has
	 made progress [etc]?”; but then the second
	 time the question is presented as a fill-in-the-blanks:
	 “___ and his collaborators have made progress on using
	 quantum computers to simulate quantum field theory.”
	 And so on, multiple different forms of the question,
	 designed so the user must always engage deeply with the
	 meaning of the question, not its superficial
	 appearance. Ultimately, we'd like to develop a library of
	 techniques for identifying when this
	 learning-the-surface-feature pattern is occurring, and for
	 remedying it.
	 
	  How to best help users when they forget the
	 answer to a question? Suppose a user can't remember
	 the answer to the question: “Who was the second
	 President of the United States?” Perhaps they think
	 it's Thomas Jefferson, and are surprised to learn it's John
	 Adams. In a typical spaced-repetition memory system this
	 would be dealt with by decreasing the time interval until
	 the question is reviewed again. But it may be more effective
	 to follow up with questions designed to help the user
	 understand some of the surrounding context. E.g.: “Who
	 was George Washington's Vice President?” (A:
	 “John Adams”). Indeed, there could be a whole
	 series of followup questions, all designed to help better
	 encode the answer to the initial question in memory.
	 
	  How to encode stories in the mnemonic
	 medium? People often find certain ideas most
	 compelling in story form. Here's a short, fun example: did
	 you know that Steve Jobs actively opposed the development of
	 the App Store in the early days of the iPhone? It was
	 instead championed by another executive at Apple, Scott
	 Forstall. Such a story carries a force not carried by
	 declarative facts alone. It's one thing to know in the
	 abstract that even the visionaries behind new technologies
	 often fail to see many of their uses. It's quite another to
	 hear of Steve Jobs arguing with Scott Forstall against what
	 is today a major use of a technology Jobs is credited with
	 inventing. Can the mnemonic medium be used to help people
	 internalize such stories? To do so would likely violate the
	 principle of atomicity, since good stories are rarely atomic
	 (though this particular example comes close). Nonetheless,
	 the benefits of such stories seem well worth violating
	 atomicity, if they can be encoded in the cards effectively.
	 
	
 
 
	It's easy to generate dozens more questions and ideas in a
	similar vein. The mnemonic medium is not a fixed form, but
	rather a platform for experimentation and continued
	improvement.
 
 
	One useful metaphor for thinking about how to improve the
	mnemonic medium is to think of each mnemonic essay as a
	conventional essay accompanied by a kind of “reflected
	essay” – the knowledge encoded by all the cards. A
	user can, with ease, choose to remember as much of that
	reflected essay as they wish. Of course, the reflection is
	imperfect. But by developing good card-making strategies we
	can make the reflected essay a nearly faithful reflection of
	all the important ideas, the ideas a reader would ideally like
	to retain.
 
 
	We said above that it's a mistake to use the simplistic model
	“good memory system = spaced repetition”. In fact,
	while spaced repetition is a helpful way to
	introduce Quantum Country, we certainly shouldn't
	pigeonhole the mnemonic medium inside the paradigm of existing
	SRM systems. Instead, it's better to go back to first
	principles, and to ask questions like: what would
	make Quantum Country a good memory system? Are there
	other powerful principles about memory which we could we build
	into the system, apart from spaced repetition?
 
 
	In fact, there are ideas about memory very different from
	spaced repetition, but of comparable power. One such idea is
	elaborative encoding. Roughly speaking, this is the
	idea that the richer the associations we have to a concept,
	the better we will remember it. As a consequence, we can
	improve our memory by enriching that network of associations.
 
 
	This is in some sense an obvious idea, according well with
	everyday experience. For instance, it's part of the reason
	it's so much easier to learn new facts in an area we're
	already expert in – we quickly form associations to our
	existing knowledge. But just because the idea is obvious, that
	doesn't mean it's particularly well supported by existing
	media forms. There's a lot of low-hanging fruit which we can
	actively support inside the mnemonic medium. Indeed, several
	of the suggestions above already implicitly build on the idea
	of elaborative encoding – principles like “avoid
	orphan cards” are based on this. Here's three more
	suggestions which build on elaborative encoding:
 
 
	  Provide questions and answers in multiple
	 forms: In 1971, the psychologist Allan Paivio
	 proposed the dual-coding theory, namely, the assertion that
	 verbal and non-verbal information are stored separately in
	 long-term memory. Paivio and others investigated
	 the picture superiority effect, demonstrating that
	 pictures and words together are often recalled substantially
	 better than words alone. This suggests, for instance, that
	 the question “Who was George Washington's Vice
	 President?” may have a higher recall rate if
	 accompanied by a picture of Washington, or if the answer
	 (John Adams) is accompanied by a picture of Adams. For
	 memory systems the dual-coding theory and picture
	 superiority effect suggest many questions and ideas. How
	 much benefit is there in presenting questions and answer in
	 multiple forms? Perhaps even with multiple pictures, or in
	 audio or video (perhaps with multiple speakers of different
	 genders, different accents, etc), or in computer
	 code? Perhaps in a form that demands some form of
	 interaction? And in each case: what works best?
	 
	  Vary the context: In 1978, the
	 psychologists Steven Smith, Arthur Glenberg, and Robert
	 Bjork Steven M. Smith, Arthur
	 Glenberg, and Robert
	 A. Bjork, Environmental
	 context and human memory (1978). reported several
	 experiments studying the effect of place on human memory. In
	 one of their experiments, they found that studying material
	 in two different places, instead of twice in the same place,
	 provided a 40% improvement in later recall. This is part of
	 a broader pattern of experiments showing that varying the
	 context of review promotes memory. We can use memory systems
	 to support things like: changing the location of review;
	 changing the time of day of review; changing the background
	 sound, or lack thereof, while reviewing. In each case,
	 experiments have been done suggesting an impact on
	 recall. It's not necessarily clear how robust the results
	 are, or how reproducible – it's possible some (or all)
	 are the results of other effects, uncontrolled in the
	 original experiment. Still, it seems worth building systems
	 to test and (if possible) improve on these results.
	 
	  How do the cards interact with one another? What
	 is the ideal network structure of knowledge?
	 This is a very complicated and somewhat subtle set of
	 questions. Let's give a simple example to illustrate the
	 idea. We've presented the cards in the mnemonic medium
	 as though they are standalone entities. But there are
	 connections between the cards. Suppose you have cards:
	 “Who was George Washington’s Vice
	 President?” (Answer: “John Adams”,
	 with a picture of Adams); “What did John Adams
	 look like?” (Answer: a picture of Adams); perhaps
	 a question involving a sketch of Adams and Washington
	 together at some key moment; and so on. Now, this set of
	 cards forms a network of interrelated cards. And you can
	 use a memory system like Quantum Country to
	 study that network. What happens to people's observed
	 recall if you remove a card? Are there crucial lynchpin
	 cards? Are there particularly effective network
	 structures? Particularly effective types of
	 relationship between cards? Crucially: are there general
	 principles we can identify about finding the deepest,
	 most powerful ways of representing knowledge in this
	 system?
	 
	
 
	By now it's obvious that the prototype mnemonic medium we've
	developed is the tip of a much larger iceberg. What's more,
	the suggestions we've made and questions we've asked here are
	also merely a beginning, to give you the flavor of what is
	possible.
 
 Two cheers for mnemonic techniques
 
	When we discuss memory systems with people, many immediately
	respond that we should look into mnemonic techniques. This is
	an approach to memory systems very different to Quantum
	Country, Duolingo, Anki, and the other systems we've
	discussed. You're perhaps familiar with simple mnemonic
	techniques from school. One common form is tricks such as
	remembering the colors of the rainbow as the name Roy G. Biv
	(red, orange, yellow, green, etc). Or remembering the
	periodic table of elements using
	a songAn
	enjoyable extended introduction to such techniques may be
	found in Joshua Foer's book “Moonwalking with
	Einstein” (2012)..
 
 
	A more complex variation is visualization techniques such as
	the method of loci. Suppose you want to remember your
	shopping list. To do so using the method of loci, you
	visualize yourself in some familiar location – say, your
	childhood home. And then you visualize yourself walking from
	room to room, placing an item from your shopping list
	prominently in each room. When you go shopping, you can recall
	the list by imagining yourself walking through the house
	– your so-called memory palace – and
	looking at the items in each room.
 
 
	If you've never used memory palaces this sounds like it
	couldn't possibly work. But even novices are often shocked by
	how well such techniques work, with just a small amount of
	practice. Experts who work hard developing these techniques
	can do remarkable things, like memorizing the order of a
	shuffled deck of cards, or lists of hundreds of digits. It's a
	way of using people's immensely powerful visual and spatial
	memories as a form of leverage for other types of
	memory A small minority of the
	population does not possess a mind's eye, and so cannot
	mentally visualize. This condition is known
	as aphantasia. One
	of
	us asked
	on Twitter if any aphantasics had tried using the method
	of loci, and if so how well it worked for them. The replies
	were remarkably heterogeneous (and striking), but most said
	such mnemonic techniques did not work for them. This deserves
	further study..
 
 
	Given all this, it's perhaps not surprising that we often meet
	people who tell us that mnemonic techniques are a much more
	promising approach to memory than ideas such as spaced
	repetition.
 
 
	We're enthusiastic about such mnemonic techniques. But it's
	important to understand their limitations, and not be
	bedazzled by the impressiveness of someone who can rapidly
	memorize a deck of cards.
 
 
	One caution concerns the range of what can be memorized using
	mnemonic techniques. In practice they're often quite
	specialized. Mnemonic experts will, for instance, use somewhat
	different approaches to memorize lists of digits versus decks
	of cards. Those approaches must be mastered separately –
	a heavy time investment for two narrow kinds of
	memory. Furthermore, the mnemonic techniques tend to be much
	better suited for concrete objects than abstract conceptual
	knowledge – it's difficult to store, say, the main
	points in the Treaty of Versailles in your memory palace. This
	doesn't mean it can't be done – mnemonic experts have
	developed clever techniques for converting abstract conceptual
	knowledge into concrete objects which can be stored in a
	memory palace. But, in general, an advantage of spaced
	repetition is that it works across a far broader range of
	knowledge than do any of the mnemonic techniques.
 
 
	A second caution relates to elaborative encoding. The mnemonic
	techniques are, as you have likely realized, an example of
	elaborative encoding in action, connecting the things we want
	to memorize (say, our shopping list) to something which
	already has meaning for us (say, our memory palace). By
	contrast, when an expert learns new information in their
	field, they don't make up artificial connections to their
	memory palace. Instead, they find meaningful connections to
	what they already know. Those connections are themselves
	useful expertise; they're building out a dense network of
	understanding. It's a deeper and more desirable kind of
	expertise, connections native to the subject itself, not
	artificially constructed mnemonics.
 
 
	All this makes us seem negative about mnemonic techniques. In
	fact, we're enthusiastic, and have to date certainly underused
	them in the mnemonic medium. What we've written here is merely
	meant to temper the over-enthusiasm we sometimes
	encounter. We've had people go so far as to tell us that
	mnemonics make memory a solved problem. That is simply
	false. But with their limitations understood, they're a
	powerful tool. This is particularly true for knowledge which
	has an arbitrary, ad hoc structure. For example, it's
	difficult to remember the colors of the rainbow because those
	colors are not obviously connected to anything else, unless
	you happen to have
	the spectrum
	of visible light memorized for other reasons! That makes a
	mnemonic like Roy G. Biv extremely helpful. And so mnemonic
	techniques should be thought of as a useful tool to use in
	building powerful memory systems, especially when combined
	with ideas such as spaced repetition.
 
 How important is memory, anyway?
 
	People tend to fall into two buckets when told of the mnemonic
	medium. One group is fascinated by the idea, and wants to try
	it out. The second group is skeptical or even repulsed. In
	caricature, they say: “Why should I care about memory? I
	want deeper kinds of understanding! Can't I just look stuff up
	on the internet? I want creativity! I want conceptual
	understanding! I want to know how to solve important problems!
	Only dull, detail-obsessed grinds focus on rote memory.”
 
 
	It's worth thinking hard about such objections. To develop the
	best possible memory system we need to understand and address
	the underlying concerns. In part, this means digging down far
	enough to identify the mistaken or superficial parts of these
	concerns. It also means distilling as sharply as possible the
	truth in the concerns. Doing both will help us improve and go
	beyond the current prototype mnemonic medium.
 
 
	One response to such objections is the argument from lived
	experience. In the past, one of us (MN) has often helped
	students learn technical subjects such as quantum
	mechanics. He noticed that people often think they're getting
	stuck on esoteric, complex issues. But, as suggested in the
	introduction to this essay, often what's really going on is
	that they're having a hard time with basic notation and
	terminology. It's difficult to understand quantum mechanics
	when you're unclear about every third word or piece of
	notation. Every sentence is a struggle.
 
 
	It's like they're trying to compose a beautiful sonnet in
	French, but only know 200 words of French. They're frustrated
	and think the trouble is the difficulty of finding a good
	theme, striking sentiments and images, and so on. But really
	the issue is that they have only 200 words with which to
	compose.
 
 
	At the time, MN's somewhat self-satisfied belief was that if
	people only focused more on remembering the basics, and
	worried less about the “difficult” high-level
	issues, they'd find the high-level issues took care of
	themselves. What he didn't realize is that this also applied
	to him. When he began using the memory system Anki to read
	papers in new fields, he found it almost unsettling how much
	easier Anki made learning the basics of such subjects. And it
	made him start wondering if memory was often a binding
	constraint in learning new fields
	See here
	and here
	for more on learning new fields using Anki. The last four
	paragraphs are adapted from
	Augmenting
	Long-term Memory (2018)..
 
 
	One particularly common negative response to the mnemonic
	medium is that people don't want to remember
	“unimportant details”, and are just looking for
	“a broad, conceptual understanding”. It's
	difficult to know what to make of this argument. Bluntly, it
	seems likely that such people are fooling themselves,
	confusing a sense of enjoyment with any sort of durable
	understanding.
 
 
	Imagine meeting a person who told you they “had a broad
	conceptual understanding” of how to speak French, but it
	turned out they didn’t know the meaning of
	“bonjour”, “au revoir”, or “tres
	bien”. You'd think their claim to have a broad
	conceptual understanding of French was hilarious. If you want
	to understand a subject in any real sense you need to know the
	details of the fundamentals. What's more, that means not just
	knowing them immediately after reading. It means internalizing
	them for the long termThe last two
	paragraphs are adapted from our forthcoming mnemonic essay:
	Andy Matuschak and Michael
	Nielsen, How
	quantum teleportation works (2019)..
 
 
	A better model is that conceptual mastery is actually enabled
	by a mastery of details. One user of Quantum Country
	told us that she found the experience of reading unexpectedly
	relaxing, because she “no longer had to worry”
	about whether she would remember the content. She simply
	trusted that the medium itself would ensure that she did. And
	she reported that she was instead able to spend more of her
	time on conceptual issues.
 
 
	When people respond to the mnemonic medium with “why do
	you focus on all that boring memory stuff?”, they are
	missing the point. By largely automating away the problem of
	memory, the mnemonic medium makes it easier for people to
	spend more time focusing on other parts of learning, such as
	conceptual issues.
 
 
	Another common argument against spaced repetition systems is
	that it's better to rely on natural repetition. For instance,
	if you're learning a programming language, the argument goes,
	you shouldn't memorize every detail of that language. Instead,
	as you use the language in real projects you'll naturally
	repeatedly use, and eventually commit to memory, those parts
	of the language most important to learn.
 
 
	There are important partial truths in this. It is good to use
	what you're learning as part of your creative
	projects. Indeed, an ideal memory system might help that
	happen, prompting you as you work, rather than in an
	artificial card-based environment. Furthermore, a common
	failure mode with memory systems is that people attempt to
	memorize things they're unlikely to ever have any use for. For
	instance, it's no good (but surprisingly common) for someone
	to memorize lots of details of a programming language they
	plan to use for just one small project. Or to memorize details
	“just in case” they ever need them. These patterns
	are mistakes.
 
 
	But the truths of the last paragraph also have limits. If
	you're learning French, but don't know any French speakers,
	then waiting for “natural opportunities” to speak
	just won't work. And even if you do have (or create)
	opportunities to speak, it's desirable to accelerate the
	awkward, uncomfortable early stages that form such a barrier
	to using the language.
 
 
	It's in this phase that memory systems shine. They can
	accelerate people through the awkward early stages of learning
	a subject. Ideally, they'll support and enable work on
	creative projects. For this to work well takes good heuristics
	for what any given person should commit to memory; what is
	good for one person to memorize may be bad for
	another. Working such heuristics out is an ongoing challenge
	in the design of memory systems.
 
 
	(Incidentally, a surprising number of people say they are
	“repulsed”, or some similarly strong word, by
	spaced-repetition memory systems. Their line of argument is
	usually some variant on: it is claimed that spaced-repetition
	systems help with memory; if that is true I must use
	the systems; but I hate using the systems. The response is to
	deny the first step of the argument. Of course, the mistake is
	elsewhere: there is absolutely no reason anyone
	“should” use such systems, even if they help with
	memory. Someone who hates using them should simply choose not
	to do so. Using memory systems is not a moral imperative!)
 
 
	An immense amount of research has been done on the
	relationship of memory to mastery. Much of this research is
	detailed and context specific. But at the level of broader
	conclusions, one especially interesting series of studies was
	done in the 1970s by Herbert Simon and his collaborators. They
	studied chess players, and discovered
	See, e.g., William G. Chase and Herbert
	A. Simon, Perception in
	Chess (1973). Some fascinating earlier work in a related
	vein was done by Adrian D. de Groot, and summarized in his
	book Thought and Choice in Chess (1965). that
	when master chess players look at a position in chess they
	don't see it in terms of the individual pieces, a rook here, a
	pawn there. Instead, over years of playing and analyzing games
	the players learn to recognize somewhere between 25,000 and
	100,000 patterns of chess pieces. These much more elaborate
	“chunks” are combinations of pieces that the
	players perceive as a unity, and are able to reason about at a
	higher level of abstraction than the individual pieces. At
	least in part it's the ability to recognize and reason about
	these chunks which made their gameplay so much better than
	novices. Furthermore, although Simon did this work in the
	context of chess, subsequent studies have found
	similar
	 results in other areas of
	 expertise We've met many
	 mathematicians and physicists who say that one reason they
	 went into mathematics or physics is because they hated the
	 rote memorization common in many subjects, and preferred
	 subjects where it is possible to derive everything from
	 scratch. But in conversation it quickly becomes evident that
	 they have memorized an enormous number of concepts,
	 connections, and facts in their discipline. It's fascinating
	 these people are so blind to the central role memory plays
	 in their own thinking.. It seems plausible, though
	 needs further study, that the mnemonic medium can help speed
	 up the acquisition of such chunks, and so the acquisition of
	 mastery.
 
 
 
	So, does all this mean we're fans of rote memory, the kind of
	forced memorization common schools?
 
 
	Of course not. What we do believe is that many people's
	dislike of rote memorization has led them to a generalized
	dislike of memory, and consequently to underrate the role it
	plays in cognition. Memory is, in fact, a central part of
	cognition. But the right response to this is not immense
	amounts of dreary rote memorization. Rather, it's to use good
	tools and good judgment to memorize what truly matters.
 
 
	We've identified some ways in which criticisms of memory
	systems are mistaken or miss the point. But what about the
	ways in which those criticisms are insightful? What are the
	shortcomings of memory systems? In what ways should we be wary
	of them?
 
 
	We've already implicitly mentioned a few points in this
	vein. Think about problems like the need to avoid orphan
	questions. Or to make sure that users don't merely learn
	surface features of questions. These are ways in which memory
	systems can fail, if used poorly. Here's a few more key
	concerns about memory systems:
 
 
	  Memory systems don't make it easy to decide what
	 to memorize: Most obviously, we meet a lot of
	 people who use memory systems for poorly chosen
	 purposes. The following is surprisingly close to a
	 transcript of a conversation we've both had many times:
	 
	 “I don’t like [memory system]. I tried to memorize
	 the countries in Africa, and it was boring.”
	 “Why were you trying to remember the countries in
	 Africa?”
	 [blank look of confusion.]
	 
	 It's easy to poke fun at this kind of thing. But we've
	 both done the equivalent in our own memory practices. Even
	 some users of Quantum Country seem to be going
	 through the motions out of some misplaced sense of
	 duty. The question “what will be beneficial to
	 memorize” is fundamental, and answering that
	 question well is not trivial.
	 
	  What's the real impact of the mnemonic medium on
	 people's cognition? How does it change people's
	 behavior? A famous boxer is supposed to have
	 said that everyone has a plan until they get punched in
	 the face. Regular users of memory systems sometimes
	 report that while they can remembers answers when being
	 tested by their system, that doesn't mean they can
	 recall them when they really need them. There can be a
	 tip-of-the-tongue feeling of “Oh, I know
	 this”, but not actual recall, much less the fluent
	 facility one ultimately wants for effective
	 action. Furthermore, the user may not even recognize
	 opportunities to use what they have learned. More
	 broadly: memory is not an end-goal in itself. It's
	 embedded in a larger context: things like creative
	 problem-solving, problem-finding, and all the many ways
	 there are of taking action in the world. We suspect the
	 impact of memory systems will vary a lot, depending on
	 their design. They may be used as crutches for people to
	 lean on. Or they may be used to greatly enable people to
	 develop other parts of their cognition. We don't yet
	 understand very well how to ensure they're enablers,
	 rather than crutches. But later in the essay we'll
	 describe some other tools for thought that, when
	 integrated with memory systems, may better enable this
	 transition to more effective action.
	 
	
 
 
 
 How to invent Hindu-Arabic numerals?
 
	Let's briefly get away from memory systems. Imagine you're a
	designer living in ancient Rome, working for MDC (Mathematical
	Designs Corporation). A client comes in one day, expressing a
	desire to improve on Roman numerals. Of course, that's not
	literally how they describe their problem to you – more
	likely it's a tax collector wanting to tabulate taxes more
	efficiently, and having some vague notion that MDC may be able
	to help. But to you, an experienced designer, it seems that an
	improved system of numerals may be what they need.
 
 
	How should you respond to this request? From our modern
	vantage point we know a vastly better system of numerals is
	possible, the Hindu-Arabic numerals. Hindu-Arabic numerals
	were, in fact, a great leap in the history of tools for
	thought. Could you, as a designer, have made that leap? What
	creative steps would be needed to invent Hindu-Arabic
	numerals, starting from the Roman numerals? Is there a
	creative practice in which such steps would be likely to
	occur?
 
 
	To be clear: this is a somewhat fanciful thought
	experiment. Many of the ideas needed to get to Hindu-Arabic
	numerals were, in fact, known earlier to the Babylonians, to
	the Greeks, and in other cultures. They were also inchoate in
	the abacus and similar devices. And so we're not asking a
	literal historical question. Rather, it's a question meant to
	stimulate thought: what design process could take you
	from Roman numerals to Hindu-Arabic numerals?
 
 
	We can't know the answer to this question for sure. But it's
	worth pointing out that the Hindu-Arabic numerals aren't just
	an extraordinary piece of design. They're also an
	extraordinary mathematical insight. They involve many
	non-obvious ideas, if all you know is Roman numerals. Perhaps
	most remarkably, the meaning of a numeral
	actually changes, depending on its position within a
	number. Also remarkable, consider that when we add the numbers
	72 and 83 we at some point will likely use 2+3=5; similarly,
	when we add 27 and 38 we will also use 2+3=5, despite the fact
	that the meaning of 2 and 3 in the second sum is completely
	different than in the first sum. In modern user interface
	terms, the numerals have the same affordances, despite their
	meaning being very different in the two cases. We take this
	for granted, but this similarity in behavior is a consequence
	of deep facts about the number system: commutativity,
	associativity, and distributivityThe
	same phenomenon occurs in the conventional grade-school
	algorithms for multiplication and division. One of us has spun
	a short piece
	of discovery
	fiction discussing in more detail the way a hypothetical
	designer might have arrived at these ideas.. All these
	properties (and many more) point to the design and
	mathematical insights being inextricably entangled: the
	mathematical insights are, in some sense, design insights, and
	vice versa.
 
 
	Indeed, it seems fair to say that any person who could invent
	Hindu-Arabic numerals, starting from the Roman numerals, would
	be both one of the great mathematical geniuses who ever lived,
	and one of the great design geniuses who ever lived. They'd
	have to be extraordinarily capable in both domains, capable of
	an insight-through-making loop which used the evolving system
	of numerals to improve not just their own mathematical ideas,
	but to have original, world-class insights into mathematics;
	and also to use those mathematical insights to improve their
	evolving system of numerals.
 
 
	This is rather sobering if we compare to conventional modern
	design practice. In a typical practice, you'd interview domain
	experts (in this case, mathematicians), and read any relevant
	literature. You'd talk to users of existing systems, and
	analyze serious behavior, both individually and at scale. In
	short, you'd do what people in the design community refer to
	as immersing themselves in the target field.
 
 
	This is a powerful practice. At its best it causes systems to
	come into existence which would otherwise be inconceivable. If
	applied to Roman numerals (in hypothetical ancient Rome, not
	todayOf course, a designer who spoke
	to an expert on, say, Babylonian mathematics, might well have
	come across some of these ideas. We'll ignore that, since it
	depends on the oddity that many excellent prior ideas about
	numeral systems had been displaced in Roman culture.)
	this practice would likely improve them a great deal. But it
	would not provide anywhere near the level of mathematical
	insight needed to arrive at Hindu-Arabic numerals.
 
 
	Our story about Hindu-Arabic numerals and mathematics is
	fanciful. But it expresses a general truth: the most
	powerful tools for thought express deep insights into the
	underlying subject matter. In the case of memory
	systems, this means they're not just “applied cognitive
	science”, a collage of existing ideas from cognitive
	science pasted together using modern design practice. Rather,
	they will express deep original insights into memory, insights
	no-one else in the world has ever had. A truly great memory
	system will be cognitive science of the highest order.
 
 
	From this discussion, we take away a warning and an
	aspiration.
 
 
	The warning is this: conventional tech industry product
	practice will not produce deep enough subject matter insights
	to create transformative tools for thought. Indeed, that's
	part of the reason there's been so little progress from the
	tech industry on tools for thought. This sounds like a knock
	on conventional product practice, but it's not. That practice
	has been astoundingly successful at its purpose: creating
	great businesses. But it's also what Alan Kay has dubbed a pop
	culture, not a research culture. To build transformative tools
	for thought we need to go beyond that pop culture.
 
 
	The aspiration is for any team serious about making
	transformative tools for thought. It's to create a culture
	that combines the best parts of modern product practice with
	the best parts of the (very different) modern research
	culture. 
	You need the insight-through-making loop to operate, whereby
	deep, original insights about the subject feed back to change
	and improve the system, and changes to the system result in
	deep, original insights about the subject.
 
   
	Note that we are not making the common argument that making
	new tools can lead to new subject matter insights for the
	toolmaker, and vice versa. This is correct, but is much weaker
	than what we are saying. Rather: making new tools can lead to
	new subject matter insights for humanity as a whole
	(i.e., significant original research insights), and vice
	versa, and this would ideally be a rapidly-turning loop to
	develop the most transformative tools.
 
 
 
	Doing this is a cultural struggle. It seems to be
	extraordinarily rare to find the insight-through-making loop
	working at full throttle. People with expertise on one side of
	the loop often have trouble perceiving (much less
	understanding and participating in) the nature of the work
	that goes on on the other side of the loop. You have
	researchers, brilliant in their domain, who think of making as
	something essentially trivial, “just a matter of
	implementation”. And you have makers who don't
	understand research at all, who see it as merely a rather slow
	and dysfunctional (and unprofitable) making process. This is
	certainly true in Silicon Valley, where it's common to meet
	accomplished technical makers who, after reading a few stories
	from Richard Hamming and Richard Feynman, think they
	understand research well enough that they can “create
	the new Bell Labs”. Usually they're victims of
	Dunning-Krugeritis, so ignorant they're not even aware of
	their ignorance.
 
 
	Of course, we've got a long way to go with Quantum
	Country. It's not yet generating nearly deep enough ideas
	about memory and cognition; it's not yet one of the world's
	foremost memory laboratories. And considered as a product,
	it's also in the very earliest days; we're not yet iterating
	nearly fast enough, nor learning nearly fast enough from the
	system. Getting the insight-through-making loop to operate at
	full throttle will mean reinventing parts of both research
	culture and conventional product development culture; it will
	mean new norms and a new type of person involved in key
	decision making. But that's the aspiration, and what we
	believe is necessary to develop transformative tools for
	thought.
 
 
 
 
 
	We've examined the mnemonic medium in some depth. The intent
	was to show you the early stages in the development of a
	specific tool for thought, and some of the thinking enabled by
	that development. In this second part of the essay we explore
	more broadly, briefly sketching ideas for several other tools
	for thought. And we'll address some broader questions,
	especially around why there hasn't been more work on tools for
	thought.
 
 Mnemonic video
 
	In 2014, the digital artist Eric Wernquist released an
	extraordinary short video entitled
	“Wanderers”. The
	video provides a first-person glimpse of what it would be like
	to explore the Solar System:
 
	
 
	
 
	We hear the narrator's (Carl Sagan) wonder and awe, and cannot
	help but empathize with his deep belief in the value of
	exploration. We get a sense of how many mysteries and how much
	beauty there is in our own cosmic neighborhood. The music
	begins with a wistful nostalgia for those of our ancestors who
	dared to explore, and then changes to convey excitement and
	danger and the boldness of those members of our and future
	generations who continue that exploration.
 
 
	It's interesting to contrast the video to the video's script,
	a short text by Carl
	Sagan. The text
	is beautiful, but reading it is a much more remote and
	cerebral experience, conveying a much less visceral emotional
	understanding.
 
 
	We have a friend, Grant Sanderson, who makes astonishing
	mathematics videos on his YouTube
	channel, 3Blue1Brown. Below
	is one of our favorites, a video sketching a proof of a
	relatively recent research result in geometry, using ideas
	from algebraic topology. This sounds fearsome, but the video
	is beautiful and accessible, and has been viewed more than 1.2
	million times.
 
 
	
 
	As with Wanderers, watching this video is a
	remarkable emotional experience. It's obvious the person
	narrating the video loves mathematics, and you cannot help but
	empathize. As you watch, you experience repeated
	“Ahah!” moments, moments of surprising insight, as
	connections that were formerly invisible become obvious. It
	shows mathematics as something beautiful, containing
	extraordinary ideas and intriguing mysteries, while at the same
	time showing that doing mathematics is not mysterious, that it
	is something anyone can understand and even do.
 
 
	It's tempting to overlook or undervalue this kind of emotional
	connection to a subject. But it's the foundation of all
	effective learning and of all effective action. And it is much
	easier to create such an emotional connection using video than
	using text.
 
 
	There's a flipside to this emotional connection,
	however. We've often heard people describe Sanderson's videos
	as about “teaching mathematics”. But in
	conversation he's told us he doesn't think more than a small
	fraction of viewers are taking away much detailed
	understanding of mathematics. We suspect this is generally
	true, that high affect videos usually do little to change
	people's detailed intellectual understanding. Rather, the
	extraordinary value of the videos lies in the emotional
	connection they create.
 
 
	Is it possible to create a medium which blends the best
	qualities of both video and text?
 
 
	In particular, is it possible to create a medium which has the
	emotional range possible in video – a range which can be
	used to convey awe and mystery and surprise and beauty? But
	which can also firmly ground that emotional connection in
	detailed understanding, the mastery of details which is the
	raison d'être of both conventional text and, perhaps even
	moreso, of the mnemonic medium?
 
 
	We believe this may be possible, and we plan to develop a
	mnemonic video form that provides both the emotional
	connection possible in video, and the mastery of details
	possible in the mnemonic medium.
 
 
	Creating such a form is challenging. Many MOOC platforms have
	attempted something a little in this vein. The typical approach
	is to have a low-affect talking head video, with the videos
	interrupted occasionally for brief quizzesThese images excerpted from Astronomy:
	Exploring Time and Space.. Here's how it works on
	one MOOC platform, Coursera:
	
	
 
	Other MOOCs differ in the details. But the overall emotional
	experience may be summed up as:
 
	
 
	The very best parts of the video may be emotionally
	compelling, though it's rare that they achieve the emotional
	range and connection of the best videos from people like Grant
	Sanderson. And the overall emotional experience is
	disjointed, almost repellent. Is it possible to create an
	integrated medium, with a unified and carefully crafted
	emotional and intellectual experience? Ideally, is it possible
	to create something like the following?
 
	
 
	In MOOCs, questions are typically presented in a very dry
	form, detached from context. In the mnemonic video the
	narrator would explain why the questions are important, and
	why the user will benefit from participating, as a seamless
	part of the overall narration. Done right – perhaps with
	appropriate music, and a sense of urgency or play in the
	narration – it would create a real sense of the
	stakes. At the same time, the video player can be modified so
	the user can respond directly to questions, as part of the
	spaced-repetition experience. The result would be much softer
	transitions between the high-affect core narration and the
	moderate-affect questions.
 
 
	Here's a short sketch of one approach to doing
	thisThe narrator's script in this
	sketch is adapted from Cosmos (1984), episode
	4., showing how the narrator could ask questions aloud
	as an integrated part of the overall narration:
 
	
 
	It seems likely that the rhythm of mnemonic videos would be
	quite different to mnemonic essays. In particularly, the
	frequency and density of questions would be lower than in the
	mnemonic medium, and it would be necessary to test different
	beats and cadences to ensure a good balance of emotional and
	intellectual experience. Even high-affect video typically has
	quieter moments; it achieves the high affect in part by
	contrast to the lower-intensity moments. Think about the way a
	good action movie or thriller needs lulls; if it's too
	high-intensity all the time, eventually our emotional response
	is dulled. We could design mnemonic video so the questions
	help fill this lower-intensity emotional beat.
 
 
	Of course, this is merely a quick sketch of one approach to
	the design of mnemonic video. Ideally, there would also be a
	spaced-repetition component, perhaps with the questions asked
	in text instead of video. This sort of sketch seems to us a
	promising direction, but needs considerable development and
	intense testing. In particular, we need to do detailed,
	second-by-second user experience testing, to understand and
	shape users' emotional and intellectual experience. That would
	continue until we were confident that our target users were
	having the desired experience. Ideally, we'd also generate
	several more very different designs, and try to understand how
	each approach would impact the user's emotional and
	intellectual experience.
 
 
	The broader point here is about taking emotion
	seriously. Historically, a lot of work on tools for thought
	has either ignored emotion, or treated it as no more than a
	secondary concern. Instead, that work has focused on new
	skills acquired, on what the user “learns”.
	They've been designing for Spock, when emotional connection is
	a high-order bit. Do users feel disinterested? Afraid?
	Hostile? Anxious? Or do they internalize a sense of
	excitement, of beauty, perhaps even an expansion in their own
	goals, an expansion of their self?
 
 
	By contrast, media forms such as movies and music and (often)
	video games do take emotion seriously. The designers of such
	forms often have incredibly elaborate models of user's
	emotional responses. These models range from detailed,
	second-by-second understanding, to deep thinking about a
	user's overall emotional journey. We believe it's possible and
	desirable to use such approaches in the development of tools
	for thought.
 
 
	At the same time, a positive emotional experience alone is not
	enough. For tools for thought to attain enduring power, the
	user must experience a real growth in mastery, an expansion in
	their ability to act. And so we'd like to take both the
	emotional and intellectual experience of tools for thought
	seriously. Mnemonic video is a good venue for such
	exploration. To paraphrase Einstein, attaining a detailed
	understanding without forming an emotional connection is lame;
	while forming an emotional connection without detailed
	understanding has no enduring power.
 
 
 Why isn't there more work on tools for thought today?
 
	If tools for thought are so great, why isn't more work being
	done on them? Why aren't they a major industry?
 
 
	As noted in the introduction, there's certainly a lot of lip
	service paid. It is, for instance, common to hear
	technologists allude to Steve Jobs's metaphor of computers as
	“bicycles for the mind”. But in practice it's
	rarely more than lip service. Many pioneers of computing have
	been deeply disappointed in the limited use of computers as
	tools to improve human cognition. Douglas
	Engelbart disparaged
	the “dangerous, disappointing, narrow, path that we seem
	to be stuck with following”. When asked in 2006 how much
	of his vision had been achieved, Engelbart replied
	facetiously “about
	2.8 percent”. Alan Kay gives talks
	asserting “The
	real computer revolution hasn't happened yet” and in
	an
	interview has described the modern web as
	“reinventing the flat tire… at least give us what
	Engelbart did, for Christ's sake.”
 
 
	Our experience is that many of today's technology leaders
	genuinely venerate Engelbart, Kay, and their colleagues. Many
	even feel that computers have huge potential as tools for
	improving human thinking. But they don't see how to build good
	businesses around developing new tools for thought. And
	without such business opportunities, work languishes.
 
 
	What makes it difficult to build companies that develop tools
	for thought? To answer this, consider Adobe, one of the few
	large companies serious about developing new tools for
	thoughtAnother large company which
	takes tools for thought extremely seriously is AutoDesk. A
	similar story could be told about it.. It's poured
	money into developing new mediums The
	plural of medium is, of course, media. However, in this
	context media would usually mean many pieces of new
	content. That's not what we mean: we mean multiple different
	new mediums
	(Illustrator, Photoshop etc). We'll
	reserve the unusual pluralization for this somewhat unusual
	meaning. for designers and artists – programs
	such as Illustrator, Photoshop, and so
	on. These mediums are remarkable tools for thought.
 
 
	Unfortunately for Adobe, such mediums are extremely expensive
	to develop, and it's difficult to prevent other companies from
	cheaply copying the ideas or developing
	near-equivalents. Consider, for example, the way the
	program Sketch has eaten into Adobe's market share,
	after duplicating many of the best features from several of
	Adobe's products, perhaps most notably
	Illustrator. And consider the way Figma is
	now eating into both Sketch
	and Illustrator's market share. Both Sketch
	and Figma have done this without needing to make an
	enormous investment in research. That's a big advantage they
	have over Adobe.
 
 
	As Marc Andreessen has observed In
	 Elad Gil's “High Growth Handbook”
	 (2018).:
 
 
	true defensibility purely at the product level is really rare
	in [Silicon] Valley, because there are a lot of really good
	engineers… And then there's the issue of
	leap-frogging. The next team has the opportunity to learn from
	what you did and then build something better.
 
 
	Put another way, many tools for thought are public goods. They
	often cost a lot to develop initially, but it's easy for
	others to duplicate and improve on them, free riding on the
	initial investment. While such duplication and improvement is
	good for our society as a whole, it's bad for the companies
	that make that initial investment. And so such tools for
	thought suffer the fate of many public goods: our society
	collectively underinvests in them, relative to the benefits
	they provideOf course, it does cost
	money for companies such as Sketch and Figma
	to duplicate features originating in Illustrator, and
	they have introduced some improvements. So our
	characterization as a public good is only
	approximate. .
 
 
	Earlier, we argued that modern design practice generally isn't
	up to the challenge of producing genuinely transformative
	tools for thought. On the surface, that process-level argument
	appears very different to the public goods argument we just
	made. In fact, the process-level explanation is a consequence
	of the public goods explanation: companies don't use the
	necessary processes because there's little value to them in
	doing so. By contrast, in “harder-tech” industries
	– say, chip design – companies have much more
	incentive to do deep research work. In those industries it's
	considerably harder for other companies to duplicate or
	capture the value of that research.
 
 
	It's illuminating to contrast with video games. Game companies
	develop many genuinely new interface ideas. This perhaps seems
	surprising, since you'd expect such interface ideas to also
	suffer from the public goods problem: game designers need to
	invest enormous effort to develop those interface ideas, and
	they are often immediately copied (and improved on) by other
	companies, at little cost. In that sense, they are public
	goods, and enrich the entire video game ecosystem.
 
 
	But there's a big difference between video game companies and
	companies such as Adobe. Many video games make most of their
	money from the first few months of sales. While other
	companies can (and do) come in and copy or riff on any new
	ideas, it often does little to affect revenue from the
	original game, which has already made most of its
	money In fact, cloning is a real
	issue in gaming, especially in very technically simple
	games. An example is the game Threes, which took the
	developers more than a year to make. Much of that time was
	spent developing beautiful new interface ideas. The resulting
	game was so simple that clones and near-clones began appearing
	within days. One near clone, a game called 2048,
	sparked a mini-craze, and became far more successful
	than Threes. At the other extreme, some game
	companies prolong the revenue-generating lifetime of their
	games with re-releases, long-lived online versions, and so
	on. This is particularly common for capital-intensive AAA
	games, such as the Grand Theft Auto series. In such
	cases the business model relies less on clever new ideas, and
	more on improved artwork (for re-release), network effects
	(for online versions), and branding.
	. While this copying is no doubt irritating for the
	companies being copied, it's still worth it for them to make
	the up-front investment.
 
 
	The net result is that in gaming, clever new interface ideas
	can be distinguishing features which become a game's primary
	advantage in the marketplace. Indeed, new interface ideas may
	even help games become classics – consider the many
	original (at the time) ideas in games ranging from Space
	Invaders to Wolfenstein 3D to Braid
	to Monument Valley. As a result, rather than
	underinvesting, many companies make sizeable investments in
	developing new interface ideas, even though they then become
	public goods. In this way the video game industry has largely
	solved the public goods problems.
 
 
	By contrast, a company like Adobe builds their business around
	distribution and long-term lock in. They convince people
	– indeed, entire organizations – to make long-term
	commitments to their products. Schools offer classes so people
	can call themselves “Photoshop experts”
	or “Illustrator experts”. Companies
	designate their design departments as “Adobe
	shops”. So while Adobe does invest in developing clever
	new interface ideas (for them, unlike the video game
	companies, this genuinely means tools for thought), it's less
	central to their competitive advantage, and they invest less
	than they would if it was their central advantage. And Adobe
	does perhaps as much or more work developing tools for thought
	as any company.
 
 
	It's encouraging that the video game industry can make inroads
	on the public goods problem. Is there a solution for tools for
	thought? Unfortunately, the novelty-based short-term revenue
	approach of the game industry doesn't work. You want people to
	really master the best new tools for thought, developing
	virtuoso skill, not spend a few dozen hours (as with most
	games) getting pretty good, and then moving onto something
	new.
 
 
	Another plausible solution to the public goods problem is
	patents, granting a temporary monopoly over use of an
	invention. Many software companies, including Adobe, develop a
	large patent portfolio. However, the current patent system is
	not a solution for this problem. In 2017, Dana Rao, Adobe's
	Vice President for Intellectual Property and Litigation,
	posted
	a call
	for major reforms to the patent system, stating that:
 
 
	[the patent] system is broken… What happened? A patent
	gold rush built by patent profiteers… Their value lies
	not in the innovation behind the patent but in the vagueness
	of the patent's claims and the ability to enforce it in a
	plaintiff-friendly forum… Where did the material for
	these bad patents come from? The advent of software…
	This led to idea-only patents being granted with broad and
	often invalid claims, and eager patent profiteers were only
	too glad to take advantage.
 
 
	Adobe shares in common with many other software companies that
	much of their patenting is defensive: they patent ideas so
	patent trolls cannot sue them for similar ideas. The situation
	is almost exactly the reverse of what you'd like. Innovative
	companies can easily be attacked by patent trolls who have
	made broad and often rather vague claims in a huge portfolio
	of patents, none of which they've worked out in much
	detail. But when the innovative companies develop (at much
	greater cost) and ship a genuinely good new idea, others can
	often copy the essential core of that idea, while varying it
	enough to plausibly evade any patent. The patent system is not
	protecting the right things.
 
 
	Switching away from the viewpoint of individual companies, and
	to the viewpoint of society as a whole, not only do we want to
	incentivize invention, we also want ideas to move reasonably
	rapidly into the public domain. Think about fundamental tools
	for thought such as writing and the number system. Obviously,
	it's good that those spread throughout society, unencumbered
	by IP concerns! More broadly, many tools of thought become
	more valuable for society as they become more
	ubiquitous. Again, here the modern patent system has numerous
	well-known problems, striking a poor balance between private
	and public interest. While a well-designed patent system might
	very well help solve the public goods problem, the patent
	system we actually have seems poorly adapted to the problem.
 
 
	Is it possible to avoid the public goods problem altogether?
	Here's three classes of tools for thought which do:
 
 
	 Search engines such as Google are tools for
	 thought. They avoid the public goods problem because their
	 value is in their brand and in hard-to-duplicate and
	 capital intensive backend elements (including their data
	 centers, proprietary algorithms, ad network, and
	 distribution), not in their interface ideas.
	 
	 A service such as Twitter can be considered a tool for
	 collective thought. While the interface is easily copied,
	 the company is hard to duplicate, due to network effects.
	 
	 
	 Novel hardware devices (e.g., for VR, or the Wii remote,
	 or for new musical instruments) can be used as the basis
	 for new tools for thought. While hardware can be
	 duplicated, it's often much more expensive than
	 duplicating software. And, in any case, the advantage for
	 such companies is often in distribution, marketing, and
	 relationships with vendors who make products for the
	 platform.
	 
	
 
 
 
	While these suggestions all avoid the public goods problem,
	they don't directly solve the public goods problem. And many
	promising directions – including ideas such as the
	mnemonic medium and mnemonic videos – involve a
	substantial public goods element. Is it possible to solve the
	public goods problem in such cases? The two most promising
	approaches seem to us to be:
 
 
	 Philanthropic funding for research. This approach was
	 used, for instance, by the field of computer animation and
	 animated movies. Decades of public research work on computer
	 animation resulted in a large number of powerful and (in
	 many cases) publicly available ideas. This, in turn, helped
	 prepare the way for companies such as Pixar and Dreamworks,
	 which developed many of the ideas further, and took them to
	 scale.
	 
	 The model used by Adobe and similar companies, in which
	 new tools for thought are a central part of the company's
	 operations, but not the core of their competitive moat. That
	 moat may instead be built around training, marketing,
	 documentation, and so on.
	 
	
 
 Questioning our basic premises
 
	There are three important premises we've taken for granted up
	to now. First is the assertion that we're still in the early
	days, that many more transformative tools for thought are yet
	to be discovered. Second is the assertion that work on tools
	for thought is stalled, that there's not lots of interesting
	work going on. And third, a kind of meta-premise, is the
	assertion that this kind of work is worth doing, relative to
	the current fashion for related ideas such as artificial
	general intelligence and brain-computer interfaces. In this
	section we discuss these premises.
 
 What if the best tools for thought have already been discovered?
 
	In other words, perhaps the 1960s and 1970s were an
	unrepeatable golden age, and all we can expect in the future
	is gradual incremental improvement, and perhaps the occasional
	major breakthrough, at a decreasing frequency?
 
 
	There's a plausible story suggesting this is true. Tech is an
	enormous industry, well funded, with many bright, ambitious,
	talented people. Surely if there were major ideas to discover,
	people would do so? This argument is reinforced by the fact
	that, at the individual level, we meet many brilliant people
	who are fascinated by (and often working on) tools for
	thought, but who nonetheless seem to be making slow progress.
 
 
	But while this story has a superficial appeal, it's
	misleading. Really difficult problems – problems like
	inventing Hindu-Arabic numerals – aren't solved by good
	intentions and interest alone. A major thing missing is
	foundational ideas powerful enough to make progress. In the
	earliest days of a discipline – the proto-disciplinary
	stage – a few extraordinary people – people like
	Ivan Sutherland, Doug Engelbart, Alan Kay, and Bret Victor
	– may be able to make progress. But it's a very bespoke,
	individual progress, difficult to help others become
	proficient in, or to scale out to a community. It's not yet
	really a discipline. What's needed is the development of a
	powerful praxis, a set of core ideas which are explicit and
	powerful enough that new people can rapidly assimilate them,
	and begin to develop their own practice. We're not yet at that
	stage with tools for thought. But we believe that we're not so
	far away either.
 
 
	While that argument is helpful context, it doesn't address the
	core point: it doesn't mean there are a lot of new
	transformative tools for thought waiting to be discovered.
	Again: maybe the most important tools for thought have already
	been discovered?
 
 
	We can't predict the future, so it's not possible to answer
	this question with certainty. But it seems to us that the
	human race just hasn't really tried very hard yet. When small
	groups of motivated people do – as in pioneering labs
	such as PARC, SRI, and other DARPA-inspired early efforts, as
	well as modern labs such as Dynamicland – they make
	rapid progress. It's extremely encouraging that those efforts
	– tiny efforts, in the scheme of humanity's overall
	research effort – make such rapid progress. To us, that
	suggests scaling them up, becoming much more ambitious.
 
 Isn't this what the tech industry does? Isn't there a lot of ongoing progress on tools for thought?
 
	In particular, aren't there already a lot of imaginative,
	determined, well-funded people working on this? Isn't tech in
	considerable part already about developing new tools for
	thought?
 
 
	Part of this question is caused by a confusion in
	terms. Obviously, many tech companies build special-purpose
	tools for solving specific problems. But while those may be
	valuable tools, they're certainly not “tools for
	thought” in the broad sense we're discussing – not
	like language or writing or, for that matter, Illustrator.
 
 
	Still, there are tech companies which really do develop tools
	for thought. We already discussed some examples where
	companies have partially or totally avoided the public goods
	problem, tools such as: Illustrator, Google Search, Twitter,
	Slack, Google Docs, programmer
	toolsProgrammer tools are a case
	where the insight-through-making loop operates quite well.
	The avoidance of the public goods problem is enabled by a
	complex set of factors, including the fact that this is a case
	where many highly skilled researchers are simultaneously also
	accomplished makers, and it's either their job to produce
	public goods (researchers) or an enjoyable hobby. To a lesser
	extent the same is true in the artistic and music
	communities., and so on. All really are significant
	tools for thought.
 
 
	But consider our most fundamental tools for thought –
	language, writing, music, etc. Those are public
	goods. No-one owns language; to the extent that it is owned
	(trademarks and so on) it may actually limit the utility of
	language. These tools are all about introducing fundamental
	new mental representations and mental operations. Those aren't
	owned by any company, they're patterns owned by humanity.
 
 
	This argument makes it seem likely that many of the most
	fundamental and powerful tools for thought do suffer the
	public goods problem. And that means tech companies focus
	elsewhere; it means many imaginative and ambitious people
	decide to focus elsewhere; it means we haven't developed the
	powerful practices needed to do work in the area, and a result
	the field is still in a pre-disciplinary stage. The result,
	ultimately, is that it means the most fundamental and powerful
	tools for thought are undersupplied.
 
 Why not
	work on artificial general intelligence (AGI) or
	brain-computer interfaces (BCI) instead?
 
 
	We're often asked: why don't you work on AGI
	or BCI instead of tools for thought? Aren't those more
	important and more exciting? And for AGI, in particular, many
	of the skills required seem related.
 
 
	They certainly are important and exciting subjects. What's
	more, at present AGI and BCI are far more fashionable (and
	better funded). As a reader, you may be rolling your eyes,
	supposing our thinking here is pre-determined: we wouldn't be
	writing this essay if we didn't favor work on tools for
	thought. But these are questions we've wrestled hard with in
	deciding how to spend our own lives. One of us wrote a book
	about artificial intelligence before deciding to focus
	primarily on tools for thought; it was not a decision made
	lightly, and it's one he revisits from time to time. Indeed,
	given the ongoing excitement about AGI and BCI, it would be
	surprising if people working on tools for thought didn't
	regularly have a little voice inside their head saying
	“hey, shouldn't you be over there instead?”
	Fashion is seductive.
 
 
	One striking difference is that AGI and BCI are based on
	relatively specific, well-defined goals. By contrast, work on
	tools for thought is much less clearly defined. For the most
	part we can't point to well-defined, long-range goals; rather,
	we have long-range visions and aspirations, almost
	evocations. The work is really about exploration of an
	open-ended question: how can we develop tools that change and
	expand the range of thoughts human beings can think?
 
 
	Culturally, tech is dominated by an engineering, goal-driven
	mindset. It's much easier to set KPIs, evaluate OKRs, and
	manage deliverables, when you have a very specific end-goal in
	mind. And so it's perhaps not surprising that tech culture is
	much more sympathetic to AGI and BCI as overall programs of
	work.
 
 
	But historically it's not the case that humanity's biggest
	breakthroughs have come about in this goal-driven way. The
	creation of language – the ur tool for thought –
	is perhaps the most important occurrence of humanity's
	existence. And although the origin of language is hotly
	debated and uncertain, it seems extremely unlikely to have
	been the result of a goal-driven process. It's amusing to try
	imagining some prehistoric quarterly OKRs leading to the
	development of language. What sort of goals could one possibly
	set? Perhaps a quota of new irregular verbs? It's
	inconceivable!
 
 
	Similarly, the invention of other tools for thought –
	writing, the printing press, and so on – are among our
	greatest ever breakthroughs. And, as far as we know, all
	emerged primarily out of open-ended exploration, not in a
	primarily goal-driven way. Even the computer itself came out
	of an exploration that would be regarded as ridiculously
	speculative and poorly-defined in tech today. Someone didn't
	sit down and think “I need to invent the
	computer”; that's not a thought they had any frame of
	reference for. Rather, pioneers such as Alan Turing and Alonzo
	Church were exploring extremely basic and fundamental (and
	seemingly esoteric) questions about logic, mathematics, and
	the nature of what is provable. Out of those explorations the
	idea of a computer emerged, after many years; it was a
	discovered concept, not a goal. Fundamental, open-ended
	questions seem to be at least as good a source of
	breakthroughs as goals, no matter how ambitious. This is
	difficult to imagine or convince others of in Silicon Valley's
	goal-driven culture. Indeed, we ourselves feel the attraction
	of a goal-driven culture. But empirically open-ended
	exploration can be just as, or more successful.
 
 
	“What will new tools for thought be like?” is a
	question we hear often. And yet, almost by definition, we
	cannot say. As we noted earlier, if we could communicate the
	experience in an essay, then the tools would be failing at
	their job; they would not be transforming a person's thinking,
	or even their consciousness. Concretely: to understand the
	mnemonic medium you must use it intensively over an extended
	period. And even then you may not be conscious of the effect;
	we've done interviews with users who are apparently unaware of
	the incredible level of recall they have of material in the
	essay they have read. One of the most famous papers in the
	philosophy of consciousness is entitled “What is it like
	to be a bat?” Each tool for thought poses a similar
	question, near impossible to answer without immersion in the
	tool: “What is it like to be a language user? A
	musician?” and so on.
 
 
	It seems plausible to us that work on tools for thought will
	be, over the next few decades, more important than work on AGI
	and BCI. And, given how fashionable and well-funded work on
	AGI and BCI currently is, it seems nearly certain that work on
	tools for thought offers vastly greater benefit, at the
	margin.
 
 
	What about the longer term? There, the situation is less
	clear. It seems likely that the three fields will merge, or at
	least feed strongly into one another. Together with Shan
	Carter, one of us has argued that one of the most promising
	applications for AI is as a way of discovering new tools for
	thought Shan Carter and Michael
	Nielsen, Using
	Artificial Intelligence to Augment Human Intelligence
	(2017)..
 
 
	BCI seems likely to be even more closely related. BCI is
	sometimes described using ideas like a memory chip for
	long-term memories, or some way of increasing short-term
	working memory. Such ideas may well become important. But it
	also seems possible that BCIs will be used to enable new
	mental operations, new mental representations, and new
	affordances for thought; in short, the same kind of things as
	are involved in developing non-BCI tools for thought. Perhaps
	we'll develop the capacity to directly imagine ourselves in 4
	or 5 or more dimensions; or traversing a Riemann manifold; or
	the ability to have multiple tracks of conscious
	attention. These are about changing the interface for thought,
	the basic abstractions and operations which are allowed. And
	so it seems plausible that work today on tools for thought
	will directly impact the way we use BCIs in the future.
 
	
 Executable books
 
	The skill of writing is to create a context in which other
	people can think.— Edwin Schlossberg
 
 
	The computer scientist Peter Norvig has written
	an interactive
	essay discussing the distribution of wealth in
	society. Norvig's essay is a Jupyter notebook which expresses
	many of the ideas in running Python code. That code sets up a
	population of agents, with an initial distribution of
	wealth. Agents randomly (and repeatedly) meet one another in
	pairs, and engage in simple economic transactions. More
	concretely: a simple transaction model could be that when two
	people meet, their joint wealth is pooled, and then randomly
	divided between the two of them. That model is just to give
	you the gist – more complex transaction models are, of
	course, possible. The notebook simulates how the distribution
	of wealth evolves over time.
 
 
	Part of what makes Norvig's essay beautiful is that with just
	a few lines of Python code Norvig is able to show some
	surprising results about wealth inequality. For instance, his
	results suggest that the initial distribution of wealth in the
	economy doesn't much affect the long-run distribution of
	wealth. Rather, it's the nature of the transactions which
	determines the long-run distribution of wealth. This likely
	violates at least some user's intuitions. As another example,
	his results also suggest that constraining agents to trade
	only with people geographically near them makes little
	difference to the final distribution of wealth.
 
 
	Results like these will challenge the intuition of some
	users. But instead of those challenges being on the basis of
	easily-ignored abstract arguments, users can immediately
	engage with Norvig's model. Suppose someone doesn't like the
	idea that the initial distribution of wealth doesn't affect
	long-run wealth inequality. They're challenged to find a
	counterexample, an initial distribution of wealth which does
	affect long-run inequality. They can experiment easily, making
	simple modifications to just one or a few lines of Python
	code, trying to find instances where the initial distribution
	matters. No matter whether they succeed or fail, they will
	build a better understanding of the
	problemSufficiently motivated
	reasoners will, of course, ignore any conclusion they don't
	like. Such people are most likely a lost cause to serious
	thought..
 
 
	Suppose the content of Norvig's essay had instead been
	presented in a more conventional static form. For a reader to
	extend or interrogate the results would require total mastery
	of the material, and a high level of mathematical
	competence. But in the notebook format it's much easier for
	the reader to experiment. Their exploration is scaffolded,
	they can make small modifications and see the results, even
	the answers to questions Norvig did not anticipate. This kind
	of scaffolded exploration is a way to build up their own
	understanding, and perhaps even push the frontiers of
	knowledge.
 
 
 
	Norvig's essay is one of thousands (or perhaps even millions)
	of Jupyter notebooks that have been created. Of course, most
	such notebooks are hastily and poorly written. But in the
	hands of an excellent writer and thinker like Norvig,
	notebooks can become remarkable environments for thought, both
	individual and shared. It's tempting to regard them as merely
	a mashup of essay and code. But really they're a new media
	form, with different possibilities from either essays or code,
	and with striking opportunities to go much further. In this
	section we explore those
	opportunitiesOf course, systems like
	Jupyter go back decades. There are antecedents in Knuth's
	notion of literate programming, in Mathematica notebooks, in
	PARC's Learning Research Group, in the PLATO system (and, more
	broadly, computer-assisted instruction), to name but a few. In
	the discussion below we emphasize opportunities that seem to
	us undervalued in many of these prior systems..
 
 
	We described Norvig's essay as an “interactive
	essay”. It's useful to have a more specific term, to
	distinguish it from other interactive forms, like the mnemonic
	medium. In this essay, we'll use the term “executable
	book”This is, of course, a
	placeholder term. “Executable essay” would in many
	ways be more natural, though unfortunately that term is
	perhaps even more naturally interpreted as meaning “web
	page”, in the current context.. We won't define
	this precisely here; definition is not the point. Rather, the
	point is to try to better understand the potential of media
	forms which combine prose and code in something like this
	form.
 
 Tools for thought must be developed while doing
	serious work. The aspiration to canonical content
 
 
	Seymour Papert, one of the principal creators of the Logo
	programming language, had a remarkable aspiration for
	Logo. Logo is sometimes described as a “programming
	language for children”, and people sometimes think
	Papert was mostly interested in helping children learn how to
	program. But that wasn't Papert's principal intent. Rather,
	Papert wanted to create an immersive environment – a
	kind of “Mathland” – in which children could
	be immersed in mathematical ideas. In essence, children could
	learn differential geometry by going to Mathland.
 
 
	It's a beautiful aspiration, and Logo contains many striking
	ideas. But as far as we know, no professional differential
	geometer (or, more generally, mathematician) uses Logo
	seriously as a tool in their work. And upon reflection that
	seems troubling. If Logo genuinely expresses the ideas of
	differential geometry, why don't differential geometers use
	it? You start to wonder: might it be that Logo leaves out
	important ideas about differential geometry, maybe even the
	most important ideas about differential geometry? After all,
	while mathematically trained, Papert wasn't himself an
	accomplished differential geometer. How would he even have
	known what to include? And certainly most of the people
	interested in Logo aren't qualified to make that judgment.
 
 
	There's a standard retort to this, which we've heard from
	within the Logo community. It's to talk about the
	“floor” and “ceiling” of different
	environments for thought. In this account, Logo has a low
	floor (meaning anyone can use it) and a low ceiling (so it's
	not well adapted for the sort of advanced work a professional
	would want to do).
 
 
	At first this sounds plausible. But upon reflection it's
	difficult to make much sense of. How do the creators of Logo
	know that mastering Logo helps later with understanding real
	(forgive us!) differential geometry? What's the criterion for
	success? One of us (MN) worked for several years doing
	research in the closely related field of Riemannian
	geometry. While Logo is enjoyable to use and contains many fun
	ideas, MN has trouble seeing that learning Logo would help
	much in learning differential geometry.
 
 
	At the end of Norvig's economics essay is a short afterword
	explaining how he came to write the essay. Shortly before
	writing the essay he'd heard about the kinds of economic
	models discussed in the notebook, and he wanted to explore
	several questions about them. After talking it over with some
	colleagues they decided to each independently attack the
	problems, and to compare notes. Although Norvig's essay is, in
	some sense, “educational”, Norvig's intent was to
	explore a set of problems he himself was genuinely curious
	about. The educational aspect was a byproduct.
 
 
	And so what you have is a world-class research scientist who
	wanted to explore a set of questions. He used the Jupyter
	medium to do those explorations, and then to share that
	exploration with the world. And he shared it in a form where
	others could immediately build upon and extend his thinking.
 
 
	There's a lot of work on tools for thought that takes the form
	of toys, or “educational” environments. Tools for
	writing that aren't used by actual writers. Tools for
	mathematics that aren't used by actual mathematicians. And so
	on. Even though the creators of such tools have good
	intentions, it's difficult not to be suspicious of this
	pattern. It's very easy to slip into a cargo cult mode, doing
	work that seems (say) mathematical, but which actually avoids
	engagement with the heart of the subject. Often the creators
	of these toys have not ever done serious original work in the
	subjects for which they are supposedly building tools. How can
	they know what needs to be included?
 
 
	Concretely: suppose you want to build tools for subject X (say
	X = differential geometry). Unless you are deeply involved in
	practicing that subject, it's going to be extremely difficult
	to build good tools. It'll be much like trying to build new
	tools for carpentry without actually doing any carpentry
	yourself. This is perhaps part of why tools
	like Mathematica work quite well – the
	principal designer, Stephen Wolfram, has genuine research
	interests in mathematics and physics. Of course, not all parts
	of Mathematica work equally well; some parts feel
	like toys, and it seems likely those are the ones not
	being used seriously internal to the company.
 
 
	There's a general principle here: good tools for thought
	 arise mostly as a byproduct of doing original work on
	 serious problems. They tend either be created by the
	 people doing that work, or by people working very closely to
	 them, people who are genuinely bought
	 in A related argument has been made
	 in Eric von Hippel's book “Democratizing
	 Innovation” (2005), which identifies many instances
	 where what appears to be commercial product development is
	 based in large or considerable part on innovations from
	 users.. Furthermore, the problems themselves are
	 typically of intense personal interest to the
	 problem-solvers. They're not working on the problem for a
	 paycheck; they're working on it because they desperately
	 want to know the answer.
 
 
	Many people have asked why we wrote our first mnemonic essay
	about quantum computing. If we'd chosen an easier subject we
	could have attracted a much larger audience. But we also
	wanted the essay to be authentic, to be about problems we
	wanted to solve. One of us (MN) has done a lot of original
	research work on quantum computing. The essay reflects that
	thinking. Indeed, the framing of the essay is about answering
	a question MN personally wanted to answer: if humans ever
	discovered aliens, would they have computers, and if so, what
	types of computer would they have? This perhaps sounds like a
	contrived question, but it's quite serious, and turns out to
	be a deep question with a nontrivial answer: writing the essay
	helped MN substantially improve his understanding of the
	question.
 
 
	That said, answering that question wasn't the principal point
	of creating the essay: making the mnemonic medium was. And for
	future work on tools for thought, it'd be valuable to push
	much harder on questions we'd genuinely like to answer
	ourselves. That's a way of keeping yourself honest, ensuring
	you're not just building a flashy toy, but something genuinely
	useful for solving real problems that are of independent
	interest.
 
 
	In serious mediums, there's a notion of canonical
	 media. By this, we mean instances of the medium that
	 expand its range, and set a new standard widely known
	 amongst creators in that medium. For instance, Citizen
	 Kane, The Godfather, and 2001 all
	 expanded the range of film, and inspired later film
	 makers. It's also true in new media. YouTubers like Grant
	 Sanderson have created canonical videos: they expand the
	 range of what people think is possible in the video
	 form. And something like the Feynman Lectures on
	 Physics does it for textbooks. In each case one gets
	 the sense of people deeply committed to what they're
	 doing. In many of his lectures it's obvious that Feynman
	 isn't just educating: he's reporting the results of a
	 lifelong personal obsession with understanding how the world
	 works. It's thrilling, and it expands the form.
 
 
	We've been disappointed by how unambitious people are in this
	sense with Jupyter notebooks. They haven't pushed the
	medium all that hard; there is no Citizen Kane of
	Jupyter notebooks. Indeed, we're barely beyond the Lumière
	brothers. Examples like Norvig's notebook are fine work, but
	seem disappointing when evaluated as leading examples of the
	medium.
 
 
	Aspiring to canonicity, one fun project would be to take the
	most recent IPCC climate assessment report (perhaps starting
	with a small part), and develop a version which is
	executable. Instead of a report full of assertions and
	references, you'd have a live climate model – actually,
	many interrelated models – for people to explore. If it
	was good enough, people would teach classes from it; if it was
	really superb, not only would they teach classes from it, it
	could perhaps become the creative working environment for many
	climate scientists.
 
 
	One promising exploration in this direction
	is  The
	Structure and Interpretation of Classical Mechanics ,
	a beautiful executable book building up classical
	mechanics. Many theorems of classical mechanics aren't just
	expressed in static form, on the page, but live, as code which
	can be modified by the user. Theorems become APIs, which can
	literally be applied to other objects, and chained
	together. It uses a much more powerful underlying model than
	Jupyter, developing a new symbolic language as part of the
	book. It has many flaws – among them, the book doesn't
	run live in the browser, making it difficult for users to
	experiment. And while the book is well written, the authors
	do not understand classical mechanics as deeply as the authors
	of some other books. But it's nonetheless an inspiring
	evocation of what is possible. And it hints at what is
	possible when authors use executable books for a serious
	purpose, and aspire toward canonical media.
 
 Stronger emotional connection through an inverted writing structure
 
 
	Consider an author writing a popular book about quantum
	mechanics. Such an author is in a strong position: they can
	begin their book with astonishing phenomena such as black hole
	evaporation, quantum teleportation, and the role of quantum
	fluctuations in the early universe. Or, if they wish, they can
	start with some of the deepest mysteries known to humanity:
	the relationship between quantum mechanics and gravity, or the
	quantum measurement problem. There is no shortage of
	extraordinary phenomena and beautiful mysteries. These are the
	kind of things which touch a chord inside many, perhaps most
	people. And so it's relatively easy to draw readers in, to get
	them engaged, and keep them connected.
 
 
	By contrast, consider a typical technical book about quantum
	mechanics. It's very unlikely to start with black hole
	evaporation or quantum teleportation – and if it does,
	such a discussion will be perfunctory. Instead, it will start
	out drily, with technical minutiae. Complex
	numbers. Wavefunctions. Many different types of differential
	equations, and how to solve them. Hermitian and unitary
	operators. And so on, piece by piece slowly building up all
	the machinery needed to solve quantum mechanical problems. It
	may be tens or even hundreds of pages before the book begins
	to connect to the exciting problems which form the
	bread-and-butter of popular accounts.
 
 
	People with little experience doing good technical writing
	often complain about this dry, bottom-up approach. They will
	complain that writers should instead stay closer to the fun
	material, and use less technical notation and
	nomenclature. But when competent writers attempt to follow
	this prescription, invariably it works poorly.
 
 
	One problem is that a person can spend years reading analogies
	about black hole evaporation, quantum teleportation, and so
	on. And at the end of all that reading they typically
	have… not much genuine understanding to show for it.
	The analogies and heuristic reasoning simply don't go
	far. They may be entertaining and produce some feeling of
	understanding. But the reasoning won't scale out; it can't be
	applied to other phenomena, at least not without lots of
	caveats, caveats the reader is in no position to understand or
	apply. As a result, good technical writers instead mostly
	build things up from first principles, with occasional
	digressions to the broader motivating picture. And that means
	starting with a lot of detailed, technical minutiae.
 
 
	It's striking to contrast conventional technical books with
	the possibilities enabled by executable books. You can imagine
	starting an executable book with, say, quantum teleportation,
	right on the first page. You'd provide an interface –
	perhaps a library is imported – that would let users
	teleport quantum systems immediately. They could experiment
	with different parts of the quantum teleportation protocol,
	illustrating immediately the most striking ideas about it. The
	user wouldn't necessarily understand all that was going
	on. But they'd begin to internalize an accurate picture of the
	meaning of teleportation. And over time, at leisure, the
	author could unpack some of what might a priori seem
	to be the drier details. Except by that point the reader will
	be bought into those details, and they won't be so
	dryA similar argument has been made
	by Rachel
	Thomas, Providing
	a Good Education in Deep Learning (2016).
 
 
	In other words, you could begin an executable book with
	material the users already care about, can connect to easily,
	and find motivating. For instance, you could begin by
	exploring teleportation or the Big Bang. But such an opening
	won't suffer the drawback of popular science, of being vague
	and imprecise. Rather, the interface would be completely well
	specified. And, with some care, the interface could be scaled
	out, applied in ever-expanding contexts. The understanding
	would be transferable. Even a user who has understood only a
	tiny part of the material could begin tinkering, building up
	an understanding based on play and exploration. It's common to
	dismiss such an approach as leading to a toy understanding; we
	believe, on the contrary, that with well enough designed
	scaffolding it can lead to a deep
	understandingThe masters of this are
	video game designers. See, for example, Dan
	Cook, Building a Princess Saving
	App (2008), and Jonathan Blow and Marc ten
	Bosch, Designing
	to Reveal the Nature of the Universe
	(2011). . Developed in enough depth, such an
	environment may even be used to explore novel research
	ideas. To our knowledge this kind of project has never been
	seriously pursued. But it'd be fun to try.
 
 
 Summary and Conclusion
 
	We've covered a lot, and it's helpful to distill the main
	takeaways – general principles, questions, beliefs, and
	aspirations. Let's begin with memory systems, particularly
	the mnemonic medium:
 
 
 
	 Memory systems make memory into a choice, rather
	 than an event left up to chance: This changes the
	 relationship to what we're learning, reduces worry, and
	 frees up attention to focus on other kinds of learning,
	 including conceptual, problem-solving, and creative.
	
	 Memory systems are in their infancy: it
	 is possible to increase effective human memory by an order
	 of magnitude, even beyond what existing memory systems do;
	 and systems such as the mnemonic medium may help expand the
	 range of subjects users can comprehend at all.
	
	 What would a virtuoso use of the mnemonic medium
	 look like? There's some sense in which the
	 mnemonic medium is “just” flash cards. The
	 right conclusion isn't that it's therefore trivial; it's
	 that flash cards are greatly underrated. In
	 writing Quantum Country we treated the writing of
	 the cards with reverence; ideally, authors would take card
	 writing as seriously as Nabokov took sentence writing. Of
	 course, we didn't reach that level, but the aspiration
	 expands the reach of the medium. What would virtuoso or
	 even canonical uses of the mnemonic medium look like?
	
	 Memory systems can be used to build genuine
 	 conceptual understanding, not just learn facts:
 	 In Quantum Country we achieve this in part
 	 through the aspiration to virtuoso card writing, and in
 	 part through a narrative embedding of spaced repetition
 	 that gradually builds context and understanding.
	
	 Mnemonic techniques such as memory palaces are
	 great, but not versatile enough to build genuine
	 conceptual understanding: Such techniques are
	 very specialized, and emphasize artificial connections,
	 not the inherent connections present in much conceptual
	 knowledge. The mnemonic techniques are, however, useful
	 for bootstrapping knowledge with an ad hoc
	 structure.
	
	 Memory is far more important than people tend to
	 think: It plays a role in nearly every part of
	 cognition, including problem-solving, creative work, and
	 meta-cognition. The flip side is that memory systems
	 themselves want to grow into other types of tools –
	 tools for reading, tools for problem-solving, tools for
	 creating, tools for attention management. That said, we
	 don't yet know what memory systems want to be. To
	 reiterate: memory systems are in their infancy.
	
 
 
 
	The mnemonic medium is merely one prototype tool for
	thought. We also discussed several other ideas, including
	mnemonic video and executable books. Here are some key
	takeaways:
 
 
 
	 What practices would lead to tools for thought as
	 transformative as Hindu-Arabic numerals? And in what ways does
	 modern design practice and tech industry product practice
	 fall short? To be successful, you need
	 an insight-through-making loop to be operating at
	 full throttle, combining the best of deep research
	 culture with the best of Silicon Valley product culture.
	
	
 	 Tools for thought are (mostly) public goods, and
	 as a result are undersupplied: That said, there
	 are closely-related models of production which have
	 succeeded (the games industry, Adobe, AutoDesk,
	 Pixar). These models should be studied, emulated where
	 possible, and used as inspiration to find more such
	 models.
	
	 Take emotion seriously: Historically,
	work on tools for thought has focused principally on
	cognition; much of the work has been stuck in Spock-space. But
	it should take emotion as seriously as the best musicians,
	movie directors, and video game designers. Mnemonic video is a
	promising vehicle for such explorations, possibly combining
	both deep emotional connection with the detailed intellectual
	mastery the mnemonic medium aspires toward.
	
	 Tools for thought must be developed in tandem with
	deep, original creative work: Much work on tools for
	thought focuses on toy problems and toy environments. This is
	useful when prototyping, but to be successful such tools must
	ultimately be used to do serious, original creative
	work. That's a baseline litmus test for whether the tools are
	genuinely working, or merely telling a good story. Ideally,
	for any such tool there will be a stream of canonical media
	expanding the form, and entering the consciousness of other
	creators.
	
 
 
	Let's return to the question that began the essay: how to
	build transformative tools for thought? Of course, we haven't
	even precisely defined what such transformative tools are! But
	they're the kind of tools where relatively low cost changes in
	practice produce transformative changes in outcome –
	non-linear returns and qualitative shifts in thinking. This is
	in contrast with the usual situation, where a small change in
	practice causes a small change in results.
 
 
	Historically, humans have invented many such transformative
	tools for thought. Writing and music are ancient examples; in
	modern times, tools such as Photoshop
	and AutoCAD qualify. Although it's very early days,
	we believe the mnemonic medium shows much promise. It needs
	to be developed much further, along the lines we've described,
	and likely requires additional powerful ideas. But we believe
	it's possible for humanity to have a widespread memory
	practice that radically changes the way we think.
 
 
	More broadly, we hope the principles in this essay will help
	support the creation of more transformative tools for
	thought. Historically, most invention of tools for thought has
	been done bespoke, by inspired individuals and groups. But we
	believe that in the future there will be an established
	community that routinely does this kind of invention.
How can we develop transformative tools for thought?
https://numinous.productions/ttft/
https://rootsofprogress.org/why-did-we-wait-so-long-for-the-threshing-machine
Why did we wait so long for the threshing machine?
2021-06-29
None
<div><div>
 
 
 <figure></figure>
<p>When ripe wheat is harvested, the edible seed is encased in an outer husk. Before the seed can be ground into flour, or boiled into porridge, or planted in the field to produce next year’s harvest, it must be removed from the husk. This process is called threshing.</p>
<p>As the husk is quite hard, threshing is a violent process. Traditionally, it was done with a tool called a flail, which is simply a short stick attached by a cord to a longer handle. The grain was spread out on the ground (yes, disgusting) and beaten with the stick to open the casings.</p>
<figure>
 <a href="/img/threshing-by-hand-flailing.jpg">
 
 </a>
 <figcaption>
 
 
 
 
 <a href="https://www.loc.gov/resource/fsa.8e04614/">The Growth of Industrial Art, 1892</a>
 
 
 
 </figcaption>
</figure>
<p>Other methods included “treading”, in which livestock trampled the grain with their hooves (yes, even more disgusting) or dragged a sledge over the grain (the Latin word for this sledge is <em>tribulum,</em> from which we get the world “tribulation”).</p>
<figure>
 <a href="/img/threshing-by-horse-treading.jpg">
 
 </a>
 <figcaption>
 Horse treading
 
 
 
 <a href="https://www.loc.gov/resource/fsa.8e04614/">The Growth of Industrial Art, 1892</a>
 
 
 
 </figcaption>
</figure>
<p>Occasionally the grain would be rubbed against a wire screen, or placed in a sack and beaten with rocks. It’s no coincidence that the word “thrashing” is similar: it is an archaic spelling of the same term.</p>
<p>As one of the more labor-intensive stages of wheat farming, threshing was a natural candidate to be automated by machinery. And the threshing machine was a relatively simple device: like <a href="/why-did-we-wait-so-long-for-the-cotton-gin">the cotton gin</a>, <a href="https://medium.com/@antonhowes/is-innovation-in-human-nature-48c2578e27ba">the flying shuttle</a>, or <a href="/why-did-we-wait-so-long-for-the-bicycle">the bicycle</a>, it was a mechanical invention that did not depend on any scientific discoveries. Still, threshing machines were not used to any significant degree until the late 1700s in the UK and the early 1800s in the US.</p>
<p>Once again, the question arises: <a href="/why-did-it-take-so-long">why did we wait so long?</a></p>
<h2>Early claims and concepts</h2>
<p>First, it’s <em>not</em> because no one had the idea to mechanize the threshing process. Although we think of the Industrial Revolution as a uniquely inventive age, and the era before it seems blind to the possibility of innovation by comparison, we must remember that there <em>were</em> inventions prior to 1700, many of which were broadly adopted, especially when they helped with essential economic activities. The loom and the printing press predated the threshing machine by centuries, and neither seems far less complex.</p>
<p>The threshing machine was referred to in an English patent as early as 1636 (for historical context, this is not long after the major works of Galileo and Francis Bacon). Sir John Christopher van Berg, a knight of Moravia (part of the present-day Czech Republic), fleeing religious wars in Germany and seeking protection from King Charles I, offered to Charles and to England his observations of “diverse mechanical instruments and frames operating by weights.” <a href="/van-berg-patent">The patent</a> is extremely broad and vague by today’s standards, listing a literally incredible variety of machines: for pumping water, washing clothes, cooking meat, working bellows, dredging rivers, raising sunken ships, measuring distances and depths, and for pretty much any mechanical operation. Towards the end of the list is mentioned an invention “to be agitated by wind, water, or horses, for the clean threshing of corn [grain], whereby much corn that is now left may be saved, and the straw made near as good as hay.” At the time, patents did not require detailed descriptions, diagrams, or models, and there is no more insight than this into how the machine might have worked, if indeed the invention was even real. The point is that by the early 1600s, the threshing machine was seen as possible and desirable.</p>
<p>And why not? The grinding of grain into flour had been mechanized by watermills in ancient Rome. The threshing process was similar enough that some type of mill could plausibly accomplish the task. A century after Van Berg, inventors really started working on the problem, with one Michael Menzies of Scotland patenting a threshing machine in 1734, and several more efforts throughout the 1700s.</p>
<p>So why did none of these machines gain wide adoption?</p>
<h2>Challenges with reliability</h2>
<p>One clue comes from reports of trials of the early machines.</p>
<p>Some of the machines simply broke. Of Menzies’s machines, one historian wrote that “owing to the velocity required to do the work perfectly, they soon broke, and the invention fell into disgrace” (<a href="https://www.google.com/books/edition/General_View_of_the_Agriculture_of_East/AfEx0wgDmugC?hl=en&amp;gbpv=1">Somerville 1805</a>, p. 75). Another machine was trialed in Scotland in the late 1700s, but “in a few minutes the model was torn to pieces” (<a href="https://www.amazon.com/Grain-Harvesters-Graeme-R-Quick/dp/0916150135?tag=jasocraw-20">Quick 1978</a>, p. 45).</p>
<p>When the machines did not break, they often failed to do the job. In one trial in 1753, it was found that the machine “broke off the ears of barley and wheat instead of clearing them of the grain, and that, at best, it was only fit for oats” (<a href="https://archive.org/details/implementsofagri00ransiala/page/140/mode/1up?ref=ol&amp;view=theater">Ransome 1843</a>, p. 140). Another was attempted in the late 1770s, but: “Upon trial this machine was also found defective, as along with its doing very little work in a given time, it bruised the grain, and so materially hurt its appearance, as to lessen its value considerably in the market” (<a href="https://www.google.com/books/edition/General_View_of_the_Agriculture_of_East/AfEx0wgDmugC?hl=en&amp;gbpv=1">Somerville 1805</a>, p. 76).</p>
<p>In general, <a href="https://www.amazon.com/Sowing-Modernity-Americas-Agricultural-Revolution/dp/0801433266?tag=jasocraw-20">McClelland (1997)</a> says that threshing machines in the late 18th and early 19th century were “among the most complicated and expensive of all agricultural implements in American and Britain. High price and frequent breakdowns did not bode well for rapid adoption when ‘the least derangement … is death to the whole machine’” (p. 172). He reports that Washington and Jefferson were both interested in threshing machines, but (pp 175-6):</p>
<blockquote>
 <p>Washington’s machine… built using the plans of William Booker soon proved unsatisfactory; a Maryland machine constructed with the aid of Colonel Anderson’s plans developed a warped wheel and was abandoned; in 1802 an immigrant from Edinburgh introduced into the Mid-Atlantic states “six or seven” based upon “the Scotch principle,” but they soon developed problems and “common workmen” could not repair them; the machine patented in 1803 by Jedediah Turner, based “upon entirely new, and very plain principles” was never commercially produced.… Washington perhaps spoke for a number of would-be agrarian pioneers when he voiced his frustration with this prospective new technology in 1793: <strong>“I have seen so much of beginning and ending of new inventions, that I have almost resolved to go on in the old way of treading.”</strong></p>
</blockquote>
<h2>What was the problem?</h2>
<figure></figure>
<figure></figure>
<p>One challenge was to find the right basic idea for the design of the machine. As is often the case, some of the early attempts tried to mimic human motion too closely: these machines automated the motion of the flail, but continued to beat the grain on the ground, an idea known as the “beating” principle. “All machines based upon the beating principle suffered from the same problem. The violence of the mechanical action used to separate the grain soon produced broken parts and an implement in disrepair” (<a href="https://www.amazon.com/Sowing-Modernity-Americas-Agricultural-Revolution/dp/0801433266?tag=jasocraw-20">McClelland 1997</a>, p. 170). This was the problem with Menzies’s 1734 machine.</p>
<figure>
 <a href="/img/british-flailing-machine.jpg">
 
 </a>
 <figcaption>
 Design for a “beating” type machine
 
 
 
 <a href="https://archive.org/details/pennsylvaniamaga11775phil/page/n84/mode/1up?view=theater">Thomas Paine's Pennsylvania Magazine, February 1775</a>
 
 
 
 </figcaption>
</figure>
<figure>
 <a href="/img/american-flailing-machine.jpg">
 
 </a>
 <figcaption>
 Another “beating” concept
 
 
 
 <a href="https://www.loc.gov/resource/fsa.8e04614/">The Growth of Industrial Art, 1892</a>
 
 
 
 </figcaption>
</figure>
<figure></figure>
<figure></figure>
<p>A better idea was the “rubbing” principle. This design was more like a mill or a cotton gin: the grain was fed into a rotating drum that had pegs or other protrusions to grind the husk off of the seed. While “beating” vs. “rubbing” was <a href="https://archive.org/details/geneseefarmer01roch/page/132/mode/1up?ref=ol&amp;view=theater">still debated as late as the 1830s</a>, all successful machines were of the rubbing type.</p>
<figure>
 <a href="/img/meikle-threshing-machine.jpg">
 
 </a>
 <figcaption>
 Meikle's threshing machine, 1786
 
 
 
 <a href="https://commons.wikimedia.org/wiki/File:Meikle.png">Wikimedia</a>
 
 
 
 </figcaption>
</figure>
<figure>
 <a href="/img/threshing-machine-1823.jpg">
 
 </a>
 <figcaption>
 Pope's threshing machine
 
 
 
 <a href="https://babel.hathitrust.org/cgi/pt?id=uma.ark:/13960/t04x5m96m&amp;view=1up&amp;seq=35">American Farmer, 1823</a>
 
 
 
 </figcaption>
</figure>
<p>But the threshing machine didn’t seem to take off after one core design breakthrough. There isn’t a single individual who is always called out as “the inventor of the threshing machine,” nor a single date that stands out as an inflection point. Instead, one finds series of important inventions across decades: Andrew Meikle built the first successful machine in Scotland in 1786, but Joseph Pope invented a popular one in the US in 1820, and the Pitts brothers made improvements to power and to winnowing (separating the wheat from the chaff) in the 1830s. Nor does there seem to have been a single design that became dominant. Instead, it’s a story of gradual improvements in effectiveness, reliability, and cost that led to gradual adoption by farmers. This pattern of iterative improvement applies to most inventions—even ones that have a “heroic” story in which a single invention and inventor are traditionally highlighted—but even so, the history of the threshing machine seems unusually long and incremental.</p>
<p>My hypothesis is that the threshing machine was just past a certain threshold of the combination of the <em>amount of force required</em> and the <em>delicacy of the operation</em>. A loom is a somewhat complex machine performing an intricate process, but not one that uses a high degree of force. A flour mill, or a trip hammer at an iron works, is a high-force application, but not one that is particularly subtle or delicate. Both of these were in use long before the Industrial Revolution. But a threshing machine seems to require enough of both characteristics that <em>manufacturing quality became critical to meet the standard of reliability that was needed for practicality and adoption.</em> A simple, reliable design helped with this, but competent workmanship was at least as important.</p>
<h2>Challenges in manufacturing</h2>
<p>Here’s some evidence for this from historical sources.</p>
<p>The American farm periodical <a href="https://archive.org/details/geneseefarmer01roch/page/132/mode/1up?ref=ol&amp;view=theater"><em>Genesee Farmer,</em> April 1831</a>, in an article on the the threshing machine, writes:</p>
<blockquote>
 <p>… one of the great and principal causes of failures, in many kinds of machines, is the <strong>flimsy, cheap, and do-for the-present manner in which they are made.</strong> They are not unfrequently constructed by carpenters, or rather by those who are only an apology for a good one, and who could hardly construct a button to a barn-door…</p>
 <p>By the operation of these causes the farmer often gets an <strong>ill-constructed, weak, and rickety machine, which needs wedging, nailing, and bracing, at every revolution…</strong></p>
</blockquote>
<p>He continues with specific advice on construction and especially on materials:</p>
<blockquote>
 <p>The machinery that generates the motion, whether horse or water power, ought to be as well constructed, and of as good materials, as a flouring mill; and <strong>it is worse than useless to make the main wheel and pinion gearing of wood. Nothing but cast iron,</strong> and that of the softest and best kind, can be depended upon.</p>
</blockquote>
<p>(Using iron instead of wood has also been noted as a key step taken by Henry Maudslay to improve the precision of machine tools.)</p>
<p>On the British side, <a href="https://archive.org/details/implementsofagri00ransiala/page/153/mode/1up?ref=ol&amp;view=theater">Ransome (1843)</a> defends the concept of the threshing machine against the bad reputation it might acquire from poor workmanship:</p>
<blockquote>
 <p>It has been urged against these machines, that they are apt to break the straw, and that they bruise and nib the barley so as to render it unfit for malting; but <strong>these faults are not so much attributable to the principles of the machines, as to the manner in which they are frequently turned out of the hands of the workman;</strong> and sometimes to the want of skill and judgment in the parties who have the management of them.</p>
 <p>Many of these machines are made by persons who possess little claim to any mechanical knowledge, and who, purchasing the unfitted castings, by the help of village artisans, produce an imitation of those which are considered good. As <strong>the perfection of these machines must depend upon mathematical accuracy in the adjustment of all their parts, and in the truth and precision of their fittings, it is unreasonable to expect that this can be accomplished where no facilities exist beyond the forge and the work-bench;</strong> and hence arises a degree of discredit, which is unfairly thrown upon the principles upon which the machine is formed.</p>
</blockquote>
<p>So, if manufacturing quality was a critical factor—how were the early machines made?</p>
<h2>Manufacturing systems</h2>
<p>Today, agricultural equipment is manufactured by large, specialized firms who distribute their product worldwide. But in the 1700s and even into the 1800s, there was no such thing. “Agricultural equipment” meant plows, scythes, and wagons, and it was made by local craftsmen, such as the town carpenter, blacksmith, or cartwright. Not only were there very few large manufacturing enterprises in the world, but until the growth of the railroads in the mid-1800s, there were also no efficient transportation networks to distribute their products to farmers throughout the countryside. (See also <a href="https://www.jstor.org/stable/40273647">MacDonald 1975</a> on this point.)</p>
<p>In the promotions of early threshing machine inventors, we can see how they expected their machines to be acquired.</p>
<figure>
 <a href="/img/hobday-thresher-ad.jpg">
 
 </a>
 <figcaption>
 
 
 
 
 <a href="https://www.newspapers.com/clip/79888324/hobdays-threshing-machine-1772/">The Virginia Gazette, Williamsburg, 1772</a>
 
 
 
 </figcaption>
</figure>
<p>In many cases, an inventor merely proposed to supply plans and/or models, assuming that the machine would be built by a local craftsman, typically a carpenter or especially a millwright (millwrights were the closest thing to mechanical engineers in the 1700s). A <a href="https://www.newspapers.com/clip/79888324/hobdays-threshing-machine-1772/">1772 advertisement in <em>The Virginia Gazette</em></a> promotes an invention that “may be carried into execution by any tolerable carpenter” or even “by gentlemen’s own servants.” The inventor, one John Hobday, does not even seem to expect to sell the plans, let alone the machines: he proposes that a subscription be raised for him. More than fifty years later, in 1823, not much has changed: an <a href="https://babel.hathitrust.org/cgi/pt?id=uma.ark:/13960/t04x5m96m&amp;view=1up&amp;seq=35">article in the <em>American Farmer</em></a> on Joseph Pope’s threshing machine quotes a letter from the son of the deceased inventor as saying that the machine “can be constructed at little expense, the materials, including the shears, cost $13, and it can be made by a good workman, (say a joiner or carpenter) in 12 days.”</p>
<p>In other cases, inventors offered not the plans but the parts, “some assembly required.” In 1735, Andrew Good Wright in Edinburgh <a href="https://www.newspapers.com/clip/80186234/andrew-good-wright-threshing-machine/">advertised a threshing machine</a>, saying: “Those who want them, must send for them from Edinburgh, and a millwright to receive them, that he may know how to set them up; which any millwright will be able to do, after he has seen one going.” He also mentions a licensing opportunity: “Millwrights recommended by gentlemen for their integrity, if they come to Edinburgh, and understand the machine, shall have liberty from the patentee to make them in the country.” A <a href="https://www.newspapers.com/clip/80229031/threshing-machine-ad-that-offers-parts/">1796 advertisement</a> from “John Jubb, millwright and machine-maker,” offers both options: he will “send it to any part of the kingdom” for 25 guineas plus shipping and handling, or he will sell the plans along with “the machine made at Leeds, taken to pieces, properly marked, and sent off… so that any workman may with ease put it together.”</p>
<figure>
 <a href="/img/pope-thresher-ad.jpg">
 
 </a>
 <figcaption>
 
 
 
 
 <a href="https://www.newspapers.com/clip/80185462/joseph-pope-threshing-machine/">The National Gazette, Philadelphia, 1823</a>
 
 
 
 </figcaption>
</figure>
<p>There is some evidence that threshing machines thrived in places and times when they were made by the inventor himself, or by a skilled mechanic who decided to specialize in them. <a href="https://www.google.co.uk/books/edition/The_Commercial_Agricultural_and_Manufact/45xfAAAAcAAJ?hl=en&amp;gbpv=1">One source (pp 3–4)</a> reported as of 1800 that “almost all the threshing machines in England” had been built by one “industrious workman, of the name of Stevenson”. The article noted approvingly that his machines moved easily by hand and showed little friction, “a sure warrant of accurate workmanship.” As for Joseph Pope’s machine, soon after the article quoted above, he contracted with an engine maker to be his manufacturer. From May through October 1823, the Philadelphia <em>National Gazette</em> ran <a href="https://www.newspapers.com/clip/80185462/joseph-pope-threshing-machine/">advertisements</a> stating that “Messrs. S. V. Merrick &amp; Co. Engine Manufacturers, Philadelphia,” had been invested with the privilege of making and selling his machine; prospective customers were encouraged to “apply” to them to obtain one. An engine manufacturer, at that time, would have had a well-equipped machine shop with the precision tools and skilled workers needed to make reliable machines.</p>
<p>Bottom line: In its early decades, the threshing machine suffered from inconsistent quality due to distributed, unspecialized workmanship.</p>
<h2>Contrast with the reaper</h2>
<p>In contrast, consider a closely related piece of agricultural equipment invented decades later: the mechanical reaper.</p>
<figure></figure>
<p>The reaper cut down stalks of grain in the field, automating the step just prior to threshing. As with the thresher, reliability was a crucial challenge for the reaper: early models failed to cut tangled thickets of grain, or clogged when the stalks were damp, or performed poorly on hills.</p>
<figure>
 <a href="/img/sciam-reaper-1.jpg">
 
 </a>
 <figcaption>
 McCormick reaper, 1831
 
 
 
 <a href="https://archive.org/details/scientific-american-1896-07-25/page/n27/mode/1up">Scientific American</a>
 
 
 
 </figcaption>
</figure>
<p>One of the most successful reapers was created by Cyrus McCormick. He began selling them in the 1840s, but he relied neither on local craftsmen nor on manufacturing partners. He made the first machines himself, and eventually created a large, central factory.</p>
<p>McCormick sought a nationwide market, but the primitive cargo networks were a barrier. <a href="http://www.gutenberg.org/files/41953/41953-h/41953-h.htm">Casson (1909)</a>, one of his biographers, writes (p. 62):</p>
<blockquote>
 <p>To get the seven Reapers [sold in 1844] to the West, they had first to be carried in wagons to Scottsville, then by canal to Richmond, re-shipped down the James River to the Atlantic Ocean and around Florida to New Orleans, transferred here to a river boat that went up the Mississippi and Ohio Rivers to Cincinnati, and from Cincinnati in various directions to the expectant farmers. Four of these Reapers arrived too late for the harvest of 1844, and two of them were not paid for.</p>
</blockquote>
<p>McCormick later moved from Virginia to Chicago, setting up shop at the heart of a transportation network that integrated railroads and waterways—and in a location that was closer to his biggest market, in the Midwest (where the lands were flat and labor was scarce).</p>
<p>He also innovated on business practices to sell his machines: he advertised heavily in newspapers, organized field demonstrations, offered farmers a written guarantee and free credit until the next harvest, and built out a network of distributors in each region.</p>
<p>No one, from what I’ve read, did any of the above for the threshing machine, at least not before 1830 or so, when the machines were already common.</p>
<h2>Why did it take so long?</h2>
<p>The question of the threshing machine is a microcosm of the bigger question I’m interested in: Why did we wait so long for the Industrial Revolution?</p>
<p>Some inventions depended on theoretical concepts that were not discovered before the Scientific Revolution: the electric generator, for instance, or <a href="/science-and-steam">even the steam engine</a>. But others did not: the threshing machine, the <a href="/why-did-we-wait-so-long-for-the-cotton-gin">cotton gin</a>, the spinning jenny. What is the difference between those inventions, and others of seemingly similar complexity and importance that were adopted centuries earlier: the loom, the spinning wheel, the printing press?</p>
<p>One theme we see here is the manufacturing capability required to make affordable, reliable machines. This was probably also important for the bicycle, I think even more so than I realized or emphasized in <a href="/why-did-we-wait-so-long-for-the-bicycle">my previous essay on that topic</a>. For another example, <a href="https://www.amazon.com/Industrial-Revolution-Perspective-Approaches-Economic-ebook/dp/B00FF76RPK?tag=jasocraw-20">Robert Allen (2009)</a> says that precision manufacturing from the watch industry was necessary for an early thread-spinning machine known as the “water frame”: “The watch industry was the source of gears—brass gears in particular—and they were the precision parts in the water frame…. Without watch-makers, the water frame could not have been designed” (p. 204).</p>
<p>A secondary theme here is that there are prerequisites to market creation. If specialized manufacturing is required, then a centralized manufacturer must be able to reach a wide market. This requires communications infrastructure, such as newspapers, in order to market the goods, and transportation infrastructure, such as railroads, to ship them.</p>
<p>One reason perhaps (I’m speculating here) that mechanization came to textile manufacturing decades before it came to agriculture is that the business model was different. Richard Arkwright made his spinning machines, not to sell them as McCormick sold his reapers, but to <em>use</em> them to make cheap thread. He was his own market for the machines; they were a capital investment in a new type of business. This business model wasn’t really available to early agricultural equipment makers.</p>
<p>Another instructive comparison is to the steam engine, which also required high-quality, precision manufacturing. Boulton &amp; Watt adopted the centralized, specialized factory in the 1770s, decades before this model came to agricultural equipment. My speculative hypothesis here is that farmers, as a market, were a large number of smaller and more geographically distributed customers, vs. the coal mines, ironworks, breweries, etc. who were the customers for Watt’s engines. Thus the challenges of marketing and distribution were greater in the agricultural market. (Not to mention that industrial concerns such as mines might have had more ability than small farmers to finance a large expenditure on capital equipment.)</p>
<h2>Flywheels</h2>
<p>Maybe all this can be summarized as <em>infrastructure.</em> Manufacturing capability is infrastructure, as are railroads and newspapers (or today, trucking and the Internet). Infrastructure lowers the activation energy of any particular development. Sometimes these developments themselves create more/better infrastructure, creating a reinforcing cycle that generates <a href="/exponential-growth-is-the-baseline">exponential growth</a>.</p>
<p>Is this effect enough to explain the pace of progress throughout history? Do we still need to posit cultural causes—a general factor of inventiveness—Joel Mokyr’s “idea of progress” vs. “ancestor-worship”, Deirdre McCloskey’s “honor and prestige for the bourgeoisie”? It seems obvious that it at least <em>matters</em> whether people believe that progress is possible and desirable (and if it doesn’t, then it certainly seems like an enormous coincidence that the Industrial Revolution happened not long after Bacon and the Scientific Revolution, and in the same part of the world). But now that I more vividly understand the challenges facing 18th-century threshing machine inventors, I find myself shifting a bit more towards infrastructure as a cause.</p>
<p>Or, more broadly, I’m seeing that there is a set of overlapping flywheels, each creating a virtuous cycle as it gets going. Better infrastructure enables more progress, which in turn creates more and better infrastructure. The belief in progress leads to actual progress, which reinforces the belief. And there are others: Surplus wealth allows us to invest in progress, which creates more surplus. Science ultimately leads to progress, which then helps advance science. And since all these cycles intersect at progress itself, they all reinforce each other, indirectly if not directly.</p>
<hr />
<h2>Bibliography</h2>
<p><a href="https://www.amazon.com/Industrial-Revolution-Perspective-Approaches-Economic-ebook/dp/B00FF76RPK?tag=jasocraw-20">Allen, Robert (2009). <em>The British Industrial Revolution in Global Perspective</em></a></p>
<p><a href="http://www.gutenberg.org/files/41953/41953-h/41953-h.htm">Casson, Herbert Newton (1909). <em>Cyrus Hall McCormick: His Life and Work</em></a></p>
<p><a href="https://www.jstor.org/stable/40273647">MacDonald, Stuart (1975). “The Progress of the Early Threshing Machine”</a></p>
<p><a href="https://www.amazon.com/Sowing-Modernity-Americas-Agricultural-Revolution/dp/0801433266?tag=jasocraw-20">McClelland, Peter D. (1997). <em>Sowing Modernity: America’s First Agricultural Revolution</em></a></p>
<p><a href="https://www.amazon.com/Grain-Harvesters-Graeme-R-Quick/dp/0916150135?tag=jasocraw-20">Quick, Graeme and Wesley Buchele (1978). <em>The Grain Harvesters</em></a></p>
<p><a href="https://archive.org/details/implementsofagri00ransiala/page/137/mode/1up?ref=ol&amp;view=theater">Ransome, J. Allen (1843). <em>The Implements of Agriculture</em></a></p>
<p><a href="https://www.google.com/books/edition/General_View_of_the_Agriculture_of_East/AfEx0wgDmugC?hl=en&amp;gbpv=1">Somerville, R. (1805). <em>General View of the Agriculture of East Lothian</em></a></p>
<p>Key excerpts from the above, as well as citations from newspapers and other corroborating sources I did not reference, can be found in <a href="/threshing-machine-sources">this summary of primary/historical sources</a>.</p>
<p><a href="/van-berg-patent">Van Berg’s 1636 patent</a></p>
<hr />
<p><em>Thanks to <a href="https://www.antonhowes.com/">Anton Howes</a> for conversations, help finding sources, and comments on this essay, all of which contributed greatly to it.</em></p>
 
 <p>
 
 
 
 
 
 
 Comment: <a href="https://www.lesswrong.com/posts/K7wtTqTEoKXC9Kb24">LessWrong</a>, <a href="https://www.reddit.com/r/rootsofprogress/comments/oafjh8">Reddit</a>
 
 </p>
 
 </div>||||I|||| The Roots of Progress

 About • Writing • Speaking • Bibliography • Subscribe • Community • Support

 Why did we wait so long for the threshing machine?

 by Jason Crawford · June 29, 2021 · 14 min read

 When ripe wheat is harvested, the edible seed is encased in an outer husk. Before the seed can be ground into flour, or boiled into porridge, or planted in the field to produce next year’s harvest, it must be removed from the husk. This process is called threshing.

 As the husk is quite hard, threshing is a violent process. Traditionally, it was done with a tool called a flail, which is simply a short stick attached by a cord to a longer handle. The grain was spread out on the ground (yes, disgusting) and beaten with the stick to open the casings.

 The Growth of Industrial Art, 1892

 Other methods included “treading”, in which livestock trampled the grain with their hooves (yes, even more disgusting) or dragged a sledge over the grain (the Latin word for this sledge is tribulum, from which we get the world “tribulation”).

 Horse treading The Growth of Industrial Art, 1892

 Occasionally the grain would be rubbed against a wire screen, or placed in a sack and beaten with rocks. It’s no coincidence that the word “thrashing” is similar: it is an archaic spelling of the same term.

 As one of the more labor-intensive stages of wheat farming, threshing was a natural candidate to be automated by machinery. And the threshing machine was a relatively simple device: like the cotton gin, the flying shuttle, or the bicycle, it was a mechanical invention that did not depend on any scientific discoveries. Still, threshing machines were not used to any significant degree until the late 1700s in the UK and the early 1800s in the US.

 Once again, the question arises: why did we wait so long?

 Early claims and concepts

 First, it’s not because no one had the idea to mechanize the threshing process. Although we think of the Industrial Revolution as a uniquely inventive age, and the era before it seems blind to the possibility of innovation by comparison, we must remember that there were inventions prior to 1700, many of which were broadly adopted, especially when they helped with essential economic activities. The loom and the printing press predated the threshing machine by centuries, and neither seems far less complex.

 The threshing machine was referred to in an English patent as early as 1636 (for historical context, this is not long after the major works of Galileo and Francis Bacon). Sir John Christopher van Berg, a knight of Moravia (part of the present-day Czech Republic), fleeing religious wars in Germany and seeking protection from King Charles I, offered to Charles and to England his observations of “diverse mechanical instruments and frames operating by weights.” The patent is extremely broad and vague by today’s standards, listing a literally incredible variety of machines: for pumping water, washing clothes, cooking meat, working bellows, dredging rivers, raising sunken ships, measuring distances and depths, and for pretty much any mechanical operation. Towards the end of the list is mentioned an invention “to be agitated by wind, water, or horses, for the clean threshing of corn [grain], whereby much corn that is now left may be saved, and the straw made near as good as hay.” At the time, patents did not require detailed descriptions, diagrams, or models, and there is no more insight than this into how the machine might have worked, if indeed the invention was even real. The point is that by the early 1600s, the threshing machine was seen as possible and desirable.

 And why not? The grinding of grain into flour had been mechanized by watermills in ancient Rome. The threshing process was similar enough that some type of mill could plausibly accomplish the task. A century after Van Berg, inventors really started working on the problem, with one Michael Menzies of Scotland patenting a threshing machine in 1734, and several more efforts throughout the 1700s.

 So why did none of these machines gain wide adoption?

 Challenges with reliability

 One clue comes from reports of trials of the early machines.

 Some of the machines simply broke. Of Menzies’s machines, one historian wrote that “owing to the velocity required to do the work perfectly, they soon broke, and the invention fell into disgrace” (Somerville 1805, p. 75). Another machine was trialed in Scotland in the late 1700s, but “in a few minutes the model was torn to pieces” (Quick 1978, p. 45).

 When the machines did not break, they often failed to do the job. In one trial in 1753, it was found that the machine “broke off the ears of barley and wheat instead of clearing them of the grain, and that, at best, it was only fit for oats” (Ransome 1843, p. 140). Another was attempted in the late 1770s, but: “Upon trial this machine was also found defective, as along with its doing very little work in a given time, it bruised the grain, and so materially hurt its appearance, as to lessen its value considerably in the market” (Somerville 1805, p. 76).

 In general, McClelland (1997) says that threshing machines in the late 18th and early 19th century were “among the most complicated and expensive of all agricultural implements in American and Britain. High price and frequent breakdowns did not bode well for rapid adoption when ‘the least derangement … is death to the whole machine’” (p. 172). He reports that Washington and Jefferson were both interested in threshing machines, but (pp 175-6):

 Washington’s machine… built using the plans of William Booker soon proved unsatisfactory; a Maryland machine constructed with the aid of Colonel Anderson’s plans developed a warped wheel and was abandoned; in 1802 an immigrant from Edinburgh introduced into the Mid-Atlantic states “six or seven” based upon “the Scotch principle,” but they soon developed problems and “common workmen” could not repair them; the machine patented in 1803 by Jedediah Turner, based “upon entirely new, and very plain principles” was never commercially produced.… Washington perhaps spoke for a number of would-be agrarian pioneers when he voiced his frustration with this prospective new technology in 1793: “I have seen so much of beginning and ending of new inventions, that I have almost resolved to go on in the old way of treading.”

 What was the problem?

 One challenge was to find the right basic idea for the design of the machine. As is often the case, some of the early attempts tried to mimic human motion too closely: these machines automated the motion of the flail, but continued to beat the grain on the ground, an idea known as the “beating” principle. “All machines based upon the beating principle suffered from the same problem. The violence of the mechanical action used to separate the grain soon produced broken parts and an implement in disrepair” (McClelland 1997, p. 170). This was the problem with Menzies’s 1734 machine.

 Design for a “beating” type machine Thomas Paine's Pennsylvania Magazine, February 1775

 Another “beating” concept The Growth of Industrial Art, 1892

 A better idea was the “rubbing” principle. This design was more like a mill or a cotton gin: the grain was fed into a rotating drum that had pegs or other protrusions to grind the husk off of the seed. While “beating” vs. “rubbing” was still debated as late as the 1830s, all successful machines were of the rubbing type.

 Meikle's threshing machine, 1786 Wikimedia

 Pope's threshing machine American Farmer, 1823

 But the threshing machine didn’t seem to take off after one core design breakthrough. There isn’t a single individual who is always called out as “the inventor of the threshing machine,” nor a single date that stands out as an inflection point. Instead, one finds series of important inventions across decades: Andrew Meikle built the first successful machine in Scotland in 1786, but Joseph Pope invented a popular one in the US in 1820, and the Pitts brothers made improvements to power and to winnowing (separating the wheat from the chaff) in the 1830s. Nor does there seem to have been a single design that became dominant. Instead, it’s a story of gradual improvements in effectiveness, reliability, and cost that led to gradual adoption by farmers. This pattern of iterative improvement applies to most inventions—even ones that have a “heroic” story in which a single invention and inventor are traditionally highlighted—but even so, the history of the threshing machine seems unusually long and incremental.

 My hypothesis is that the threshing machine was just past a certain threshold of the combination of the amount of force required and the delicacy of the operation. A loom is a somewhat complex machine performing an intricate process, but not one that uses a high degree of force. A flour mill, or a trip hammer at an iron works, is a high-force application, but not one that is particularly subtle or delicate. Both of these were in use long before the Industrial Revolution. But a threshing machine seems to require enough of both characteristics that manufacturing quality became critical to meet the standard of reliability that was needed for practicality and adoption. A simple, reliable design helped with this, but competent workmanship was at least as important.

 Challenges in manufacturing

 Here’s some evidence for this from historical sources.

 The American farm periodical Genesee Farmer, April 1831, in an article on the the threshing machine, writes:

 … one of the great and principal causes of failures, in many kinds of machines, is the flimsy, cheap, and do-for the-present manner in which they are made. They are not unfrequently constructed by carpenters, or rather by those who are only an apology for a good one, and who could hardly construct a button to a barn-door…

 By the operation of these causes the farmer often gets an ill-constructed, weak, and rickety machine, which needs wedging, nailing, and bracing, at every revolution…

 He continues with specific advice on construction and especially on materials:

 The machinery that generates the motion, whether horse or water power, ought to be as well constructed, and of as good materials, as a flouring mill; and it is worse than useless to make the main wheel and pinion gearing of wood. Nothing but cast iron, and that of the softest and best kind, can be depended upon.

 (Using iron instead of wood has also been noted as a key step taken by Henry Maudslay to improve the precision of machine tools.)

 On the British side, Ransome (1843) defends the concept of the threshing machine against the bad reputation it might acquire from poor workmanship:

 It has been urged against these machines, that they are apt to break the straw, and that they bruise and nib the barley so as to render it unfit for malting; but these faults are not so much attributable to the principles of the machines, as to the manner in which they are frequently turned out of the hands of the workman; and sometimes to the want of skill and judgment in the parties who have the management of them.

 Many of these machines are made by persons who possess little claim to any mechanical knowledge, and who, purchasing the unfitted castings, by the help of village artisans, produce an imitation of those which are considered good. As the perfection of these machines must depend upon mathematical accuracy in the adjustment of all their parts, and in the truth and precision of their fittings, it is unreasonable to expect that this can be accomplished where no facilities exist beyond the forge and the work-bench; and hence arises a degree of discredit, which is unfairly thrown upon the principles upon which the machine is formed.

 So, if manufacturing quality was a critical factor—how were the early machines made?

 Manufacturing systems

 Today, agricultural equipment is manufactured by large, specialized firms who distribute their product worldwide. But in the 1700s and even into the 1800s, there was no such thing. “Agricultural equipment” meant plows, scythes, and wagons, and it was made by local craftsmen, such as the town carpenter, blacksmith, or cartwright. Not only were there very few large manufacturing enterprises in the world, but until the growth of the railroads in the mid-1800s, there were also no efficient transportation networks to distribute their products to farmers throughout the countryside. (See also MacDonald 1975 on this point.)

 In the promotions of early threshing machine inventors, we can see how they expected their machines to be acquired.

 The Virginia Gazette, Williamsburg, 1772

 In many cases, an inventor merely proposed to supply plans and/or models, assuming that the machine would be built by a local craftsman, typically a carpenter or especially a millwright (millwrights were the closest thing to mechanical engineers in the 1700s). A 1772 advertisement in The Virginia Gazette promotes an invention that “may be carried into execution by any tolerable carpenter” or even “by gentlemen’s own servants.” The inventor, one John Hobday, does not even seem to expect to sell the plans, let alone the machines: he proposes that a subscription be raised for him. More than fifty years later, in 1823, not much has changed: an article in the American Farmer on Joseph Pope’s threshing machine quotes a letter from the son of the deceased inventor as saying that the machine “can be constructed at little expense, the materials, including the shears, cost $13, and it can be made by a good workman, (say a joiner or carpenter) in 12 days.”

 In other cases, inventors offered not the plans but the parts, “some assembly required.” In 1735, Andrew Good Wright in Edinburgh advertised a threshing machine, saying: “Those who want them, must send for them from Edinburgh, and a millwright to receive them, that he may know how to set them up; which any millwright will be able to do, after he has seen one going.” He also mentions a licensing opportunity: “Millwrights recommended by gentlemen for their integrity, if they come to Edinburgh, and understand the machine, shall have liberty from the patentee to make them in the country.” A 1796 advertisement from “John Jubb, millwright and machine-maker,” offers both options: he will “send it to any part of the kingdom” for 25 guineas plus shipping and handling, or he will sell the plans along with “the machine made at Leeds, taken to pieces, properly marked, and sent off… so that any workman may with ease put it together.”

 The National Gazette, Philadelphia, 1823

 There is some evidence that threshing machines thrived in places and times when they were made by the inventor himself, or by a skilled mechanic who decided to specialize in them. One source (pp 3–4) reported as of 1800 that “almost all the threshing machines in England” had been built by one “industrious workman, of the name of Stevenson”. The article noted approvingly that his machines moved easily by hand and showed little friction, “a sure warrant of accurate workmanship.” As for Joseph Pope’s machine, soon after the article quoted above, he contracted with an engine maker to be his manufacturer. From May through October 1823, the Philadelphia National Gazette ran advertisements stating that “Messrs. S. V. Merrick & Co. Engine Manufacturers, Philadelphia,” had been invested with the privilege of making and selling his machine; prospective customers were encouraged to “apply” to them to obtain one. An engine manufacturer, at that time, would have had a well-equipped machine shop with the precision tools and skilled workers needed to make reliable machines.

 Bottom line: In its early decades, the threshing machine suffered from inconsistent quality due to distributed, unspecialized workmanship.

 Contrast with the reaper

 In contrast, consider a closely related piece of agricultural equipment invented decades later: the mechanical reaper.

 The reaper cut down stalks of grain in the field, automating the step just prior to threshing. As with the thresher, reliability was a crucial challenge for the reaper: early models failed to cut tangled thickets of grain, or clogged when the stalks were damp, or performed poorly on hills.

 McCormick reaper, 1831 Scientific American

 One of the most successful reapers was created by Cyrus McCormick. He began selling them in the 1840s, but he relied neither on local craftsmen nor on manufacturing partners. He made the first machines himself, and eventually created a large, central factory.

 McCormick sought a nationwide market, but the primitive cargo networks were a barrier. Casson (1909), one of his biographers, writes (p. 62):

 To get the seven Reapers [sold in 1844] to the West, they had first to be carried in wagons to Scottsville, then by canal to Richmond, re-shipped down the James River to the Atlantic Ocean and around Florida to New Orleans, transferred here to a river boat that went up the Mississippi and Ohio Rivers to Cincinnati, and from Cincinnati in various directions to the expectant farmers. Four of these Reapers arrived too late for the harvest of 1844, and two of them were not paid for.

 McCormick later moved from Virginia to Chicago, setting up shop at the heart of a transportation network that integrated railroads and waterways—and in a location that was closer to his biggest market, in the Midwest (where the lands were flat and labor was scarce).

 He also innovated on business practices to sell his machines: he advertised heavily in newspapers, organized field demonstrations, offered farmers a written guarantee and free credit until the next harvest, and built out a network of distributors in each region.

 No one, from what I’ve read, did any of the above for the threshing machine, at least not before 1830 or so, when the machines were already common.

 Why did it take so long?

 The question of the threshing machine is a microcosm of the bigger question I’m interested in: Why did we wait so long for the Industrial Revolution?

 Some inventions depended on theoretical concepts that were not discovered before the Scientific Revolution: the electric generator, for instance, or even the steam engine. But others did not: the threshing machine, the cotton gin, the spinning jenny. What is the difference between those inventions, and others of seemingly similar complexity and importance that were adopted centuries earlier: the loom, the spinning wheel, the printing press?

 One theme we see here is the manufacturing capability required to make affordable, reliable machines. This was probably also important for the bicycle, I think even more so than I realized or emphasized in my previous essay on that topic. For another example, Robert Allen (2009) says that precision manufacturing from the watch industry was necessary for an early thread-spinning machine known as the “water frame”: “The watch industry was the source of gears—brass gears in particular—and they were the precision parts in the water frame…. Without watch-makers, the water frame could not have been designed” (p. 204).

 A secondary theme here is that there are prerequisites to market creation. If specialized manufacturing is required, then a centralized manufacturer must be able to reach a wide market. This requires communications infrastructure, such as newspapers, in order to market the goods, and transportation infrastructure, such as railroads, to ship them.

 One reason perhaps (I’m speculating here) that mechanization came to textile manufacturing decades before it came to agriculture is that the business model was different. Richard Arkwright made his spinning machines, not to sell them as McCormick sold his reapers, but to use them to make cheap thread. He was his own market for the machines; they were a capital investment in a new type of business. This business model wasn’t really available to early agricultural equipment makers.

 Another instructive comparison is to the steam engine, which also required high-quality, precision manufacturing. Boulton & Watt adopted the centralized, specialized factory in the 1770s, decades before this model came to agricultural equipment. My speculative hypothesis here is that farmers, as a market, were a large number of smaller and more geographically distributed customers, vs. the coal mines, ironworks, breweries, etc. who were the customers for Watt’s engines. Thus the challenges of marketing and distribution were greater in the agricultural market. (Not to mention that industrial concerns such as mines might have had more ability than small farmers to finance a large expenditure on capital equipment.)

 Flywheels

 Maybe all this can be summarized as infrastructure. Manufacturing capability is infrastructure, as are railroads and newspapers (or today, trucking and the Internet). Infrastructure lowers the activation energy of any particular development. Sometimes these developments themselves create more/better infrastructure, creating a reinforcing cycle that generates exponential growth.

 Is this effect enough to explain the pace of progress throughout history? Do we still need to posit cultural causes—a general factor of inventiveness—Joel Mokyr’s “idea of progress” vs. “ancestor-worship”, Deirdre McCloskey’s “honor and prestige for the bourgeoisie”? It seems obvious that it at least matters whether people believe that progress is possible and desirable (and if it doesn’t, then it certainly seems like an enormous coincidence that the Industrial Revolution happened not long after Bacon and the Scientific Revolution, and in the same part of the world). But now that I more vividly understand the challenges facing 18th-century threshing machine inventors, I find myself shifting a bit more towards infrastructure as a cause.

 Or, more broadly, I’m seeing that there is a set of overlapping flywheels, each creating a virtuous cycle as it gets going. Better infrastructure enables more progress, which in turn creates more and better infrastructure. The belief in progress leads to actual progress, which reinforces the belief. And there are others: Surplus wealth allows us to invest in progress, which creates more surplus. Science ultimately leads to progress, which then helps advance science. And since all these cycles intersect at progress itself, they all reinforce each other, indirectly if not directly.

 Bibliography

 Allen, Robert (2009). The British Industrial Revolution in Global Perspective

 Casson, Herbert Newton (1909). Cyrus Hall McCormick: His Life and Work

 MacDonald, Stuart (1975). “The Progress of the Early Threshing Machine”

 McClelland, Peter D. (1997). Sowing Modernity: America’s First Agricultural Revolution

 Quick, Graeme and Wesley Buchele (1978). The Grain Harvesters

 Ransome, J. Allen (1843). The Implements of Agriculture

 Somerville, R. (1805). General View of the Agriculture of East Lothian

 Key excerpts from the above, as well as citations from newspapers and other corroborating sources I did not reference, can be found in this summary of primary/historical sources.

 Van Berg’s 1636 patent

 Thanks to Anton Howes for conversations, help finding sources, and comments on this essay, all of which contributed greatly to it.

 Comment: LessWrong, Reddit

« Tickets available for Session 4 of The Story of Industrial Civilization Winston Churchill, futurist »

 Get posts by email:

 Copyright © Jason Crawford. Some rights reserved: CC BY-ND 4.0

 Get posts by email

 ╳

 Don't show again

When ripe wheat is harvested, the edible seed is encased in an outer husk. Before the seed can be ground into flour, or boiled into porridge, or planted in the field to produce next year’s harvest, it must be removed from the husk. This process is called threshing.
As the husk is quite hard, threshing is a violent process. Traditionally, it was done with a tool called a flail, which is simply a short stick attached by a cord to a longer handle. The grain was spread out on the ground (yes, disgusting) and beaten with the stick to open the casings.
 
 
 
 
 
 
 
 
 The Growth of Industrial Art, 1892
 
 
 
 
Other methods included “treading”, in which livestock trampled the grain with their hooves (yes, even more disgusting) or dragged a sledge over the grain (the Latin word for this sledge is tribulum, from which we get the world “tribulation”).
 
 
 
 
 Horse treading
 
 
 
 The Growth of Industrial Art, 1892
 
 
 
 
Occasionally the grain would be rubbed against a wire screen, or placed in a sack and beaten with rocks. It’s no coincidence that the word “thrashing” is similar: it is an archaic spelling of the same term.
As one of the more labor-intensive stages of wheat farming, threshing was a natural candidate to be automated by machinery. And the threshing machine was a relatively simple device: like the cotton gin, the flying shuttle, or the bicycle, it was a mechanical invention that did not depend on any scientific discoveries. Still, threshing machines were not used to any significant degree until the late 1700s in the UK and the early 1800s in the US.
Once again, the question arises: why did we wait so long? 
Early claims and concepts
First, it’s not because no one had the idea to mechanize the threshing process. Although we think of the Industrial Revolution as a uniquely inventive age, and the era before it seems blind to the possibility of innovation by comparison, we must remember that there were inventions prior to 1700, many of which were broadly adopted, especially when they helped with essential economic activities. The loom and the printing press predated the threshing machine by centuries, and neither seems far less complex.
The threshing machine was referred to in an English patent as early as 1636 (for historical context, this is not long after the major works of Galileo and Francis Bacon). Sir John Christopher van Berg, a knight of Moravia (part of the present-day Czech Republic), fleeing religious wars in Germany and seeking protection from King Charles I, offered to Charles and to England his observations of “diverse mechanical instruments and frames operating by weights.” The patent is extremely broad and vague by today’s standards, listing a literally incredible variety of machines: for pumping water, washing clothes, cooking meat, working bellows, dredging rivers, raising sunken ships, measuring distances and depths, and for pretty much any mechanical operation. Towards the end of the list is mentioned an invention “to be agitated by wind, water, or horses, for the clean threshing of corn [grain], whereby much corn that is now left may be saved, and the straw made near as good as hay.” At the time, patents did not require detailed descriptions, diagrams, or models, and there is no more insight than this into how the machine might have worked, if indeed the invention was even real. The point is that by the early 1600s, the threshing machine was seen as possible and desirable.
And why not? The grinding of grain into flour had been mechanized by watermills in ancient Rome. The threshing process was similar enough that some type of mill could plausibly accomplish the task. A century after Van Berg, inventors really started working on the problem, with one Michael Menzies of Scotland patenting a threshing machine in 1734, and several more efforts throughout the 1700s.
So why did none of these machines gain wide adoption?
Challenges with reliability
One clue comes from reports of trials of the early machines.
Some of the machines simply broke. Of Menzies’s machines, one historian wrote that “owing to the velocity required to do the work perfectly, they soon broke, and the invention fell into disgrace” (Somerville 1805, p. 75). Another machine was trialed in Scotland in the late 1700s, but “in a few minutes the model was torn to pieces” (Quick 1978, p. 45).
When the machines did not break, they often failed to do the job. In one trial in 1753, it was found that the machine “broke off the ears of barley and wheat instead of clearing them of the grain, and that, at best, it was only fit for oats” (Ransome 1843, p. 140). Another was attempted in the late 1770s, but: “Upon trial this machine was also found defective, as along with its doing very little work in a given time, it bruised the grain, and so materially hurt its appearance, as to lessen its value considerably in the market” (Somerville 1805, p. 76).
In general, McClelland (1997) says that threshing machines in the late 18th and early 19th century were “among the most complicated and expensive of all agricultural implements in American and Britain. High price and frequent breakdowns did not bode well for rapid adoption when ‘the least derangement … is death to the whole machine’” (p. 172). He reports that Washington and Jefferson were both interested in threshing machines, but (pp 175-6):
 Washington’s machine… built using the plans of William Booker soon proved unsatisfactory; a Maryland machine constructed with the aid of Colonel Anderson’s plans developed a warped wheel and was abandoned; in 1802 an immigrant from Edinburgh introduced into the Mid-Atlantic states “six or seven” based upon “the Scotch principle,” but they soon developed problems and “common workmen” could not repair them; the machine patented in 1803 by Jedediah Turner, based “upon entirely new, and very plain principles” was never commercially produced.… Washington perhaps spoke for a number of would-be agrarian pioneers when he voiced his frustration with this prospective new technology in 1793: “I have seen so much of beginning and ending of new inventions, that I have almost resolved to go on in the old way of treading.” 
What was the problem?
 
 
One challenge was to find the right basic idea for the design of the machine. As is often the case, some of the early attempts tried to mimic human motion too closely: these machines automated the motion of the flail, but continued to beat the grain on the ground, an idea known as the “beating” principle. “All machines based upon the beating principle suffered from the same problem. The violence of the mechanical action used to separate the grain soon produced broken parts and an implement in disrepair” (McClelland 1997, p. 170). This was the problem with Menzies’s 1734 machine.
 
 
 
 
 Design for a “beating” type machine
 
 
 
 Thomas Paine's Pennsylvania Magazine, February 1775
 
 
 
 
 
 
 
 
 Another “beating” concept
 
 
 
 The Growth of Industrial Art, 1892
 
 
 
 
 
 
A better idea was the “rubbing” principle. This design was more like a mill or a cotton gin: the grain was fed into a rotating drum that had pegs or other protrusions to grind the husk off of the seed. While “beating” vs. “rubbing” was still debated as late as the 1830s, all successful machines were of the rubbing type.
 
 
 
 
 Meikle's threshing machine, 1786
 
 
 
 Wikimedia
 
 
 
 
 
 
 
 
 Pope's threshing machine
 
 
 
 American Farmer, 1823
 
 
 
 
But the threshing machine didn’t seem to take off after one core design breakthrough. There isn’t a single individual who is always called out as “the inventor of the threshing machine,” nor a single date that stands out as an inflection point. Instead, one finds series of important inventions across decades: Andrew Meikle built the first successful machine in Scotland in 1786, but Joseph Pope invented a popular one in the US in 1820, and the Pitts brothers made improvements to power and to winnowing (separating the wheat from the chaff) in the 1830s. Nor does there seem to have been a single design that became dominant. Instead, it’s a story of gradual improvements in effectiveness, reliability, and cost that led to gradual adoption by farmers. This pattern of iterative improvement applies to most inventions—even ones that have a “heroic” story in which a single invention and inventor are traditionally highlighted—but even so, the history of the threshing machine seems unusually long and incremental.
My hypothesis is that the threshing machine was just past a certain threshold of the combination of the amount of force required and the delicacy of the operation. A loom is a somewhat complex machine performing an intricate process, but not one that uses a high degree of force. A flour mill, or a trip hammer at an iron works, is a high-force application, but not one that is particularly subtle or delicate. Both of these were in use long before the Industrial Revolution. But a threshing machine seems to require enough of both characteristics that manufacturing quality became critical to meet the standard of reliability that was needed for practicality and adoption. A simple, reliable design helped with this, but competent workmanship was at least as important.
Challenges in manufacturing
Here’s some evidence for this from historical sources.
The American farm periodical  Genesee Farmer, April 1831, in an article on the the threshing machine, writes:
 … one of the great and principal causes of failures, in many kinds of machines, is the flimsy, cheap, and do-for the-present manner in which they are made. They are not unfrequently constructed by carpenters, or rather by those who are only an apology for a good one, and who could hardly construct a button to a barn-door…
 By the operation of these causes the farmer often gets an ill-constructed, weak, and rickety machine, which needs wedging, nailing, and bracing, at every revolution… 
He continues with specific advice on construction and especially on materials:
 The machinery that generates the motion, whether horse or water power, ought to be as well constructed, and of as good materials, as a flouring mill; and it is worse than useless to make the main wheel and pinion gearing of wood. Nothing but cast iron, and that of the softest and best kind, can be depended upon.
(Using iron instead of wood has also been noted as a key step taken by Henry Maudslay to improve the precision of machine tools.)
On the British side, Ransome (1843) defends the concept of the threshing machine against the bad reputation it might acquire from poor workmanship:
 It has been urged against these machines, that they are apt to break the straw, and that they bruise and nib the barley so as to render it unfit for malting; but these faults are not so much attributable to the principles of the machines, as to the manner in which they are frequently turned out of the hands of the workman; and sometimes to the want of skill and judgment in the parties who have the management of them.
 Many of these machines are made by persons who possess little claim to any mechanical knowledge, and who, purchasing the unfitted castings, by the help of village artisans, produce an imitation of those which are considered good. As the perfection of these machines must depend upon mathematical accuracy in the adjustment of all their parts, and in the truth and precision of their fittings, it is unreasonable to expect that this can be accomplished where no facilities exist beyond the forge and the work-bench; and hence arises a degree of discredit, which is unfairly thrown upon the principles upon which the machine is formed.
So, if manufacturing quality was a critical factor—how were the early machines made?
Manufacturing systems
Today, agricultural equipment is manufactured by large, specialized firms who distribute their product worldwide. But in the 1700s and even into the 1800s, there was no such thing. “Agricultural equipment” meant plows, scythes, and wagons, and it was made by local craftsmen, such as the town carpenter, blacksmith, or cartwright. Not only were there very few large manufacturing enterprises in the world, but until the growth of the railroads in the mid-1800s, there were also no efficient transportation networks to distribute their products to farmers throughout the countryside. (See also MacDonald 1975 on this point.)
In the promotions of early threshing machine inventors, we can see how they expected their machines to be acquired.
 
 
 
 
 
 
 
 
 The Virginia Gazette, Williamsburg, 1772
 
 
 
 
In many cases, an inventor merely proposed to supply plans and/or models, assuming that the machine would be built by a local craftsman, typically a carpenter or especially a millwright (millwrights were the closest thing to mechanical engineers in the 1700s). A 1772 advertisement in The Virginia Gazette  promotes an invention that “may be carried into execution by any tolerable carpenter” or even “by gentlemen’s own servants.” The inventor, one John Hobday, does not even seem to expect to sell the plans, let alone the machines: he proposes that a subscription be raised for him. More than fifty years later, in 1823, not much has changed: an article in the American Farmer  on Joseph Pope’s threshing machine quotes a letter from the son of the deceased inventor as saying that the machine “can be constructed at little expense, the materials, including the shears, cost $13, and it can be made by a good workman, (say a joiner or carpenter) in 12 days.”
In other cases, inventors offered not the plans but the parts, “some assembly required.” In 1735, Andrew Good Wright in Edinburgh advertised a threshing machine, saying: “Those who want them, must send for them from Edinburgh, and a millwright to receive them, that he may know how to set them up; which any millwright will be able to do, after he has seen one going.” He also mentions a licensing opportunity: “Millwrights recommended by gentlemen for their integrity, if they come to Edinburgh, and understand the machine, shall have liberty from the patentee to make them in the country.” A 1796 advertisement from “John Jubb, millwright and machine-maker,” offers both options: he will “send it to any part of the kingdom” for 25 guineas plus shipping and handling, or he will sell the plans along with “the machine made at Leeds, taken to pieces, properly marked, and sent off… so that any workman may with ease put it together.”
 
 
 
 
 
 
 
 
 The National Gazette, Philadelphia, 1823
 
 
 
 
There is some evidence that threshing machines thrived in places and times when they were made by the inventor himself, or by a skilled mechanic who decided to specialize in them. One source (pp 3–4) reported as of 1800 that “almost all the threshing machines in England” had been built by one “industrious workman, of the name of Stevenson”. The article noted approvingly that his machines moved easily by hand and showed little friction, “a sure warrant of accurate workmanship.” As for Joseph Pope’s machine, soon after the article quoted above, he contracted with an engine maker to be his manufacturer. From May through October 1823, the Philadelphia National Gazette ran advertisements stating that “Messrs. S. V. Merrick &amp; Co. Engine Manufacturers, Philadelphia,” had been invested with the privilege of making and selling his machine; prospective customers were encouraged to “apply” to them to obtain one. An engine manufacturer, at that time, would have had a well-equipped machine shop with the precision tools and skilled workers needed to make reliable machines.
Bottom line: In its early decades, the threshing machine suffered from inconsistent quality due to distributed, unspecialized workmanship.
Contrast with the reaper
In contrast, consider a closely related piece of agricultural equipment invented decades later: the mechanical reaper.
 
The reaper cut down stalks of grain in the field, automating the step just prior to threshing. As with the thresher, reliability was a crucial challenge for the reaper: early models failed to cut tangled thickets of grain, or clogged when the stalks were damp, or performed poorly on hills.
 
 
 
 
 McCormick reaper, 1831
 
 
 
 Scientific American
 
 
 
 
One of the most successful reapers was created by Cyrus McCormick. He began selling them in the 1840s, but he relied neither on local craftsmen nor on manufacturing partners. He made the first machines himself, and eventually created a large, central factory.
McCormick sought a nationwide market, but the primitive cargo networks were a barrier. Casson (1909), one of his biographers, writes (p. 62):
 To get the seven Reapers [sold in 1844] to the West, they had first to be carried in wagons to Scottsville, then by canal to Richmond, re-shipped down the James River to the Atlantic Ocean and around Florida to New Orleans, transferred here to a river boat that went up the Mississippi and Ohio Rivers to Cincinnati, and from Cincinnati in various directions to the expectant farmers. Four of these Reapers arrived too late for the harvest of 1844, and two of them were not paid for.
McCormick later moved from Virginia to Chicago, setting up shop at the heart of a transportation network that integrated railroads and waterways—and in a location that was closer to his biggest market, in the Midwest (where the lands were flat and labor was scarce).
He also innovated on business practices to sell his machines: he advertised heavily in newspapers, organized field demonstrations, offered farmers a written guarantee and free credit until the next harvest, and built out a network of distributors in each region.
No one, from what I’ve read, did any of the above for the threshing machine, at least not before 1830 or so, when the machines were already common.
Why did it take so long?
The question of the threshing machine is a microcosm of the bigger question I’m interested in: Why did we wait so long for the Industrial Revolution?
Some inventions depended on theoretical concepts that were not discovered before the Scientific Revolution: the electric generator, for instance, or even the steam engine. But others did not: the threshing machine, the cotton gin, the spinning jenny. What is the difference between those inventions, and others of seemingly similar complexity and importance that were adopted centuries earlier: the loom, the spinning wheel, the printing press?
One theme we see here is the manufacturing capability required to make affordable, reliable machines. This was probably also important for the bicycle, I think even more so than I realized or emphasized in my previous essay on that topic. For another example, Robert Allen (2009) says that precision manufacturing from the watch industry was necessary for an early thread-spinning machine known as the “water frame”: “The watch industry was the source of gears—brass gears in particular—and they were the precision parts in the water frame…. Without watch-makers, the water frame could not have been designed” (p. 204).
A secondary theme here is that there are prerequisites to market creation. If specialized manufacturing is required, then a centralized manufacturer must be able to reach a wide market. This requires communications infrastructure, such as newspapers, in order to market the goods, and transportation infrastructure, such as railroads, to ship them.
One reason perhaps (I’m speculating here) that mechanization came to textile manufacturing decades before it came to agriculture is that the business model was different. Richard Arkwright made his spinning machines, not to sell them as McCormick sold his reapers, but to use them to make cheap thread. He was his own market for the machines; they were a capital investment in a new type of business. This business model wasn’t really available to early agricultural equipment makers.
Another instructive comparison is to the steam engine, which also required high-quality, precision manufacturing. Boulton &amp; Watt adopted the centralized, specialized factory in the 1770s, decades before this model came to agricultural equipment. My speculative hypothesis here is that farmers, as a market, were a large number of smaller and more geographically distributed customers, vs. the coal mines, ironworks, breweries, etc. who were the customers for Watt’s engines. Thus the challenges of marketing and distribution were greater in the agricultural market. (Not to mention that industrial concerns such as mines might have had more ability than small farmers to finance a large expenditure on capital equipment.)
Flywheels
Maybe all this can be summarized as infrastructure. Manufacturing capability is infrastructure, as are railroads and newspapers (or today, trucking and the Internet). Infrastructure lowers the activation energy of any particular development. Sometimes these developments themselves create more/better infrastructure, creating a reinforcing cycle that generates exponential growth.
Is this effect enough to explain the pace of progress throughout history? Do we still need to posit cultural causes—a general factor of inventiveness—Joel Mokyr’s “idea of progress” vs. “ancestor-worship”, Deirdre McCloskey’s “honor and prestige for the bourgeoisie”? It seems obvious that it at least matters whether people believe that progress is possible and desirable (and if it doesn’t, then it certainly seems like an enormous coincidence that the Industrial Revolution happened not long after Bacon and the Scientific Revolution, and in the same part of the world). But now that I more vividly understand the challenges facing 18th-century threshing machine inventors, I find myself shifting a bit more towards infrastructure as a cause.
Or, more broadly, I’m seeing that there is a set of overlapping flywheels, each creating a virtuous cycle as it gets going. Better infrastructure enables more progress, which in turn creates more and better infrastructure. The belief in progress leads to actual progress, which reinforces the belief. And there are others: Surplus wealth allows us to invest in progress, which creates more surplus. Science ultimately leads to progress, which then helps advance science. And since all these cycles intersect at progress itself, they all reinforce each other, indirectly if not directly.
Bibliography
 Allen, Robert (2009). The British Industrial Revolution in Global Perspective  
 Casson, Herbert Newton (1909). Cyrus Hall McCormick: His Life and Work  
 MacDonald, Stuart (1975). “The Progress of the Early Threshing Machine” 
 McClelland, Peter D. (1997). Sowing Modernity: America’s First Agricultural Revolution  
 Quick, Graeme and Wesley Buchele (1978). The Grain Harvesters  
 Ransome, J. Allen (1843). The Implements of Agriculture  
 Somerville, R. (1805). General View of the Agriculture of East Lothian  
Key excerpts from the above, as well as citations from newspapers and other corroborating sources I did not reference, can be found in this summary of primary/historical sources.
 Van Berg’s 1636 patent 
 Thanks to Anton Howes for conversations, help finding sources, and comments on this essay, all of which contributed greatly to it. 
 
 
 
 
 
 
 
 
 Comment: LessWrong, Reddit
Why did we wait so long for the threshing machine?
https://rootsofprogress.org/why-did-we-wait-so-long-for-the-threshing-machine
https://markmanson.net/why-you-should-quit-the-news
Why You Should Quit the News
2020-01-20
Mark Manson
<div><div>
	<section><p>Everyone kind of already knows that the news sucks. In all my life, I can’t think of anyone who seems to <em>enjoy</em> reading or watching the news every day. It’s a kind of bitter responsibility or endured necessity for people. News is like the societal version of flossing: it’s not fun, yet we continue to do it every day anyway, as a sort of obligation to prevent decay of the social order.</p>
</section>
<p>We all get it: yes, the news is overly negative, and yes, it often gets some things wrong. But in the end, we’re better off for it. That’s how the argument usually goes. Some even say we <em>need</em> the news, that democracy is impossible without it.</p>
<p>In this article, I’m going to challenge that idea. In fact, I’m going to argue that the news doesn’t just <em>seem</em> to be horrible, in its current form it <em>is</em> horrible. It is actively damaging our culture. And most of what we <a href="https://markmanson.net/social-media">blame on social media</a> or <a href="https://markmanson.net/internet">the internet</a>—the upticks in stress, anxiety, pessimism, and polarization—is misplaced. News media is the true culprit.</p>
<p>Obviously, this is a bold argument. And this article will be quite long. We’ll investigate what makes some information valuable and other information worthless. We’ll look at how networks of information emerge and how news has traditionally traveled throughout human history. We will look at how the internet and social media have affected those traditional networks, and, of course, I’ll take a stab at explaining why our current media environment is a shit sauna.</p>
<p>Finally, I’ll discuss what I feel are better alternatives for staying informed than consuming news. There will be lots of links.</p>
<p>So, take a deep breath… maybe grab a stress ball. This is going to take a while.</p>
<div>
<p>Let me ask you this: when was the last time you made a <a href="https://markmanson.net/decision-making">major life decision</a> based on a news story? Or: when was the last time information from a news story directly impacted your life? And I don’t mean, “It made me really, <em>really</em> upset,” impacted. I mean, like, actually impacted you.</p>
<p>Chances are you can’t remember. That’s because the vast majority of news is irrelevant. Car chases. Bank robberies. Celebrity overdoses. Sports scores. Homeless people throwing feces at one another. None of us ever really have the experience of seeing a news story and immediately thinking, “Man, I need to change what I’m doing.” If it’s ever happened to me, I don’t remember it.</p>
<p>The news does not tell you <a href="https://markmanson.net/how-to-find-the-perfect-career">what job to take</a> or how to <a href="https://markmanson.net/deal-with-conflict">deal with your angry husband</a> or what the best brand of headphones is or what to feed your kids in the morning. It doesn’t help you to be a <a href="https://markmanson.net/what-real-friends-look-like">better friend</a> or explain why you have panic attacks or advise on the best way to deal with an unruly child.</p>
<p>The news is pretty much useless for helping you determine <a href="https://markmanson.net/life-purpose">what to do with your life</a>. You may argue that the news helps you determine who you vote for, but research suggests that even that’s not true—we are far more swayed by our friends, family, and day-to-day circumstances than any information we may find on the news.<a href="#footnote-1"><sup>1</sup></a> If you want to know a politician’s platform today, you can just go to their website or hear them speak directly on YouTube. You don’t need a talking suit on television or an op-ed section to tell you what to think.</p>
<p>If we’re being honest, most of the value we get from consuming news is having something to talk about with other people who consume the news. And considering this has a 50% chance of making you hate each other, even that value becomes questionable.</p>
<p>So, if the news isn’t sharing useful facts or information that is directly affecting millions of people’s lives, what exactly is the news trying to do?</p>
<p>Well, that’s easy. The goal of the news is to motivate you to keep consuming news.<a href="#footnote-2"><sup>2</sup></a></p>
<p>They do this in a few ways:</p>
<ul>
<li>They present <a href="https://markmanson.net/the-life-cycle-of-outrage">emotionally-charged information that <em>feels</em> important</a>, even though it isn’t (a politician making a gaffe, for instance).</li>
<li>They catastrophize everything to make you believe this is a once-in-a-lifetime event and nothing will ever be the same ever again… until something else happens. The <a href="https://en.wikipedia.org/wiki/2020_Baghdad_International_Airport_airstrike">assassination of Soleimani in Iraq</a> in January 2020 is an example.<a href="#footnote-3"><sup>3</sup></a></li>
<li>Once they’ve overblown the severity of an event, they shuttle in a small group of “experts” to help you process the significance of the event. Unsurprisingly, most of these “experts” end up saying, “We’ll have to wait and see.”</li>
<li>They obsess over pointless details to human interest stories—what kidnap victims were wearing, who were the mass shooter’s parents, how many tacos Joe Biden ate yesterday, etc.</li>
<li>They aim to entertain rather than inform. CNN now has entire segments where they simply show popular YouTube videos and then laugh about them.</li>
</ul>
<p>These strategies produce the net effect of causing the consumer to <em>feel</em> like they’re being informed, when actually, we’re getting highly skewed perspectives on (mostly) irrelevant events that have been blown out of proportion to get you emotionally worked up and upset. So of course you tune in again tomorrow to see what happened next!</p>
<figure><figcaption>This is why we can’t have nice things…</figcaption></figure>
<p>Now, these complaints have existed for decades. You’ve probably heard them before. And you probably justify them as necessary evils. I mean, you’ve got to wrap the important stuff in some gruesome entertainment, right? It’s the sugar that helps the medicine go down.</p>
<p>But I disagree. Not only is it not necessary, but this style of <a href="https://markmanson.net/the-life-cycle-of-outrage">outrage porn</a> harms us and our culture.</p>
<ol>
<li><strong>Recurring Trauma</strong> – Studies find that many people who view catastrophic events (terrorist attacks, mass shootings, natural disasters) repeated over and over again on television can develop PTSD-like symptoms, even though they had no direct experience of the catastrophe.<a href="#footnote-4"><sup>4</sup></a></li>
<li><strong>Agenda Setting</strong> – When the news media becomes obsessed with a topic and shows it repeatedly, people begin to believe that topic is important, whether it actually is or not (hello, <a href="https://en.wikipedia.org/wiki/Clinton%E2%80%93Lewinsky_scandal">Monica Lewinsky</a>).<a href="#footnote-5"><sup>5</sup></a> Similarly, negative news coverage causes people to overestimate problems, believing they’re far more widespread than they actually are. What’s worse, this seems to only be true for negative news. Positive news does not cause the same biases in our thinking.<a href="#footnote-6"><sup>6</sup></a></li>
<li><strong>Cultivation Theory</strong> – Studies report that people who watch more news tend to overestimate the amount of crime and violence that happens in the real world. They also generally distrust their fellow citizens more often, becoming paranoid that everyone is out to get them.<a href="#footnote-7"><sup>7</sup></a></li>
<li><strong>Misinformation and Propaganda</strong> – People who watch partisan cable news (e.g., Fox News and MSNBC in the United States) <a href="https://www.poynter.org/reporting-editing/2012/survey-nprs-listeners-best-informed-fox-news-viewers-worst-informed/">are actually worse-informed</a> than someone who watches <em>no</em> news. Large swaths of the news industry are actively misinforming people.</li>
<li><strong>Stress and Anxiety</strong> – News consumption harms our mental health. Consuming news generates a greater sense of pessimism about not just the world, but your own life.<a href="#footnote-8"><sup>8</sup></a> It also increases stress and symptoms of generalized <a href="https://markmanson.net/5-books-for-dealing-with-anxiety-and-depression">anxiety</a>.<a href="#footnote-9"><sup>9</sup></a></li>
</ol>
<p>To sum up: we have highly emotive content of little relevance and utility promoting skewed and inaccurate perceptions of the world, of other people, and of ourselves. It generates stress and anxiety, causes greater distrust in others, and can actually make us less informed about the world. Oh, and it runs 24/7, on every social feed, every homepage, in every airport, every hotel, all day, every day, forever and ever…</p>
<p>Gag me with a fucking Flintstones spoon.</p>
<figure><figcaption>It shall be an honorable death…</figcaption></figure>
<p>Yet, news still has this reputation as something that <em>needs</em> to be watched. Like the proverbial veggies for our cultural diet. The broccoli that allows us to enjoy the dessert.</p>
<p>Bullshit.</p>
<p>Last summer, I wrote a long article called <a href="https://markmanson.net/how-to-improve-focus">The Attention Diet</a>. In that post, I argued that the most important personal challenge we face as individuals today is to learn how to manage and care for our own attention. To take our attentional health seriously the same way we would care for our body (or not). This is especially relevant, because in the age of Twitter diplomacy, pretty much <em>everything</em> is designed to <a href="https://markmanson.net/attention-economy">hijack our attention</a> and trigger <a href="https://markmanson.net/how-to-be-patient">compulsive and repetitive behaviors in consumers</a>.<a href="#footnote-10"><sup>10</sup></a> When I wrote that article, I lumped news media in as one of many culprits. Just as I decided to cut most of the video games, social media, and funny cat memes from my life, I lumped news into that category as something I probably spent too much time on.</p>
<p>I didn’t think much of it. I’ve read the news daily my whole life. The second half of 2019 was the first time I had ever experimented with cutting it out of my life—and it also happened to be peak cultural insanity to boot.<a href="#footnote-11"><sup>11</sup></a></p>
<p>At first, my friends thought I was a bit crazy. People challenged me saying I had a moral duty to follow the news. That the news is what makes democracy function (man, whoever sold this line to the world surely made a killing). That I would be left out of conversations. That I wouldn’t understand what was going on.</p>
<p>A few months went by… and something funny began to happen. Yes, I became less stressed and anxious. I kind of expected that. I also became more <a href="https://markmanson.net/how-to-be-more-productive">productive</a> (which was nice). But something <em>very</em> unexpected happened.</p>
<p>I became optimistic.</p>
<p>For the first time since I can remember, I actually think the world is doing all right. Better than all right: great.</p>
<p>Now, you might think, “Well, ignorance is bliss, you <em>clearly</em> haven’t been paying attention.” But, no, actually… I have.</p>
<p>I don’t believe I’m less informed than I was before. In fact, oddly, I believe I’m <em>more</em> informed. Now, when events occur and I hear about them through friends, I’m able to situate them in their proper historical and global context. I’m not swayed by the mood of the moment. I don’t freak out when Donald Trump says or does something abominable—we’ve had many, many shithead presidents. And we’ve survived.</p>
<p>Instead of feeling out of the loop, I’ve begun to feel like the only one <em>in the loop</em>. Instead of my friends explaining the world to me, I often find myself in the position of talking them off the ledge, explaining that no, this isn’t new or unique; yes, these things happen all the time; no, we’re not going to war; yes, it sucks—but we’ve survived worse. Somehow, despite not reading dozens of op-ed screeds or watching hours of talking heads bobble about the crisis <em>du jour</em>, I’ve found myself <em>more informed</em>. Calmer. More level-headed… than almost anyone I know.</p>
<p>A lot has been made about the harmful effects of social media in the past decade. People have claimed that it was causing anxiety, depression, and suicides.<a href="#footnote-12"><sup>12</sup></a> But study after study has come out showing that social media is probably not the massive danger we all think it is.<a href="#footnote-13"><sup>13</sup></a> And, in fact, when used well, social media can actually increase wellbeing.<a href="#footnote-14"><sup>14</sup></a></p>
<p>I’ve become convinced that we’ve mistaken the dangers of social media for the dangers of news consumption. After all, most of us consume our news through social media, so it’s easy to forget that they aren’t the same thing.<a href="#footnote-15"><sup>15</sup></a> Over and over, I’ve looked into the social media research to find nothing. But when I look at the research on news consumption, there it is.<a href="#footnote-16"><sup>16</sup></a></p>
<p>News is not the proverbial vegetables of our social body. News is like a bacon cheeseburger with two donuts for buns.</p>
<figure><figcaption>The culinary equivalent of a full day of cable news coverage.</figcaption></figure>
<p>And while that perfect mixture of salty and sweet sure does taste good going down, it makes us sick and nauseous later. It’s an information nutritional nightmare. And worse, it’s addictive.<a href="#footnote-17"><sup>17</sup></a></p>
<p>But has it always been this way? After all, we used to eat more vegetables a few generations ago. And nobody had even heard of a donut bacon cheeseburger. Is it the same for news?</p>
<p>The rest of this article is going to look at two things:</p>
<ol>
<li>How has news become the cultural trainwreck that it’s become?</li>
<li>What can we replace it with to be a happier, healthier, more functional citizen body?</li>
</ol>
<p>The next few sections are going to get philosophical. We’re going to look at questions about how information spreads through human societies, how technology affects the ease with which information passes through networks, and how the internet scrambled those networks like a basket of cheap eggs.</p>
</div>
<div>
<p>Thought experiment: let’s say it’s 5,000 years ago and you live in a little village with a few hundred other smelly, hairy humans. What’s the best way to spread an important piece of information so that everyone knows it, as quickly as possible?</p>
<p>One idea is to stand in the middle of the town square and scream whatever you need to say a few dozen times. It’d be annoying. And your vocal cords would get sore. But it’d probably be effective—it’d get the news out, that’s for sure. Let’s call this the <strong>proclamation</strong> form of information-sharing. It starts with one central person announcing information to many, many people.</p>
<p>But that’s not the only way to get the news out. Another would be to walk up to a few people individually, tell them the news, and then kindly ask them to pass it on to others. Ideally, if the news is juicy and exciting enough, it will spread from person to person, like syphilis in a brothel, until everyone is, err… infected… What I mean is that the information would organically work its way through the networks of people until everyone knows it. Let’s call this the <strong>hearsay</strong> form of information-sharing.<a href="#footnote-18"><sup>18</sup></a></p>
<p>Proclamations can be pretty effective. But not anybody can just get up and start screaming whatever they want. Generally, throughout history, you needed the permission of the king or the lord or whatever. What’s more, generally, if you’re going to stand in the town square screaming something, you can only scream something that would make Mr. King happy. Therefore, we can say that proclamations depend on <em>appealing to the elites</em>.</p>
<p>Hearsay is also effective. You just talk some shit and see if it spreads. The hardest part about hearsay is sharing something exciting enough to get other people to talk about it. Discussing the recent ratio of rice-to-wheat harvest is unlikely to get many people chatting in the local tavern. But mention that so-and-so’s <a href="https://markmanson.net/why-people-cheat">cheating</a> on you-know-who, and pretty soon nobody can shut the fuck up about it. Therefore, we can say that hearsay depends on <em>appealing to the masses</em>. Or, at least, appealing to specific sub-groups.</p>
<p>It turns out, these two methods—proclamations and hearsay—are how information has been disseminated throughout human history. You’d be chilling in the local tavern having some mead, and merchants would arrive with news from the neighboring city to trade for some news from where you were. Networks of kin and neighbors traded info about other farmers—who was marriage-material, who was screwing up the harvest, and, of course, word of marauders, bandits and the like.</p>
<p>Hearsay is information that is spread across <strong>decentralized networks</strong> of individuals. Your crop fails because of some new fungus. You tell your neighbor. Your neighbor tells the local merchant. The merchant tells the buyers in the next town over: “No wheat this year. Tough titties.” And the information continues from person to person until it saturates the network of interested people (or it runs out of connections).</p>
<p>Nobody is completely independent. People clump themselves into groups and tribes. So decentralized networks depend on these clusters of people to spread information. If you want to spread the news, you don’t just tell a random Joe Schmoe on the street. You find the leaders and heads of communities, people who are networked and connected. Decentralized networks depend on humans clustering themselves into groups to receive and transmit info. This fact will be important for later.</p>
<p>But then, let’s say, someone pisses in the king’s cornflakes and he decides that everyone needs to hear about it. So, the king would send someone down to the town square and this person would stand there and scream what the king wanted everyone to know until his eyes bled and everyone begged him to shut up. These people were literally known as “town criers” and believe it or not, they held a lot of prestige for much of human history. They were the <a href="https://en.wikipedia.org/wiki/Walter_Cronkite">OG Walter Cronkite</a> and <a href="https://en.wikipedia.org/wiki/Edward_R._Murrow">Edward R. Murrow</a>. When they showed up in town, people knew something serious was brewing. Town criers were rare but major events. The proclamation of a war, a new tax, a royal wedding. It was a big deal.</p>
<p></p>
<p>Proclamations are information that spread through <strong>centralized networks</strong>. It’s info that comes down from the top.</p>
<p>Centralized and decentralized networks still spread information today. In fact, not much has changed. People talk shit in their little clusters on social media, saying the prime minister is a weenie, then Mr. Prime Minister does a press conference and everyone shuts their traps and listens.</p>
<p>Other times, decentralized networks overrule centralized networks of authority. Kings and emperors are overrun by the disaffected masses. Civil rights and social justice movements organically emerge from overlapping groups with no central leader and change major policies. Both centralized and decentralized networks possess power. Both proclamations and hearsay spread information through society simultaneously and in their own ways.</p>
</div>
<div>
<p>Throughout history, some technological innovations have made proclamations (centralized networks) more effective ways of transmitting information. Like, whoever invented the first bullhorn suddenly made their town crier like three times more effective. That, in turn, would make whoever has control of screaming in the town square more influential and powerful than before.</p>
<p>Similarly, whoever invented the bulletin board increased the power of decentralized networks. Now you walk into the local tavern and read what dozens of people needed and cared about without having to find them and talk to them. This means whoever is best able to write effective notes on the local bulletin board would gain power.<a href="#footnote-19"><sup>19</sup></a></p>
<p>You could almost look at world history as a tug-o-war between centralized networks and decentralized networks, based on whatever the predominant communications technology was at the time. When centralized networks are prominent, power tends to be concentrated and autocratic, centered on the few people who have access to the bullhorn. But when decentralized networks are bolstered, it’s all about what groups can organize themselves and inspire followers. Centralized networks produce the tyrants and decentralized networks produce the revolutions that overthrow them and reset the game. And on goes the seesaw of human history.</p>
<p>Below are some of the most impactful technological developments and how they affected centralized and decentralized networks of information. As you can probably guess, we’re going to end up at the internet.</p>
<p><strong>Invented:</strong> Around 3,200 BCE<br />
<strong>Promoted:</strong> Centralized networks (Proclamations)</p>
<p>Writing was the first and most important technological innovation in the world because it allowed knowledge to be shared between people who never met each other—including people from past generations with future generations.</p>
<p>On the surface, this sounds like it would promote decentralized networks. Everyone would start writing each other notes, chiseling their own clay tablets and handing them out like candy at a Thanksgiving Day parade.</p>
<p>But that’s not what happened. Back in the day, writing was expensive. Clay tablets, man! Shit ain’t cheap. On top of that, only the elites were educated enough to read and write, therefore, writing mostly served as a way to create proclamations given by the king.</p>
<p>It’s no coincidence that it’s around this point in human history that empires began to emerge. The written word allowed kings to coordinate their subjects over vast distances. It allowed them to unify millions of disparate peoples under a common code of laws and ethics.<a href="#footnote-20"><sup>20</sup></a></p>
<p>Later, the centralized control of information would grant the great religions dominion over most of the planet. Revolutions were minimal, if non-existent. The vast majority of the population couldn’t read or write, much less coordinate themselves across wide areas. Unless you were highly educated <em>and</em> highly connected, you weren’t able to write anything of importance or usefulness. And if you were highly educated and connected, you were part of the elite anyway, so why cause trouble? Censorship was absolute. Saying the wrong thing could get you killed. Everything went through your king or priest, no questions asked.</p>
<p>This period was no fun.</p>
<p><strong>Invented:</strong> 1440 CE<br />
<strong>Promoted:</strong> Decentralized networks (Hearsay)</p>
<p>It’s a sad irony that the invention most likely responsible for creating the modern world resulted in the inventor going broke, being exiled, and dying a disgraced nobody. Johannes Gutenberg created the printing press, a machine that could mass-produce books and pamphlets affordably. What, at first, seemed like a convenient way to reproduce bibles and some poetry for a quick buck, would soon revolutionize the world.</p>
<figure><figcaption>The next time you think *you* were under-appreciated, just think of Johannes Gutenberg.</figcaption></figure>
<p>The reason writing initially favored centralized networks was because it was expensive, both in terms of labor and money. Teams of monks would spend months copying bibles by hand. Only kings, emperors, and popes had access to scribes to write out hundreds of copies of whatever they wanted to say.</p>
<p>But the printing press changed all that. Now anybody could approach a printer and have their ideas reproduced. Whereas the written word expanded the reach of the centralized networks, the printing press would similarly expand the reach of decentralized networks. And this expansion would shift the balance of power in the world.</p>
<p>That shift didn’t take long. In 1517, Martin Luther published his “<a href="https://www.history.com/topics/reformation/martin-luther-and-the-95-theses">95 Theses</a>” criticizing the Catholic Church and launching the Reformation. For centuries, the Catholic Church had murdered any dissenters like Luther. But Luther had reprinted hundreds of copies of his theses. Anyone could pick up and read them. Killing Luther was no longer sufficient. You’d have to kill all his followers. Thus began war.</p>
<p>Luther’s <a href="https://en.wikipedia.org/wiki/Reformation">Reformation</a> would eventually spark over a hundred years of bloodshed across the European continent. It would inspire monarchs to distance themselves from the church. And it would help inspire small, holy enclaves to seek religious freedom in the New World.</p>
<p>But the printing press wasn’t done. By the 1600s, independent scholars were publishing their own work and promoting a new idea called “<a href="https://en.wikipedia.org/wiki/Scientific_method">the scientific method</a>.” The scientific method was decentralized—anyone could use it and arrive at their own conclusions. Ideas about inalienable human rights, the rule of law, and democracy would take hold. The ease of publishing and promoting work across decentralized networks allowed the Enlightenment to flourish.</p>
<p>The first newspapers emerged around the same time,<a href="#footnote-21"><sup>21</sup></a> independently determining important information and disseminating it through local merchant networks, circumventing the king’s proclamations and town criers. This group of publishers was seen as an ally of the new Enlightenment ideas, as they helped coordinate social movements in the 17th and 18th centuries that would lead to democratic revolutions. The American Revolution relied heavily on the printing and dissemination of political pamphlets.<a href="#footnote-22"><sup>22</sup></a> Similarly, revolutionaries in France coordinated across the country and, for the first time, realized they were getting royally screwed by the king (pun totally intended). As Robert Darnton, a historian from Princeton put it, “Without the press, the French Revolution was impossible and unthinkable.”<a href="#footnote-23"><sup>23</sup></a></p>
<p>That said, news media in this era was also a shitshow. Fake news proliferated. There were practically as many newspapers as there were people. And you think people are polarized today? Hoo boy… There were <em>stabbings</em> in Congress. Politicians shot at each other. Their supporters bombed and lynched and killed each other over nothing but rumors and insults… or rather, that is, hearsay.</p>
<p><strong>Invented:</strong> 1895 CE (radio); 1925 CE (television)<br />
<strong>Promoted:</strong> Centralized networks (Proclamations)</p>
<p>I lumped radio and television together here because they both occurred at similar times and the same companies who started radio stations ended up also dominating television.</p>
<p>Radio/TV once again favored centralized networks for two reasons:</p>
<ol>
<li>Like writing in the ancient world, running a television/radio station was prohibitively expensive and difficult and only the elites would have access to the resources to do so.</li>
<li>These technologies allowed someone to hit millions and millions of people with the exact same information simultaneously. It was as if a mutant town crier with a hundred vocal cords could scream for thousands of miles in every direction.</li>
</ol>
<p>The 20th century presented a hard return of centralized power: it was peak colonialism,<a href="#footnote-24"><sup>24</sup></a> the rise of totalitarian governments (of which <a href="https://en.wikipedia.org/wiki/Politics_of_North_Korea">North Korea</a> is still a holdout), two world wars and that whole ugly <a href="https://en.wikipedia.org/wiki/Nazi_Germany">Nazi thing</a>.</p>
<p>Radio and television launched the industries of advertising and public relations. When millions of eyeballs are watching the same exact thing, this grants a huge opportunity for businesses (and governments) to <a href="https://markmanson.net/insecurity">persuade people to think or feel a certain way</a>. This is where the ability to manipulate and influence people into certain opinions or desires became formalized into the practice of propaganda, or as it’s known today, marketing.<a href="#footnote-25"><sup>25</sup></a></p>
<figure><figcaption>The Nazis were one of the first governments to master the art of radio propaganda.</figcaption></figure>
<p>Radio and television likely created the most cultural solidarity we have ever seen in human history. Everyone was watching the same shows, listening to the same music and following the same events. The wars of the 20th century pitted half the planet against the other and everyone held on for their dear life. From World War I, up through the end of the Cold War, there was always relative political conformity and trust, especially because no matter what country you were in, you were likely united against some large external enemy.</p>
<p><strong>Invented:</strong> 1983 CE<br />
<strong>Promoted:</strong> Decentralized networks (Hearsay)</p>
<p>The originators of the internet saw it as a great liberating force on humanity.<a href="#footnote-26"><sup>26</sup></a> At the time, they lived with the media gatekeepers of the television and radio stations, limiting the vast majority of the population’s ideas from being widely expressed.</p>
<p>But now, with the power of networked computers, <em>anybody</em> could get online, start a website, and start spitballing their bullshit into the wind, hoping somebody would stop awhile and listen.</p>
<p>Whereas TV and radio expanded the reach of centralized networks to worldwide audiences, the internet would expand the reach of decentralized networks worldwide. A professor in Canada could exchange emails with a student in India and come up with an idea that would be tested in China and written up in newspapers in Australia.</p>
<figure><figcaption>Obligatory cat meme goes here…</figcaption></figure>
<p>It sounded like heaven. And while the internet has granted us incredible amounts of information and perspective, it hasn’t quite lived up to its utopian expectations. In fact, public discourse has kind of gone to shit. And it’s done so for a few reasons:</p>
<p>First, the old guard of media has faced the economic realities: for better or worse, pretty much any kid with a computer can compete with them now. This caused revenue to plummet and led them to cut large chunks of their staff—most notably, investigative journalists and foreign correspondents, the people who made them reliable and trustworthy in the first place.<a href="#footnote-27"><sup>27</sup></a> Expensive news that took years of study and months to report was no longer economical. So the traditional media giants began to look more and more like their amateur competitors, while their amateur competitors leveraged marketing strategies to make themselves appear far more reliable than they actually were.</p>
<p>Second, because <a href="https://markmanson.net/smartphones">everyone’s phone is on them all the time</a> and we’re all compulsively checking social media and email, emotionally impactful information (regardless of its truth or utility) spreads extremely fast. Thus, it’s easier than ever to make a quick buck by indulging in the worst human tendencies.  You want to net a bunch of traffic by cooking up a fake pedophile ring in a pizza shop?<a href="#footnote-28"><sup>28</sup></a> Hell yeah, go for it!</p>
<p>Third, by giving voice to <em>everybody,</em> the internet unintentionally gave voice back to the uglier sub-cultures of society that were once barred from any mainstream attention: the bigots, the conspiracy theorists, and the cranks. In our idealism of giving the whole world a way to express themselves, <em>we forgot that most people say really awful shit. </em></p>
<p>The internet is like the modern printing press.<a href="#footnote-29"><sup>29</sup></a> It allows anyone to express themselves, challenge authority, and share information on informal networks outside of the traditional pipelines. On the one hand, this is great. It grants greater freedom of expression and the opportunity for self-actualization to everyone. On the other hand, it’s a fucking mess, because as a society, we don’t know who to listen to or what to believe anymore.</p>
<p>Like the period after the printing press, we are seeing more explicit challenges to the establishment. We are seeing a resurgence of tribal and religious affiliations. We are seeing emerging conflicts over the identity and direction of country and culture. The times are tense. The internet has complicated… well, everything. But if we’re willing to look hard enough, we can spot iridescent flashes of collective brilliance breaking through an otherwise endless stream of bullshit.</p>
<div>
<h3>To Sum Up</h3>
<ul>
<li>When centralized networks of information have the advantage, we get smaller quantities of highly relevant information. The danger of these networks of information is that they can easily be corrupted for tyrannical purposes like the Catholic Church in the Dark Ages or the totalitarian governments of the 20th century.</li>
<li>Decentralized networks promote higher quantities of low-quality information. The quantity of information generally fragments the population into warring factions based on tribal or religious identities. They only trust their preferred messengers and distrust everybody else.</li>
<li>Whereas centralized information networks promote conflict <em>between</em> political bodies, decentralized information networks promote conflict <em>within</em> political bodies.</li>
<li>Where centralized information networks solidify and reinforce cultural identity, decentralized information networks tend to overthrow and revolutionize cultural identities. Sometimes these revolutions are huge leaps forward for humanity (Enlightenment, civil rights, etc.) but other times, it’s just a bunch of arbitrary, religious-style fighting.</li>
<li>Decentralized information networks create a much larger diversity of information, but the signal-to-noise ratio is poor, making it incumbent on each individual to navigate the information and carefully choose what is worth consuming and what should be ignored.</li>
<li>To put it into plain English: the frustrations we have with news media—the negativity, the lies, the partisan bickery—it’s not going to change. It’s not going to get better. It’s on us to learn how to navigate the media environment and vote with our eyeballs. To do that, we must understand what information is actually useful and important, and what is just nonsense.</li>
</ul>
</div>
</div>
<div>
<p>OK, let’s state the obvious. Some information is important, some is not. Some information <em>feels</em> important while being totally unimportant. And some information <em>feels</em> unimportant while it’s actually quite crucial.</p>
<p><strong>The most succinct way to sum up the issue today is that news media optimizes for information that</strong> <strong><em>feels</em></strong> <strong>important with little regard to its actual importance. </strong></p>
<p>If we look back through history, this is a common state of eras where information networks are highly decentralized. What travels is what <em>feels</em> impactful, even if it’s not—and even if it’s not true! Because the news has to compete in a highly diverse market with tons of noise, they must be ruthless in capturing people’s <a href="https://markmanson.net/understanding-your-emotions">emotions</a> as effectively as possible.</p>
<p>Eras of centralized information networks have had the opposite problem: they share information that seems unimportant but is actually quite important. Like the Soviet announcements of <a href="https://en.wikipedia.org/wiki/Rationing_in_the_Soviet_Union">monthly bread rations</a> or the long, tedious Congressional committees that drone on <a href="https://www.c-span.org/">C-SPAN</a> in the United States every day, these forms of news are dull and boring. But they matter.</p>
<p>Back in the 1950s, when there were only two TV channels, they could show a 12-hour congressional committee and you really had no option. That’s what was important. That’s what people needed to know. You lived with it whether you cared or not. But as cable news and the internet emerged, the incentives for media companies pushed them more into fighting for people’s hearts rather than their minds. They profited off having the most emotional information, not the most factual.</p>
<p>Today, if someone tries to show a congressional committee, the next channel is going to bury them with pictures of dead children in Sudan and plane crashes in Iran. There’s no chance.</p>
<p>This is why we’ve all experienced the “decline” of news media over the past 30 years: we’ve lived through the slow escalation of emotional impact at the cost of veracity and utility. On the one hand, we have more choice of information than ever before. But the average quality of that information is far lower. It’s simply the trade-off that comes with the technology.<a href="#footnote-30"><sup>30</sup></a></p>
<p>Useful macro information is information that directly impacts you and/or has an outsized influence on the economy. Prescription drug price policy is important information. The safety rating of your car is important information. A tariff on sugar is important information.</p>
<p>These things are boring, but they are important because they are the first-order effects that drive almost everything else. As the old saying goes, “It’s the economy, stupid.” Cultural issues usually happen downstream.<a href="#footnote-31"><sup>31</sup></a></p>
<p>Almost everything else is hogwash. Terrorist attacks? School shooters? Neither of these would probably be such a problem if it weren’t for the news coverage in the first place. Trump said something stupid? So what? Wake me up when there’s another $350 billion tariff.</p>
<p>When you focus on first principles, the news becomes incredibly empty. I’d say 80-90% of the content out there is either: a) catastrophizing some irrelevant event, b) horse race updates when all you need to know is who won, or c) opinion and commentary dressed up as news.</p>
<p>You can skip all this stuff. You don’t care about the events. You don’t need to know about the horse race, just the winner. You don’t need more opinions—you need data.</p>
</div>
<div>
<p>There are certain biases within our minds that we all fall victim to and the news media leverages them to keep us engaged and wanting more. There’s the <a href="https://markmanson.net/cognitive-biases-that-hurt-relationships#negativity-bias">negativity bias</a>, a well-documented effect where people pay more attention and have more intense emotional responses to negative events than positive events.<a href="#footnote-32"><sup>32</sup></a> There’s the <a href="https://en.wikipedia.org/wiki/Present_bias">present bias</a>, the tendency to put more weight and importance on events that have happened recently, as opposed to far in the past or the future. There’s the <a href="https://www.psychologytoday.com/us/blog/science-choice/201612/what-is-narrative-bias">narrative bias</a>, the desire to arrange information in terms of a coherent story structure (i.e., “The stock market dropped 2% today because it rained in Michigan”) even if it’s not true. And finally, there is the almighty <a href="https://markmanson.net/cognitive-biases-that-hurt-relationships#confirmation-bias">confirmation bias</a>, the propensity to arrange new information so that it agrees with our previously-held beliefs.</p>
<p><strong>Chances are if you think about the news you’ve consumed lately, pretty much all of it is: negative, recent, story-driven, and supports your previously held beliefs.</strong></p>
<p>News media leverages our biases to keep us hooked and consuming more information, regardless of its utility. Especially in our decentralized world of the internet, <a href="https://markmanson.net/attention-economy">the name of the game is attention</a>, and the news media will use every tool to leverage more of yours. As such, news media specializes in information that is immediate, fast-moving, narrative-driven and highly visible.<a href="#footnote-33"><sup>33</sup></a></p>
<p>The problem is that the most important information is usually long-term, slow-moving, impersonal, abstract and invisible, and not necessarily negative:</p>
<ul>
<li>Car accidents kill more people than terrorism, mass shootings, and natural disasters combined. How much reporting do you see on car accidents or driver safety?</li>
<li>The <a href="https://en.wikipedia.org/wiki/Human_Genome_Project">sequencing of the human genome</a> was arguably more important than any other scientific development in the 1990s, opening a path to predicting disease and conditions before they occur, altering and improving the health of millions, if not billions, of people over the coming generations. How many weekly updates did you hear about it?</li>
<li>More than 1 billion people have been lifted out of extreme poverty worldwide,<a href="#footnote-34"><sup>34</sup></a> due to a combination of greater access to health care, sanitation, and the opening of markets for greater commerce. Who knew?</li>
<li>Despite the fact the first international climate agreement was signed in 1992 by 154 countries, news media has just only recently started covering climate change with any significance—and if we’re being honest, it’s probably just to go along with all the natural disasters that they’ve been milking.</li>
</ul>
<p>Paradoxically, in its effort to inform wide audiences, news media positions itself to misinform and over-inflate extreme outlier stories.</p>
<p>But this is not the news media’s fault. It’s simply how information scales in large decentralized human networks. The information with the greatest emotional valence will travel the furthest and be broadcast the loudest.<a href="#footnote-35"><sup>35</sup></a> Gone are the days where we have reliable journalistic mainstays who can consistently nail what is most significant. The structures simply aren’t there.</p>
<p>Instead, we must rely upon ourselves. After all, in the internet age, we are the ones who are empowered: the user. We must screen out crap information and consciously seek out useful information. We must develop an awareness of <a href="https://markmanson.net/attention-economy">how our attention is being seduced</a> and prevent ourselves from getting sucked into the social media vortex of never-ending comment threads.</p>
<p>For the first time in human history, our relationship with news <em>depends on us. </em></p>
</div>
<div>
<p>Here’s the short version: Seek out the highest-value content you can find, and then pay for it. Limit your consumption to a few pieces of content per week. Go for quality over quantity.</p>
<p>By high-value content, ask yourself how much that content cost in terms of the labor, research, expertise, production, etc. Long-form content is more expensive than short-form. Research and citations are more difficult to put together than just some old dude ranting. Something that explains the historical context and acknowledges multiple sides of an issue is more difficult and time-consuming than a partisan rant.</p>
<p>I focus on the expensive thing because right now, the news media industry is going through a slow-moving crisis of their own design (and ironically, it’s too slow-moving for them to report on it). Research, fact-checking, traveling to some crazy war-torn country, and not getting your shit blown up—these are all expensive. Hiring a journalist with decades of knowledge on a region is expensive. Interviewing professors and digging into research is expensive.<a href="#footnote-36"><sup>36</sup></a> Meanwhile, online advertising revenue pays next to nothing. So, more and more, media outlets are subtly and secretly pushing out cheap content that looks expensive—uninformed 24-year-olds who can sorta, kinda sound like an experienced journalist with a lot of Google-fu.</p>
<p>There are only a few publications that have the luxury of not playing the Clickbait Headline Game, and that’s because they charge enough money that they don’t have to. Publications such as <a href="http://www.economist.com/">The Economist</a>,<a href="#footnote-37"><sup>37</sup></a> <a href="http://ft.com/">Financial Times</a> or <a href="http://www.foreignpolicy.com/">Foreign Policy</a> don’t give a shit if their articles are fun or not. They’re not writing for the general public. They’re writing for professionals who depend on the information to make big, expensive decisions. This is where you want to insert yourself into the information pipeline. Long, boring, thoughtful pieces, put together by long, boring, thoughtful people. One good piece of long-form journalism that focuses on real data is worth dozens of crappy short-form pieces you find elsewhere.</p>
<p>When you get into the less expensive publications, you start to run into a lot of bias.<a href="#footnote-38"><sup>38</sup></a> You can feel the articles getting more frantic and the headlines more hyperbolic. Then, once you’re into the cheap (or free) seats, you’re basically being spoon-fed what to think and feel.</p>
<p><strong>The point to remember is that you want your news to be a little bit dull. If you’re getting excited or worked up reading something, chances are you’re being taken for a ride. </strong></p>
<p>If you can’t afford more expensive news publications, then I recommend skipping news entirely and simply reading books. In terms of quality information per dollar, books are the best bang for your buck. These days, I aim to make books account for at least 80% of my total reading time.</p>
<p>You may worry that by only reading books, you’ll “fall behind.” But I think you’ll actually find the opposite. If you take the hours and minutes that you used to spend reading news and instead read books about the same topics, within a few months, the “news” that everyone is freaking out about will seem trifling to you. You will see how slowly the world truly moves. You will have more context and clarity than most and you will recognize that<em> almost nothing</em> needs to be known the moment it happens.</p>
<p>Finally, I’d be remiss if I didn’t mention blogs and podcasts. Blogs/podcasts are interesting in that they don’t succumb to the same economic realities of mainstream news media—they don’t demand huge overhead expenses to maintain, and they rely on the personal trust of their readers to continue to exist. Also, most prominent bloggers and podcasters gained their expertise elsewhere, therefore allowing them to offer high-value information at a huge discount.</p>
<p>Put simply: most blogs and podcasts suck. But the best ones are arguably better than any other content on the planet right now.<a href="#footnote-39"><sup>39</sup></a></p>
<p>If you can find one of these experts, they can often be a gold mine of non-partisan, on-the-ground information. Bloggers such as <a href="http://waitbutwhy.com/">Tim Urban</a>, <a href="http://stratechery.com/">Ben Thompson</a>, <a href="http://marginalrevolution.com/">Tyler Cowen</a>, <a href="http://slatestarcodex.com/">Scott Alexander</a>, and <a href="http://danwang.co/">Dan Wang</a> provide a ton of value for little to no cost at all.</p>
</div>
<div>
<p>But perhaps more important than finding sources of high-quality information is <em>cutting out</em> the sources of low-quality information. And there is <em>a lot</em> of low-quality information out there. If you get nothing else from this behemoth of an article, please consider cutting the following from your information diet entirely:</p>
<ul>
<li><strong>STOP Watching News on Television – </strong>In surveys, cable news is consistently the least informative and most biased source of information. The nature of television demands constant stimulation and therefore undermines pretty much any serious discussion of a topic.<strong><br />
</strong></li>
</ul>
<ul>
<li><strong>STOP Following News Sites on Social Media</strong> – Trust me, if something is important enough, you’ll hear about it anyway. People will link to it. They’ll email you. Whatever. When you follow news sites on social media, you are opting in to the attention game. You are allowing yourself to be triggered and fought over by egregious headlines, <a href="https://markmanson.net/the-life-cycle-of-outrage">moral outrage</a>, and clickbait. Just don’t play it. Unfollow all news sources on your social media. Not only will it make your news experience better, but it will make your social media experience a hundred times better.</li>
</ul>
<ul>
<li><strong>STOP Clicking on Clickbait</strong> – I know it’s easier said than done. But seriously, get into the habit of <em>actively seeking out information</em> instead of passively receiving it on your newsfeeds or apps. The algorithms on these platforms <em>do not</em> work in your favor. Cut them all out and then when you need to know about a subject, go looking for the info yourself, preferably in a book.</li>
</ul>
<ul>
<li><strong>STOP Consuming News That Requires Less than 20 Minutes</strong> – Most short-form news is garbage. If something requires more than 20 minutes to consume, that likely means it took weeks or months (years?) to produce. This means skip the hot takes, the quick interview clips, the editorials, and the breaking news alerts. Instead, download long-form articles, watch documentaries, listen to long-form interviews and podcasts where the person has time to explain themselves and get into the nuance of their positions. The world is complicated. You need time to sort it out.</li>
</ul>
<ul>
<li><strong>STOP Getting All Your Information from One Source</strong> – Make a conscious effort to check both left-wing and right-wing sources. Get info from minorities and women and foreigners. Follow professors and follow really smart kids with too much time on their hands. There’s a lot to be said for diversity. <em>But the real diversity is diversity of thought</em>. The wider the range of ideas you expose yourself to, the less you’ll ever be caught off guard by whatever new event happens.</li>
</ul>
</div>
<div>
<p>I was in Australia over the New Year. As I’m sure you know, there are huge bushfires that have consumed huge stretches of land there, destroying natural habitats, and displacing thousands of people.</p>
<p>We saw the smoke when we were there. We rode a train through the Blue Mountains and saw acres of charred forests. On our first day in Sydney, the pollution level was the same as a small coal-mining Chinese city… i.e., toxic to even be outside. We talked to the locals in Adelaide about the burnt wine country. But all in all, it was like anywhere else I’ve been after a disaster: it’s sad, but life goes on. People are getting on with it.</p>
<p>Then I started getting emails and messages from friends and family back home. “Are you OK? Have you been hurt?” They said, “You should come home! It’s not safe!” I looked up from my phone: sunny skies and beautiful beaches. What in the hell were they going on about?</p>
<p>I was then subjected to what I could only call insanity. News headlines claiming that Australians’ <a href="https://www.wsj.com/articles/australias-catastrophic-fires-will-change-the-way-people-live-11578297077">way of life is forever lost</a>, that <a href="https://www.commondreams.org/news/2019/12/21/everything-burning-australian-inferno-continues-choking-access-cities-across-country">everything is burning</a>, that over <a href="https://www.cnet.com/how-to/australia-fires-have-killed-more-than-a-billion-animals-so-far-how-you-can-help/">a billion animals</a> (an absurd number) have been killed. One article even went so far as to imply that <a href="https://www.theatlantic.com/science/archive/2020/01/only-way-confront-australias-wildfires/604546/">the entire country will soon have to be abandoned</a>.<a href="#footnote-40"><sup>40</sup></a></p>
<figure><figcaption>This map is fake. But that didn’t stop it from being posted on approximately eight billion news sites.</figcaption></figure>
<p>I’m as alarmed as anyone about climate change, but <em>come the fuck on</em>.</p>
<p>Every few months it seems as though there’s something like this. Whether it’s <a href="https://www.nytimes.com/2016/06/25/world/europe/britain-brexit-european-union-referendum.html">Brexit</a> or <a href="https://www.history.com/this-day-in-history/president-trump-impeached-house-of-representatives">Trump’s impeachment</a> or <a href="https://www.bbc.com/news/world-latin-america-53893161">the Amazon burning</a> or <a href="https://www.theatlantic.com/international/archive/2018/08/china-pathologizing-uighur-muslims-mental-illness/568525/">China’s internment of a million Uighurs</a>—we live in the Age of Hyperventilation.</p>
<p>But while there’s an endless stream of tragedy in the world, there is so much that the news never teaches you or shows you.</p>
<p>The news doesn’t show you how bad humans are at predicting the future. How pretty much every projection, scientific or not, ends up incredibly inaccurate. It doesn’t tell you that we’ve been predicting civilization-leveling catastrophes since pretty much the dawn of human thought—and every time, we have been wrong.</p>
<p>The news doesn’t teach you that technology doesn’t develop linearly—how it comes in unexpected leaps, and then trounces prior assumptions under the weight of its efficiency.</p>
<p>The news doesn’t tell you that natural disasters are just that: natural. Yes, they may be occurring slightly more often and be more severe, but they are, and have always been, the rule, not the exception.</p>
<p>The news doesn’t show that the vast majority of people are good. They will help if they can. They care even if they’re confused about how to care or why. The news doesn’t teach you that most people won’t hurt you and even if they do, <a href="https://markmanson.net/how-to-grow-from-your-pain">you will recover and be fine and be stronger than before</a>.</p>
<p>But most of all, the news doesn’t teach you that moral outrage, while strangely satisfying, gives people a sense of accomplishing something without actually accomplishing anything. Posting on social media or writing a scathing op-ed may <em>feel</em> as though it’s contributing to some grand movement, some great push for humanity, but that push is, more often than not, grounded in nothing.</p>
<p>Nowadays, I briefly check a handful of news sites, maybe once a week.<a href="#footnote-41"><sup>41</sup></a> I read maybe five news articles in total. They are articles I choose, based on my own perception of what’s important and what matters. The rest is books, blogs, and podcasts.</p>
<p>Months ago, my friends felt sorry for me. They worried that I’d be out of touch, that I’d lose my connection to society and the world, and that I’d become disempowered to change it.</p>
<p>But the opposite has happened. I have realized that it is <em>they</em> who are out of touch, disconnected, and disempowered.</p>
<p>And this is the ultimate failure of the news media as currently constructed: it creates a passivity, a false feeling that we’re not somehow part of the narrative that we are also watching unfold. It lulls us into thinking that we don’t—that we <em>can’t—</em>have some role to play in the drama. It convinces us that our obstacles are too great, that our fears are too deep, that we are incapable of overcoming the same challenges of the thousands of generations who have come before us.</p>
<p>And this is what was lost. The town crier was never meant as a mere announcer. He was an instigator. He was a call to arms to leave our domestic safety and take up the business of the world that is ours. He reminded us to not simply receive the news but to go out and be the news as well.</p>
<p>What news will you be?</p>
</div>
	</div>||||I|||| Skip to content

 Menu

 * Articles
 * Books
 * Courses
 * Newsletter
 * Log In
 Log In Sign Up

 Become a member

 Get a single membership for just $6.67 per month

 Sign Up

 Why You Should Quit the News

 Written By Mark Manson – filed under Decision Making | Focus | Happiness

 E veryone kind of already knows that the news sucks. In all my life, I can’t think of anyone who seems to enjoy reading or watching the news every day. It’s a kind of bitter responsibility or endured necessity for people. News is like the societal version of flossing: it’s not fun, yet we continue to do it every day anyway, as a sort of obligation to prevent decay of the social order.

 We all get it: yes, the news is overly negative, and yes, it often gets some things wrong. But in the end, we’re better off for it. That’s how the argument usually goes. Some even say we need the news, that democracy is impossible without it.

 In this article, I’m going to challenge that idea. In fact, I’m going to argue that the news doesn’t just seem to be horrible, in its current form it is horrible. It is actively damaging our culture. And most of what we blame on social media or the internet—the upticks in stress, anxiety, pessimism, and polarization—is misplaced. News media is the true culprit.

 Obviously, this is a bold argument. And this article will be quite long. We’ll investigate what makes some information valuable and other information worthless. We’ll look at how networks of information emerge and how news has traditionally traveled throughout human history. We will look at how the internet and social media have affected those traditional networks, and, of course, I’ll take a stab at explaining why our current media environment is a shit sauna.

 Finally, I’ll discuss what I feel are better alternatives for staying informed than consuming news. There will be lots of links.

 So, take a deep breath… maybe grab a stress ball. This is going to take a while.

 Table of Contents

 * Why Is the News So Terrible?
 * Human Networks of Information
 * What Makes Information Useful?
 * How the News Media Hijacks Your Attention
 * What to Do
 * What to NOT Do

 Why Is the News So Terrible?

 Let me ask you this: when was the last time you made a major life decision based on a news story? Or: when was the last time information from a news story directly impacted your life? And I don’t mean, “It made me really, really upset,” impacted. I mean, like, actually impacted you.

 Chances are you can’t remember. That’s because the vast majority of news is irrelevant. Car chases. Bank robberies. Celebrity overdoses. Sports scores. Homeless people throwing feces at one another. None of us ever really have the experience of seeing a news story and immediately thinking, “Man, I need to change what I’m doing.” If it’s ever happened to me, I don’t remember it.

 The news does not tell you what job to take or how to deal with your angry husband or what the best brand of headphones is or what to feed your kids in the morning. It doesn’t help you to be a better friend or explain why you have panic attacks or advise on the best way to deal with an unruly child.

 The news is pretty much useless for helping you determine what to do with your life. You may argue that the news helps you determine who you vote for, but research suggests that even that’s not true—we are far more swayed by our friends, family, and day-to-day circumstances than any information we may find on the news.1 If you want to know a politician’s platform today, you can just go to their website or hear them speak directly on YouTube. You don’t need a talking suit on television or an op-ed section to tell you what to think.

 If we’re being honest, most of the value we get from consuming news is having something to talk about with other people who consume the news. And considering this has a 50% chance of making you hate each other, even that value becomes questionable.

 So, if the news isn’t sharing useful facts or information that is directly affecting millions of people’s lives, what exactly is the news trying to do?

 Well, that’s easy. The goal of the news is to motivate you to keep consuming news.2

 They do this in a few ways:

 * They present emotionally-charged information that feels important, even though it isn’t (a politician making a gaffe, for instance).
 * They catastrophize everything to make you believe this is a once-in-a-lifetime event and nothing will ever be the same ever again… until something else happens. The assassination of Soleimani in Iraq in January 2020 is an example.3
 * Once they’ve overblown the severity of an event, they shuttle in a small group of “experts” to help you process the significance of the event. Unsurprisingly, most of these “experts” end up saying, “We’ll have to wait and see.”
 * They obsess over pointless details to human interest stories—what kidnap victims were wearing, who were the mass shooter’s parents, how many tacos Joe Biden ate yesterday, etc.
 * They aim to entertain rather than inform. CNN now has entire segments where they simply show popular YouTube videos and then laugh about them.

 These strategies produce the net effect of causing the consumer to feel like they’re being informed, when actually, we’re getting highly skewed perspectives on (mostly) irrelevant events that have been blown out of proportion to get you emotionally worked up and upset. So of course you tune in again tomorrow to see what happened next!

 This is why we can’t have nice things…

 Now, these complaints have existed for decades. You’ve probably heard them before. And you probably justify them as necessary evils. I mean, you’ve got to wrap the important stuff in some gruesome entertainment, right? It’s the sugar that helps the medicine go down.

 But I disagree. Not only is it not necessary, but this style of outrage porn harms us and our culture.

 1. Recurring Trauma – Studies find that many people who view catastrophic events (terrorist attacks, mass shootings, natural disasters) repeated over and over again on television can develop PTSD-like symptoms, even though they had no direct experience of the catastrophe.4
 2. Agenda Setting – When the news media becomes obsessed with a topic and shows it repeatedly, people begin to believe that topic is important, whether it actually is or not (hello, Monica Lewinsky).5 Similarly, negative news coverage causes people to overestimate problems, believing they’re far more widespread than they actually are. What’s worse, this seems to only be true for negative news. Positive news does not cause the same biases in our thinking.6
 3. Cultivation Theory – Studies report that people who watch more news tend to overestimate the amount of crime and violence that happens in the real world. They also generally distrust their fellow citizens more often, becoming paranoid that everyone is out to get them.7
 4. Misinformation and Propaganda – People who watch partisan cable news (e.g., Fox News and MSNBC in the United States) are actually worse-informed than someone who watches no news. Large swaths of the news industry are actively misinforming people.
 5. Stress and Anxiety – News consumption harms our mental health. Consuming news generates a greater sense of pessimism about not just the world, but your own life.8 It also increases stress and symptoms of generalized anxiety.9

 To sum up: we have highly emotive content of little relevance and utility promoting skewed and inaccurate perceptions of the world, of other people, and of ourselves. It generates stress and anxiety, causes greater distrust in others, and can actually make us less informed about the world. Oh, and it runs 24/7, on every social feed, every homepage, in every airport, every hotel, all day, every day, forever and ever…

 Gag me with a fucking Flintstones spoon.

 It shall be an honorable death…

 Yet, news still has this reputation as something that needs to be watched. Like the proverbial veggies for our cultural diet. The broccoli that allows us to enjoy the dessert.

 Bullshit.

 Last summer, I wrote a long article called The Attention Diet. In that post, I argued that the most important personal challenge we face as individuals today is to learn how to manage and care for our own attention. To take our attentional health seriously the same way we would care for our body (or not). This is especially relevant, because in the age of Twitter diplomacy, pretty much everything is designed to hijack our attention and trigger compulsive and repetitive behaviors in consumers.10 When I wrote that article, I lumped news media in as one of many culprits. Just as I decided to cut most of the video games, social media, and funny cat memes from my life, I lumped news into that category as something I probably spent too much time on.

 I didn’t think much of it. I’ve read the news daily my whole life. The second half of 2019 was the first time I had ever experimented with cutting it out of my life—and it also happened to be peak cultural insanity to boot.11

 At first, my friends thought I was a bit crazy. People challenged me saying I had a moral duty to follow the news. That the news is what makes democracy function (man, whoever sold this line to the world surely made a killing). That I would be left out of conversations. That I wouldn’t understand what was going on.

 A few months went by… and something funny began to happen. Yes, I became less stressed and anxious. I kind of expected that. I also became more productive (which was nice). But something very unexpected happened.

 I became optimistic.

 For the first time since I can remember, I actually think the world is doing all right. Better than all right: great.

 Now, you might think, “Well, ignorance is bliss, you clearly haven’t been paying attention.” But, no, actually… I have.

 I don’t believe I’m less informed than I was before. In fact, oddly, I believe I’m more informed. Now, when events occur and I hear about them through friends, I’m able to situate them in their proper historical and global context. I’m not swayed by the mood of the moment. I don’t freak out when Donald Trump says or does something abominable—we’ve had many, many shithead presidents. And we’ve survived.

 Instead of feeling out of the loop, I’ve begun to feel like the only one in the loop. Instead of my friends explaining the world to me, I often find myself in the position of talking them off the ledge, explaining that no, this isn’t new or unique; yes, these things happen all the time; no, we’re not going to war; yes, it sucks—but we’ve survived worse. Somehow, despite not reading dozens of op-ed screeds or watching hours of talking heads bobble about the crisis du jour, I’ve found myself more informed. Calmer. More level-headed… than almost anyone I know.

 A lot has been made about the harmful effects of social media in the past decade. People have claimed that it was causing anxiety, depression, and suicides.12 But study after study has come out showing that social media is probably not the massive danger we all think it is.13 And, in fact, when used well, social media can actually increase wellbeing.14

 I’ve become convinced that we’ve mistaken the dangers of social media for the dangers of news consumption. After all, most of us consume our news through social media, so it’s easy to forget that they aren’t the same thing.15 Over and over, I’ve looked into the social media research to find nothing. But when I look at the research on news consumption, there it is.16

 News is not the proverbial vegetables of our social body. News is like a bacon cheeseburger with two donuts for buns.

 The culinary equivalent of a full day of cable news coverage.

 And while that perfect mixture of salty and sweet sure does taste good going down, it makes us sick and nauseous later. It’s an information nutritional nightmare. And worse, it’s addictive.17

 But has it always been this way? After all, we used to eat more vegetables a few generations ago. And nobody had even heard of a donut bacon cheeseburger. Is it the same for news?

 The rest of this article is going to look at two things:

 1. How has news become the cultural trainwreck that it’s become?
 2. What can we replace it with to be a happier, healthier, more functional citizen body?

 The next few sections are going to get philosophical. We’re going to look at questions about how information spreads through human societies, how technology affects the ease with which information passes through networks, and how the internet scrambled those networks like a basket of cheap eggs.

 Human Networks of Information

 Thought experiment: let’s say it’s 5,000 years ago and you live in a little village with a few hundred other smelly, hairy humans. What’s the best way to spread an important piece of information so that everyone knows it, as quickly as possible?

 One idea is to stand in the middle of the town square and scream whatever you need to say a few dozen times. It’d be annoying. And your vocal cords would get sore. But it’d probably be effective—it’d get the news out, that’s for sure. Let’s call this the proclamation form of information-sharing. It starts with one central person announcing information to many, many people.

 But that’s not the only way to get the news out. Another would be to walk up to a few people individually, tell them the news, and then kindly ask them to pass it on to others. Ideally, if the news is juicy and exciting enough, it will spread from person to person, like syphilis in a brothel, until everyone is, err… infected… What I mean is that the information would organically work its way through the networks of people until everyone knows it. Let’s call this the hearsay form of information-sharing.18

 Proclamations can be pretty effective. But not anybody can just get up and start screaming whatever they want. Generally, throughout history, you needed the permission of the king or the lord or whatever. What’s more, generally, if you’re going to stand in the town square screaming something, you can only scream something that would make Mr. King happy. Therefore, we can say that proclamations depend on appealing to the elites.

 Hearsay is also effective. You just talk some shit and see if it spreads. The hardest part about hearsay is sharing something exciting enough to get other people to talk about it. Discussing the recent ratio of rice-to-wheat harvest is unlikely to get many people chatting in the local tavern. But mention that so-and-so’s cheating on you-know-who, and pretty soon nobody can shut the fuck up about it. Therefore, we can say that hearsay depends on appealing to the masses. Or, at least, appealing to specific sub-groups.

 It turns out, these two methods—proclamations and hearsay—are how information has been disseminated throughout human history. You’d be chilling in the local tavern having some mead, and merchants would arrive with news from the neighboring city to trade for some news from where you were. Networks of kin and neighbors traded info about other farmers—who was marriage-material, who was screwing up the harvest, and, of course, word of marauders, bandits and the like.

 Hearsay is information that is spread across decentralized networks of individuals. Your crop fails because of some new fungus. You tell your neighbor. Your neighbor tells the local merchant. The merchant tells the buyers in the next town over: “No wheat this year. Tough titties.” And the information continues from person to person until it saturates the network of interested people (or it runs out of connections).

 Nobody is completely independent. People clump themselves into groups and tribes. So decentralized networks depend on these clusters of people to spread information. If you want to spread the news, you don’t just tell a random Joe Schmoe on the street. You find the leaders and heads of communities, people who are networked and connected. Decentralized networks depend on humans clustering themselves into groups to receive and transmit info. This fact will be important for later.

 But then, let’s say, someone pisses in the king’s cornflakes and he decides that everyone needs to hear about it. So, the king would send someone down to the town square and this person would stand there and scream what the king wanted everyone to know until his eyes bled and everyone begged him to shut up. These people were literally known as “town criers” and believe it or not, they held a lot of prestige for much of human history. They were the OG Walter Cronkite and Edward R. Murrow. When they showed up in town, people knew something serious was brewing. Town criers were rare but major events. The proclamation of a war, a new tax, a royal wedding. It was a big deal.

 Proclamations are information that spread through centralized networks. It’s info that comes down from the top.

 Centralized and decentralized networks still spread information today. In fact, not much has changed. People talk shit in their little clusters on social media, saying the prime minister is a weenie, then Mr. Prime Minister does a press conference and everyone shuts their traps and listens.

 Other times, decentralized networks overrule centralized networks of authority. Kings and emperors are overrun by the disaffected masses. Civil rights and social justice movements organically emerge from overlapping groups with no central leader and change major policies. Both centralized and decentralized networks possess power. Both proclamations and hearsay spread information through society simultaneously and in their own ways.

 Tyranny and Revolution: The Evolution of Human Networks

 Throughout history, some technological innovations have made proclamations (centralized networks) more effective ways of transmitting information. Like, whoever invented the first bullhorn suddenly made their town crier like three times more effective. That, in turn, would make whoever has control of screaming in the town square more influential and powerful than before.

 Similarly, whoever invented the bulletin board increased the power of decentralized networks. Now you walk into the local tavern and read what dozens of people needed and cared about without having to find them and talk to them. This means whoever is best able to write effective notes on the local bulletin board would gain power.19

 You could almost look at world history as a tug-o-war between centralized networks and decentralized networks, based on whatever the predominant communications technology was at the time. When centralized networks are prominent, power tends to be concentrated and autocratic, centered on the few people who have access to the bullhorn. But when decentralized networks are bolstered, it’s all about what groups can organize themselves and inspire followers. Centralized networks produce the tyrants and decentralized networks produce the revolutions that overthrow them and reset the game. And on goes the seesaw of human history.

 Below are some of the most impactful technological developments and how they affected centralized and decentralized networks of information. As you can probably guess, we’re going to end up at the internet.

 Written Word

 Invented: Around 3,200 BCE
 Promoted: Centralized networks (Proclamations)

 Writing was the first and most important technological innovation in the world because it allowed knowledge to be shared between people who never met each other—including people from past generations with future generations.

 On the surface, this sounds like it would promote decentralized networks. Everyone would start writing each other notes, chiseling their own clay tablets and handing them out like candy at a Thanksgiving Day parade.

 But that’s not what happened. Back in the day, writing was expensive. Clay tablets, man! Shit ain’t cheap. On top of that, only the elites were educated enough to read and write, therefore, writing mostly served as a way to create proclamations given by the king.

 It’s no coincidence that it’s around this point in human history that empires began to emerge. The written word allowed kings to coordinate their subjects over vast distances. It allowed them to unify millions of disparate peoples under a common code of laws and ethics.20

 Later, the centralized control of information would grant the great religions dominion over most of the planet. Revolutions were minimal, if non-existent. The vast majority of the population couldn’t read or write, much less coordinate themselves across wide areas. Unless you were highly educated and highly connected, you weren’t able to write anything of importance or usefulness. And if you were highly educated and connected, you were part of the elite anyway, so why cause trouble? Censorship was absolute. Saying the wrong thing could get you killed. Everything went through your king or priest, no questions asked.

 This period was no fun.

 The Printing Press

 Invented: 1440 CE
 Promoted: Decentralized networks (Hearsay)

 It’s a sad irony that the invention most likely responsible for creating the modern world resulted in the inventor going broke, being exiled, and dying a disgraced nobody. Johannes Gutenberg created the printing press, a machine that could mass-produce books and pamphlets affordably. What, at first, seemed like a convenient way to reproduce bibles and some poetry for a quick buck, would soon revolutionize the world.

 The next time you think *you* were under-appreciated, just think of Johannes Gutenberg.

 The reason writing initially favored centralized networks was because it was expensive, both in terms of labor and money. Teams of monks would spend months copying bibles by hand. Only kings, emperors, and popes had access to scribes to write out hundreds of copies of whatever they wanted to say.

 But the printing press changed all that. Now anybody could approach a printer and have their ideas reproduced. Whereas the written word expanded the reach of the centralized networks, the printing press would similarly expand the reach of decentralized networks. And this expansion would shift the balance of power in the world.

 That shift didn’t take long. In 1517, Martin Luther published his “95 Theses” criticizing the Catholic Church and launching the Reformation. For centuries, the Catholic Church had murdered any dissenters like Luther. But Luther had reprinted hundreds of copies of his theses. Anyone could pick up and read them. Killing Luther was no longer sufficient. You’d have to kill all his followers. Thus began war.

 Luther’s Reformation would eventually spark over a hundred years of bloodshed across the European continent. It would inspire monarchs to distance themselves from the church. And it would help inspire small, holy enclaves to seek religious freedom in the New World.

 But the printing press wasn’t done. By the 1600s, independent scholars were publishing their own work and promoting a new idea called “the scientific method.” The scientific method was decentralized—anyone could use it and arrive at their own conclusions. Ideas about inalienable human rights, the rule of law, and democracy would take hold. The ease of publishing and promoting work across decentralized networks allowed the Enlightenment to flourish.

 The first newspapers emerged around the same time,21 independently determining important information and disseminating it through local merchant networks, circumventing the king’s proclamations and town criers. This group of publishers was seen as an ally of the new Enlightenment ideas, as they helped coordinate social movements in the 17th and 18th centuries that would lead to democratic revolutions. The American Revolution relied heavily on the printing and dissemination of political pamphlets.22 Similarly, revolutionaries in France coordinated across the country and, for the first time, realized they were getting royally screwed by the king (pun totally intended). As Robert Darnton, a historian from Princeton put it, “Without the press, the French Revolution was impossible and unthinkable.”23

 That said, news media in this era was also a shitshow. Fake news proliferated. There were practically as many newspapers as there were people. And you think people are polarized today? Hoo boy… There were stabbings in Congress. Politicians shot at each other. Their supporters bombed and lynched and killed each other over nothing but rumors and insults… or rather, that is, hearsay.

 Radio and Television

 Invented: 1895 CE (radio); 1925 CE (television)
 Promoted: Centralized networks (Proclamations)

 I lumped radio and television together here because they both occurred at similar times and the same companies who started radio stations ended up also dominating television.

 Radio/TV once again favored centralized networks for two reasons:

 1. Like writing in the ancient world, running a television/radio station was prohibitively expensive and difficult and only the elites would have access to the resources to do so.
 2. These technologies allowed someone to hit millions and millions of people with the exact same information simultaneously. It was as if a mutant town crier with a hundred vocal cords could scream for thousands of miles in every direction.

 The 20th century presented a hard return of centralized power: it was peak colonialism,24 the rise of totalitarian governments (of which North Korea is still a holdout), two world wars and that whole ugly Nazi thing.

 Radio and television launched the industries of advertising and public relations. When millions of eyeballs are watching the same exact thing, this grants a huge opportunity for businesses (and governments) to persuade people to think or feel a certain way. This is where the ability to manipulate and influence people into certain opinions or desires became formalized into the practice of propaganda, or as it’s known today, marketing.25

 The Nazis were one of the first governments to master the art of radio propaganda.

 Radio and television likely created the most cultural solidarity we have ever seen in human history. Everyone was watching the same shows, listening to the same music and following the same events. The wars of the 20th century pitted half the planet against the other and everyone held on for their dear life. From World War I, up through the end of the Cold War, there was always relative political conformity and trust, especially because no matter what country you were in, you were likely united against some large external enemy.

 The Internet!

 Invented: 1983 CE
 Promoted: Decentralized networks (Hearsay)

 The originators of the internet saw it as a great liberating force on humanity.26 At the time, they lived with the media gatekeepers of the television and radio stations, limiting the vast majority of the population’s ideas from being widely expressed.

 But now, with the power of networked computers, anybody could get online, start a website, and start spitballing their bullshit into the wind, hoping somebody would stop awhile and listen.

 Whereas TV and radio expanded the reach of centralized networks to worldwide audiences, the internet would expand the reach of decentralized networks worldwide. A professor in Canada could exchange emails with a student in India and come up with an idea that would be tested in China and written up in newspapers in Australia.

 Obligatory cat meme goes here…

 It sounded like heaven. And while the internet has granted us incredible amounts of information and perspective, it hasn’t quite lived up to its utopian expectations. In fact, public discourse has kind of gone to shit. And it’s done so for a few reasons:

 First, the old guard of media has faced the economic realities: for better or worse, pretty much any kid with a computer can compete with them now. This caused revenue to plummet and led them to cut large chunks of their staff—most notably, investigative journalists and foreign correspondents, the people who made them reliable and trustworthy in the first place.27 Expensive news that took years of study and months to report was no longer economical. So the traditional media giants began to look more and more like their amateur competitors, while their amateur competitors leveraged marketing strategies to make themselves appear far more reliable than they actually were.

 Second, because everyone’s phone is on them all the time and we’re all compulsively checking social media and email, emotionally impactful information (regardless of its truth or utility) spreads extremely fast. Thus, it’s easier than ever to make a quick buck by indulging in the worst human tendencies. You want to net a bunch of traffic by cooking up a fake pedophile ring in a pizza shop?28 Hell yeah, go for it!

 Third, by giving voice to everybody, the internet unintentionally gave voice back to the uglier sub-cultures of society that were once barred from any mainstream attention: the bigots, the conspiracy theorists, and the cranks. In our idealism of giving the whole world a way to express themselves, we forgot that most people say really awful shit.

 The internet is like the modern printing press.29 It allows anyone to express themselves, challenge authority, and share information on informal networks outside of the traditional pipelines. On the one hand, this is great. It grants greater freedom of expression and the opportunity for self-actualization to everyone. On the other hand, it’s a fucking mess, because as a society, we don’t know who to listen to or what to believe anymore.

 Like the period after the printing press, we are seeing more explicit challenges to the establishment. We are seeing a resurgence of tribal and religious affiliations. We are seeing emerging conflicts over the identity and direction of country and culture. The times are tense. The internet has complicated… well, everything. But if we’re willing to look hard enough, we can spot iridescent flashes of collective brilliance breaking through an otherwise endless stream of bullshit.

 To Sum Up

 * When centralized networks of information have the advantage, we get smaller quantities of highly relevant information. The danger of these networks of information is that they can easily be corrupted for tyrannical purposes like the Catholic Church in the Dark Ages or the totalitarian governments of the 20th century.
 * Decentralized networks promote higher quantities of low-quality information. The quantity of information generally fragments the population into warring factions based on tribal or religious identities. They only trust their preferred messengers and distrust everybody else.
 * Whereas centralized information networks promote conflict between political bodies, decentralized information networks promote conflict within political bodies.
 * Where centralized information networks solidify and reinforce cultural identity, decentralized information networks tend to overthrow and revolutionize cultural identities. Sometimes these revolutions are huge leaps forward for humanity (Enlightenment, civil rights, etc.) but other times, it’s just a bunch of arbitrary, religious-style fighting.
 * Decentralized information networks create a much larger diversity of information, but the signal-to-noise ratio is poor, making it incumbent on each individual to navigate the information and carefully choose what is worth consuming and what should be ignored.
 * To put it into plain English: the frustrations we have with news media—the negativity, the lies, the partisan bickery—it’s not going to change. It’s not going to get better. It’s on us to learn how to navigate the media environment and vote with our eyeballs. To do that, we must understand what information is actually useful and important, and what is just nonsense.

 What Makes Information Useful?

 OK, let’s state the obvious. Some information is important, some is not. Some information feels important while being totally unimportant. And some information feels unimportant while it’s actually quite crucial.

 The most succinct way to sum up the issue today is that news media optimizes for information that feels important with little regard to its actual importance.

 If we look back through history, this is a common state of eras where information networks are highly decentralized. What travels is what feels impactful, even if it’s not—and even if it’s not true! Because the news has to compete in a highly diverse market with tons of noise, they must be ruthless in capturing people’s emotions as effectively as possible.

 Eras of centralized information networks have had the opposite problem: they share information that seems unimportant but is actually quite important. Like the Soviet announcements of monthly bread rations or the long, tedious Congressional committees that drone on C-SPAN in the United States every day, these forms of news are dull and boring. But they matter.

 Back in the 1950s, when there were only two TV channels, they could show a 12-hour congressional committee and you really had no option. That’s what was important. That’s what people needed to know. You lived with it whether you cared or not. But as cable news and the internet emerged, the incentives for media companies pushed them more into fighting for people’s hearts rather than their minds. They profited off having the most emotional information, not the most factual.

 Today, if someone tries to show a congressional committee, the next channel is going to bury them with pictures of dead children in Sudan and plane crashes in Iran. There’s no chance.

 This is why we’ve all experienced the “decline” of news media over the past 30 years: we’ve lived through the slow escalation of emotional impact at the cost of veracity and utility. On the one hand, we have more choice of information than ever before. But the average quality of that information is far lower. It’s simply the trade-off that comes with the technology.30

 Useful macro information is information that directly impacts you and/or has an outsized influence on the economy. Prescription drug price policy is important information. The safety rating of your car is important information. A tariff on sugar is important information.

 These things are boring, but they are important because they are the first-order effects that drive almost everything else. As the old saying goes, “It’s the economy, stupid.” Cultural issues usually happen downstream.31

 Almost everything else is hogwash. Terrorist attacks? School shooters? Neither of these would probably be such a problem if it weren’t for the news coverage in the first place. Trump said something stupid? So what? Wake me up when there’s another $350 billion tariff.

 When you focus on first principles, the news becomes incredibly empty. I’d say 80-90% of the content out there is either: a) catastrophizing some irrelevant event, b) horse race updates when all you need to know is who won, or c) opinion and commentary dressed up as news.

 You can skip all this stuff. You don’t care about the events. You don’t need to know about the horse race, just the winner. You don’t need more opinions—you need data.

 How the News Media Hijacks Your Attention

 There are certain biases within our minds that we all fall victim to and the news media leverages them to keep us engaged and wanting more. There’s the negativity bias, a well-documented effect where people pay more attention and have more intense emotional responses to negative events than positive events.32 There’s the present bias, the tendency to put more weight and importance on events that have happened recently, as opposed to far in the past or the future. There’s the narrative bias, the desire to arrange information in terms of a coherent story structure (i.e., “The stock market dropped 2% today because it rained in Michigan”) even if it’s not true. And finally, there is the almighty confirmation bias, the propensity to arrange new information so that it agrees with our previously-held beliefs.

 Chances are if you think about the news you’ve consumed lately, pretty much all of it is: negative, recent, story-driven, and supports your previously held beliefs.

 News media leverages our biases to keep us hooked and consuming more information, regardless of its utility. Especially in our decentralized world of the internet, the name of the game is attention, and the news media will use every tool to leverage more of yours. As such, news media specializes in information that is immediate, fast-moving, narrative-driven and highly visible.33

 The problem is that the most important information is usually long-term, slow-moving, impersonal, abstract and invisible, and not necessarily negative:

 * Car accidents kill more people than terrorism, mass shootings, and natural disasters combined. How much reporting do you see on car accidents or driver safety?
 * The sequencing of the human genome was arguably more important than any other scientific development in the 1990s, opening a path to predicting disease and conditions before they occur, altering and improving the health of millions, if not billions, of people over the coming generations. How many weekly updates did you hear about it?
 * More than 1 billion people have been lifted out of extreme poverty worldwide,34 due to a combination of greater access to health care, sanitation, and the opening of markets for greater commerce. Who knew?
 * Despite the fact the first international climate agreement was signed in 1992 by 154 countries, news media has just only recently started covering climate change with any significance—and if we’re being honest, it’s probably just to go along with all the natural disasters that they’ve been milking.

 Paradoxically, in its effort to inform wide audiences, news media positions itself to misinform and over-inflate extreme outlier stories.

 But this is not the news media’s fault. It’s simply how information scales in large decentralized human networks. The information with the greatest emotional valence will travel the furthest and be broadcast the loudest.35 Gone are the days where we have reliable journalistic mainstays who can consistently nail what is most significant. The structures simply aren’t there.

 Instead, we must rely upon ourselves. After all, in the internet age, we are the ones who are empowered: the user. We must screen out crap information and consciously seek out useful information. We must develop an awareness of how our attention is being seduced and prevent ourselves from getting sucked into the social media vortex of never-ending comment threads.

 For the first time in human history, our relationship with news depends on us.

 What to Do

 Here’s the short version: Seek out the highest-value content you can find, and then pay for it. Limit your consumption to a few pieces of content per week. Go for quality over quantity.

 By high-value content, ask yourself how much that content cost in terms of the labor, research, expertise, production, etc. Long-form content is more expensive than short-form. Research and citations are more difficult to put together than just some old dude ranting. Something that explains the historical context and acknowledges multiple sides of an issue is more difficult and time-consuming than a partisan rant.

 I focus on the expensive thing because right now, the news media industry is going through a slow-moving crisis of their own design (and ironically, it’s too slow-moving for them to report on it). Research, fact-checking, traveling to some crazy war-torn country, and not getting your shit blown up—these are all expensive. Hiring a journalist with decades of knowledge on a region is expensive. Interviewing professors and digging into research is expensive.36 Meanwhile, online advertising revenue pays next to nothing. So, more and more, media outlets are subtly and secretly pushing out cheap content that looks expensive—uninformed 24-year-olds who can sorta, kinda sound like an experienced journalist with a lot of Google-fu.

 There are only a few publications that have the luxury of not playing the Clickbait Headline Game, and that’s because they charge enough money that they don’t have to. Publications such as The Economist,37 Financial Times or Foreign Policy don’t give a shit if their articles are fun or not. They’re not writing for the general public. They’re writing for professionals who depend on the information to make big, expensive decisions. This is where you want to insert yourself into the information pipeline. Long, boring, thoughtful pieces, put together by long, boring, thoughtful people. One good piece of long-form journalism that focuses on real data is worth dozens of crappy short-form pieces you find elsewhere.

 When you get into the less expensive publications, you start to run into a lot of bias.38 You can feel the articles getting more frantic and the headlines more hyperbolic. Then, once you’re into the cheap (or free) seats, you’re basically being spoon-fed what to think and feel.

 The point to remember is that you want your news to be a little bit dull. If you’re getting excited or worked up reading something, chances are you’re being taken for a ride.

 If you can’t afford more expensive news publications, then I recommend skipping news entirely and simply reading books. In terms of quality information per dollar, books are the best bang for your buck. These days, I aim to make books account for at least 80% of my total reading time.

 You may worry that by only reading books, you’ll “fall behind.” But I think you’ll actually find the opposite. If you take the hours and minutes that you used to spend reading news and instead read books about the same topics, within a few months, the “news” that everyone is freaking out about will seem trifling to you. You will see how slowly the world truly moves. You will have more context and clarity than most and you will recognize that almost nothing needs to be known the moment it happens.

 Finally, I’d be remiss if I didn’t mention blogs and podcasts. Blogs/podcasts are interesting in that they don’t succumb to the same economic realities of mainstream news media—they don’t demand huge overhead expenses to maintain, and they rely on the personal trust of their readers to continue to exist. Also, most prominent bloggers and podcasters gained their expertise elsewhere, therefore allowing them to offer high-value information at a huge discount.

 Put simply: most blogs and podcasts suck. But the best ones are arguably better than any other content on the planet right now.39

 If you can find one of these experts, they can often be a gold mine of non-partisan, on-the-ground information. Bloggers such as Tim Urban, Ben Thompson, Tyler Cowen, Scott Alexander, and Dan Wang provide a ton of value for little to no cost at all.

 What to NOT Do

 But perhaps more important than finding sources of high-quality information is cutting out the sources of low-quality information. And there is a lot of low-quality information out there. If you get nothing else from this behemoth of an article, please consider cutting the following from your information diet entirely:

 * STOP Watching News on Television – In surveys, cable news is consistently the least informative and most biased source of information. The nature of television demands constant stimulation and therefore undermines pretty much any serious discussion of a topic.
 * STOP Following News Sites on Social Media – Trust me, if something is important enough, you’ll hear about it anyway. People will link to it. They’ll email you. Whatever. When you follow news sites on social media, you are opting in to the attention game. You are allowing yourself to be triggered and fought over by egregious headlines, moral outrage, and clickbait. Just don’t play it. Unfollow all news sources on your social media. Not only will it make your news experience better, but it will make your social media experience a hundred times better.
 * STOP Clicking on Clickbait – I know it’s easier said than done. But seriously, get into the habit of actively seeking out information instead of passively receiving it on your newsfeeds or apps. The algorithms on these platforms do not work in your favor. Cut them all out and then when you need to know about a subject, go looking for the info yourself, preferably in a book.
 * STOP Consuming News That Requires Less than 20 Minutes – Most short-form news is garbage. If something requires more than 20 minutes to consume, that likely means it took weeks or months (years?) to produce. This means skip the hot takes, the quick interview clips, the editorials, and the breaking news alerts. Instead, download long-form articles, watch documentaries, listen to long-form interviews and podcasts where the person has time to explain themselves and get into the nuance of their positions. The world is complicated. You need time to sort it out.
 * STOP Getting All Your Information from One Source – Make a conscious effort to check both left-wing and right-wing sources. Get info from minorities and women and foreigners. Follow professors and follow really smart kids with too much time on their hands. There’s a lot to be said for diversity. But the real diversity is diversity of thought. The wider the range of ideas you expose yourself to, the less you’ll ever be caught off guard by whatever new event happens.

 Knowing the World’s Not Going to End

 I was in Australia over the New Year. As I’m sure you know, there are huge bushfires that have consumed huge stretches of land there, destroying natural habitats, and displacing thousands of people.

 We saw the smoke when we were there. We rode a train through the Blue Mountains and saw acres of charred forests. On our first day in Sydney, the pollution level was the same as a small coal-mining Chinese city… i.e., toxic to even be outside. We talked to the locals in Adelaide about the burnt wine country. But all in all, it was like anywhere else I’ve been after a disaster: it’s sad, but life goes on. People are getting on with it.

 Then I started getting emails and messages from friends and family back home. “Are you OK? Have you been hurt?” They said, “You should come home! It’s not safe!” I looked up from my phone: sunny skies and beautiful beaches. What in the hell were they going on about?

 I was then subjected to what I could only call insanity. News headlines claiming that Australians’ way of life is forever lost, that everything is burning, that over a billion animals (an absurd number) have been killed. One article even went so far as to imply that the entire country will soon have to be abandoned.40

 This map is fake. But that didn’t stop it from being posted on approximately eight billion news sites.

 I’m as alarmed as anyone about climate change, but come the fuck on.

 Every few months it seems as though there’s something like this. Whether it’s Brexit or Trump’s impeachment or the Amazon burning or China’s internment of a million Uighurs—we live in the Age of Hyperventilation.

 But while there’s an endless stream of tragedy in the world, there is so much that the news never teaches you or shows you.

 The news doesn’t show you how bad humans are at predicting the future. How pretty much every projection, scientific or not, ends up incredibly inaccurate. It doesn’t tell you that we’ve been predicting civilization-leveling catastrophes since pretty much the dawn of human thought—and every time, we have been wrong.

 The news doesn’t teach you that technology doesn’t develop linearly—how it comes in unexpected leaps, and then trounces prior assumptions under the weight of its efficiency.

 The news doesn’t tell you that natural disasters are just that: natural. Yes, they may be occurring slightly more often and be more severe, but they are, and have always been, the rule, not the exception.

 The news doesn’t show that the vast majority of people are good. They will help if they can. They care even if they’re confused about how to care or why. The news doesn’t teach you that most people won’t hurt you and even if they do, you will recover and be fine and be stronger than before.

 But most of all, the news doesn’t teach you that moral outrage, while strangely satisfying, gives people a sense of accomplishing something without actually accomplishing anything. Posting on social media or writing a scathing op-ed may feel as though it’s contributing to some grand movement, some great push for humanity, but that push is, more often than not, grounded in nothing.

 Nowadays, I briefly check a handful of news sites, maybe once a week.41 I read maybe five news articles in total. They are articles I choose, based on my own perception of what’s important and what matters. The rest is books, blogs, and podcasts.

 Months ago, my friends felt sorry for me. They worried that I’d be out of touch, that I’d lose my connection to society and the world, and that I’d become disempowered to change it.

 But the opposite has happened. I have realized that it is they who are out of touch, disconnected, and disempowered.

 And this is the ultimate failure of the news media as currently constructed: it creates a passivity, a false feeling that we’re not somehow part of the narrative that we are also watching unfold. It lulls us into thinking that we don’t—that we can’t—have some role to play in the drama. It convinces us that our obstacles are too great, that our fears are too deep, that we are incapable of overcoming the same challenges of the thousands of generations who have come before us.

 And this is what was lost. The town crier was never meant as a mere announcer. He was an instigator. He was a call to arms to leave our domestic safety and take up the business of the world that is ours. He reminded us to not simply receive the news but to go out and be the news as well.

 What news will you be?

 Want more actionable ideas every week?

 Join millions of readers and subscribe to The Breakthrough newsletter. Enter your email below to receive new ideas and exercises that could change your life each week.

 No spam or unexpected emails. Ever.

 Footnotes
 1. Beck, P., Dalton R., Greene, S. & Huckfeldt, R. (2002). The Social Calculus of Voting: Interpersonal, Media, and Organizational Influences on Presidential Choices. American Political Science Review, 96(1), 57-73.↵
 2. By the way, this is true of any information source, including this blog. (P.S. Subscribe to my weekly newsletter!)↵
 3. Yep, clearly the news of 2020.↵
 4. Mark A. Schuster, et al., (2001) A National Survey of Stress Reactions After the September 11, 2011, Terrorist Attacks, New England Journal of Medicine, 345(20).↵
 5. Dearing, J; Rogers, E (1988). “Agenda-setting research: Where has it been, where is it going?“. Communication Yearbook. 11: 555–594.↵
 6. Wanta, W., Golan, G., & Lee, C. (2004). Agenda Setting and International News: Media Influence on Public Perceptions of Foreign Nations. Journalism & Mass Communication Quarterly, 81(2), 364–377.↵
 7. Riddle, K. (2009). Cultivation Theory Revisited: The Impact of Childhood Television Viewing Levels on Social Reality Beliefs and Construct Accessibility in Adulthood. International Communication Association. pp. 1–29.↵
 8. Johnston, W. M., & Davey, G. C. L. (1997). The psychological impact of negative TV news bulletins: The catastrophizing of personal worries. British Journal of Psychology, 88(1), 85–91.↵
 9. Mary E. McNaughton-cassill (2001) The news media and psychological distress, Anxiety, Stress & Coping, 14:2, 193-211.↵
 10. You know things are bad when the inventors of the Facebook “like” button are themselves coming out against it, with one describing likes as “bright dings of pseudo-pleasure” that can be as hollow as they are seductive.↵
 11. If 2019 feels like a lifetime ago, here are 22 things that happened for the first time that year.↵
 12. The most famous claims have come from the popular (and controversial) researcher Jean Twenge. But her data is all correlative. Yes, depression, anxiety, and suicide have all increased since the advent of social media. But that doesn’t mean social media is the cause. See her scary-sounding study: Twenge, J. M., Joiner, T. E., Rogers, M. L., & Martin, G. N. (2017). Increases in Depressive Symptoms, Suicide-Related Outcomes, and Suicide Rates Among U.S. Adolescents After 2010 and Links to Increased New Media Screen Time. Clinical Psychological Science, 6(1), 3–17.↵
 13. A longitudinal study that followed people over eight years found little negative effect from social media use. See: Coyne, S. M., Rogers, A. A., Zurcher, J. D., Stockdale, L., & Booth, M. (2019). Does Time Spent Using Social Media Impact Mental Health?: An Eight-Year Longitudinal Study. Computers in Human Behavior, 106160. Also see: See: Collis, A., & Eggers, F. (2020, January 14). Effects of Restricting Social Media Usage.↵
 14. OK, I’m going to get pilloried for this. But a meta-analysis of 124 studies on social media use found that when used primarily to keep in touch with relationships and entertainment, social media actually improves well-being. See: Liu, D., Baumeister, R. F., Yang, C., & Hu, B. (2019). Digital Communication Media Use and Psychological Well-Being: A Meta-Analysis. Journal of Computer-Mediated Communication, 24(5), 259–273.↵
 15. Amanda Zantal-Wiener, 68% of Americans Still Get Their News on Social Media, Even If They Don’t Trust It, HubSpot, Originally published Sep 14, 2018, updated Dec 11, 2019.↵
 16. Another example: Bodas, M., Siman-Tov, M., Peleg, K., & Solomon, Z. (2015). Anxiety-Inducing Media: The Effect of Constant News Broadcasting on the Well-Being of Israeli Television Viewers. Psychiatry, 78(3), 265–276.↵
 17. I have to give credit to Rolf Dobelli’s fantastic essay, “Avoid News: Towards a Healthy News Diet” for the original comparison of news to the sugar of our informational diet.↵
 18. Many of the ideas from this section are drawn from Niall Ferguson’s book, The Tower and the Square: Networks and Power. Just to make things even more confusing, Ferguson uses “the square” to represent decentralized networks whereas I am going to use the town square to demonstrate centralized networks.↵
 19. See: A History of Mass Communication: Six Information Revolutions by Irving Fang, PhD.↵
 20. See: Hammurabi’s Code.↵
 21. The Encyclopedia Britannica credits Relation of Strasbourg, first printed in 1609, as the first newspaper. This was quickly followed by several others and, according to the encyclopedia, provides evidence of “fairly sudden demand” for newspapers at the start of the 17th century, a demand that was soon well-established.↵
 22. One of these tireless publishers was named Benjamin Franklin.↵
 23. Bernstein, Richard. (Feb 18, 1989). “How the Storming of the Bastille Liberated the Printing Press, Too.” The New York Times.↵
 24. Kicked off by the Scramble for Africa that saw the 10 percent of Africa that was under formal European control in 1870 increase to almost 90 percent by 1914. For the entire history of colonialism, see here.↵
 25. McGarry, E. D. (1958). The Propaganda Function in Marketing. Journal of Marketing, 23(2), 131–139.↵
 26. Many researchers since have investigated the role of the internet as a “liberation technology”. See for example: Diamond, L. (2010). Liberation Technology. Journal of Democracy, 21(3), 69–83.↵
 27. According to Pew Research Center, overall newsroom employment in the US dropped by 23% from 2008 to 2019.↵
 28. This actually happened. See: Pizzagate Conspiracy Theory.↵
 29. A lot of research out there attempts to compare the internet to the printing press. See for example: Bawden, D., & Robinson, L. (2000). A distant mirror?; the Internet and the printing press. Aslib Proceedings, 52(2), 51–57.↵
 30. It is important to note that I would argue that the ceiling on the quality of information has gone up. Extremely high-quality information is available for anybody today. It’s just incredibly difficult to find.↵
 31. There are a few significant exceptions to this: racism/sexism, for instance. But for the most part, economics drives culture, not the other way around.↵
 32. Vaish, A., Grossmann, T., & Woodward, A. (2008). Not all emotions are created equal: the negativity bias in social-emotional development. Psychological Bulletin, 134(3), 383–403.↵
 33. This longitudinal study finds that despite the decline of aviation incidents over time, relative media attention has increased. Go figure. A tragic plane crash—however regrettable—sells news like few other things do.↵
 34. You can see the data at PovcalNet: the online tool for poverty measurement developed by the Development Research Group of the World Bank.↵
 35. Choi, H.-Y., Kensinger, E. A., & Rajaram, S. (2017). Mnemonic transmission, social contagion, and emergence of collective memory: Influence of emotional valence, group structure, and information distribution. Journal of Experimental Psychology: General, 146(9), 1247–1265.↵
 36. If you want to dive into how news is produced in the digital age, you can start with this article.↵
 37. Coincidentally, when I went to the Economist’s website to link it here, the first article on the front page was about how car accidents are more dangerous than almost anything else and how easily they can be prevented.↵
 38. This includes, but is not limited to, The New York Times (sadly), The Washington Post, The Guardian, Slate, Breitbart, Town Hall, and so on.↵
 39. Yes, I’m totally biased.↵
 40. I posted a couple of long responses to Aussies about these types of news articles and the fires in general on my Facebook Page. You can read them here and here.↵
 41. Research shows that small amounts of focused news consumption is optimal. See: Bourkes, M. & Vliegenthart, R. (2017) News Consumption and Its Unpleasant Side Effect Studying the Effect of Hard and Soft News Exposure on Mental Well-Being Over Time. Journal of Media Psychology, 29, pp. 137–147.↵

 About the Author

 Mark is the three-time #1 New York Times bestselling author of The Subtle Art of Not Giving a F*ck as well as other titles. His books have sold around 20 million copies, been translated into more than 65 languages, and reached number one in more than a dozen countries. In 2023, a feature film about his life and ideas was released worldwide by Universal Pictures.

 Learn More About Mark

 MORE ARTICLES

 * How to Break Up With Someone
 * Interviewing Like a Boss
 * Why It Can Be Good to Doubt Yourself
 * 9 Subtle Lies We All Tell Ourselves
 * 7 Books That Will Change How You See the World
 * Are You Entertained—Or Addicted?
 VIEW ALL ARTICLES

 Mark is the three-time #1 New York Times bestselling author of The Subtle Art of Not Giving a F*ck, as well as other titles. His books have sold around 20 million copies, been translated into more than 65 languages, and reached number one in more than a dozen countries. In 2023, a feature film about his life and ideas was released worldwide by Universal Pictures.

 Learn More About Mark
 © 2023 Infinity Squared Media LLC
 * Articles
 * Books
 * Courses
 * Newsletter
 * About Mark
 * Sign Up
 Privacy Policy Terms and Conditions
 © 2023 Infinity Squared Media LLC

 Wait! Here Are 3 Ideas That Could Change Your Life

 Learn about the idea that transformed a depressed deadbeat into one of the most important philosophers who ever lived. Read about it in my free 19-page ebook.

 I'm not interested.
Everyone kind of already knows that the news sucks. In all my life, I can’t think of anyone who seems to enjoy reading or watching the news every day. It’s a kind of bitter responsibility or endured necessity for people. News is like the societal version of flossing: it’s not fun, yet we continue to do it every day anyway, as a sort of obligation to prevent decay of the social order.
We all get it: yes, the news is overly negative, and yes, it often gets some things wrong. But in the end, we’re better off for it. That’s how the argument usually goes. Some even say we need the news, that democracy is impossible without it.
In this article, I’m going to challenge that idea. In fact, I’m going to argue that the news doesn’t just seem to be horrible, in its current form it is horrible. It is actively damaging our culture. And most of what we blame on social media or the internet—the upticks in stress, anxiety, pessimism, and polarization—is misplaced. News media is the true culprit.
Obviously, this is a bold argument. And this article will be quite long. We’ll investigate what makes some information valuable and other information worthless. We’ll look at how networks of information emerge and how news has traditionally traveled throughout human history. We will look at how the internet and social media have affected those traditional networks, and, of course, I’ll take a stab at explaining why our current media environment is a shit sauna.
Finally, I’ll discuss what I feel are better alternatives for staying informed than consuming news. There will be lots of links.
So, take a deep breath… maybe grab a stress ball. This is going to take a while.
Let me ask you this: when was the last time you made a major life decision based on a news story? Or: when was the last time information from a news story directly impacted your life? And I don’t mean, “It made me really, really upset,” impacted. I mean, like, actually impacted you.
Chances are you can’t remember. That’s because the vast majority of news is irrelevant. Car chases. Bank robberies. Celebrity overdoses. Sports scores. Homeless people throwing feces at one another. None of us ever really have the experience of seeing a news story and immediately thinking, “Man, I need to change what I’m doing.” If it’s ever happened to me, I don’t remember it.
The news does not tell you what job to take or how to deal with your angry husband or what the best brand of headphones is or what to feed your kids in the morning. It doesn’t help you to be a better friend or explain why you have panic attacks or advise on the best way to deal with an unruly child.
The news is pretty much useless for helping you determine what to do with your life. You may argue that the news helps you determine who you vote for, but research suggests that even that’s not true—we are far more swayed by our friends, family, and day-to-day circumstances than any information we may find on the news. 1  If you want to know a politician’s platform today, you can just go to their website or hear them speak directly on YouTube. You don’t need a talking suit on television or an op-ed section to tell you what to think.
If we’re being honest, most of the value we get from consuming news is having something to talk about with other people who consume the news. And considering this has a 50% chance of making you hate each other, even that value becomes questionable.
So, if the news isn’t sharing useful facts or information that is directly affecting millions of people’s lives, what exactly is the news trying to do?
Well, that’s easy. The goal of the news is to motivate you to keep consuming news. 2  
They do this in a few ways:
They present emotionally-charged information that feels important, even though it isn’t (a politician making a gaffe, for instance).
They catastrophize everything to make you believe this is a once-in-a-lifetime event and nothing will ever be the same ever again… until something else happens. The assassination of Soleimani in Iraq in January 2020 is an example. 3  
Once they’ve overblown the severity of an event, they shuttle in a small group of “experts” to help you process the significance of the event. Unsurprisingly, most of these “experts” end up saying, “We’ll have to wait and see.”
They obsess over pointless details to human interest stories—what kidnap victims were wearing, who were the mass shooter’s parents, how many tacos Joe Biden ate yesterday, etc.
They aim to entertain rather than inform. CNN now has entire segments where they simply show popular YouTube videos and then laugh about them.
These strategies produce the net effect of causing the consumer to feel like they’re being informed, when actually, we’re getting highly skewed perspectives on (mostly) irrelevant events that have been blown out of proportion to get you emotionally worked up and upset. So of course you tune in again tomorrow to see what happened next!
 This is why we can’t have nice things… 
Now, these complaints have existed for decades. You’ve probably heard them before. And you probably justify them as necessary evils. I mean, you’ve got to wrap the important stuff in some gruesome entertainment, right? It’s the sugar that helps the medicine go down.
But I disagree. Not only is it not necessary, but this style of outrage porn harms us and our culture.
 Recurring Trauma – Studies find that many people who view catastrophic events (terrorist attacks, mass shootings, natural disasters) repeated over and over again on television can develop PTSD-like symptoms, even though they had no direct experience of the catastrophe. 4  
 Agenda Setting – When the news media becomes obsessed with a topic and shows it repeatedly, people begin to believe that topic is important, whether it actually is or not (hello, Monica Lewinsky). 5  Similarly, negative news coverage causes people to overestimate problems, believing they’re far more widespread than they actually are. What’s worse, this seems to only be true for negative news. Positive news does not cause the same biases in our thinking. 6  
 Cultivation Theory – Studies report that people who watch more news tend to overestimate the amount of crime and violence that happens in the real world. They also generally distrust their fellow citizens more often, becoming paranoid that everyone is out to get them. 7  
 Misinformation and Propaganda – People who watch partisan cable news (e.g., Fox News and MSNBC in the United States) are actually worse-informed than someone who watches no news. Large swaths of the news industry are actively misinforming people.
 Stress and Anxiety – News consumption harms our mental health. Consuming news generates a greater sense of pessimism about not just the world, but your own life. 8  It also increases stress and symptoms of generalized anxiety. 9  
To sum up: we have highly emotive content of little relevance and utility promoting skewed and inaccurate perceptions of the world, of other people, and of ourselves. It generates stress and anxiety, causes greater distrust in others, and can actually make us less informed about the world. Oh, and it runs 24/7, on every social feed, every homepage, in every airport, every hotel, all day, every day, forever and ever…
Gag me with a fucking Flintstones spoon.
 It shall be an honorable death… 
Yet, news still has this reputation as something that needs to be watched. Like the proverbial veggies for our cultural diet. The broccoli that allows us to enjoy the dessert.
Bullshit.
Last summer, I wrote a long article called The Attention Diet. In that post, I argued that the most important personal challenge we face as individuals today is to learn how to manage and care for our own attention. To take our attentional health seriously the same way we would care for our body (or not). This is especially relevant, because in the age of Twitter diplomacy, pretty much everything is designed to hijack our attention and trigger compulsive and repetitive behaviors in consumers. 10  When I wrote that article, I lumped news media in as one of many culprits. Just as I decided to cut most of the video games, social media, and funny cat memes from my life, I lumped news into that category as something I probably spent too much time on.
I didn’t think much of it. I’ve read the news daily my whole life. The second half of 2019 was the first time I had ever experimented with cutting it out of my life—and it also happened to be peak cultural insanity to boot. 11  
At first, my friends thought I was a bit crazy. People challenged me saying I had a moral duty to follow the news. That the news is what makes democracy function (man, whoever sold this line to the world surely made a killing). That I would be left out of conversations. That I wouldn’t understand what was going on.
A few months went by… and something funny began to happen. Yes, I became less stressed and anxious. I kind of expected that. I also became more productive (which was nice). But something very unexpected happened.
I became optimistic.
For the first time since I can remember, I actually think the world is doing all right. Better than all right: great.
Now, you might think, “Well, ignorance is bliss, you clearly haven’t been paying attention.” But, no, actually… I have.
I don’t believe I’m less informed than I was before. In fact, oddly, I believe I’m more informed. Now, when events occur and I hear about them through friends, I’m able to situate them in their proper historical and global context. I’m not swayed by the mood of the moment. I don’t freak out when Donald Trump says or does something abominable—we’ve had many, many shithead presidents. And we’ve survived.
Instead of feeling out of the loop, I’ve begun to feel like the only one in the loop. Instead of my friends explaining the world to me, I often find myself in the position of talking them off the ledge, explaining that no, this isn’t new or unique; yes, these things happen all the time; no, we’re not going to war; yes, it sucks—but we’ve survived worse. Somehow, despite not reading dozens of op-ed screeds or watching hours of talking heads bobble about the crisis du jour, I’ve found myself more informed. Calmer. More level-headed… than almost anyone I know.
A lot has been made about the harmful effects of social media in the past decade. People have claimed that it was causing anxiety, depression, and suicides. 12  But study after study has come out showing that social media is probably not the massive danger we all think it is. 13  And, in fact, when used well, social media can actually increase wellbeing. 14  
I’ve become convinced that we’ve mistaken the dangers of social media for the dangers of news consumption. After all, most of us consume our news through social media, so it’s easy to forget that they aren’t the same thing. 15  Over and over, I’ve looked into the social media research to find nothing. But when I look at the research on news consumption, there it is. 16  
News is not the proverbial vegetables of our social body. News is like a bacon cheeseburger with two donuts for buns.
 The culinary equivalent of a full day of cable news coverage. 
And while that perfect mixture of salty and sweet sure does taste good going down, it makes us sick and nauseous later. It’s an information nutritional nightmare. And worse, it’s addictive. 17  
But has it always been this way? After all, we used to eat more vegetables a few generations ago. And nobody had even heard of a donut bacon cheeseburger. Is it the same for news?
The rest of this article is going to look at two things:
How has news become the cultural trainwreck that it’s become?
What can we replace it with to be a happier, healthier, more functional citizen body?
The next few sections are going to get philosophical. We’re going to look at questions about how information spreads through human societies, how technology affects the ease with which information passes through networks, and how the internet scrambled those networks like a basket of cheap eggs.
Thought experiment: let’s say it’s 5,000 years ago and you live in a little village with a few hundred other smelly, hairy humans. What’s the best way to spread an important piece of information so that everyone knows it, as quickly as possible?
One idea is to stand in the middle of the town square and scream whatever you need to say a few dozen times. It’d be annoying. And your vocal cords would get sore. But it’d probably be effective—it’d get the news out, that’s for sure. Let’s call this the proclamation form of information-sharing. It starts with one central person announcing information to many, many people.
But that’s not the only way to get the news out. Another would be to walk up to a few people individually, tell them the news, and then kindly ask them to pass it on to others. Ideally, if the news is juicy and exciting enough, it will spread from person to person, like syphilis in a brothel, until everyone is, err… infected… What I mean is that the information would organically work its way through the networks of people until everyone knows it. Let’s call this the hearsay form of information-sharing. 18  
Proclamations can be pretty effective. But not anybody can just get up and start screaming whatever they want. Generally, throughout history, you needed the permission of the king or the lord or whatever. What’s more, generally, if you’re going to stand in the town square screaming something, you can only scream something that would make Mr. King happy. Therefore, we can say that proclamations depend on appealing to the elites.
Hearsay is also effective. You just talk some shit and see if it spreads. The hardest part about hearsay is sharing something exciting enough to get other people to talk about it. Discussing the recent ratio of rice-to-wheat harvest is unlikely to get many people chatting in the local tavern. But mention that so-and-so’s cheating on you-know-who, and pretty soon nobody can shut the fuck up about it. Therefore, we can say that hearsay depends on appealing to the masses. Or, at least, appealing to specific sub-groups.
It turns out, these two methods—proclamations and hearsay—are how information has been disseminated throughout human history. You’d be chilling in the local tavern having some mead, and merchants would arrive with news from the neighboring city to trade for some news from where you were. Networks of kin and neighbors traded info about other farmers—who was marriage-material, who was screwing up the harvest, and, of course, word of marauders, bandits and the like.
Hearsay is information that is spread across decentralized networks of individuals. Your crop fails because of some new fungus. You tell your neighbor. Your neighbor tells the local merchant. The merchant tells the buyers in the next town over: “No wheat this year. Tough titties.” And the information continues from person to person until it saturates the network of interested people (or it runs out of connections).
Nobody is completely independent. People clump themselves into groups and tribes. So decentralized networks depend on these clusters of people to spread information. If you want to spread the news, you don’t just tell a random Joe Schmoe on the street. You find the leaders and heads of communities, people who are networked and connected. Decentralized networks depend on humans clustering themselves into groups to receive and transmit info. This fact will be important for later.
But then, let’s say, someone pisses in the king’s cornflakes and he decides that everyone needs to hear about it. So, the king would send someone down to the town square and this person would stand there and scream what the king wanted everyone to know until his eyes bled and everyone begged him to shut up. These people were literally known as “town criers” and believe it or not, they held a lot of prestige for much of human history. They were the OG Walter Cronkite and Edward R. Murrow. When they showed up in town, people knew something serious was brewing. Town criers were rare but major events. The proclamation of a war, a new tax, a royal wedding. It was a big deal.
 
Proclamations are information that spread through centralized networks. It’s info that comes down from the top.
Centralized and decentralized networks still spread information today. In fact, not much has changed. People talk shit in their little clusters on social media, saying the prime minister is a weenie, then Mr. Prime Minister does a press conference and everyone shuts their traps and listens.
Other times, decentralized networks overrule centralized networks of authority. Kings and emperors are overrun by the disaffected masses. Civil rights and social justice movements organically emerge from overlapping groups with no central leader and change major policies. Both centralized and decentralized networks possess power. Both proclamations and hearsay spread information through society simultaneously and in their own ways.
Throughout history, some technological innovations have made proclamations (centralized networks) more effective ways of transmitting information. Like, whoever invented the first bullhorn suddenly made their town crier like three times more effective. That, in turn, would make whoever has control of screaming in the town square more influential and powerful than before.
Similarly, whoever invented the bulletin board increased the power of decentralized networks. Now you walk into the local tavern and read what dozens of people needed and cared about without having to find them and talk to them. This means whoever is best able to write effective notes on the local bulletin board would gain power. 19  
You could almost look at world history as a tug-o-war between centralized networks and decentralized networks, based on whatever the predominant communications technology was at the time. When centralized networks are prominent, power tends to be concentrated and autocratic, centered on the few people who have access to the bullhorn. But when decentralized networks are bolstered, it’s all about what groups can organize themselves and inspire followers. Centralized networks produce the tyrants and decentralized networks produce the revolutions that overthrow them and reset the game. And on goes the seesaw of human history.
Below are some of the most impactful technological developments and how they affected centralized and decentralized networks of information. As you can probably guess, we’re going to end up at the internet.
 Invented: Around 3,200 BCE
Promoted: Centralized networks (Proclamations)
Writing was the first and most important technological innovation in the world because it allowed knowledge to be shared between people who never met each other—including people from past generations with future generations.
On the surface, this sounds like it would promote decentralized networks. Everyone would start writing each other notes, chiseling their own clay tablets and handing them out like candy at a Thanksgiving Day parade.
But that’s not what happened. Back in the day, writing was expensive. Clay tablets, man! Shit ain’t cheap. On top of that, only the elites were educated enough to read and write, therefore, writing mostly served as a way to create proclamations given by the king.
It’s no coincidence that it’s around this point in human history that empires began to emerge. The written word allowed kings to coordinate their subjects over vast distances. It allowed them to unify millions of disparate peoples under a common code of laws and ethics. 20  
Later, the centralized control of information would grant the great religions dominion over most of the planet. Revolutions were minimal, if non-existent. The vast majority of the population couldn’t read or write, much less coordinate themselves across wide areas. Unless you were highly educated and highly connected, you weren’t able to write anything of importance or usefulness. And if you were highly educated and connected, you were part of the elite anyway, so why cause trouble? Censorship was absolute. Saying the wrong thing could get you killed. Everything went through your king or priest, no questions asked.
This period was no fun.
 Invented: 1440 CE
Promoted: Decentralized networks (Hearsay)
It’s a sad irony that the invention most likely responsible for creating the modern world resulted in the inventor going broke, being exiled, and dying a disgraced nobody. Johannes Gutenberg created the printing press, a machine that could mass-produce books and pamphlets affordably. What, at first, seemed like a convenient way to reproduce bibles and some poetry for a quick buck, would soon revolutionize the world.
 The next time you think *you* were under-appreciated, just think of Johannes Gutenberg. 
The reason writing initially favored centralized networks was because it was expensive, both in terms of labor and money. Teams of monks would spend months copying bibles by hand. Only kings, emperors, and popes had access to scribes to write out hundreds of copies of whatever they wanted to say.
But the printing press changed all that. Now anybody could approach a printer and have their ideas reproduced. Whereas the written word expanded the reach of the centralized networks, the printing press would similarly expand the reach of decentralized networks. And this expansion would shift the balance of power in the world.
That shift didn’t take long. In 1517, Martin Luther published his “95 Theses” criticizing the Catholic Church and launching the Reformation. For centuries, the Catholic Church had murdered any dissenters like Luther. But Luther had reprinted hundreds of copies of his theses. Anyone could pick up and read them. Killing Luther was no longer sufficient. You’d have to kill all his followers. Thus began war.
Luther’s Reformation would eventually spark over a hundred years of bloodshed across the European continent. It would inspire monarchs to distance themselves from the church. And it would help inspire small, holy enclaves to seek religious freedom in the New World.
But the printing press wasn’t done. By the 1600s, independent scholars were publishing their own work and promoting a new idea called “the scientific method.” The scientific method was decentralized—anyone could use it and arrive at their own conclusions. Ideas about inalienable human rights, the rule of law, and democracy would take hold. The ease of publishing and promoting work across decentralized networks allowed the Enlightenment to flourish.
The first newspapers emerged around the same time, 21  independently determining important information and disseminating it through local merchant networks, circumventing the king’s proclamations and town criers. This group of publishers was seen as an ally of the new Enlightenment ideas, as they helped coordinate social movements in the 17th and 18th centuries that would lead to democratic revolutions. The American Revolution relied heavily on the printing and dissemination of political pamphlets. 22  Similarly, revolutionaries in France coordinated across the country and, for the first time, realized they were getting royally screwed by the king (pun totally intended). As Robert Darnton, a historian from Princeton put it, “Without the press, the French Revolution was impossible and unthinkable.” 23  
That said, news media in this era was also a shitshow. Fake news proliferated. There were practically as many newspapers as there were people. And you think people are polarized today? Hoo boy… There were stabbings in Congress. Politicians shot at each other. Their supporters bombed and lynched and killed each other over nothing but rumors and insults… or rather, that is, hearsay.
 Invented: 1895 CE (radio); 1925 CE (television)
Promoted: Centralized networks (Proclamations)
I lumped radio and television together here because they both occurred at similar times and the same companies who started radio stations ended up also dominating television.
Radio/TV once again favored centralized networks for two reasons:
Like writing in the ancient world, running a television/radio station was prohibitively expensive and difficult and only the elites would have access to the resources to do so.
These technologies allowed someone to hit millions and millions of people with the exact same information simultaneously. It was as if a mutant town crier with a hundred vocal cords could scream for thousands of miles in every direction.
The 20th century presented a hard return of centralized power: it was peak colonialism, 24  the rise of totalitarian governments (of which North Korea is still a holdout), two world wars and that whole ugly Nazi thing.
Radio and television launched the industries of advertising and public relations. When millions of eyeballs are watching the same exact thing, this grants a huge opportunity for businesses (and governments) to persuade people to think or feel a certain way. This is where the ability to manipulate and influence people into certain opinions or desires became formalized into the practice of propaganda, or as it’s known today, marketing. 25  
 The Nazis were one of the first governments to master the art of radio propaganda. 
Radio and television likely created the most cultural solidarity we have ever seen in human history. Everyone was watching the same shows, listening to the same music and following the same events. The wars of the 20th century pitted half the planet against the other and everyone held on for their dear life. From World War I, up through the end of the Cold War, there was always relative political conformity and trust, especially because no matter what country you were in, you were likely united against some large external enemy.
 Invented: 1983 CE
Promoted: Decentralized networks (Hearsay)
The originators of the internet saw it as a great liberating force on humanity. 26  At the time, they lived with the media gatekeepers of the television and radio stations, limiting the vast majority of the population’s ideas from being widely expressed.
But now, with the power of networked computers, anybody could get online, start a website, and start spitballing their bullshit into the wind, hoping somebody would stop awhile and listen.
Whereas TV and radio expanded the reach of centralized networks to worldwide audiences, the internet would expand the reach of decentralized networks worldwide. A professor in Canada could exchange emails with a student in India and come up with an idea that would be tested in China and written up in newspapers in Australia.
 Obligatory cat meme goes here… 
It sounded like heaven. And while the internet has granted us incredible amounts of information and perspective, it hasn’t quite lived up to its utopian expectations. In fact, public discourse has kind of gone to shit. And it’s done so for a few reasons:
First, the old guard of media has faced the economic realities: for better or worse, pretty much any kid with a computer can compete with them now. This caused revenue to plummet and led them to cut large chunks of their staff—most notably, investigative journalists and foreign correspondents, the people who made them reliable and trustworthy in the first place. 27  Expensive news that took years of study and months to report was no longer economical. So the traditional media giants began to look more and more like their amateur competitors, while their amateur competitors leveraged marketing strategies to make themselves appear far more reliable than they actually were.
Second, because everyone’s phone is on them all the time and we’re all compulsively checking social media and email, emotionally impactful information (regardless of its truth or utility) spreads extremely fast. Thus, it’s easier than ever to make a quick buck by indulging in the worst human tendencies.  You want to net a bunch of traffic by cooking up a fake pedophile ring in a pizza shop? 28  Hell yeah, go for it!
Third, by giving voice to everybody, the internet unintentionally gave voice back to the uglier sub-cultures of society that were once barred from any mainstream attention: the bigots, the conspiracy theorists, and the cranks. In our idealism of giving the whole world a way to express themselves, we forgot that most people say really awful shit.  
The internet is like the modern printing press. 29  It allows anyone to express themselves, challenge authority, and share information on informal networks outside of the traditional pipelines. On the one hand, this is great. It grants greater freedom of expression and the opportunity for self-actualization to everyone. On the other hand, it’s a fucking mess, because as a society, we don’t know who to listen to or what to believe anymore.
Like the period after the printing press, we are seeing more explicit challenges to the establishment. We are seeing a resurgence of tribal and religious affiliations. We are seeing emerging conflicts over the identity and direction of country and culture. The times are tense. The internet has complicated… well, everything. But if we’re willing to look hard enough, we can spot iridescent flashes of collective brilliance breaking through an otherwise endless stream of bullshit.
To Sum Up
When centralized networks of information have the advantage, we get smaller quantities of highly relevant information. The danger of these networks of information is that they can easily be corrupted for tyrannical purposes like the Catholic Church in the Dark Ages or the totalitarian governments of the 20th century.
Decentralized networks promote higher quantities of low-quality information. The quantity of information generally fragments the population into warring factions based on tribal or religious identities. They only trust their preferred messengers and distrust everybody else.
Whereas centralized information networks promote conflict between political bodies, decentralized information networks promote conflict within political bodies.
Where centralized information networks solidify and reinforce cultural identity, decentralized information networks tend to overthrow and revolutionize cultural identities. Sometimes these revolutions are huge leaps forward for humanity (Enlightenment, civil rights, etc.) but other times, it’s just a bunch of arbitrary, religious-style fighting.
Decentralized information networks create a much larger diversity of information, but the signal-to-noise ratio is poor, making it incumbent on each individual to navigate the information and carefully choose what is worth consuming and what should be ignored.
To put it into plain English: the frustrations we have with news media—the negativity, the lies, the partisan bickery—it’s not going to change. It’s not going to get better. It’s on us to learn how to navigate the media environment and vote with our eyeballs. To do that, we must understand what information is actually useful and important, and what is just nonsense.
OK, let’s state the obvious. Some information is important, some is not. Some information feels important while being totally unimportant. And some information feels unimportant while it’s actually quite crucial.
 The most succinct way to sum up the issue today is that news media optimizes for information that  feels  important with little regard to its actual importance.  
If we look back through history, this is a common state of eras where information networks are highly decentralized. What travels is what feels impactful, even if it’s not—and even if it’s not true! Because the news has to compete in a highly diverse market with tons of noise, they must be ruthless in capturing people’s emotions as effectively as possible.
Eras of centralized information networks have had the opposite problem: they share information that seems unimportant but is actually quite important. Like the Soviet announcements of monthly bread rations or the long, tedious Congressional committees that drone on C-SPAN in the United States every day, these forms of news are dull and boring. But they matter.
Back in the 1950s, when there were only two TV channels, they could show a 12-hour congressional committee and you really had no option. That’s what was important. That’s what people needed to know. You lived with it whether you cared or not. But as cable news and the internet emerged, the incentives for media companies pushed them more into fighting for people’s hearts rather than their minds. They profited off having the most emotional information, not the most factual.
Today, if someone tries to show a congressional committee, the next channel is going to bury them with pictures of dead children in Sudan and plane crashes in Iran. There’s no chance.
This is why we’ve all experienced the “decline” of news media over the past 30 years: we’ve lived through the slow escalation of emotional impact at the cost of veracity and utility. On the one hand, we have more choice of information than ever before. But the average quality of that information is far lower. It’s simply the trade-off that comes with the technology. 30  
Useful macro information is information that directly impacts you and/or has an outsized influence on the economy. Prescription drug price policy is important information. The safety rating of your car is important information. A tariff on sugar is important information.
These things are boring, but they are important because they are the first-order effects that drive almost everything else. As the old saying goes, “It’s the economy, stupid.” Cultural issues usually happen downstream. 31  
Almost everything else is hogwash. Terrorist attacks? School shooters? Neither of these would probably be such a problem if it weren’t for the news coverage in the first place. Trump said something stupid? So what? Wake me up when there’s another $350 billion tariff.
When you focus on first principles, the news becomes incredibly empty. I’d say 80-90% of the content out there is either: a) catastrophizing some irrelevant event, b) horse race updates when all you need to know is who won, or c) opinion and commentary dressed up as news.
You can skip all this stuff. You don’t care about the events. You don’t need to know about the horse race, just the winner. You don’t need more opinions—you need data.
There are certain biases within our minds that we all fall victim to and the news media leverages them to keep us engaged and wanting more. There’s the negativity bias, a well-documented effect where people pay more attention and have more intense emotional responses to negative events than positive events. 32  There’s the present bias, the tendency to put more weight and importance on events that have happened recently, as opposed to far in the past or the future. There’s the narrative bias, the desire to arrange information in terms of a coherent story structure (i.e., “The stock market dropped 2% today because it rained in Michigan”) even if it’s not true. And finally, there is the almighty confirmation bias, the propensity to arrange new information so that it agrees with our previously-held beliefs.
 Chances are if you think about the news you’ve consumed lately, pretty much all of it is: negative, recent, story-driven, and supports your previously held beliefs. 
News media leverages our biases to keep us hooked and consuming more information, regardless of its utility. Especially in our decentralized world of the internet, the name of the game is attention, and the news media will use every tool to leverage more of yours. As such, news media specializes in information that is immediate, fast-moving, narrative-driven and highly visible. 33  
The problem is that the most important information is usually long-term, slow-moving, impersonal, abstract and invisible, and not necessarily negative:
Car accidents kill more people than terrorism, mass shootings, and natural disasters combined. How much reporting do you see on car accidents or driver safety?
The sequencing of the human genome was arguably more important than any other scientific development in the 1990s, opening a path to predicting disease and conditions before they occur, altering and improving the health of millions, if not billions, of people over the coming generations. How many weekly updates did you hear about it?
More than 1 billion people have been lifted out of extreme poverty worldwide, 34  due to a combination of greater access to health care, sanitation, and the opening of markets for greater commerce. Who knew?
Despite the fact the first international climate agreement was signed in 1992 by 154 countries, news media has just only recently started covering climate change with any significance—and if we’re being honest, it’s probably just to go along with all the natural disasters that they’ve been milking.
Paradoxically, in its effort to inform wide audiences, news media positions itself to misinform and over-inflate extreme outlier stories.
But this is not the news media’s fault. It’s simply how information scales in large decentralized human networks. The information with the greatest emotional valence will travel the furthest and be broadcast the loudest. 35  Gone are the days where we have reliable journalistic mainstays who can consistently nail what is most significant. The structures simply aren’t there.
Instead, we must rely upon ourselves. After all, in the internet age, we are the ones who are empowered: the user. We must screen out crap information and consciously seek out useful information. We must develop an awareness of how our attention is being seduced and prevent ourselves from getting sucked into the social media vortex of never-ending comment threads.
For the first time in human history, our relationship with news depends on us.  
Here’s the short version: Seek out the highest-value content you can find, and then pay for it. Limit your consumption to a few pieces of content per week. Go for quality over quantity.
By high-value content, ask yourself how much that content cost in terms of the labor, research, expertise, production, etc. Long-form content is more expensive than short-form. Research and citations are more difficult to put together than just some old dude ranting. Something that explains the historical context and acknowledges multiple sides of an issue is more difficult and time-consuming than a partisan rant.
I focus on the expensive thing because right now, the news media industry is going through a slow-moving crisis of their own design (and ironically, it’s too slow-moving for them to report on it). Research, fact-checking, traveling to some crazy war-torn country, and not getting your shit blown up—these are all expensive. Hiring a journalist with decades of knowledge on a region is expensive. Interviewing professors and digging into research is expensive. 36  Meanwhile, online advertising revenue pays next to nothing. So, more and more, media outlets are subtly and secretly pushing out cheap content that looks expensive—uninformed 24-year-olds who can sorta, kinda sound like an experienced journalist with a lot of Google-fu.
There are only a few publications that have the luxury of not playing the Clickbait Headline Game, and that’s because they charge enough money that they don’t have to. Publications such as The Economist, 37  Financial Times or Foreign Policy don’t give a shit if their articles are fun or not. They’re not writing for the general public. They’re writing for professionals who depend on the information to make big, expensive decisions. This is where you want to insert yourself into the information pipeline. Long, boring, thoughtful pieces, put together by long, boring, thoughtful people. One good piece of long-form journalism that focuses on real data is worth dozens of crappy short-form pieces you find elsewhere.
When you get into the less expensive publications, you start to run into a lot of bias. 38  You can feel the articles getting more frantic and the headlines more hyperbolic. Then, once you’re into the cheap (or free) seats, you’re basically being spoon-fed what to think and feel.
 The point to remember is that you want your news to be a little bit dull. If you’re getting excited or worked up reading something, chances are you’re being taken for a ride.  
If you can’t afford more expensive news publications, then I recommend skipping news entirely and simply reading books. In terms of quality information per dollar, books are the best bang for your buck. These days, I aim to make books account for at least 80% of my total reading time.
You may worry that by only reading books, you’ll “fall behind.” But I think you’ll actually find the opposite. If you take the hours and minutes that you used to spend reading news and instead read books about the same topics, within a few months, the “news” that everyone is freaking out about will seem trifling to you. You will see how slowly the world truly moves. You will have more context and clarity than most and you will recognize that almost nothing needs to be known the moment it happens.
Finally, I’d be remiss if I didn’t mention blogs and podcasts. Blogs/podcasts are interesting in that they don’t succumb to the same economic realities of mainstream news media—they don’t demand huge overhead expenses to maintain, and they rely on the personal trust of their readers to continue to exist. Also, most prominent bloggers and podcasters gained their expertise elsewhere, therefore allowing them to offer high-value information at a huge discount.
Put simply: most blogs and podcasts suck. But the best ones are arguably better than any other content on the planet right now. 39  
If you can find one of these experts, they can often be a gold mine of non-partisan, on-the-ground information. Bloggers such as Tim Urban, Ben Thompson, Tyler Cowen, Scott Alexander, and Dan Wang provide a ton of value for little to no cost at all.
But perhaps more important than finding sources of high-quality information is cutting out the sources of low-quality information. And there is a lot of low-quality information out there. If you get nothing else from this behemoth of an article, please consider cutting the following from your information diet entirely:
 STOP Watching News on Television – In surveys, cable news is consistently the least informative and most biased source of information. The nature of television demands constant stimulation and therefore undermines pretty much any serious discussion of a topic. 
 
 STOP Following News Sites on Social Media – Trust me, if something is important enough, you’ll hear about it anyway. People will link to it. They’ll email you. Whatever. When you follow news sites on social media, you are opting in to the attention game. You are allowing yourself to be triggered and fought over by egregious headlines, moral outrage, and clickbait. Just don’t play it. Unfollow all news sources on your social media. Not only will it make your news experience better, but it will make your social media experience a hundred times better.
 STOP Clicking on Clickbait – I know it’s easier said than done. But seriously, get into the habit of actively seeking out information instead of passively receiving it on your newsfeeds or apps. The algorithms on these platforms do not work in your favor. Cut them all out and then when you need to know about a subject, go looking for the info yourself, preferably in a book.
 STOP Consuming News That Requires Less than 20 Minutes – Most short-form news is garbage. If something requires more than 20 minutes to consume, that likely means it took weeks or months (years?) to produce. This means skip the hot takes, the quick interview clips, the editorials, and the breaking news alerts. Instead, download long-form articles, watch documentaries, listen to long-form interviews and podcasts where the person has time to explain themselves and get into the nuance of their positions. The world is complicated. You need time to sort it out.
 STOP Getting All Your Information from One Source – Make a conscious effort to check both left-wing and right-wing sources. Get info from minorities and women and foreigners. Follow professors and follow really smart kids with too much time on their hands. There’s a lot to be said for diversity. But the real diversity is diversity of thought. The wider the range of ideas you expose yourself to, the less you’ll ever be caught off guard by whatever new event happens.
I was in Australia over the New Year. As I’m sure you know, there are huge bushfires that have consumed huge stretches of land there, destroying natural habitats, and displacing thousands of people.
We saw the smoke when we were there. We rode a train through the Blue Mountains and saw acres of charred forests. On our first day in Sydney, the pollution level was the same as a small coal-mining Chinese city… i.e., toxic to even be outside. We talked to the locals in Adelaide about the burnt wine country. But all in all, it was like anywhere else I’ve been after a disaster: it’s sad, but life goes on. People are getting on with it.
Then I started getting emails and messages from friends and family back home. “Are you OK? Have you been hurt?” They said, “You should come home! It’s not safe!” I looked up from my phone: sunny skies and beautiful beaches. What in the hell were they going on about?
I was then subjected to what I could only call insanity. News headlines claiming that Australians’ way of life is forever lost, that everything is burning, that over a billion animals (an absurd number) have been killed. One article even went so far as to imply that the entire country will soon have to be abandoned. 40  
 This map is fake. But that didn’t stop it from being posted on approximately eight billion news sites. 
I’m as alarmed as anyone about climate change, but come the fuck on.
Every few months it seems as though there’s something like this. Whether it’s Brexit or Trump’s impeachment or the Amazon burning or China’s internment of a million Uighurs—we live in the Age of Hyperventilation.
But while there’s an endless stream of tragedy in the world, there is so much that the news never teaches you or shows you.
The news doesn’t show you how bad humans are at predicting the future. How pretty much every projection, scientific or not, ends up incredibly inaccurate. It doesn’t tell you that we’ve been predicting civilization-leveling catastrophes since pretty much the dawn of human thought—and every time, we have been wrong.
The news doesn’t teach you that technology doesn’t develop linearly—how it comes in unexpected leaps, and then trounces prior assumptions under the weight of its efficiency.
The news doesn’t tell you that natural disasters are just that: natural. Yes, they may be occurring slightly more often and be more severe, but they are, and have always been, the rule, not the exception.
The news doesn’t show that the vast majority of people are good. They will help if they can. They care even if they’re confused about how to care or why. The news doesn’t teach you that most people won’t hurt you and even if they do, you will recover and be fine and be stronger than before.
But most of all, the news doesn’t teach you that moral outrage, while strangely satisfying, gives people a sense of accomplishing something without actually accomplishing anything. Posting on social media or writing a scathing op-ed may feel as though it’s contributing to some grand movement, some great push for humanity, but that push is, more often than not, grounded in nothing.
Nowadays, I briefly check a handful of news sites, maybe once a week. 41  I read maybe five news articles in total. They are articles I choose, based on my own perception of what’s important and what matters. The rest is books, blogs, and podcasts.
Months ago, my friends felt sorry for me. They worried that I’d be out of touch, that I’d lose my connection to society and the world, and that I’d become disempowered to change it.
But the opposite has happened. I have realized that it is they who are out of touch, disconnected, and disempowered.
And this is the ultimate failure of the news media as currently constructed: it creates a passivity, a false feeling that we’re not somehow part of the narrative that we are also watching unfold. It lulls us into thinking that we don’t—that we can’t—have some role to play in the drama. It convinces us that our obstacles are too great, that our fears are too deep, that we are incapable of overcoming the same challenges of the thousands of generations who have come before us.
And this is what was lost. The town crier was never meant as a mere announcer. He was an instigator. He was a call to arms to leave our domestic safety and take up the business of the world that is ours. He reminded us to not simply receive the news but to go out and be the news as well.
What news will you be?
Why You Should Quit the News
https://markmanson.net/why-you-should-quit-the-news
https://markmanson.net/attention-economy
In The Future, Our Attention Will Be Sold
2014-12-04
Mark Manson
<div><div>
	<section><p>Last week, I logged onto Facebook to see a story about a man who got drunk, cut off his friend’s penis, and then fed it to a dog. This was followed by a story of a 100-year-old woman who had never seen the ocean before. Then eight ways I can totally know I’m a 90’s kid. Then 11 steps to make me a “smarter Black Friday shopper,” an oxymoron if I ever saw one.</p>
</section>
<p>This is life now: one constant, never-ending stream of non sequiturs and self-referential garbage that passes in through our eyes and out of our brains at the speed of a touchscreen.</p>
<p>Last month, Kim Kardashian “broke the internet.” And by “broke the internet,” I mean she photoshopped a picture of her bare ass and put it on <a href="http://www.glamourmagazine.co.uk/news/celebrity/2014/11/12/kim-kardashian-naked-for-papermag-break-the-internet">a magazine cover</a>.</p>
<p>But, of course, because it’s a naked ass, and because that ass happens to belong to Kim Kardashian, the photo was republished umpteen bazillion times, as adults rolled their eyes and guffawed at the insanity of it all, while the teenagers giggled and masturbated.<a href="#footnote-1"><sup>1</sup></a></p>
<p>The media brought out the obligatory screeds about how lewd or classless the pictures were. People then complained that the media gives this woman—a woman who has accomplished nothing in her life other than being born rich and fucking rappers—way too much airtime.</p>
<p>Then people pointed out that criticizing the media for publicizing Kardashian ironically just gives Kardashian <em>even more</em> attention and cultural presence. Being the internet, these people were then berated for being neckbeard virgins and their mothers were threatened with sexual violence. The neckbeards then responded by creating ironic memes that involved Kardashian’s bare ass and an espresso machine. Everyone LOLZ’d and moved on with their lives.</p>
<p>And this is why she’s still famous.</p>
<p></p>
<p>Kardashian has come to represent pretty much everything we loathe about the social media age: bite-sized, meaningless content that you hate looking at, but for some reason can’t look away from.</p>
<p>And because nobody has the self-control to look away, the bite-sized meaningless content spreads like wildfire, creating an online experience of a never-ending series of cultural car wrecks where we all gawk, rubberneck, discuss, and/or make fun of something for 12 minutes until distracted by the next oncoming collision.</p>
<p>There are three common complaints against social media and the internet in general: 1) that it’s making us all narcissistic and shallow,<a href="#footnote-2"><sup>2</sup></a> 2) that it’s crapping on our ability to maintain <a href="https://markmanson.net/relationship-advice">meaningful relationships</a> and therefore making us lonelier,<a href="#footnote-3"><sup>3</sup></a> and 3) that it’s interfering with our ability to focus and get quote, unquote “more important shit” done in our lives.<a href="#footnote-4"><sup>4</sup></a></p>
<p>Interestingly, after quite a bit of research, it turns out that none of these claims are completely accurate. <a href="https://markmanson.net/social-media">Social media doesn’t necessarily <em>cause</em> people to become more narcissistic</a>, it just gives narcissistic people more opportunities to indulge their narcissism, and to a larger audience.<a href="#footnote-5"><sup>5</sup></a> It’s not interfering with either the closeness we feel to others or how many people we feel close to, it simply expands our network of casual acquaintances and the quantity of our casual social interactions.<a href="#footnote-6"><sup>6</sup></a> And while technology does present more opportunities for distraction (which we’ll get to), it also presents easier transmission of information, tools for collaboration, and opportunities for organization.<a href="#footnote-7"><sup>7</sup></a></p>
<p>What I’m saying is that the whole “the internet is ruining us” argument is a big whiff. It’s likely just the anxiety that’s always wrought by new technologies. When TV and radio were invented, people complained that everyone’s brain was going to go to mush. When the printing press was invented, people thought it was going to destroy our ability to speak eloquently. Complaints about the minds of children being ruined by technology are as old as technology itself.<a href="#footnote-8"><sup>8</sup></a></p>
<p>Modern technology isn’t changing us. It’s changing society. There’s a difference. One is how <em>we</em> are, and one is simply how we react every day to the world around us. The <a href="https://markmanson.net/social-media">social media age</a> is changing the basic economics of our day-to-day lives. It’s changing them in profound ways, ways that most of us likely don’t notice. And surprisingly, it’s people like Kim Kardashian who are taking advantage of it.</p>
<div>
<p>If you’ve ever <a href="https://markmanson.net/5-life-lessons-5-years-traveling-world">spent time</a> in a really poor country or with people who grew up in awful poverty, you’ll notice how much they talk about food—their favorite foods, what they’re going to eat this weekend, how they like this and don’t like that, and so on.</p>
<p>Much of these people’s lives and conversations revolve around food for the simple reason that the scarcity of food makes it appear incredibly important. The fact you prefer strawberries to oranges matters a lot when you can rarely afford to have either. But in rich societies where food is never an issue, discussions of food among most people are superficial and usually over within a few seconds.</p>
<p>For most of human history, the big economic scarcity in the world was <strong>land</strong>. There was a limited amount of productive land, therefore there was a limited amount of food. And because there was a limited amount of food, most day-to-day economic concerns and political squabbles involved land. Most people spent their lives contemplating what land they were going to work, what they were going to grow, what kind of harvest to expect, and so on. Food was always at the top of people’s mind.<a href="#footnote-9"><sup>9</sup></a></p>
<p>Eventually, when the industrial revolution hit, the primary scarcity was no longer land, as machines could now help cultivate more than enough food for everybody. Now the big scarcity was <strong>labor</strong>. You needed trained people to run all of these machines that did all of the cool new shit so you could make money and get rich. Thus, for a few hundred years, the organizing principle in society was based on labor—who you worked for, how much you made, and so on.</p>
<p>Then, in the 20th century, there was more being produced than any one person would ever need or could ever purchase. The new scarcity in society was no longer labor or land, the scarcity was now <strong>knowledge</strong>. People had so many choices of what to purchase with their hard-earned money, but they didn’t <em>know</em> what to purchase. Thus, people spent most of their day-to-day existence trying to figure out what the best toothpaste was, what a toaster oven could do, how to spend their bonus money over the holidays and so on. The fields of advertising and marketing were invented and came to dominate society, as they were the means of disseminating the information people needed to allocate their resources appropriately.</p>
<p>Still with me? Because <em>this</em> is where the internet and <a href="https://markmanson.net/smartphones">smartphones</a> have fucked everything up—or, ahem, where they’ve “disrupted” everything.</p>
<p>With the advent of the internet, the primary scarcity in society is no longer information. In fact, there is now more information than any of us could possibly know what to do with. If you want to know about a new product, you can have the Wikipedia article and 500 Amazon reviews up within 10 seconds. If you want a refresher on the process of photosynthesis, you can have it figured out within a few minutes. If you need to know every actor Kevin Spacey has ever worked with, you can figure it out in mere seconds.</p>
<p>The scarcity in our world is no longer knowledge. There’s an abundance of knowledge, just as there’s an abundance of labor and an abundance of land.</p>
<p>No, the new scarcity in the internet age is <strong>attention</strong>. Since there is a surplus of information, more information flowing through our society than <em>any of us</em> could ever hope to process or understand, the new bottleneck on our economy is attention. We now live in an attention-based economy.<a href="#footnote-10"><sup>10</sup></a></p>
<p>This is why today we are each bombarded with over 3,000 advertising messages per day. This is why these advertisements get zanier and more nonsensical—like the Geico gecko or the Old Spice guy—because the goal of advertisements is no longer information but simply attention.</p>
<p></p>
<p>This is why social media is plastered with ridiculous article headlines such as, “I Thought I Was Going To Die, But Then You’ll Never Believe How This Polar Bear Saved My Life,” and when you click it, it takes you to a series of stupid animated GIFs or a YouTube video that has nothing to do with polar bears but is instead slathered in advertisements.</p>
<p>This is why politics is becoming less about actual policies and more about dramatic actions designed to draw either positive or negative attention to various actors and political parties.<a href="#footnote-11"><sup>11</sup></a></p>
<p>This is why everything is becoming a version of softcore porn: music videos, commercials, movies, and reality TV shows. And when it’s not softcore porn, it’s some other kind of porn: food porn, murder porn, disaster porn, or actual, like <a href="https://markmanson.net/pornography-can-ruin-your-sex-life">real-life porn</a>. Porn gets attention. And today, attention is what sells.</p>
<p>This is why Kim Kardashian is famous and has continued to be famous for the better part of a decade for no other reason than—you guessed it—she’s already famous! This woman has contributed absolutely nothing to humanity. Yet in the age of attention, she is goddamned Master Yoda. She’s got it all figured out.</p>
<p>Kim Kardashian is a genius. Not an Einstein-like genius. Not a “solves differential equations in her head but can’t tie her shoes” genius. But she’s a genius. A savant. She’s like the Rain Man of attention whoredom. The same way an autistic prodigy can count 2,318 toothpicks dumped on the floor just by looking at them, Kim Kardashian can command the attention of tens of millions of people with the crack of her ass.</p>
<p>The quality of that attention doesn’t matter. What matters is the attention. That attention is an asset, the most valuable asset in the new economy. Millions of eyeballs follow her wherever she goes and she leverages the shit out of it. She earns millions off a crappy iPhone app that does nothing and a TV show that shows nothing. Just the fact she stands in a nightclub allows that club to charge $2,500 for entry. She gets paid more for public appearances than Nobel Prize winners. <a href="http://www.forbes.com/sites/clareoconnor/2014/06/30/kim-kardashians-28-million-year-how-she-made-more-than-ever-before/">Forbes estimates</a> her income last year at $28 million.</p>
<p></p>
<p>But making stupid people rich and famous isn’t anything new in our culture. While the attention economy may exacerbate the problem, it didn’t create it. But when we apply the attention economy to the other areas of our lives, we run into some problems.</p>
</div>
<div>
<h2>How the Attention Economy Promotes Extremism</h2>
<p>Social networks are the business model of the attention economy. They are wholly dependent on eyeballs and clicks to make all of their revenue. To do this, they design algorithms that show you the most interesting and attention-grabbing information available in your social network.</p>
<p>If your newsfeed was full of the boring and drab day-to-day stuff, you’d stop looking at it. So instead, Facebook shows you the most extreme occurrences in your social network for the simple reason that the extreme events draw the most attention.</p>
<p>This has drastic effects not only on our perceptions of society as a whole, but also on how we perceive our personal lives.</p>
<ol>
<li>If it seems like “everybody” is getting married or having kids or having amazing trips around the world or doing something cool and fun and sexy, it’s only because we are exposed to these events in disproportionate numbers. It’s not that everyone is having amazing life experiences all the time, it’s that we’re always <em>shown</em> people’s amazing life experiences all the time. As a result, many of us begin to feel a constant gnaw of <a href="https://markmanson.net/you-cant-have-it-all">somehow “missing out”</a> when really, we just have a heavily-biased perception of what’s going on in our peers’ lives.</li>
<li>The attention economy rewards people who are narcissistic and self-promotional because these people excel at getting attention. Therefore, it <em>seems</em> that everyone is becoming more shallow and self-absorbed, when in fact, we are merely becoming more exposed to other people’s self-promotion.</li>
<li>Politically, the most extreme, radical, and ignorant views get the most airtime because they’re the most unique and they grab the most attention. Therefore, it appears as if the world is spiraling into a festering shithole, when really, we’re just getting exposed to the people on the fringes more often than ever before.</li>
<li>Threats such as Ebola or terrorism become sensationalized, not because they’re actually that threatening, but because of their extremity and how much attention they garner. You’re more likely to get eaten by a shark while getting struck by lightning than die from a terrorist attack. You’re more likely to die from the flu this year than you are from Ebola, ever. Yet, in our culture, it feels as though the world is in a constant state of imminent collapse.</li>
<li>Pointless but dramatic events such as nipple slips, gaffes, errant interviews, and celebrities doing stupid celebrity stuff seem as though they are taking on a much greater cultural significance than they actually are. If Kim Kardashian got hit by a truck tomorrow and died, I’m sure you’d see the usual media-driven mourn fest and televised funeral, but would anyone really miss her? Would anybody’s life be altered? Probably not. In fact, we’d probably get another really good Kanye album.</li>
</ol>
<p>When you look at all of the complaints about social media, smartphones, and the internet at large, most of the complaints boil down to one thing: attention. People don’t have <a href="https://markmanson.net/how-to-improve-focus">any attention span</a> anymore. People don’t focus on what’s in front of them anymore. People don’t even fucking talk to you at dinner anymore.</p>
<figure><figcaption>Typical “date night” these days.</figcaption></figure>
<p>The problem is that the attention economy <em>makes it economical</em> to spread one’s attention across eight different interests and 23 different friends each day. And because we’re all spreading our attention so thin, many of us are losing the all-important life skill of <strong>focus</strong>.</p>
<p>Focus is what generates long-term <a href="https://markmanson.net/5-mindsets-that-create-success">success</a>. Focus leads to deeper and more meaningful relationships. Focus determines how well we can improve at something. Yet our current economy is constantly providing incentive away from focus and towards—whoa, did you see that video of the guy on the motorcycle who landed on the car? That was crazy!</p>
<p></p>
<p>Anyway, what was I talking about? Oh yeah… focus. The new age presents problems of attention, not of happiness or narcissism or loneliness. And as the technology’s critics point out, this issue is not going to go away, it’s simply going to get worse.</p>
</div>
<div>
<p>But the attention economy brings with it a host of social benefits, many of which are not immediately obvious to us. In fact, some of the benefits are actually painful to bear in some ways.</p>
<h3>Salience of Political Issues</h3>
<p>Take, for example, the recent exposure that <a href="http://www.vulture.com/2014/09/timeline-of-the-abuse-charges-against-cosby.html">Bill Cosby is (very likely) a rapist</a>. This is a man that was a role model for decades and took that role seriously, traveling around the country lecturing people on <a href="https://markmanson.net/personal-values">values</a> and responsibilities. Women had been accusing him of rape for years, yet it was the attention economy that finally allowed the allegations to make their way into the public consciousness in a significant way.</p>
<p>Pre-internet, things such as sexual assault or gay rights or intransigent racism or the failed drug war, they were all abstract concepts that we never had to confront in any tangible way. They were fairy tales of some far away land. They had nothing to do with us.</p>
<p>But new technologies bring these issues right in front of our face, over and over again, whether we like them or not. Sexual assault does happen and most of the perpetrators do go unpunished. LGBT people are humans too, and they deserve to be treated like humans. The drug war is a dismal failure and ruining millions of people’s lives. And because these experiences are now easily transmittable and experienced by large swaths of the population, we see these social policies changing faster than ever before.</p>
<p>The past five years have seen a wave of protests and political mobilization, mostly throughout the developing world. From Egypt to Turkey to Brazil to Mexico, millions of people are able to react immediately upon major political issues and demand changes from long-corrupt governments. Sure, most of them don’t work. But some of them do. And at the least, they alter the political discourse.</p>
<p>Just look at what’s been going on lately with racism in the US and <a href="https://en.wikipedia.org/wiki/Ferguson_unrest">Ferguson</a> and <a href="http://time.com/3016326/eric-garner-video-police-chokehold-death/">Eric Garner</a>. Regardless of what you think, the issue of racism and police brutality is forced on you. You can’t turn away. There are videos. There are dead kids. It’s impossible to <em>not</em> see it, and therefore an issue that has lasted for as long as the country has existed is being confronted more seriously than it has been in decades.</p>
<h3>Ease of Relationships</h3>
<p>But beyond politics, technology does benefit our relationships, even if we don’t always recognize it.</p>
<p>We are the first generation that is still within easy contact of old friends from grade school, summer camp, and college. We are capable of being in constant contact with one another, for better or worse. It’s never been easier to transmit important life events and emergency situations worldwide. Travel and <a href="https://markmanson.net/long-distance-relationships">long distance relationships</a>—romantic or otherwise—are easier than ever before. One can spend months or even years abroad and never feel “out of touch” with what’s going on at home.</p>
<p>Obviously, technology shouldn’t replace face time and in-person interactions. But it can definitely supplement them.</p>
<h3>Limitless Knowledge</h3>
<p>And, of course, there is simply the breadth of information available at our fingertips. For all of the headaches the attention economy can cause, it’s the cost of having limitless information accessible at any time. You used to have to go to the library to do research. You used to have to call stores to find out if they had something in stock. You used to have to buy a map and spend five minutes studying it to figure out where you wanted to go.</p>
<p>We forget so easily the whole point of all of this in the first place: the availability of limitless and free knowledge. These benefits are so widespread and ubiquitous that <em>we can’t even remember what it was like to not have them</em>. And as a result, we overestimate how much these technologies are hurting us and underestimate how much they’re helping us.</p>
<p>Yes, the attention economy has come with new social challenges like identity theft and cyberbullying and assholes who text while they drive. But let’s not throw the baby out with the bathwater here.</p>
</div>
<div>
<p>These are the new challenges new generations must face. Our grandparents had to learn to master their time and their energy in order to take advantage of the labor economy. Our parents had to master their minds and their problem-solving in order to take advantage of the knowledge economy. We must learn to <a href="https://markmanson.net/how-to-improve-focus">master our focus</a> and our <a href="https://markmanson.net/self-awareness">self-awareness</a> to properly take advantage of the attention economy.</p>
<p>Limitless access to knowledge brings limitless opportunity. But only to those who learn to manage the new currency: their attention. In the new economy, the most valuable asset you can accumulate may not be money, may not be wealth, may not even be knowledge, but rather, the ability to control your own attention, and to focus.</p>
<p>Because until you are able to limit your attention, until you are able to turn away, at will, from all of the shiny things and nipple slips, until you are able to consciously choose what has value to you and what does not, you and I and everyone else will continue to be served up garbage indefinitely. And it will not get better, it will get worse.</p>
<p>In the future, your attention will be sold. And it may be that the only people able to capitalize on it, are the people who can control their own.</p>
<p>Now, if you’ll excuse me, I have some stupid cat videos to watch.</p>
</div>
	</div>||||I|||| Skip to content

 Menu

 * Articles
 * Books
 * Courses
 * Newsletter
 * Log In
 Log In Sign Up

 Become a member

 Get a single membership for just $6.67 per month

 Sign Up

 In the Future, Our Attention Will Be Sold

 Written By Mark Manson – filed under Focus

 L ast week, I logged onto Facebook to see a story about a man who got drunk, cut off his friend’s penis, and then fed it to a dog. This was followed by a story of a 100-year-old woman who had never seen the ocean before. Then eight ways I can totally know I’m a 90’s kid. Then 11 steps to make me a “smarter Black Friday shopper,” an oxymoron if I ever saw one.

 This is life now: one constant, never-ending stream of non sequiturs and self-referential garbage that passes in through our eyes and out of our brains at the speed of a touchscreen.

 Last month, Kim Kardashian “broke the internet.” And by “broke the internet,” I mean she photoshopped a picture of her bare ass and put it on a magazine cover.

 But, of course, because it’s a naked ass, and because that ass happens to belong to Kim Kardashian, the photo was republished umpteen bazillion times, as adults rolled their eyes and guffawed at the insanity of it all, while the teenagers giggled and masturbated.1

 The media brought out the obligatory screeds about how lewd or classless the pictures were. People then complained that the media gives this woman—a woman who has accomplished nothing in her life other than being born rich and fucking rappers—way too much airtime.

 Then people pointed out that criticizing the media for publicizing Kardashian ironically just gives Kardashian even more attention and cultural presence. Being the internet, these people were then berated for being neckbeard virgins and their mothers were threatened with sexual violence. The neckbeards then responded by creating ironic memes that involved Kardashian’s bare ass and an espresso machine. Everyone LOLZ’d and moved on with their lives.

 And this is why she’s still famous.

 Kardashian has come to represent pretty much everything we loathe about the social media age: bite-sized, meaningless content that you hate looking at, but for some reason can’t look away from.

 And because nobody has the self-control to look away, the bite-sized meaningless content spreads like wildfire, creating an online experience of a never-ending series of cultural car wrecks where we all gawk, rubberneck, discuss, and/or make fun of something for 12 minutes until distracted by the next oncoming collision.

 There are three common complaints against social media and the internet in general: 1) that it’s making us all narcissistic and shallow,2 2) that it’s crapping on our ability to maintain meaningful relationships and therefore making us lonelier,3 and 3) that it’s interfering with our ability to focus and get quote, unquote “more important shit” done in our lives.4

 Interestingly, after quite a bit of research, it turns out that none of these claims are completely accurate. Social media doesn’t necessarily cause people to become more narcissistic, it just gives narcissistic people more opportunities to indulge their narcissism, and to a larger audience.5 It’s not interfering with either the closeness we feel to others or how many people we feel close to, it simply expands our network of casual acquaintances and the quantity of our casual social interactions.6 And while technology does present more opportunities for distraction (which we’ll get to), it also presents easier transmission of information, tools for collaboration, and opportunities for organization.7

 What I’m saying is that the whole “the internet is ruining us” argument is a big whiff. It’s likely just the anxiety that’s always wrought by new technologies. When TV and radio were invented, people complained that everyone’s brain was going to go to mush. When the printing press was invented, people thought it was going to destroy our ability to speak eloquently. Complaints about the minds of children being ruined by technology are as old as technology itself.8

 Modern technology isn’t changing us. It’s changing society. There’s a difference. One is how we are, and one is simply how we react every day to the world around us. The social media age is changing the basic economics of our day-to-day lives. It’s changing them in profound ways, ways that most of us likely don’t notice. And surprisingly, it’s people like Kim Kardashian who are taking advantage of it.

 Table of Contents

 * The Attention Economy
 * The Attention Economy and Extremism
 * The Silent Benefits of the Attention Economy
 * Taking Advantage of the Attention Economy

 The Attention Economy

 If you’ve ever spent time in a really poor country or with people who grew up in awful poverty, you’ll notice how much they talk about food—their favorite foods, what they’re going to eat this weekend, how they like this and don’t like that, and so on.

 Much of these people’s lives and conversations revolve around food for the simple reason that the scarcity of food makes it appear incredibly important. The fact you prefer strawberries to oranges matters a lot when you can rarely afford to have either. But in rich societies where food is never an issue, discussions of food among most people are superficial and usually over within a few seconds.

 For most of human history, the big economic scarcity in the world was land. There was a limited amount of productive land, therefore there was a limited amount of food. And because there was a limited amount of food, most day-to-day economic concerns and political squabbles involved land. Most people spent their lives contemplating what land they were going to work, what they were going to grow, what kind of harvest to expect, and so on. Food was always at the top of people’s mind.9

 Eventually, when the industrial revolution hit, the primary scarcity was no longer land, as machines could now help cultivate more than enough food for everybody. Now the big scarcity was labor. You needed trained people to run all of these machines that did all of the cool new shit so you could make money and get rich. Thus, for a few hundred years, the organizing principle in society was based on labor—who you worked for, how much you made, and so on.

 Then, in the 20th century, there was more being produced than any one person would ever need or could ever purchase. The new scarcity in society was no longer labor or land, the scarcity was now knowledge. People had so many choices of what to purchase with their hard-earned money, but they didn’t know what to purchase. Thus, people spent most of their day-to-day existence trying to figure out what the best toothpaste was, what a toaster oven could do, how to spend their bonus money over the holidays and so on. The fields of advertising and marketing were invented and came to dominate society, as they were the means of disseminating the information people needed to allocate their resources appropriately.

 Still with me? Because this is where the internet and smartphones have fucked everything up—or, ahem, where they’ve “disrupted” everything.

 With the advent of the internet, the primary scarcity in society is no longer information. In fact, there is now more information than any of us could possibly know what to do with. If you want to know about a new product, you can have the Wikipedia article and 500 Amazon reviews up within 10 seconds. If you want a refresher on the process of photosynthesis, you can have it figured out within a few minutes. If you need to know every actor Kevin Spacey has ever worked with, you can figure it out in mere seconds.

 The scarcity in our world is no longer knowledge. There’s an abundance of knowledge, just as there’s an abundance of labor and an abundance of land.

 No, the new scarcity in the internet age is attention. Since there is a surplus of information, more information flowing through our society than any of us could ever hope to process or understand, the new bottleneck on our economy is attention. We now live in an attention-based economy.10

 This is why today we are each bombarded with over 3,000 advertising messages per day. This is why these advertisements get zanier and more nonsensical—like the Geico gecko or the Old Spice guy—because the goal of advertisements is no longer information but simply attention.

 This is why social media is plastered with ridiculous article headlines such as, “I Thought I Was Going To Die, But Then You’ll Never Believe How This Polar Bear Saved My Life,” and when you click it, it takes you to a series of stupid animated GIFs or a YouTube video that has nothing to do with polar bears but is instead slathered in advertisements.

 This is why politics is becoming less about actual policies and more about dramatic actions designed to draw either positive or negative attention to various actors and political parties.11

 This is why everything is becoming a version of softcore porn: music videos, commercials, movies, and reality TV shows. And when it’s not softcore porn, it’s some other kind of porn: food porn, murder porn, disaster porn, or actual, like real-life porn. Porn gets attention. And today, attention is what sells.

 This is why Kim Kardashian is famous and has continued to be famous for the better part of a decade for no other reason than—you guessed it—she’s already famous! This woman has contributed absolutely nothing to humanity. Yet in the age of attention, she is goddamned Master Yoda. She’s got it all figured out.

 Kim Kardashian is a genius. Not an Einstein-like genius. Not a “solves differential equations in her head but can’t tie her shoes” genius. But she’s a genius. A savant. She’s like the Rain Man of attention whoredom. The same way an autistic prodigy can count 2,318 toothpicks dumped on the floor just by looking at them, Kim Kardashian can command the attention of tens of millions of people with the crack of her ass.

 The quality of that attention doesn’t matter. What matters is the attention. That attention is an asset, the most valuable asset in the new economy. Millions of eyeballs follow her wherever she goes and she leverages the shit out of it. She earns millions off a crappy iPhone app that does nothing and a TV show that shows nothing. Just the fact she stands in a nightclub allows that club to charge $2,500 for entry. She gets paid more for public appearances than Nobel Prize winners. Forbes estimates her income last year at $28 million.

 But making stupid people rich and famous isn’t anything new in our culture. While the attention economy may exacerbate the problem, it didn’t create it. But when we apply the attention economy to the other areas of our lives, we run into some problems.

 How the Attention Economy Promotes Extremism

 Social networks are the business model of the attention economy. They are wholly dependent on eyeballs and clicks to make all of their revenue. To do this, they design algorithms that show you the most interesting and attention-grabbing information available in your social network.

 If your newsfeed was full of the boring and drab day-to-day stuff, you’d stop looking at it. So instead, Facebook shows you the most extreme occurrences in your social network for the simple reason that the extreme events draw the most attention.

 This has drastic effects not only on our perceptions of society as a whole, but also on how we perceive our personal lives.

 1. If it seems like “everybody” is getting married or having kids or having amazing trips around the world or doing something cool and fun and sexy, it’s only because we are exposed to these events in disproportionate numbers. It’s not that everyone is having amazing life experiences all the time, it’s that we’re always shown people’s amazing life experiences all the time. As a result, many of us begin to feel a constant gnaw of somehow “missing out” when really, we just have a heavily-biased perception of what’s going on in our peers’ lives.
 2. The attention economy rewards people who are narcissistic and self-promotional because these people excel at getting attention. Therefore, it seems that everyone is becoming more shallow and self-absorbed, when in fact, we are merely becoming more exposed to other people’s self-promotion.
 3. Politically, the most extreme, radical, and ignorant views get the most airtime because they’re the most unique and they grab the most attention. Therefore, it appears as if the world is spiraling into a festering shithole, when really, we’re just getting exposed to the people on the fringes more often than ever before.
 4. Threats such as Ebola or terrorism become sensationalized, not because they’re actually that threatening, but because of their extremity and how much attention they garner. You’re more likely to get eaten by a shark while getting struck by lightning than die from a terrorist attack. You’re more likely to die from the flu this year than you are from Ebola, ever. Yet, in our culture, it feels as though the world is in a constant state of imminent collapse.
 5. Pointless but dramatic events such as nipple slips, gaffes, errant interviews, and celebrities doing stupid celebrity stuff seem as though they are taking on a much greater cultural significance than they actually are. If Kim Kardashian got hit by a truck tomorrow and died, I’m sure you’d see the usual media-driven mourn fest and televised funeral, but would anyone really miss her? Would anybody’s life be altered? Probably not. In fact, we’d probably get another really good Kanye album.

 When you look at all of the complaints about social media, smartphones, and the internet at large, most of the complaints boil down to one thing: attention. People don’t have any attention span anymore. People don’t focus on what’s in front of them anymore. People don’t even fucking talk to you at dinner anymore.

 Typical “date night” these days.

 The problem is that the attention economy makes it economical to spread one’s attention across eight different interests and 23 different friends each day. And because we’re all spreading our attention so thin, many of us are losing the all-important life skill of focus.

 Focus is what generates long-term success. Focus leads to deeper and more meaningful relationships. Focus determines how well we can improve at something. Yet our current economy is constantly providing incentive away from focus and towards—whoa, did you see that video of the guy on the motorcycle who landed on the car? That was crazy!

 Anyway, what was I talking about? Oh yeah… focus. The new age presents problems of attention, not of happiness or narcissism or loneliness. And as the technology’s critics point out, this issue is not going to go away, it’s simply going to get worse.

 The Silent Benefits of the Attention Economy

 But the attention economy brings with it a host of social benefits, many of which are not immediately obvious to us. In fact, some of the benefits are actually painful to bear in some ways.

 Salience of Political Issues

 Take, for example, the recent exposure that Bill Cosby is (very likely) a rapist. This is a man that was a role model for decades and took that role seriously, traveling around the country lecturing people on values and responsibilities. Women had been accusing him of rape for years, yet it was the attention economy that finally allowed the allegations to make their way into the public consciousness in a significant way.

 Pre-internet, things such as sexual assault or gay rights or intransigent racism or the failed drug war, they were all abstract concepts that we never had to confront in any tangible way. They were fairy tales of some far away land. They had nothing to do with us.

 But new technologies bring these issues right in front of our face, over and over again, whether we like them or not. Sexual assault does happen and most of the perpetrators do go unpunished. LGBT people are humans too, and they deserve to be treated like humans. The drug war is a dismal failure and ruining millions of people’s lives. And because these experiences are now easily transmittable and experienced by large swaths of the population, we see these social policies changing faster than ever before.

 The past five years have seen a wave of protests and political mobilization, mostly throughout the developing world. From Egypt to Turkey to Brazil to Mexico, millions of people are able to react immediately upon major political issues and demand changes from long-corrupt governments. Sure, most of them don’t work. But some of them do. And at the least, they alter the political discourse.

 Just look at what’s been going on lately with racism in the US and Ferguson and Eric Garner. Regardless of what you think, the issue of racism and police brutality is forced on you. You can’t turn away. There are videos. There are dead kids. It’s impossible to not see it, and therefore an issue that has lasted for as long as the country has existed is being confronted more seriously than it has been in decades.

 Ease of Relationships

 But beyond politics, technology does benefit our relationships, even if we don’t always recognize it.

 We are the first generation that is still within easy contact of old friends from grade school, summer camp, and college. We are capable of being in constant contact with one another, for better or worse. It’s never been easier to transmit important life events and emergency situations worldwide. Travel and long distance relationships—romantic or otherwise—are easier than ever before. One can spend months or even years abroad and never feel “out of touch” with what’s going on at home.

 Obviously, technology shouldn’t replace face time and in-person interactions. But it can definitely supplement them.

 Limitless Knowledge

 And, of course, there is simply the breadth of information available at our fingertips. For all of the headaches the attention economy can cause, it’s the cost of having limitless information accessible at any time. You used to have to go to the library to do research. You used to have to call stores to find out if they had something in stock. You used to have to buy a map and spend five minutes studying it to figure out where you wanted to go.

 We forget so easily the whole point of all of this in the first place: the availability of limitless and free knowledge. These benefits are so widespread and ubiquitous that we can’t even remember what it was like to not have them. And as a result, we overestimate how much these technologies are hurting us and underestimate how much they’re helping us.

 Yes, the attention economy has come with new social challenges like identity theft and cyberbullying and assholes who text while they drive. But let’s not throw the baby out with the bathwater here.

 Taking Advantage of the Attention Economy

 The problem is not the technology itself, it’s how we choose to use the technology. Is it serving us or are we serving it?

 These are the new challenges new generations must face. Our grandparents had to learn to master their time and their energy in order to take advantage of the labor economy. Our parents had to master their minds and their problem-solving in order to take advantage of the knowledge economy. We must learn to master our focus and our self-awareness to properly take advantage of the attention economy.

 Limitless access to knowledge brings limitless opportunity. But only to those who learn to manage the new currency: their attention. In the new economy, the most valuable asset you can accumulate may not be money, may not be wealth, may not even be knowledge, but rather, the ability to control your own attention, and to focus.

 Because until you are able to limit your attention, until you are able to turn away, at will, from all of the shiny things and nipple slips, until you are able to consciously choose what has value to you and what does not, you and I and everyone else will continue to be served up garbage indefinitely. And it will not get better, it will get worse.

 In the future, your attention will be sold. And it may be that the only people able to capitalize on it, are the people who can control their own.

 Now, if you’ll excuse me, I have some stupid cat videos to watch.

 Want more actionable ideas every week?

 Join millions of readers and subscribe to The Breakthrough newsletter. Enter your email below to receive new ideas and exercises that could change your life each week.

 No spam or unexpected emails. Ever.

 Footnotes
 1. As a general rule, teenagers are always giggling and masturbating.↵
 2. Buffardi, L. E., & Campbell, W. K. (2008). Narcissism and Social Networking Web Sites. Personality and Social Psychology Bulletin, 34(10), 1303–1314.↵
 3. Marche, S. (2012, May). Is Facebook Making Us Lonely? The Atlantic.↵
 4. Vitak, J., Crouse, J., & LaRose, R. (2011). Personal Internet use at work: Understanding cyberslacking. Computers in Human Behavior, 27(5), 1751–1759. Quotes are totally made up by me and not theirs.↵
 5. Ryan, T., & Xenos, S. (2011). Who uses Facebook? An investigation into the relationship between the Big Five, shyness, narcissism, loneliness, and Facebook usage. Computers in Human Behavior, 27(5), 1658–1664.↵
 6. Vitak, J., Ellison, N. B., & Steinfield, C. (2011). The Ties That Bond: Re-Examining the Relationship between Facebook Use and Bonding Social Capital. In 2011 44th Hawaii International Conference on System Sciences (HICSS) (pp. 1–10).↵
 7. Leif Jarle Gressgård. (2011). Virtual team collaboration and innovation in organizations. Team Performance Management: An International Journal, 17(1/2), 102–119.↵
 8. Superstar Harvard psychologist Steven Pinker has a great op-ed in the New York Times about this: Mind Over Mass Media. He also discusses this in his new book↵
 9. Many of the ideas in this section are from The Fourth Economy: Inventing Western Civilization by Ron Davison.↵
 10. I was so proud of myself years ago when I thought I came up with this idea—of attention as the new organizing principle in the coming century. But of course, somebody smarter than me was there like 10 years before I was. Michael Goldhaber wrote a famous article in Wired in 1998 called “Attention Shoppers!” where he first described the attention economy. It’s been a “thing” ever since.↵
 11. Example just from a couple days ago: seventeen Republican-led states are now suing Obama over his immigration executive orders. This is a clear headline-grab with a case that likely has no substance or chance of changing anything.↵

 About the Author

 Mark is the three-time #1 New York Times bestselling author of The Subtle Art of Not Giving a F*ck as well as other titles. His books have sold around 20 million copies, been translated into more than 65 languages, and reached number one in more than a dozen countries. In 2023, a feature film about his life and ideas was released worldwide by Universal Pictures.

 Learn More About Mark

 MORE ARTICLES

 * Maybe You Don't Know What Love Is
 * Why Everyone on the Internet Is an Asshole
 * How to Be Patient in an Impatient World
 * Why We Stay in Bad Relationships
 * The Paradox of Success
 * Woman Lovers and Woman Haters
 VIEW ALL ARTICLES

 Mark is the three-time #1 New York Times bestselling author of The Subtle Art of Not Giving a F*ck, as well as other titles. His books have sold around 20 million copies, been translated into more than 65 languages, and reached number one in more than a dozen countries. In 2023, a feature film about his life and ideas was released worldwide by Universal Pictures.

 Learn More About Mark
 © 2023 Infinity Squared Media LLC
 * Articles
 * Books
 * Courses
 * Newsletter
 * About Mark
 * Sign Up
 Privacy Policy Terms and Conditions
 © 2023 Infinity Squared Media LLC

 Wait! Here Are 3 Ideas That Could Change Your Life

 Learn about the idea that transformed a depressed deadbeat into one of the most important philosophers who ever lived. Read about it in my free 19-page ebook.

 I'm not interested.
Last week, I logged onto Facebook to see a story about a man who got drunk, cut off his friend’s penis, and then fed it to a dog. This was followed by a story of a 100-year-old woman who had never seen the ocean before. Then eight ways I can totally know I’m a 90’s kid. Then 11 steps to make me a “smarter Black Friday shopper,” an oxymoron if I ever saw one.
This is life now: one constant, never-ending stream of non sequiturs and self-referential garbage that passes in through our eyes and out of our brains at the speed of a touchscreen.
Last month, Kim Kardashian “broke the internet.” And by “broke the internet,” I mean she photoshopped a picture of her bare ass and put it on a magazine cover.
But, of course, because it’s a naked ass, and because that ass happens to belong to Kim Kardashian, the photo was republished umpteen bazillion times, as adults rolled their eyes and guffawed at the insanity of it all, while the teenagers giggled and masturbated. 1  
The media brought out the obligatory screeds about how lewd or classless the pictures were. People then complained that the media gives this woman—a woman who has accomplished nothing in her life other than being born rich and fucking rappers—way too much airtime.
Then people pointed out that criticizing the media for publicizing Kardashian ironically just gives Kardashian even more attention and cultural presence. Being the internet, these people were then berated for being neckbeard virgins and their mothers were threatened with sexual violence. The neckbeards then responded by creating ironic memes that involved Kardashian’s bare ass and an espresso machine. Everyone LOLZ’d and moved on with their lives.
And this is why she’s still famous.
 
Kardashian has come to represent pretty much everything we loathe about the social media age: bite-sized, meaningless content that you hate looking at, but for some reason can’t look away from.
And because nobody has the self-control to look away, the bite-sized meaningless content spreads like wildfire, creating an online experience of a never-ending series of cultural car wrecks where we all gawk, rubberneck, discuss, and/or make fun of something for 12 minutes until distracted by the next oncoming collision.
There are three common complaints against social media and the internet in general: 1) that it’s making us all narcissistic and shallow, 2  2) that it’s crapping on our ability to maintain meaningful relationships and therefore making us lonelier, 3  and 3) that it’s interfering with our ability to focus and get quote, unquote “more important shit” done in our lives. 4  
Interestingly, after quite a bit of research, it turns out that none of these claims are completely accurate. Social media doesn’t necessarily cause people to become more narcissistic, it just gives narcissistic people more opportunities to indulge their narcissism, and to a larger audience. 5  It’s not interfering with either the closeness we feel to others or how many people we feel close to, it simply expands our network of casual acquaintances and the quantity of our casual social interactions. 6  And while technology does present more opportunities for distraction (which we’ll get to), it also presents easier transmission of information, tools for collaboration, and opportunities for organization. 7  
What I’m saying is that the whole “the internet is ruining us” argument is a big whiff. It’s likely just the anxiety that’s always wrought by new technologies. When TV and radio were invented, people complained that everyone’s brain was going to go to mush. When the printing press was invented, people thought it was going to destroy our ability to speak eloquently. Complaints about the minds of children being ruined by technology are as old as technology itself. 8  
Modern technology isn’t changing us. It’s changing society. There’s a difference. One is how we are, and one is simply how we react every day to the world around us. The social media age is changing the basic economics of our day-to-day lives. It’s changing them in profound ways, ways that most of us likely don’t notice. And surprisingly, it’s people like Kim Kardashian who are taking advantage of it.
If you’ve ever spent time in a really poor country or with people who grew up in awful poverty, you’ll notice how much they talk about food—their favorite foods, what they’re going to eat this weekend, how they like this and don’t like that, and so on.
Much of these people’s lives and conversations revolve around food for the simple reason that the scarcity of food makes it appear incredibly important. The fact you prefer strawberries to oranges matters a lot when you can rarely afford to have either. But in rich societies where food is never an issue, discussions of food among most people are superficial and usually over within a few seconds.
For most of human history, the big economic scarcity in the world was land. There was a limited amount of productive land, therefore there was a limited amount of food. And because there was a limited amount of food, most day-to-day economic concerns and political squabbles involved land. Most people spent their lives contemplating what land they were going to work, what they were going to grow, what kind of harvest to expect, and so on. Food was always at the top of people’s mind. 9  
Eventually, when the industrial revolution hit, the primary scarcity was no longer land, as machines could now help cultivate more than enough food for everybody. Now the big scarcity was labor. You needed trained people to run all of these machines that did all of the cool new shit so you could make money and get rich. Thus, for a few hundred years, the organizing principle in society was based on labor—who you worked for, how much you made, and so on.
Then, in the 20th century, there was more being produced than any one person would ever need or could ever purchase. The new scarcity in society was no longer labor or land, the scarcity was now knowledge. People had so many choices of what to purchase with their hard-earned money, but they didn’t know what to purchase. Thus, people spent most of their day-to-day existence trying to figure out what the best toothpaste was, what a toaster oven could do, how to spend their bonus money over the holidays and so on. The fields of advertising and marketing were invented and came to dominate society, as they were the means of disseminating the information people needed to allocate their resources appropriately.
Still with me? Because this is where the internet and smartphones have fucked everything up—or, ahem, where they’ve “disrupted” everything.
With the advent of the internet, the primary scarcity in society is no longer information. In fact, there is now more information than any of us could possibly know what to do with. If you want to know about a new product, you can have the Wikipedia article and 500 Amazon reviews up within 10 seconds. If you want a refresher on the process of photosynthesis, you can have it figured out within a few minutes. If you need to know every actor Kevin Spacey has ever worked with, you can figure it out in mere seconds.
The scarcity in our world is no longer knowledge. There’s an abundance of knowledge, just as there’s an abundance of labor and an abundance of land.
No, the new scarcity in the internet age is attention. Since there is a surplus of information, more information flowing through our society than any of us could ever hope to process or understand, the new bottleneck on our economy is attention. We now live in an attention-based economy. 10  
This is why today we are each bombarded with over 3,000 advertising messages per day. This is why these advertisements get zanier and more nonsensical—like the Geico gecko or the Old Spice guy—because the goal of advertisements is no longer information but simply attention.
 
This is why social media is plastered with ridiculous article headlines such as, “I Thought I Was Going To Die, But Then You’ll Never Believe How This Polar Bear Saved My Life,” and when you click it, it takes you to a series of stupid animated GIFs or a YouTube video that has nothing to do with polar bears but is instead slathered in advertisements.
This is why politics is becoming less about actual policies and more about dramatic actions designed to draw either positive or negative attention to various actors and political parties. 11  
This is why everything is becoming a version of softcore porn: music videos, commercials, movies, and reality TV shows. And when it’s not softcore porn, it’s some other kind of porn: food porn, murder porn, disaster porn, or actual, like real-life porn. Porn gets attention. And today, attention is what sells.
This is why Kim Kardashian is famous and has continued to be famous for the better part of a decade for no other reason than—you guessed it—she’s already famous! This woman has contributed absolutely nothing to humanity. Yet in the age of attention, she is goddamned Master Yoda. She’s got it all figured out.
Kim Kardashian is a genius. Not an Einstein-like genius. Not a “solves differential equations in her head but can’t tie her shoes” genius. But she’s a genius. A savant. She’s like the Rain Man of attention whoredom. The same way an autistic prodigy can count 2,318 toothpicks dumped on the floor just by looking at them, Kim Kardashian can command the attention of tens of millions of people with the crack of her ass.
The quality of that attention doesn’t matter. What matters is the attention. That attention is an asset, the most valuable asset in the new economy. Millions of eyeballs follow her wherever she goes and she leverages the shit out of it. She earns millions off a crappy iPhone app that does nothing and a TV show that shows nothing. Just the fact she stands in a nightclub allows that club to charge $2,500 for entry. She gets paid more for public appearances than Nobel Prize winners. Forbes estimates her income last year at $28 million.
 
But making stupid people rich and famous isn’t anything new in our culture. While the attention economy may exacerbate the problem, it didn’t create it. But when we apply the attention economy to the other areas of our lives, we run into some problems.
How the Attention Economy Promotes Extremism
Social networks are the business model of the attention economy. They are wholly dependent on eyeballs and clicks to make all of their revenue. To do this, they design algorithms that show you the most interesting and attention-grabbing information available in your social network.
If your newsfeed was full of the boring and drab day-to-day stuff, you’d stop looking at it. So instead, Facebook shows you the most extreme occurrences in your social network for the simple reason that the extreme events draw the most attention.
This has drastic effects not only on our perceptions of society as a whole, but also on how we perceive our personal lives.
If it seems like “everybody” is getting married or having kids or having amazing trips around the world or doing something cool and fun and sexy, it’s only because we are exposed to these events in disproportionate numbers. It’s not that everyone is having amazing life experiences all the time, it’s that we’re always shown people’s amazing life experiences all the time. As a result, many of us begin to feel a constant gnaw of somehow “missing out” when really, we just have a heavily-biased perception of what’s going on in our peers’ lives.
The attention economy rewards people who are narcissistic and self-promotional because these people excel at getting attention. Therefore, it seems that everyone is becoming more shallow and self-absorbed, when in fact, we are merely becoming more exposed to other people’s self-promotion.
Politically, the most extreme, radical, and ignorant views get the most airtime because they’re the most unique and they grab the most attention. Therefore, it appears as if the world is spiraling into a festering shithole, when really, we’re just getting exposed to the people on the fringes more often than ever before.
Threats such as Ebola or terrorism become sensationalized, not because they’re actually that threatening, but because of their extremity and how much attention they garner. You’re more likely to get eaten by a shark while getting struck by lightning than die from a terrorist attack. You’re more likely to die from the flu this year than you are from Ebola, ever. Yet, in our culture, it feels as though the world is in a constant state of imminent collapse.
Pointless but dramatic events such as nipple slips, gaffes, errant interviews, and celebrities doing stupid celebrity stuff seem as though they are taking on a much greater cultural significance than they actually are. If Kim Kardashian got hit by a truck tomorrow and died, I’m sure you’d see the usual media-driven mourn fest and televised funeral, but would anyone really miss her? Would anybody’s life be altered? Probably not. In fact, we’d probably get another really good Kanye album.
When you look at all of the complaints about social media, smartphones, and the internet at large, most of the complaints boil down to one thing: attention. People don’t have any attention span anymore. People don’t focus on what’s in front of them anymore. People don’t even fucking talk to you at dinner anymore.
 Typical “date night” these days. 
The problem is that the attention economy makes it economical to spread one’s attention across eight different interests and 23 different friends each day. And because we’re all spreading our attention so thin, many of us are losing the all-important life skill of focus.
Focus is what generates long-term success. Focus leads to deeper and more meaningful relationships. Focus determines how well we can improve at something. Yet our current economy is constantly providing incentive away from focus and towards—whoa, did you see that video of the guy on the motorcycle who landed on the car? That was crazy!
 
Anyway, what was I talking about? Oh yeah… focus. The new age presents problems of attention, not of happiness or narcissism or loneliness. And as the technology’s critics point out, this issue is not going to go away, it’s simply going to get worse.
But the attention economy brings with it a host of social benefits, many of which are not immediately obvious to us. In fact, some of the benefits are actually painful to bear in some ways.
Salience of Political Issues
Take, for example, the recent exposure that Bill Cosby is (very likely) a rapist. This is a man that was a role model for decades and took that role seriously, traveling around the country lecturing people on values and responsibilities. Women had been accusing him of rape for years, yet it was the attention economy that finally allowed the allegations to make their way into the public consciousness in a significant way.
Pre-internet, things such as sexual assault or gay rights or intransigent racism or the failed drug war, they were all abstract concepts that we never had to confront in any tangible way. They were fairy tales of some far away land. They had nothing to do with us.
But new technologies bring these issues right in front of our face, over and over again, whether we like them or not. Sexual assault does happen and most of the perpetrators do go unpunished. LGBT people are humans too, and they deserve to be treated like humans. The drug war is a dismal failure and ruining millions of people’s lives. And because these experiences are now easily transmittable and experienced by large swaths of the population, we see these social policies changing faster than ever before.
The past five years have seen a wave of protests and political mobilization, mostly throughout the developing world. From Egypt to Turkey to Brazil to Mexico, millions of people are able to react immediately upon major political issues and demand changes from long-corrupt governments. Sure, most of them don’t work. But some of them do. And at the least, they alter the political discourse.
Just look at what’s been going on lately with racism in the US and Ferguson and Eric Garner. Regardless of what you think, the issue of racism and police brutality is forced on you. You can’t turn away. There are videos. There are dead kids. It’s impossible to not see it, and therefore an issue that has lasted for as long as the country has existed is being confronted more seriously than it has been in decades.
Ease of Relationships
But beyond politics, technology does benefit our relationships, even if we don’t always recognize it.
We are the first generation that is still within easy contact of old friends from grade school, summer camp, and college. We are capable of being in constant contact with one another, for better or worse. It’s never been easier to transmit important life events and emergency situations worldwide. Travel and long distance relationships—romantic or otherwise—are easier than ever before. One can spend months or even years abroad and never feel “out of touch” with what’s going on at home.
Obviously, technology shouldn’t replace face time and in-person interactions. But it can definitely supplement them.
Limitless Knowledge
And, of course, there is simply the breadth of information available at our fingertips. For all of the headaches the attention economy can cause, it’s the cost of having limitless information accessible at any time. You used to have to go to the library to do research. You used to have to call stores to find out if they had something in stock. You used to have to buy a map and spend five minutes studying it to figure out where you wanted to go.
We forget so easily the whole point of all of this in the first place: the availability of limitless and free knowledge. These benefits are so widespread and ubiquitous that we can’t even remember what it was like to not have them. And as a result, we overestimate how much these technologies are hurting us and underestimate how much they’re helping us.
Yes, the attention economy has come with new social challenges like identity theft and cyberbullying and assholes who text while they drive. But let’s not throw the baby out with the bathwater here.
These are the new challenges new generations must face. Our grandparents had to learn to master their time and their energy in order to take advantage of the labor economy. Our parents had to master their minds and their problem-solving in order to take advantage of the knowledge economy. We must learn to master our focus and our self-awareness to properly take advantage of the attention economy.
Limitless access to knowledge brings limitless opportunity. But only to those who learn to manage the new currency: their attention. In the new economy, the most valuable asset you can accumulate may not be money, may not be wealth, may not even be knowledge, but rather, the ability to control your own attention, and to focus.
Because until you are able to limit your attention, until you are able to turn away, at will, from all of the shiny things and nipple slips, until you are able to consciously choose what has value to you and what does not, you and I and everyone else will continue to be served up garbage indefinitely. And it will not get better, it will get worse.
In the future, your attention will be sold. And it may be that the only people able to capitalize on it, are the people who can control their own.
Now, if you’ll excuse me, I have some stupid cat videos to watch.
In The Future, Our Attention Will Be Sold
https://markmanson.net/attention-economy
https://michaelnielsen.org/blog/fear-of-flying/#comment-235
Fear of flying
2004-09-30
Siva
<div><div>
	<p>Joel Gilmore at <a href="http://www.illuminatingscience.org/">illuminating science</a> asks an <a href="http://www.illuminatingscience.org/index.php?p=63">interesting question</a> about the test flight of SpaceShipOne that’s been in the news:</p>
<blockquote><p>
Oh, and a question of scruples for you all: The X-Prize requires a ship with one pilot and two passengers, but the passengers on this flight were replaced by ballast (as allowed by the rules.) If you’d been offered one of those passenger seats for today’s flight, would you have taken it? I have to say that I would have, despite the obvious risks with what the commentators called “test flight in its purest form.” In a room of three physicists, we had two votes for yes and one for no – what would you do?
</p></blockquote>
<p>I’d be tempted, but I’m pretty sure that, when push came to shove, I wouldn’t be up for the risk. Admittedly, if I’d been the person <em>developing</em> the rocket, I might well be interested in a flight. But just to be an invited passenger, nah. I’d rather wait for the “debugged” version.</p>
	</div>||||I|||| Skip to content
 * 
 Michael Nielsen
 * About
 * Michael’s main blog
 * Data-driven intelligence blog
 * Writing

 Fear of flying

 by Michael Nielsen on September 30, 2004

 Joel Gilmore at illuminating science asks an interesting question about the test flight of SpaceShipOne that’s been in the news:

 Oh, and a question of scruples for you all: The X-Prize requires a ship with one pilot and two passengers, but the passengers on this flight were replaced by ballast (as allowed by the rules.) If you’d been offered one of those passenger seats for today’s flight, would you have taken it? I have to say that I would have, despite the obvious risks with what the commentators called “test flight in its purest form.” In a room of three physicists, we had two votes for yes and one for no – what would you do?

 I’d be tempted, but I’m pretty sure that, when push came to shove, I wouldn’t be up for the risk. Admittedly, if I’d been the person developing the rocket, I might well be interested in a flight. But just to be an invited passenger, nah. I’d rather wait for the “debugged” version.

 From → General

 One Comment
 1. Macneil Shonle permalink

 I would love to take the flight if I were to know the outcome beforehand. Beyond that, I’d rather not hurt my family by taking the risk.

 Comments are closed.

 * Follow Michael

 + Subscribe to this blog
 + Subscribe to this blog by email
 + Michael's blog on data-driven intelligence
 + Follow Michael on Twitter

 * Other places on the web

 + GitHub
 + Delicious
 + Delicious research links for book project
 + Delicious research links for last book
 + Google Plus

 * Books


 Reinventing Discovery: The New Era of Networked Science (Errata)


 Quantum Computation and Quantum Information

 * Other projects

 + Essays
 + TEDxWaterloo video about open science
 + Quantum computing for the determined
 + Polymath wiki

 * Recent Posts

 + Is there a tension between creativity and accuracy?
 + Where will the key ideas shaping the future of scientific publishing come from?
 + How the backpropagation algorithm works
 + Reinventing Explanation
 + Neural Networks and Deep Learning: first chapter goes live

 * Recent Comments

 + Siva on Is there a tension between creativity and accuracy?
 + Reading into the landscape – drossbucket on Is there a tension between creativity and accuracy?
 + kBite on Is there a tension between creativity and accuracy?
 + Siname on Is there a tension between creativity and accuracy?
 + Florian Seidl-Schulz on Is there a tension between creativity and accuracy?

 * Linklog

 * Documentaries

 * Search

 Search

 * Archives

 + April 2017
 + May 2015
 + April 2014
 + January 2014
 + November 2013
 + February 2013
 + February 2012
 + January 2012
 + October 2011
 + September 2011
 + August 2011
 + June 2011
 + May 2011
 + April 2011
 + March 2011
 + February 2011
 + January 2011
 + December 2010
 + November 2010
 + August 2010
 + May 2010
 + April 2010
 + February 2010
 + January 2010
 + December 2009
 + November 2009
 + October 2009
 + September 2009
 + August 2009
 + July 2009
 + June 2009
 + May 2009
 + April 2009
 + March 2009
 + February 2009
 + January 2009
 + December 2008
 + November 2008
 + October 2008
 + September 2008
 + August 2008
 + July 2008
 + June 2008
 + May 2008
 + April 2008
 + March 2008
 + February 2008
 + January 2008
 + November 2007
 + October 2007
 + September 2007
 + August 2007
 + July 2007
 + June 2007
 + May 2007
 + April 2007
 + August 2006
 + July 2006
 + June 2006
 + April 2006
 + March 2006
 + October 2005
 + August 2005
 + July 2005
 + June 2005
 + May 2005
 + April 2005
 + March 2005
 + February 2005
 + January 2005
 + December 2004
 + November 2004
 + October 2004
 + September 2004
 + August 2004
 + July 2004
 + June 2004
 + May 2004
 + April 2004
 + March 2004
 + February 2004
 + January 2004
 + November 2003
 + October 2003
 + September 2003
 + August 2003
 * 
 * 
 * 

 Copyright © 2023 . Titan Theme by The Theme Foundry

Joel Gilmore at illuminating science asks an interesting question about the test flight of SpaceShipOne that’s been in the news:
 
Oh, and a question of scruples for you all: The X-Prize requires a ship with one pilot and two passengers, but the passengers on this flight were replaced by ballast (as allowed by the rules.) If you’d been offered one of those passenger seats for today’s flight, would you have taken it? I have to say that I would have, despite the obvious risks with what the commentators called “test flight in its purest form.” In a room of three physicists, we had two votes for yes and one for no – what would you do?
 
I’d be tempted, but I’m pretty sure that, when push came to shove, I wouldn’t be up for the risk. Admittedly, if I’d been the person developing the rocket, I might well be interested in a flight. But just to be an invited passenger, nah. I’d rather wait for the “debugged” version.
Fear of flying
https://michaelnielsen.org/blog/fear-of-flying/
https://blogs.nasa.gov/spacex/2021/11/24/nasa-announces-date-for-spacexs-24th-cargo-resupply-mission/
NASA Announces Date for SpaceX’s 24th Cargo Resupply Mission
2021-11-24
Author Danielle Sempsrott
<div><div>
	<p><a href="#content">Skip to content</a></p>
	<div>
	<main>
	
<article>
	<div>
	<figure><figcaption>A close-up view of the SpaceX Falcon 9 rocket with the Dragon capsule atop in the vertical position on June 2, 2021, at Launch Complex 39A at NASA’s Kennedy Space Center in Florida, in preparation for the company’s 22nd Commercial Resupply Services mission for NASA to the International Space Station. Photo credit: SpaceX</figcaption></figure>
<p>NASA and SpaceX are targeting Tuesday, Dec. 21, at 5:06 a.m. EST for launch of the 24th Commercial Resupply Services mission to the International Space Station. The Dragon spacecraft will lift off aboard a Falcon 9 rocket from Launch Complex 39A at NASA’s Kennedy Space Center in Florida.</p>
<p>Dragon will bring food, supplies, and scientific investigations to the orbiting crew, including a <a href="https://www.nasa.gov/mission_pages/station/research/experiments/explorer/Investigation.html?#id=8616">protein crystal growth study</a> that could improve the delivery of cancer treatment drugs and a <a href="https://www.nasa.gov/mission_pages/station/research/experiments/explorer/Investigation.html?#id=8552">handheld bioprinter</a> that could one day be used to print tissue directly onto wounds to faster healing.</p>
<p>The spacecraft will arrive at the station on Wednesday, Dec. 22 at approximately 4:30 a.m. and remain docked for about a month before returning to Earth.</p>
<p>The mission will be covered live on NASA TV, the NASA app, and the <a href="https://www.nasa.gov/nasalive">agency’s website</a>.</p>
	</div>
	
</article>
	<nav>
	<h2>Post navigation</h2>
	</nav>
	</main>
	
</div>
	</div>||||I|||| Skip to content

 SpaceX

 Menu
 * NASA Blogs Home

 NASA Announces Date for SpaceX’s 24th Cargo Resupply Mission

 A close-up view of the SpaceX Falcon 9 rocket with the Dragon capsule atop in the vertical position on June 2, 2021, at Launch Complex 39A at NASA’s Kennedy Space Center in Florida, in preparation for the company’s 22nd Commercial Resupply Services mission for NASA to the International Space Station. Photo credit: SpaceX

 NASA and SpaceX are targeting Tuesday, Dec. 21, at 5:06 a.m. EST for launch of the 24th Commercial Resupply Services mission to the International Space Station. The Dragon spacecraft will lift off aboard a Falcon 9 rocket from Launch Complex 39A at NASA’s Kennedy Space Center in Florida.

 Dragon will bring food, supplies, and scientific investigations to the orbiting crew, including a protein crystal growth study that could improve the delivery of cancer treatment drugs and a handheld bioprinter that could one day be used to print tissue directly onto wounds to faster healing.

 The spacecraft will arrive at the station on Wednesday, Dec. 22 at approximately 4:30 a.m. and remain docked for about a month before returning to Earth.

 The mission will be covered live on NASA TV, the NASA app, and the agency’s website.

 Author Danielle Sempsrott Posted on November 24, 2021November 24, 2021 Categories Cargo Resupply (CRS) , Commercial Resupply , Kennedy Space Center Tags CRS-24 , International Space Station , NASA

 Post navigation

 Previous Previous post: NASA, SpaceX Continue Planning for Next Crew Rotation Missions to International Space Station
 Next Next post: NASA’s SpaceX Crew-4 Trains for Upcoming Mission
 Search for: Search

 Recent Posts

 * Crew-4 Prelaunch Briefings, Weather Update
 * SpaceX Completes Falcon 9 Static Fire Test in Preparation for Crew-4 Launch
 * Crew-4 Dragon Rolls to the Pad, Completes Dry Dress Rehearsal
 * Dragon Spacecraft Mated to Falcon 9
 * NASA’s SpaceX Crew-4 Astronauts En Route to Kennedy Space Center

 Archives

 * April 2022
 * March 2022
 * November 2021
 * September 2021
 * June 2021
 * May 2021
 * December 2020
 * November 2020
 * October 2020
 * March 2020
 * February 2020
 * December 2019
 * July 2019
 * June 2019
 * May 2019
 * April 2019
 * December 2018
 * November 2018
 * June 2018
 * April 2018
 * March 2018
 * December 2017
 * November 2017
 * August 2017
 * July 2017
 * June 2017
 * May 2017
 * February 2017
 * July 2016
 * May 2016
 * April 2016
 * March 2016
 * July 2015
 * June 2015
 * April 2015
 * March 2015
 * January 2015
 * December 2014
 * November 2014
 * October 2014
 * September 2014
 * May 2014
 * April 2014
 * March 2014
 * February 2014

 Categories

 * Cargo Resupply (CRS)
 * Commercial Crew
 * Commercial Crew Program
 * Commercial Resupply
 * Expedition 64
 * Falcon Heavy
 * Kennedy Space Center
 * Uncategorized
 SpaceX National Aeronautics and Space Administration NASA Official: Brian Dunbar
 * No Fear Act
 * FOIA
 * Privacy
 * Office of Inspector General
 * Agency Financial Reports
 * Contact NASA
Skip to content 
	
	
	
	
	 A close-up view of the SpaceX Falcon 9 rocket with the Dragon capsule atop in the vertical position on June 2, 2021, at Launch Complex 39A at NASA’s Kennedy Space Center in Florida, in preparation for the company’s 22nd Commercial Resupply Services mission for NASA to the International Space Station. Photo credit: SpaceX 
NASA and SpaceX are targeting Tuesday, Dec. 21, at 5:06 a.m. EST for launch of the 24th Commercial Resupply Services mission to the International Space Station. The Dragon spacecraft will lift off aboard a Falcon 9 rocket from Launch Complex 39A at NASA’s Kennedy Space Center in Florida.
Dragon will bring food, supplies, and scientific investigations to the orbiting crew, including a protein crystal growth study that could improve the delivery of cancer treatment drugs and a handheld bioprinter that could one day be used to print tissue directly onto wounds to faster healing.
The spacecraft will arrive at the station on Wednesday, Dec. 22 at approximately 4:30 a.m. and remain docked for about a month before returning to Earth.
The mission will be covered live on NASA TV, the NASA app, and the agency’s website.
	
	
	
	Post navigation
NASA Announces Date for SpaceX’s 24th Cargo Resupply Mission
https://blogs.nasa.gov/spacex/2021/11/24/nasa-announces-date-for-spacexs-24th-cargo-resupply-mission/
https://davidgauntlett.com/tools-for-thinking/tools-for-thinking/
David Gauntlett | Tools for thinking: Introduction
2013-06-11
David Gauntlett
<div><div>
<p>It is fitting that for this first ‘proper’ post on my new blog, I’m writing about tools for thinking. I’ve been thinking about tools for thinking more and more, and for some time now, although I haven’t always used the term ‘tools for thinking’.</p>
<p>For instance, I wrote a whole book about tools for thinking, called <a href="https://davidgauntlett.com/portfolio/creative-explorations/"><em>Creative Explorations</em></a>, published by Routledge in 2007. A quick text search in the manuscript shows that the phrase ‘tools for thinking’ does appear in it, but only once, and sort of by accident.</p>
<p>So what do I mean by tools for thinking? Well, that’s what this blog post is about.</p>
<p>Because I’ve been thinking about tools for thinking, and wondering if perhaps I’ll write my next book about this subject, I searched on Amazon to see if there were actually hundreds of books already with ‘tools for thinking’ in the title.</p>
<p>Now, as an author, I obviously hope to be saying something distinctive. I don’t know where the phrase ‘tools for thinking’ came from. Did I invent it? That would be nice. Or did I just hear it somewhere? I have no idea. In self-serving terms, I hope that the number of books with ‘tools for thinking’ in the title is zero. But that’s not very likely, is it? In rational terms, ‘tools for thinking’ sound like a very obvious kind of thing that most people have a real need for, and so you’d expect that Amazon would offer a few hundred ‘tools for thinking’ books, at the very least.</p>
<p>In fact: there are very few. There’s a new book by Daniel Dennett, called <em>Intuition Pumps, and Other Tools for Thinking</em> (2013), of which more in a moment.</p>
<p>And then there’s a book on modelling in management science, called <em>Tools for Thinking: Modelling in Management Science</em> (by Michael Pidd, third edition 2009), but thankfully, as far as I’m concerned, that’s not really the same as what I’m talking about, because it’s about modelling in management science.</p>
<p>Then, on the Amazon list, after those two, that’s the end of the books with ‘tools for thinking’ in the title, and we’re on to other items such as <em>Six Thinking Hats</em> by Edward de Bono, then other de Bono books. Soon after that, we’re onto items like the ‘Gerber Suspension Multitool’, which looks like this:</p>
<p></p>
<p>That’s not really what I meant. But I’m quite impressed that Amazon was able to identify <em>Six Thinking Hats</em> as a book about tools for thinking, because that’s what it is, even though it doesn’t say so in an obvious place like the title, or even in the blurb. Edward de Bono’s <em>Six Thinking Hats</em> was first published in 1985, and was preceded by <em>De Bono’s Thinking Course: Powerful Tools to Transform Your Thinking</em>, first published in 1982.</p>
<p>That book, published more than 30 years ago, talks about ‘tools’ for thinking a <em>lot</em>. You can see that from the Amazon page previews. I’ve looked at that book before, probably, and there’s certainly a copy of <em>Six Thinking Hats</em> in the house somewhere. Edward de Bono always seemed like a bit of a hack, parcelling up his ideas into several rather-too-similar books and videos and courses, turning them into a big money machine. But maybe that’s just the sound of my brain being jealous. And did I invent the phrase ‘tools for thinking’? That’s a no.</p>
<p>So I’m going to have to order that <em>De Bono’s Thinking Course</em>, just to be comprehensive. And meanwhile, I’ve already ordered, and received, the Daniel Dennett book which is #1 in Amazon’s ‘tools for thinking’ list. Here it is, photographed for this blog post next to my own <em>Creative Explorations</em>:</p>
<p></p>
<p>Now, by coincidence (but it can’t <em>exactly</em> be a coincidence), I’ve encountered Daniel Dennett before. In fact, he’s discussed in <em>Creative Explorations</em> – not because of anything to do with ‘tools for thinking’, but in his role as a philosopher and cognitive scientist who made an argument for consciousness being a kind of illusion (to simplify greatly). Since then, Dennett has also become notable as a fundamentalist atheist in the Richard Dawkins mould, which I find unattractive (simply because dealing with religion by telling religious people that they are idiots doesn’t seem to be a very compassionate way of going about your ‘humanism’). But we can put all that aside for now.</p>
<p>So what’s Daniel Dennett up to in his ‘tools for thinking’ book?</p>
<p>Well the good news for me is that it’s definitely not the ‘tools for thinking’ book that <em>I</em> might want to write.  It’s basically about the kinds of thought experiment and thinking tricks that you can do in order to test arguments. Dennett has collected some old ones and some fresh ones, but a classic example to give here, so you can see the kind of thing we’re talking about, is the <em>Reductio Ad Absurdum</em>: taking an argument to the extreme to see if it still makes sense. Another well-known ‘tool for thinking’ of this kind is Occam’s Razor, which is the idea that you shouldn’t use a complex explanation if you’ve got a simpler one which works just as well.</p>
<p>So the ‘tools’ here are words and ideas and ways of thinking, and it would be a useful book for students of philosophy, or critical thinking, or indeed anyone who wants to get better at exploring and testing arguments. Both religious and scientific ideas could be tested with these tools.</p>
<p>But when <em>I’m</em> talking about tools for thinking – and you’ll have to forgive me for being so literal-minded – I’m talking about <em>actual tools</em>. Things you can hold in your hand. Tangible things you can shift and manipulate, which help you to think about things. (And also, their digital equivalents, and physical/digital hybrids, but that’s for a future blogpost).</p>
<p>My own best example of that is the things I’ve done with LEGO. In 2005 I became involved with LEGO Serious Play, a consultancy process developed by the LEGO Group – an activity for groups of adults, guided by a facilitator, in which participants would build metaphorical models using LEGO bricks. The models would typically represent their experiences of activities, structures and communications within their organisations, and then – having externalised this, by having built it in LEGO – they would go on to develop ideas for initiatives or strategies, in response to what they had built.</p>
<p>I worked with the LEGO Group on researching some aspects of this process, and I developed it as a social-science research tool. Specifically, I worked out how to use it as an alternative to the traditional social-science qualitative methods of interviews or focus groups. That’s what <a href="https://davidgauntlett.com/portfolio/creative-explorations/"><em>Creative Explorations</em></a> is about. Later I worked with the LEGO Group on the <a href="https://davidgauntlett.com/portfolio/lego-collaborations/">open source release</a> of LEGO Serious Play, and co-wrote the documentation.</p>
<p>It’s all about getting people to physically build stuff, representing their ideas, experiences and feelings, so that then you can look at it, reflect on it, respond to it, move things around, change it, and make new plans. That’s what I call a <em>real</em> tool for thinking.</p>
<p>Now, my use of LEGO Serious Play as a social-science research tool was intended to be a good thing for participants – giving them a stimulating, creative experience which is more interesting to do than traditional research methods. Ultimately, though, I now reflect that basically it was done for the purposes of the researcher rather than anything else – it was meant to be a good research tool, which is fair enough, but that’s not actually about helping people to solve their own problems, which is what I’m interested in now. I’ll say more about that in a future post.</p>
<p>It’s surprising, I think, that you don’t find more physical, hands-on tools for thinking. I was recently delighted to be contacted by Fanny Bissa, a final year Design student at Goldsmiths College, University of London. She was working on a whole set of hands-on tools for working through different kinds of problems and relationships (see her excellent <a href="http://learntosolve.tumblr.com/">Tumblr blog</a> and <a href="http://issuu.com/fannybissa/docs/unfoldevaluateandsolve/1">online booklet</a>). We met up a couple of times to talk about it. One of the tools can be seen in the image at the top of this blogpost. I’ll write more about that, as well, in a future post. For now, here’s a short video she made about her tools (3 minutes):</p>
<p></p>
<hr/>
<p><em>Image at the top by Fanny Bissa, reproduced by kind permission.</em></p>
</div></div>||||I|||| * Home
 * About
 * Blog
 * Stuff
 * Contact
 * Twitter
 * Insta
 * Home
 * About
 * Blog
 * Stuff
 * Contact
 * Twitter
 * Insta
 Menu
 Home Tools for thinking Tools for thinking: Introduction

 Tools for thinking: Introduction

 11 June 2013 David Gauntlett 9 Comments

 It is fitting that for this first ‘proper’ post on my new blog, I’m writing about tools for thinking. I’ve been thinking about tools for thinking more and more, and for some time now, although I haven’t always used the term ‘tools for thinking’.

 For instance, I wrote a whole book about tools for thinking, called Creative Explorations, published by Routledge in 2007. A quick text search in the manuscript shows that the phrase ‘tools for thinking’ does appear in it, but only once, and sort of by accident.

 So what do I mean by tools for thinking? Well, that’s what this blog post is about.

 Because I’ve been thinking about tools for thinking, and wondering if perhaps I’ll write my next book about this subject, I searched on Amazon to see if there were actually hundreds of books already with ‘tools for thinking’ in the title.

 Now, as an author, I obviously hope to be saying something distinctive. I don’t know where the phrase ‘tools for thinking’ came from. Did I invent it? That would be nice. Or did I just hear it somewhere? I have no idea. In self-serving terms, I hope that the number of books with ‘tools for thinking’ in the title is zero. But that’s not very likely, is it? In rational terms, ‘tools for thinking’ sound like a very obvious kind of thing that most people have a real need for, and so you’d expect that Amazon would offer a few hundred ‘tools for thinking’ books, at the very least.

 In fact: there are very few. There’s a new book by Daniel Dennett, called Intuition Pumps, and Other Tools for Thinking (2013), of which more in a moment.

 And then there’s a book on modelling in management science, called Tools for Thinking: Modelling in Management Science (by Michael Pidd, third edition 2009), but thankfully, as far as I’m concerned, that’s not really the same as what I’m talking about, because it’s about modelling in management science.

 Then, on the Amazon list, after those two, that’s the end of the books with ‘tools for thinking’ in the title, and we’re on to other items such as Six Thinking Hats by Edward de Bono, then other de Bono books. Soon after that, we’re onto items like the ‘Gerber Suspension Multitool’, which looks like this:

 That’s not really what I meant. But I’m quite impressed that Amazon was able to identify Six Thinking Hats as a book about tools for thinking, because that’s what it is, even though it doesn’t say so in an obvious place like the title, or even in the blurb. Edward de Bono’s Six Thinking Hats was first published in 1985, and was preceded by De Bono’s Thinking Course: Powerful Tools to Transform Your Thinking, first published in 1982.

 That book, published more than 30 years ago, talks about ‘tools’ for thinking a lot. You can see that from the Amazon page previews. I’ve looked at that book before, probably, and there’s certainly a copy of Six Thinking Hats in the house somewhere. Edward de Bono always seemed like a bit of a hack, parcelling up his ideas into several rather-too-similar books and videos and courses, turning them into a big money machine. But maybe that’s just the sound of my brain being jealous. And did I invent the phrase ‘tools for thinking’? That’s a no.

 So I’m going to have to order that De Bono’s Thinking Course, just to be comprehensive. And meanwhile, I’ve already ordered, and received, the Daniel Dennett book which is #1 in Amazon’s ‘tools for thinking’ list. Here it is, photographed for this blog post next to my own Creative Explorations:

 Now, by coincidence (but it can’t exactly be a coincidence), I’ve encountered Daniel Dennett before. In fact, he’s discussed in Creative Explorations – not because of anything to do with ‘tools for thinking’, but in his role as a philosopher and cognitive scientist who made an argument for consciousness being a kind of illusion (to simplify greatly). Since then, Dennett has also become notable as a fundamentalist atheist in the Richard Dawkins mould, which I find unattractive (simply because dealing with religion by telling religious people that they are idiots doesn’t seem to be a very compassionate way of going about your ‘humanism’). But we can put all that aside for now.

 So what’s Daniel Dennett up to in his ‘tools for thinking’ book?

 Well the good news for me is that it’s definitely not the ‘tools for thinking’ book that I might want to write. It’s basically about the kinds of thought experiment and thinking tricks that you can do in order to test arguments. Dennett has collected some old ones and some fresh ones, but a classic example to give here, so you can see the kind of thing we’re talking about, is the Reductio Ad Absurdum: taking an argument to the extreme to see if it still makes sense. Another well-known ‘tool for thinking’ of this kind is Occam’s Razor, which is the idea that you shouldn’t use a complex explanation if you’ve got a simpler one which works just as well.

 So the ‘tools’ here are words and ideas and ways of thinking, and it would be a useful book for students of philosophy, or critical thinking, or indeed anyone who wants to get better at exploring and testing arguments. Both religious and scientific ideas could be tested with these tools.

 But when I’m talking about tools for thinking – and you’ll have to forgive me for being so literal-minded – I’m talking about actual tools. Things you can hold in your hand. Tangible things you can shift and manipulate, which help you to think about things. (And also, their digital equivalents, and physical/digital hybrids, but that’s for a future blogpost).

 My own best example of that is the things I’ve done with LEGO. In 2005 I became involved with LEGO Serious Play, a consultancy process developed by the LEGO Group – an activity for groups of adults, guided by a facilitator, in which participants would build metaphorical models using LEGO bricks. The models would typically represent their experiences of activities, structures and communications within their organisations, and then – having externalised this, by having built it in LEGO – they would go on to develop ideas for initiatives or strategies, in response to what they had built.

 I worked with the LEGO Group on researching some aspects of this process, and I developed it as a social-science research tool. Specifically, I worked out how to use it as an alternative to the traditional social-science qualitative methods of interviews or focus groups. That’s what Creative Explorations is about. Later I worked with the LEGO Group on the open source release of LEGO Serious Play, and co-wrote the documentation.

 It’s all about getting people to physically build stuff, representing their ideas, experiences and feelings, so that then you can look at it, reflect on it, respond to it, move things around, change it, and make new plans. That’s what I call a real tool for thinking.

 Now, my use of LEGO Serious Play as a social-science research tool was intended to be a good thing for participants – giving them a stimulating, creative experience which is more interesting to do than traditional research methods. Ultimately, though, I now reflect that basically it was done for the purposes of the researcher rather than anything else – it was meant to be a good research tool, which is fair enough, but that’s not actually about helping people to solve their own problems, which is what I’m interested in now. I’ll say more about that in a future post.

 It’s surprising, I think, that you don’t find more physical, hands-on tools for thinking. I was recently delighted to be contacted by Fanny Bissa, a final year Design student at Goldsmiths College, University of London. She was working on a whole set of hands-on tools for working through different kinds of problems and relationships (see her excellent Tumblr blog and online booklet). We met up a couple of times to talk about it. One of the tools can be seen in the image at the top of this blogpost. I’ll write more about that, as well, in a future post. For now, here’s a short video she made about her tools (3 minutes):

 Image at the top by Fanny Bissa, reproduced by kind permission.

 Twitter
 Facebook
 email
 Reddit
 Linkedin
 Pinterest

 9 Responses to “Tools for thinking: Introduction”

 1. David Hendy 12 June 2013

 A fascinating post, written with your usual generosity and insight – thank you! The ‘tools for thinking’ approach is very fruitful. I’ve been starting to write recently about various old media, I.e. radio, cinema, television – as being ‘machines for thinking’. I suppose I use the machine word as a way of raising the notion if one thing that is neither fully autonomous nor fully under human control, and so therefore getting into a fascinating debate about agency. Anyway, I look forward to more posts – and congratulations on your new site. D.

 Reply
 + David Gauntlett 12 June 2013

 Hi David H!,

 Many thanks for the nice comment. I like the ‘machines for thinking’ idea … would love to hear more about that. (I wonder why it’s ‘old media’?).

 Best wishes, David G

 Reply
 2. Helene Galdin-O'Shea 12 June 2013

 Hi David,

 Delighted to see you have a new site although I’ll keep visiting Theory.org.uk too!

 I just wanted to say that you should read a little more about De Bono’s Thinking Hats and how his ideas have been ‘debunked’. A good starting point is the new book by Tom Bennett, Teacher Proof. I think you’ll find it interesting!
 http://behaviourguru.blogspot.co.uk/2013/06/teacher-proof-why-educational-research.html

 Thank you again for talking to our Park High students last term. Good luck with the new site.

 Reply
 + David Gauntlett 12 June 2013

 Hi Helene,

 Yes, I suspected de Bono of being a bit phoney, as I hinted above. On the other hand it seemed unfair to be rude about something I’ve not looked at very closely … and his basic approach, that you can have tools for thinking, mental techniques to help you get your head round things, seemed perfectly good, even if it’s based on rather pseudo-scientific claims when you look at the detail. (In which case, he’d simply be better off with the pseudo-scientific claims!).

 Many thanks for the comment. Best wishes, D

 Reply
 3. Carianne Buurmeijer 14 June 2013

 Thanks for that! Speaking of the senses, does inspiring music/muse/powernap count as tool for thinking? Or perhaps, are they just placebos?

 Reply
 + David Gauntlett 14 June 2013

 Hi Carianne,

 Well, I’m talking about actual tools that you can use to work on particular ideas … I mean the tools actually relate to the ideas. Whereas things like music or a powernap might be helpful for thought in a general way but they are not (probably?!) about working on the actual ideas. If you know what I mean. But no I don’t think it’s a placebo effect either.

 Best wishes, David

 Reply
 4. Carianne Buurmeijer 14 June 2013

 It was meant in a more general way indeed, however… For a college assignment, I once used the ‘Carnaval des animaux’ by St. Saens to make a toolbox which explained creative thinking methods, each animal representing a thinking method. Not at all a scientifical approach, but who knows, musical psychologists might have such tools.

 Reply
 + David Gauntlett 16 June 2013

 Oh, well that sounds really nice!

 Reply
 5. Mary Kay Culpepper 16 June 2013

 Thanks for writing about this topic, David; I study creativity and often think about tools that can engender it. Most of the physical tools I encounter in my research about sketching as a means to solve problems involve paper and pencil or keyboard and monitor.

 What’s so captivating about The Circles is that unlike paper, they are infinitely reusable; unlike a computer, they capture the immediacy of moving a thought from the brain to a sketch on the chalkboard. While they were designed for children, I would use The Circles for sketch-based storyboarding–beginning with the problem, ending with the desired result, and working my way to a solution from both ends.

 Like Fanny’s other tools, The Circles are novel and useful, which is a generally accepted definition of a creative product. Plus, they’re handsome, and as the design theorists say in unguarded moments, pretty never hurts.

 Reply

 Leave a Reply

 Your email address will not be published. Required fields are marked *

 Comment

 Name *

 Email *

 Website

 All blog posts

 View a complete list of posts.

 Recent Posts

 * Pecha Kucha video: ‘Identities, Creativity, Practice, Process and Change’
 * Chris E. Vargas launches ‘Trans Hirstory in 99 Objects’
 * Practice-based research is for imagining and exploring
 * Conversation with Justine Woods and Francisco-Fernando Granados on practice-based research
 * A practice-based PhD as a basket of things

 Recent Comments

 * Isabella Wesoly on William Morris, making things, Kickstarter, and LEGO
 * Simon Reynell on Drawing just one line
 * Antje Behrens on Micro LEGO Serious Play: How small can a useful tool for thinking be?
 * Holly Peacock on New video on ‘identity’ for UK A and AS Level students
 * David Gauntlett on Our Innovation Studio is not what you think

 Categories

 * A Drawing A Day
 * Creativity
 * Creativity: Seven Keys book
 * Digital DIY
 * Digital media
 * Making is Connecting
 * Making Media Studies
 * Making Things With Makers About Making
 * Research practice
 * Tools for thinking

 Navigation

 * Home
 * About
 * Blog
 * Stuff
 * Contact
 * Twitter
 * Insta

 See also

 → Further information
 → Complete list of publications
 → Complete list of blog posts

 Search

 All my stuff

 * Books
 * Videos
 * Projects
 * Blog
 * All books, videos, projects

 Site by David Gauntlett, 2013-16. All material appears under a Creative Commons BY-NC-SA 2.0 licence.

It is fitting that for this first ‘proper’ post on my new blog, I’m writing about tools for thinking. I’ve been thinking about tools for thinking more and more, and for some time now, although I haven’t always used the term ‘tools for thinking’.
For instance, I wrote a whole book about tools for thinking, called  Creative Explorations , published by Routledge in 2007. A quick text search in the manuscript shows that the phrase ‘tools for thinking’ does appear in it, but only once, and sort of by accident.
So what do I mean by tools for thinking? Well, that’s what this blog post is about.
Because I’ve been thinking about tools for thinking, and wondering if perhaps I’ll write my next book about this subject, I searched on Amazon to see if there were actually hundreds of books already with ‘tools for thinking’ in the title.
Now, as an author, I obviously hope to be saying something distinctive. I don’t know where the phrase ‘tools for thinking’ came from. Did I invent it? That would be nice. Or did I just hear it somewhere? I have no idea. In self-serving terms, I hope that the number of books with ‘tools for thinking’ in the title is zero. But that’s not very likely, is it? In rational terms, ‘tools for thinking’ sound like a very obvious kind of thing that most people have a real need for, and so you’d expect that Amazon would offer a few hundred ‘tools for thinking’ books, at the very least.
In fact: there are very few. There’s a new book by Daniel Dennett, called Intuition Pumps, and Other Tools for Thinking (2013), of which more in a moment.
And then there’s a book on modelling in management science, called Tools for Thinking: Modelling in Management Science (by Michael Pidd, third edition 2009), but thankfully, as far as I’m concerned, that’s not really the same as what I’m talking about, because it’s about modelling in management science.
Then, on the Amazon list, after those two, that’s the end of the books with ‘tools for thinking’ in the title, and we’re on to other items such as Six Thinking Hats by Edward de Bono, then other de Bono books. Soon after that, we’re onto items like the ‘Gerber Suspension Multitool’, which looks like this:
 
That’s not really what I meant. But I’m quite impressed that Amazon was able to identify Six Thinking Hats as a book about tools for thinking, because that’s what it is, even though it doesn’t say so in an obvious place like the title, or even in the blurb. Edward de Bono’s Six Thinking Hats was first published in 1985, and was preceded by De Bono’s Thinking Course: Powerful Tools to Transform Your Thinking, first published in 1982.
That book, published more than 30 years ago, talks about ‘tools’ for thinking a lot. You can see that from the Amazon page previews. I’ve looked at that book before, probably, and there’s certainly a copy of Six Thinking Hats in the house somewhere. Edward de Bono always seemed like a bit of a hack, parcelling up his ideas into several rather-too-similar books and videos and courses, turning them into a big money machine. But maybe that’s just the sound of my brain being jealous. And did I invent the phrase ‘tools for thinking’? That’s a no.
So I’m going to have to order that De Bono’s Thinking Course, just to be comprehensive. And meanwhile, I’ve already ordered, and received, the Daniel Dennett book which is #1 in Amazon’s ‘tools for thinking’ list. Here it is, photographed for this blog post next to my own Creative Explorations:
 
Now, by coincidence (but it can’t exactly be a coincidence), I’ve encountered Daniel Dennett before. In fact, he’s discussed in Creative Explorations – not because of anything to do with ‘tools for thinking’, but in his role as a philosopher and cognitive scientist who made an argument for consciousness being a kind of illusion (to simplify greatly). Since then, Dennett has also become notable as a fundamentalist atheist in the Richard Dawkins mould, which I find unattractive (simply because dealing with religion by telling religious people that they are idiots doesn’t seem to be a very compassionate way of going about your ‘humanism’). But we can put all that aside for now.
So what’s Daniel Dennett up to in his ‘tools for thinking’ book?
Well the good news for me is that it’s definitely not the ‘tools for thinking’ book that I might want to write.  It’s basically about the kinds of thought experiment and thinking tricks that you can do in order to test arguments. Dennett has collected some old ones and some fresh ones, but a classic example to give here, so you can see the kind of thing we’re talking about, is the Reductio Ad Absurdum: taking an argument to the extreme to see if it still makes sense. Another well-known ‘tool for thinking’ of this kind is Occam’s Razor, which is the idea that you shouldn’t use a complex explanation if you’ve got a simpler one which works just as well.
So the ‘tools’ here are words and ideas and ways of thinking, and it would be a useful book for students of philosophy, or critical thinking, or indeed anyone who wants to get better at exploring and testing arguments. Both religious and scientific ideas could be tested with these tools.
But when I’m talking about tools for thinking – and you’ll have to forgive me for being so literal-minded – I’m talking about actual tools. Things you can hold in your hand. Tangible things you can shift and manipulate, which help you to think about things. (And also, their digital equivalents, and physical/digital hybrids, but that’s for a future blogpost).
My own best example of that is the things I’ve done with LEGO. In 2005 I became involved with LEGO Serious Play, a consultancy process developed by the LEGO Group – an activity for groups of adults, guided by a facilitator, in which participants would build metaphorical models using LEGO bricks. The models would typically represent their experiences of activities, structures and communications within their organisations, and then – having externalised this, by having built it in LEGO – they would go on to develop ideas for initiatives or strategies, in response to what they had built.
I worked with the LEGO Group on researching some aspects of this process, and I developed it as a social-science research tool. Specifically, I worked out how to use it as an alternative to the traditional social-science qualitative methods of interviews or focus groups. That’s what  Creative Explorations  is about. Later I worked with the LEGO Group on the open source release of LEGO Serious Play, and co-wrote the documentation.
It’s all about getting people to physically build stuff, representing their ideas, experiences and feelings, so that then you can look at it, reflect on it, respond to it, move things around, change it, and make new plans. That’s what I call a real tool for thinking.
Now, my use of LEGO Serious Play as a social-science research tool was intended to be a good thing for participants – giving them a stimulating, creative experience which is more interesting to do than traditional research methods. Ultimately, though, I now reflect that basically it was done for the purposes of the researcher rather than anything else – it was meant to be a good research tool, which is fair enough, but that’s not actually about helping people to solve their own problems, which is what I’m interested in now. I’ll say more about that in a future post.
It’s surprising, I think, that you don’t find more physical, hands-on tools for thinking. I was recently delighted to be contacted by Fanny Bissa, a final year Design student at Goldsmiths College, University of London. She was working on a whole set of hands-on tools for working through different kinds of problems and relationships (see her excellent Tumblr blog and online booklet). We met up a couple of times to talk about it. One of the tools can be seen in the image at the top of this blogpost. I’ll write more about that, as well, in a future post. For now, here’s a short video she made about her tools (3 minutes):
 
 Image at the top by Fanny Bissa, reproduced by kind permission.
David Gauntlett | Tools for thinking: Introduction
https://davidgauntlett.com/tools-for-thinking/tools-for-thinking/
https://space-facts.com/googles-planet-scale-mistake/
Google's Planet Scale Mistake - Space Facts
2019-07-31
Chris
<div><section>
	<main>
	
<article>
	<div>
	
<p>I’ve <a href="https://space-facts.com/comparing-planets-moons-dwarf-planets-google/">written before</a> about Google showing planet information directly in its search results. Since then Google has begun to highlight information from particular websites more and more. Today I came across a case where this could be considered as a little misleading:</p>
<div><figure></figure></div>
<p>Google is taking the different sizes of some neat looking <a href="https://www.littleplanetfactory.com/products/the-solar-system-scale">planet models</a> and using these rather than the planet’s actual sizes. While the relative scale is correct, the Earth is of course slightly larger than the 8.4mm shown here. </p>
<div><figure></figure></div>
<p>You can currently see this at <a href="https://www.google.com/search?q=planets+to+scale">https://www.google.com/search?q=planets+to+scale</a> or by searching for “planets to scale”. I expect that this will be fixed at some point. </p>
<p>You can find the actual sizes of the planets on the <a href="https://space-facts.com/planets/">Planet Facts</a> page. </p>
	</div>
	
</article>
	</main>
	</section>||||I|||| Skip to content

 Space Facts

 * Planets
 + Terrestrial Planets
 o Mercury
 o Venus
 o Earth
 o Mars
 + Gas Giants
 o Jupiter
 o Saturn
 o Uranus
 o Neptune
 + Dwarf Planets
 o Ceres
 o Pluto
 o Haumea
 o Makemake
 o Eris
 * Solar System
 + The Sun
 + Solar Eclipses
 + Asteroid Belt
 + Kuiper Belt
 + Oort Cloud
 + Asteroids
 + Comets
 + Meteorites
 + Meteor Showers
 * Galaxies
 + Milky Way
 + Andromeda
 + Messier 87
 + Antennae
 + Magellanic Clouds
 + Pinwheel
 + Sombrero
 + Triangulum
 + Whirlpool
 + Galaxy Types
 * Moons
 + The Moon
 + Mars' Moons
 o Phobos
 o Deimos
 + Jupiter's Moons
 o Io
 o Europa
 o Ganymede
 o Callisto
 + Saturn's Moons
 o Enceladus
 o Rhea
 o Titan
 o Iapetus
 + Uranus' Moons
 o Miranda
 o Titania
 + Neptune's Moons
 o Triton
 + Pluto's Moons
 o Charon
 * Universe
 + Black Holes
 + Stars
 + Nebulae
 + Exoplanets
 * 

 Google’s Planet Scale Mistake

 July 31, 2019 Chris Blog

 I’ve written before about Google showing planet information directly in its search results. Since then Google has begun to highlight information from particular websites more and more. Today I came across a case where this could be considered as a little misleading:

 Google is taking the different sizes of some neat looking planet models and using these rather than the planet’s actual sizes. While the relative scale is correct, the Earth is of course slightly larger than the 8.4mm shown here.

 You can currently see this at https://www.google.com/search?q=planets+to+scale or by searching for “planets to scale”. I expect that this will be fixed at some point.

 You can find the actual sizes of the planets on the Planet Facts page.

 Post navigation

 Previous Post: The Ultimate List of TV Shows set in Space or on Other Planets
 Next Post: Dr James O’Donoghue’s Space Animations

 Related space facts:

 4 comments

 1. michael kemble says:
 November 6, 2019 at 7:58 am

 the guestimate of stars in the Milky Way on page 1 does seem rather vague. Astronomers in various discovery space programmes tell us a figure (which I cant remember) that is a lot more precise than 100bn – 400bn. How often do the ‘facts’ get updated?

 good site – thanks

 Reply
 2. zaria says:
 November 8, 2019 at 7:11 pm

 well that is cool i guess

 Reply
 3. Minecraft_nerd says:
 January 11, 2022 at 7:32 pm

 (^U^)

 (^_^)

 Reply
 4. Brooklyn Baca says:
 March 22, 2022 at 10:37 pm

 this is a good site helps with all of my space projects and gives me real info and great facts

 Reply

 Leave a Reply

 Your email address will not be published. Required fields are marked *

 Comment *

 Name *

 Email *

 Website

 Δ
 Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
 To find out more, including how to control cookies, see here: Cookie Policy

 Top Facts

 * Planet Facts
 * Moon Facts
 * Mars Facts
 * Saturn Facts
 * Mercury Facts
 * Earth Facts
 * Jupiter Facts
 * Neptune Facts
 * Venus Facts

 Recent Posts

 * Dr James O’Donoghue’s Space Animations
 * Google’s Planet Scale Mistake
 * The Ultimate List of TV Shows set in Space or on Other Planets
 * Jupiter’s Galilean Moons (Profile Gifs)
 * The Planets and Moons of Star Wars to Scale
 * Planet Orbits
 © Copyright 2021 Space Facts
 * Blog
 * About
 * Contact
 * Privacy Policy
I’ve written before about Google showing planet information directly in its search results. Since then Google has begun to highlight information from particular websites more and more. Today I came across a case where this could be considered as a little misleading:
   
Google is taking the different sizes of some neat looking planet models and using these rather than the planet’s actual sizes. While the relative scale is correct, the Earth is of course slightly larger than the 8.4mm shown here. 
   
You can currently see this at https://www.google.com/search?q=planets+to+scale or by searching for “planets to scale”. I expect that this will be fixed at some point. 
You can find the actual sizes of the planets on the Planet Facts page.
Google's Planet Scale Mistake - Space Facts
https://space-facts.com/googles-planet-scale-mistake/
https://space-facts.com/jupiters-galilean-moons-profile-gifs/
Jupiter's Galilean Moons (Profile Gifs) - Space Facts
2016-06-30
Chris
<div><section>
	<main>
	
<article>
	<div>
	<p>NASA’s Juno Spacecraft will arrive at Jupiter in the next few day after nearly five years in space. While it’s purpose is primarily to study Jupiter’s origins, interior structure, deep atmosphere and magnetosphere, it’s worth remembering Jupiter is also host to the largest and some of the most interesting moons in the solar system.</p>
<p>Lets have a look at Jupiter’s four Galilean moons: <a href="https://space-facts.com/moons/io/">Io</a>, <a href="https://space-facts.com/moons/europa/">Europa</a>, <a href="https://space-facts.com/moons/ganymede/">Ganymede</a> and <a href="https://space-facts.com/moons/callisto/">Callisto</a>.</p>
<p></p>
<p></p>
<p></p>
<p></p>
<p>Made using: <a href="http://www.shatters.net/celestia/">Celestia</a>, <a href="https://screentogif.codeplex.com/">Screen2Gif</a> &amp; <a href="https://www.gimp.org/">GIMP</a><br />
Based on: <a href="https://twitter.com/SpacePlasma">@spaceplasma’s</a> <a href="http://spaceplasma.tumblr.com/post/95444495299/planets-of-our-solar-system-our-solar-system">solar system gifs</a><br />
Profile sources: http://solarsystem.nasa.gov/planets, http://nssdc.gsfc.nasa.gov/planetary/factsheet/joviansatfact.html</p>
	</div>
	
</article>
	</main>
	</section>||||I|||| Skip to content

 Space Facts

 * Planets
 + Terrestrial Planets
 o Mercury
 o Venus
 o Earth
 o Mars
 + Gas Giants
 o Jupiter
 o Saturn
 o Uranus
 o Neptune
 + Dwarf Planets
 o Ceres
 o Pluto
 o Haumea
 o Makemake
 o Eris
 * Solar System
 + The Sun
 + Solar Eclipses
 + Asteroid Belt
 + Kuiper Belt
 + Oort Cloud
 + Asteroids
 + Comets
 + Meteorites
 + Meteor Showers
 * Galaxies
 + Milky Way
 + Andromeda
 + Messier 87
 + Antennae
 + Magellanic Clouds
 + Pinwheel
 + Sombrero
 + Triangulum
 + Whirlpool
 + Galaxy Types
 * Moons
 + The Moon
 + Mars' Moons
 o Phobos
 o Deimos
 + Jupiter's Moons
 o Io
 o Europa
 o Ganymede
 o Callisto
 + Saturn's Moons
 o Enceladus
 o Rhea
 o Titan
 o Iapetus
 + Uranus' Moons
 o Miranda
 o Titania
 + Neptune's Moons
 o Triton
 + Pluto's Moons
 o Charon
 * Universe
 + Black Holes
 + Stars
 + Nebulae
 + Exoplanets
 * 

 Jupiter’s Galilean Moons (Profile Gifs)

 June 30, 2016 Chris Blog

 NASA’s Juno Spacecraft will arrive at Jupiter in the next few day after nearly five years in space. While it’s purpose is primarily to study Jupiter’s origins, interior structure, deep atmosphere and magnetosphere, it’s worth remembering Jupiter is also host to the largest and some of the most interesting moons in the solar system.

 Lets have a look at Jupiter’s four Galilean moons: Io, Europa, Ganymede and Callisto.

 Made using: Celestia, Screen2Gif & GIMP
 Based on: @spaceplasma’s solar system gifs
 Profile sources: http://solarsystem.nasa.gov/planets, http://nssdc.gsfc.nasa.gov/planetary/factsheet/joviansatfact.html

 Post navigation

 Previous Post: The Planets and Moons of Star Wars to Scale
 Next Post: The Ultimate List of TV Shows set in Space or on Other Planets

 Related space facts:

 29 comments

 1. christian says:
 February 7, 2017 at 6:37 pm

 cool

 Reply
 1. Aakash says:
 January 21, 2021 at 7:01 pm

 yah it is cool

 Reply
 2. thebloodyone12 says:
 February 28, 2017 at 6:35 pm

 Wow. These planets are extrordanary.

 Reply
 3. Issac says:
 April 7, 2017 at 3:53 pm

 I like this website

 Reply
 4. Erika says:
 April 26, 2017 at 12:07 am

 try to find more facts obout Jupiter. Thank you if you are going to do it. c:

 Reply
 5. muriel keiana says:
 May 10, 2017 at 4:05 pm

 I like the videos

 Reply
 6. PlanetGeekGirl says:
 June 23, 2017 at 8:23 pm

 I love Jupiter. I see it and its moons nearly every night outside my window, and it’s beautiful.

 Reply
 1. Jacob Maps says:
 June 21, 2018 at 3:16 pm

 wow, good for you…would you kindly gimme a tip how I can see Jupiter too…I only seem to always spot venus 🙂

 Reply
 7. slime says:
 October 5, 2017 at 5:43 pm

 the IO moon is sooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo fast

 Reply
 8. bob says:
 October 18, 2017 at 8:43 pm

 … 😛 is ok

 Reply
 9. cali white says:
 November 14, 2017 at 9:05 pm

 this is so cool.

 Reply
 10. abbi says:
 November 29, 2017 at 7:07 pm

 There is amazing things but i dont like science so no thanks bu thanks.

 Reply
 11. Traetonok says:
 December 11, 2017 at 3:14 pm

 one of jupiter’s moons likes like a buger

 Reply
 12. Mimi says:
 December 12, 2017 at 3:25 pm

 Those moons like nasty, but its cool.

 Reply
 13. Ari says:
 December 30, 2017 at 4:28 am

 cool facts and stuff! 🙂

 Reply
 14. juc_meams says:
 January 19, 2018 at 6:43 pm

 ok with the oj

 Reply
 15. jaden says:
 January 22, 2018 at 1:35 pm

 i love this sight my mind is blown

 Reply
 16. Deluous says:
 January 25, 2018 at 1:01 pm

 eh, the planets are so amazing. I just wanna see what happens when we go to the galaxies.

 Reply
 17. Wandie says:
 February 19, 2018 at 7:34 pm

 Which moon has a liquid methane lake throughout it’s surface and has mountain sized diamonds floating on its surface?

 Reply
 18. rashed says:
 February 24, 2018 at 6:02 pm

 i love this website it help me with new information about earth and all of the planet

 Reply
 19. jayden says:
 March 1, 2018 at 4:53 pm

 Ganymede is soooooooooooooooooooooooooooooooooooooooo cold

 Reply
 20. anthony says:
 March 6, 2018 at 2:11 pm

 love the facts, also Ganymede has a small amount of oxygen in its atmosphere

 Reply
 21. kennedy says:
 April 3, 2018 at 10:19 pm

 the planets are cool and I have a report and this helped a lot.

 Reply
 22. lina chocoman says:
 April 24, 2018 at 10:08 pm

 I love astronamy!

 Reply
 23. destiiny says:
 May 4, 2018 at 7:47 pm

 the sun is 1,000,000 times as big as earth

 Reply
 24. Ashley Hoober says:
 October 2, 2018 at 7:37 pm

 Wow so incredibly cool, the moon and planets have always fascinated me , ever since childhood
 thanks for the post 🙂

 Reply
 25. jayden says:
 November 20, 2019 at 2:25 am

 i love sasquatches and planets, also penguins on the moon of Ganymade. It is sooo cold there, the penguins love it.

 Reply
 26. Gervais says:
 October 1, 2020 at 11:37 am

 Where can Jupiter be observed from earth in the sky. Is it from sun rise or sun set ?

 Reply
 27. Big Boy says:
 May 31, 2022 at 5:59 pm

 Cool B-)

 Reply

 Leave a Reply

 Your email address will not be published. Required fields are marked *

 Comment *

 Name *

 Email *

 Website

 Δ
 Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
 To find out more, including how to control cookies, see here: Cookie Policy

 Top Facts

 * Planet Facts
 * Moon Facts
 * Mars Facts
 * Saturn Facts
 * Earth Facts
 * Mercury Facts
 * Jupiter Facts
 * Neptune Facts
 * Venus Facts

 Recent Posts

 * Dr James O’Donoghue’s Space Animations
 * Google’s Planet Scale Mistake
 * The Ultimate List of TV Shows set in Space or on Other Planets
 * Jupiter’s Galilean Moons (Profile Gifs)
 * The Planets and Moons of Star Wars to Scale
 * Planet Orbits
 © Copyright 2021 Space Facts
 * Blog
 * About
 * Contact
 * Privacy Policy
NASA’s Juno Spacecraft will arrive at Jupiter in the next few day after nearly five years in space. While it’s purpose is primarily to study Jupiter’s origins, interior structure, deep atmosphere and magnetosphere, it’s worth remembering Jupiter is also host to the largest and some of the most interesting moons in the solar system.
Lets have a look at Jupiter’s four Galilean moons: Io, Europa, Ganymede and Callisto.
 
 
 
 
Made using: Celestia, Screen2Gif &amp; GIMP 
Based on: @spaceplasma’s solar system gifs 
Profile sources: http://solarsystem.nasa.gov/planets, http://nssdc.gsfc.nasa.gov/planetary/factsheet/joviansatfact.html
Jupiter's Galilean Moons (Profile Gifs) - Space Facts
https://space-facts.com/jupiters-galilean-moons-profile-gifs/
https://zserge.com/posts/one-time-passwords/
How do One-Time passwords work?
2021-04-28
None
<div><div><p><a href="https://en.wikipedia.org/wiki/One-time_password">One-time passwords</a> (OTP) are short sequences of numbers that authenticate user for a single login. They are often used as an additional layer of security when the 2-factor authorisation is enabled.</p><p>A picture is worth a thousand words, here’s what I’m talking about:</p><p></p><p>How are these numbers generated and can we reproduce them?</p><h2>Import the secret</h2><p>Let’s take Google’s 2-factor authorization as an example. It all starts with telling the authenticator app the secret key from your account. It is what you get when you scan the QR code during the 2FA setup for your Google account. And luckily there is an option for those who don’t have a QR reader:</p><ol><li>Go to “Google account” - <a href="https://myaccount.google.com/">https://myaccount.google.com/</a></li><li>Go to the “Security” section in the menu on the left.</li><li>Select the “2-Step Verification” section.</li><li>You would probably be asked for the password.</li><li>Select the “Change Phone” item in the “Authenticator App” section.</li><li>A pop-up should appear asking if you have an Android or iOS device, choose any of those.</li><li>When the QR code appears - select the “Can’t scan it” link below it.</li></ol><p>This should reveal the secret key used for your account OTP generator. Keep it secret.</p><p>If you wonder, the QR there contains a special link, such as <code>otpauth-migration://offline?data=...</code>. Data parameter is a base64-encoded Protobuf message, containing the hashing algorithm (SHA1/256/512 or MD5), number of digits in the OTP (six or eight), OTP type (time-based or HMAC-based), and the actual secret bytes.</p><p>We wouldn’t bother with deciphering protobuf here, instead we would just copy the plain text secret that Google offers when you choose “Can’t scan it”. Further on I will be referring to is as <code>$SECRET</code> environment variable.</p><h2>HMAC-based passwords</h2><p>HMAC-based one-time passwords are documented in <a href="https://tools.ietf.org/html/rfc4226">RFC 4226</a>. The algorithm takes as an input:</p><ul><li>A 64-bit counter, a moving factor. Some kind of value that increases over time and is synchronised between the authenticator and the server. Time is a good example, but we will talk about it further.</li><li>A unique shared secret, known for both, server and the authenticator.</li></ul><p>The algorithm itself does the following three steps. It generates a <a href="https://en.wikipedia.org/wiki/HMAC">HMAC-SHA-1</a> value, where secret is the key and counter is the message to be hashed using the key. Then it truncates the 20-byte hash string to a shorter value.</p><p>The truncation is rather uncomplicated, the last byte from the 20-byte string is taken, the lower 4 bits are considered the “offset”. This makes the offset to be always in the range from 0 to 15.</p><p>Then, the sequential 4 bytes starting from the offset are taken and converted into the 32-bit integer. Finally, the integer is printed as decimal, stripped down to the 6 digits, and that is the requested OTP.</p><p>Here’s an example from the RFC to make it clear:</p><div><pre><code>-------------------------------------------------------------
| Byte Number |
-------------------------------------------------------------
|00|01|02|03|04|05|06|07|08|09|10|11|12|13|14|15|16|17|18|19|
-------------------------------------------------------------
| Byte Value |
-------------------------------------------------------------
|1f|86|98|69|0e|02|ca|16|61|85|50|ef|7f|19|da|8e|94|5b|55|5a|
-------------------------------***********----------------++|
</code></pre></div><p>The message above is a HMAC-SHA-1 value, a string of 20 bytes. The last byte is 0x5a. The lower 4 bits are 0xa, or 10 in decimal. This means we should take the bytes 10, 11, 12 and 13 and read them as a 32-bit big-endian integer 0x50ef7f19. In decimal this integer would be 1357872921. The last 6 digits would be “872921” and that would be the OTP.</p><p>Not that complicated, huh?</p><h2>Time-based passwords</h2><p>The <a href="https://tools.ietf.org/html/rfc6238">RFC</a> that describes one-time passwords is very brief. It only mentions that the counter value from the HOTP algorithm above would be the time difference between “now” and some fixed start point (usually 1 Jan 1970), divided by some fixed time interval (often: 30 seconds).</p><p>The rest is handled by the algorithm above.</p><p>Knowing this, we can finally implement time-based OTP (here the code is in Go, but you may use the language of your choice):</p><div><pre><code>func totp(secret []byte) (string, error) {
 // Counter is UNIX time in seconds, divided by interval of 30 seconds
 counter := time.Now().Unix() / 30
 // Decode Base-32 secret key
 key := make([]byte, 64)
 if _, err := base32.StdEncoding.Decode(key, bytes.ToUpper([]byte(secret))); err != nil {
 return "", err
 }
 // Write counter as 8 bytes, big-endian
 b := make([]byte, 8)
 binary.BigEndian.PutUint64(b, uint64(counter))
 // Calcular HMAC-SHA-1
 mac160 := hmac.New(sha1.New, key)
 mac160.Write(b)
 b = mac160.Sum(nil)
 // Find offset, lower 4 bits of the last byte
 offset := b[len(b)-1] &amp; 0xf
 // Read 4 bytes from offset as 32-bit integer
 n := binary.BigEndian.Uint32(b[offset : offset+4])
 // Covert it to decimal
 s := fmt.Sprintf("%06d", int(n &amp; 0x7fffffff))
 // Return last 6 digits
 return s[len(s)-6:], nil
}
</code></pre></div><p>That’s just 16 lines of code! If you call totp() function passing it your secret key – you should get a 6-digit number that matches the one in the Google Authenticator:</p><div><pre><code>otp, err := totp([]byte(os.Getenv("SECRET")))
if err != nil {
 log.Fatal(err)
}
log.Println("OTP:", otp)
</code></pre></div><p>Now you can use this little utility to get one-time passwords quickly without reaching for the phone.</p><p>I hope you’ve enjoyed this article. You can follow – and contribute to – on <a href="https://github.com/zserge">Github</a>, <a href="https://twitter.com/zsergo">Twitter</a> or subscribe via <a href="/rss.xml">rss</a>.</p><p><em>Apr 28, 2021</em></p><p>See also:
<a href="/posts/agile-issue-tracker/">my minimalistic agile issue tracker</a> and <a href="/posts/">more</a>.</p></div>||||I||||Z
 * about
 * posts
 * @me
 * </>me

 How do One-Time passwords work?

 One-time passwords (OTP) are short sequences of numbers that authenticate user for a single login. They are often used as an additional layer of security when the 2-factor authorisation is enabled.

 A picture is worth a thousand words, here’s what I’m talking about:

 How are these numbers generated and can we reproduce them?

 Import the secret

 Let’s take Google’s 2-factor authorization as an example. It all starts with telling the authenticator app the secret key from your account. It is what you get when you scan the QR code during the 2FA setup for your Google account. And luckily there is an option for those who don’t have a QR reader:

 1. Go to “Google account” - https://myaccount.google.com/
 2. Go to the “Security” section in the menu on the left.
 3. Select the “2-Step Verification” section.
 4. You would probably be asked for the password.
 5. Select the “Change Phone” item in the “Authenticator App” section.
 6. A pop-up should appear asking if you have an Android or iOS device, choose any of those.
 7. When the QR code appears - select the “Can’t scan it” link below it.

 This should reveal the secret key used for your account OTP generator. Keep it secret.

 If you wonder, the QR there contains a special link, such as otpauth-migration://offline?data=.... Data parameter is a base64-encoded Protobuf message, containing the hashing algorithm (SHA1/256/512 or MD5), number of digits in the OTP (six or eight), OTP type (time-based or HMAC-based), and the actual secret bytes.

 We wouldn’t bother with deciphering protobuf here, instead we would just copy the plain text secret that Google offers when you choose “Can’t scan it”. Further on I will be referring to is as $SECRET environment variable.

 HMAC-based passwords

 HMAC-based one-time passwords are documented in RFC 4226. The algorithm takes as an input:

 * A 64-bit counter, a moving factor. Some kind of value that increases over time and is synchronised between the authenticator and the server. Time is a good example, but we will talk about it further.
 * A unique shared secret, known for both, server and the authenticator.

 The algorithm itself does the following three steps. It generates a HMAC-SHA-1 value, where secret is the key and counter is the message to be hashed using the key. Then it truncates the 20-byte hash string to a shorter value.

 The truncation is rather uncomplicated, the last byte from the 20-byte string is taken, the lower 4 bits are considered the “offset”. This makes the offset to be always in the range from 0 to 15.

 Then, the sequential 4 bytes starting from the offset are taken and converted into the 32-bit integer. Finally, the integer is printed as decimal, stripped down to the 6 digits, and that is the requested OTP.

 Here’s an example from the RFC to make it clear:

 -------------------------------------------------------------
 | Byte Number |
 -------------------------------------------------------------
 |00|01|02|03|04|05|06|07|08|09|10|11|12|13|14|15|16|17|18|19|
 -------------------------------------------------------------
 | Byte Value |
 -------------------------------------------------------------
 |1f|86|98|69|0e|02|ca|16|61|85|50|ef|7f|19|da|8e|94|5b|55|5a|
 -------------------------------***********----------------++|
 

 The message above is a HMAC-SHA-1 value, a string of 20 bytes. The last byte is 0x5a. The lower 4 bits are 0xa, or 10 in decimal. This means we should take the bytes 10, 11, 12 and 13 and read them as a 32-bit big-endian integer 0x50ef7f19. In decimal this integer would be 1357872921. The last 6 digits would be “872921” and that would be the OTP.

 Not that complicated, huh?

 Time-based passwords

 The RFC that describes one-time passwords is very brief. It only mentions that the counter value from the HOTP algorithm above would be the time difference between “now” and some fixed start point (usually 1 Jan 1970), divided by some fixed time interval (often: 30 seconds).

 The rest is handled by the algorithm above.

 Knowing this, we can finally implement time-based OTP (here the code is in Go, but you may use the language of your choice):

 func totp(secret []byte) (string, error) {
 // Counter is UNIX time in seconds, divided by interval of 30 seconds
 counter := time.Now().Unix() / 30
 // Decode Base-32 secret key
 key := make([]byte, 64)
 if _, err := base32.StdEncoding.Decode(key, bytes.ToUpper([]byte(secret))); err != nil {
 return "", err
 }
 // Write counter as 8 bytes, big-endian
 b := make([]byte, 8)
 binary.BigEndian.PutUint64(b, uint64(counter))
 // Calcular HMAC-SHA-1
 mac160 := hmac.New(sha1.New, key)
 mac160.Write(b)
 b = mac160.Sum(nil)
 // Find offset, lower 4 bits of the last byte
 offset := b[len(b)-1] & 0xf
 // Read 4 bytes from offset as 32-bit integer
 n := binary.BigEndian.Uint32(b[offset : offset+4])
 // Covert it to decimal
 s := fmt.Sprintf("%06d", int(n & 0x7fffffff))
 // Return last 6 digits
 return s[len(s)-6:], nil
 }
 

 That’s just 16 lines of code! If you call totp() function passing it your secret key – you should get a 6-digit number that matches the one in the Google Authenticator:

 otp, err := totp([]byte(os.Getenv("SECRET")))
 if err != nil {
 log.Fatal(err)
 }
 log.Println("OTP:", otp)
 

 Now you can use this little utility to get one-time passwords quickly without reaching for the phone.

 I hope you’ve enjoyed this article. You can follow – and contribute to – on Github, Twitter or subscribe via rss.

 Apr 28, 2021

 See also: my minimalistic agile issue tracker and more.

©2012–2022 · Serge Zaitsev · hello@zserge.com

One-time passwords (OTP) are short sequences of numbers that authenticate user for a single login. They are often used as an additional layer of security when the 2-factor authorisation is enabled. A picture is worth a thousand words, here’s what I’m talking about:   How are these numbers generated and can we reproduce them? Import the secret Let’s take Google’s 2-factor authorization as an example. It all starts with telling the authenticator app the secret key from your account. It is what you get when you scan the QR code during the 2FA setup for your Google account. And luckily there is an option for those who don’t have a QR reader:  Go to “Google account” - https://myaccount.google.com/  Go to the “Security” section in the menu on the left. Select the “2-Step Verification” section. You would probably be asked for the password. Select the “Change Phone” item in the “Authenticator App” section. A pop-up should appear asking if you have an Android or iOS device, choose any of those. When the QR code appears - select the “Can’t scan it” link below it.  This should reveal the secret key used for your account OTP generator. Keep it secret. If you wonder, the QR there contains a special link, such as otpauth-migration://offline?data=.... Data parameter is a base64-encoded Protobuf message, containing the hashing algorithm (SHA1/256/512 or MD5), number of digits in the OTP (six or eight), OTP type (time-based or HMAC-based), and the actual secret bytes. We wouldn’t bother with deciphering protobuf here, instead we would just copy the plain text secret that Google offers when you choose “Can’t scan it”. Further on I will be referring to is as $SECRET environment variable. HMAC-based passwords HMAC-based one-time passwords are documented in RFC 4226. The algorithm takes as an input:  A 64-bit counter, a moving factor. Some kind of value that increases over time and is synchronised between the authenticator and the server. Time is a good example, but we will talk about it further. A unique shared secret, known for both, server and the authenticator.  The algorithm itself does the following three steps. It generates a HMAC-SHA-1 value, where secret is the key and counter is the message to be hashed using the key. Then it truncates the 20-byte hash string to a shorter value. The truncation is rather uncomplicated, the last byte from the 20-byte string is taken, the lower 4 bits are considered the “offset”. This makes the offset to be always in the range from 0 to 15. Then, the sequential 4 bytes starting from the offset are taken and converted into the 32-bit integer. Finally, the integer is printed as decimal, stripped down to the 6 digits, and that is the requested OTP. Here’s an example from the RFC to make it clear:   -------------------------------------------------------------
| Byte Number |
-------------------------------------------------------------
|00|01|02|03|04|05|06|07|08|09|10|11|12|13|14|15|16|17|18|19|
-------------------------------------------------------------
| Byte Value |
-------------------------------------------------------------
|1f|86|98|69|0e|02|ca|16|61|85|50|ef|7f|19|da|8e|94|5b|55|5a|
-------------------------------***********----------------++|
   The message above is a HMAC-SHA-1 value, a string of 20 bytes. The last byte is 0x5a. The lower 4 bits are 0xa, or 10 in decimal. This means we should take the bytes 10, 11, 12 and 13 and read them as a 32-bit big-endian integer 0x50ef7f19. In decimal this integer would be 1357872921. The last 6 digits would be “872921” and that would be the OTP. Not that complicated, huh? Time-based passwords The RFC that describes one-time passwords is very brief. It only mentions that the counter value from the HOTP algorithm above would be the time difference between “now” and some fixed start point (usually 1 Jan 1970), divided by some fixed time interval (often: 30 seconds). The rest is handled by the algorithm above. Knowing this, we can finally implement time-based OTP (here the code is in Go, but you may use the language of your choice):   func totp(secret []byte) (string, error) {
 // Counter is UNIX time in seconds, divided by interval of 30 seconds
 counter := time.Now().Unix() / 30
 // Decode Base-32 secret key
 key := make([]byte, 64)
 if _, err := base32.StdEncoding.Decode(key, bytes.ToUpper([]byte(secret))); err != nil {
 return "", err
 }
 // Write counter as 8 bytes, big-endian
 b := make([]byte, 8)
 binary.BigEndian.PutUint64(b, uint64(counter))
 // Calcular HMAC-SHA-1
 mac160 := hmac.New(sha1.New, key)
 mac160.Write(b)
 b = mac160.Sum(nil)
 // Find offset, lower 4 bits of the last byte
 offset := b[len(b)-1] &amp; 0xf
 // Read 4 bytes from offset as 32-bit integer
 n := binary.BigEndian.Uint32(b[offset : offset+4])
 // Covert it to decimal
 s := fmt.Sprintf("%06d", int(n &amp; 0x7fffffff))
 // Return last 6 digits
 return s[len(s)-6:], nil
}
   That’s just 16 lines of code! If you call totp() function passing it your secret key – you should get a 6-digit number that matches the one in the Google Authenticator:   otp, err := totp([]byte(os.Getenv("SECRET")))
if err != nil {
 log.Fatal(err)
}
log.Println("OTP:", otp)
   Now you can use this little utility to get one-time passwords quickly without reaching for the phone. I hope you’ve enjoyed this article. You can follow – and contribute to – on Github, Twitter or subscribe via rss.  Apr 28, 2021  See also:
my minimalistic agile issue tracker and more.
How do One-Time passwords work?
https://zserge.com/posts/one-time-passwords/
https://zserge.com/posts/bfaas/
Scalable, Resilient Brainf∗ck
2021-04-01
None
<div><div><p><em>You probably shouldn’t read this article.</em></p><p>If you want to start learning a programming language - write a BF interpreter in it. For many years this has been a common suggestion to the newcomers into programming. And we all know how the typical solution would look like:</p><div><pre><code>s[999],*r=s,*d,c;main(a,b){char*v=1[d=b];for(;c=*v++%93;)for(b=c%7?a&amp;&amp;(c&amp;17?c&amp;1?
(*r-=c-44):(r+=c-61):c&amp;2?putchar(*r):(*r=getchar()),0):v;b&amp;&amp;c|a**r;v=d)main(!c,&amp;
b-1);d=v;}
</code></pre></div><p>Well, or something a bit more readable in ~100 lines of code.</p><p>But, hey, that’s not how things should be done these days! First of all, the common truth is that “C is unsafe” and thus should be avoided at all costs. We are writing an interpreter, a serious and complex software, so Java, Go, Python, at least Rust would be a good start.</p><p>Then, every junior programmer knows that computers are unreliable, and such a critical tool as interpreter should not be run on a local machine. Better move it into the cloud from the beginning. Nothing says “I treat my software seriously” better than paying for its CPU usage from your own pocket.</p><p>And of course, monoliths are so old-school. Look, BF interpreter has been waiting for years to be semantically split into self-contained microservices. How about we take one to do the parsing, another moves the pointer and the third one increments and decrements memory cells? Sounds logical, and definitely reduces the complexity of the interpreter.</p><p>When we do the parsing - we need to take care about the loops. Ideally, we should put loop beginning address onto the stack and pop it when we need to jump back and repeat the loop. However, nobody in their sane mind would be writing such complex data structure as stack manually! Fortunately, quick googling shows that Redis already has lists and stacks implemented, and it’s a software with good reputation and many githib stars, so we’d better delegate handling stacks to it. Some trivial LPUSH/LPOP commands would do all the job and our service remains small and focused.</p><p>Moving the pointer is not so simple either. Where do we store the pointer value? I suggest we take a MongoDB and put a pointer as a document there. Yes, for now it’s just a single number, but who knows what else we would need to store in the future? We are lucky here, Mongo even has an <code>$inc</code> function to increment numbers, as if it was created to be used in BF interpreters! Talking to my SRE peers proves that if one has not got any weird problems with Mongo yet - they should definitely give it a try in production!</p><p>Finally, operations on memory cells. Whoa, this is hard. Memory is big and should be reliable. How about PostgreSQL? It’s battle tested, we can set up a few replicas. In fact, let’s go one step futher and think about performance. We can’t put all memory cells into one database. That would be slow. How about we put odd addresses in one and even addresses in the other? Such sharding should speed up things a little.</p><p>Obviously, we need some kind of a load balancer at front. And, we are done!</p><p></p><p>You think I’m joking? Here’s a prototype - <a href="https://github.com/zserge/bfapi">https://github.com/zserge/bfapi</a> - and it works much like we’ve designed above. Most of the code there was copied from StackOverflow, proving that it’s correct and we can skip writing tests and save us some time. In fact, the service runs a “hello world” script in only 400ms! That’s faster than a typical page loading on the web, proving that our design is a real improvement!</p><p>Enjoy, and please, think twice before adding complexity to the systems!</p><p>I hope you’ve enjoyed this article. You can follow – and contribute to – on <a href="https://github.com/zserge">Github</a>, <a href="https://twitter.com/zsergo">Twitter</a> or subscribe via <a href="/rss.xml">rss</a>.</p><p><em>Apr 01, 2021</em></p><p>See also:
<a href="/posts/better-c-benchmark/">A "Better C" Benchmark</a> and <a href="/posts/">more</a>.</p></div>||||I||||Z
 * about
 * posts
 * @me
 * </>me

 Scalable, Resilient Brainf∗ck

 You probably shouldn’t read this article.

 If you want to start learning a programming language - write a BF interpreter in it. For many years this has been a common suggestion to the newcomers into programming. And we all know how the typical solution would look like:

 s[999],*r=s,*d,c;main(a,b){char*v=1[d=b];for(;c=*v++%93;)for(b=c%7?a&&(c&17?c&1?
 (*r-=c-44):(r+=c-61):c&2?putchar(*r):(*r=getchar()),0):v;b&&c|a**r;v=d)main(!c,&
 b-1);d=v;}
 

 Well, or something a bit more readable in ~100 lines of code.

 But, hey, that’s not how things should be done these days! First of all, the common truth is that “C is unsafe” and thus should be avoided at all costs. We are writing an interpreter, a serious and complex software, so Java, Go, Python, at least Rust would be a good start.

 Then, every junior programmer knows that computers are unreliable, and such a critical tool as interpreter should not be run on a local machine. Better move it into the cloud from the beginning. Nothing says “I treat my software seriously” better than paying for its CPU usage from your own pocket.

 And of course, monoliths are so old-school. Look, BF interpreter has been waiting for years to be semantically split into self-contained microservices. How about we take one to do the parsing, another moves the pointer and the third one increments and decrements memory cells? Sounds logical, and definitely reduces the complexity of the interpreter.

 When we do the parsing - we need to take care about the loops. Ideally, we should put loop beginning address onto the stack and pop it when we need to jump back and repeat the loop. However, nobody in their sane mind would be writing such complex data structure as stack manually! Fortunately, quick googling shows that Redis already has lists and stacks implemented, and it’s a software with good reputation and many githib stars, so we’d better delegate handling stacks to it. Some trivial LPUSH/LPOP commands would do all the job and our service remains small and focused.

 Moving the pointer is not so simple either. Where do we store the pointer value? I suggest we take a MongoDB and put a pointer as a document there. Yes, for now it’s just a single number, but who knows what else we would need to store in the future? We are lucky here, Mongo even has an $inc function to increment numbers, as if it was created to be used in BF interpreters! Talking to my SRE peers proves that if one has not got any weird problems with Mongo yet - they should definitely give it a try in production!

 Finally, operations on memory cells. Whoa, this is hard. Memory is big and should be reliable. How about PostgreSQL? It’s battle tested, we can set up a few replicas. In fact, let’s go one step futher and think about performance. We can’t put all memory cells into one database. That would be slow. How about we put odd addresses in one and even addresses in the other? Such sharding should speed up things a little.

 Obviously, we need some kind of a load balancer at front. And, we are done!

 You think I’m joking? Here’s a prototype - https://github.com/zserge/bfapi - and it works much like we’ve designed above. Most of the code there was copied from StackOverflow, proving that it’s correct and we can skip writing tests and save us some time. In fact, the service runs a “hello world” script in only 400ms! That’s faster than a typical page loading on the web, proving that our design is a real improvement!

 Enjoy, and please, think twice before adding complexity to the systems!

 I hope you’ve enjoyed this article. You can follow – and contribute to – on Github, Twitter or subscribe via rss.

 Apr 01, 2021

 See also: A "Better C" Benchmark and more.

©2012–2022 · Serge Zaitsev · hello@zserge.com

You probably shouldn’t read this article.  If you want to start learning a programming language - write a BF interpreter in it. For many years this has been a common suggestion to the newcomers into programming. And we all know how the typical solution would look like:   s[999],*r=s,*d,c;main(a,b){char*v=1[d=b];for(;c=*v++%93;)for(b=c%7?a&amp;&amp;(c&amp;17?c&amp;1?
(*r-=c-44):(r+=c-61):c&amp;2?putchar(*r):(*r=getchar()),0):v;b&amp;&amp;c|a**r;v=d)main(!c,&amp;
b-1);d=v;}
   Well, or something a bit more readable in ~100 lines of code. But, hey, that’s not how things should be done these days! First of all, the common truth is that “C is unsafe” and thus should be avoided at all costs. We are writing an interpreter, a serious and complex software, so Java, Go, Python, at least Rust would be a good start. Then, every junior programmer knows that computers are unreliable, and such a critical tool as interpreter should not be run on a local machine. Better move it into the cloud from the beginning. Nothing says “I treat my software seriously” better than paying for its CPU usage from your own pocket. And of course, monoliths are so old-school. Look, BF interpreter has been waiting for years to be semantically split into self-contained microservices. How about we take one to do the parsing, another moves the pointer and the third one increments and decrements memory cells? Sounds logical, and definitely reduces the complexity of the interpreter. When we do the parsing - we need to take care about the loops. Ideally, we should put loop beginning address onto the stack and pop it when we need to jump back and repeat the loop. However, nobody in their sane mind would be writing such complex data structure as stack manually! Fortunately, quick googling shows that Redis already has lists and stacks implemented, and it’s a software with good reputation and many githib stars, so we’d better delegate handling stacks to it. Some trivial LPUSH/LPOP commands would do all the job and our service remains small and focused. Moving the pointer is not so simple either. Where do we store the pointer value? I suggest we take a MongoDB and put a pointer as a document there. Yes, for now it’s just a single number, but who knows what else we would need to store in the future? We are lucky here, Mongo even has an $inc function to increment numbers, as if it was created to be used in BF interpreters! Talking to my SRE peers proves that if one has not got any weird problems with Mongo yet - they should definitely give it a try in production! Finally, operations on memory cells. Whoa, this is hard. Memory is big and should be reliable. How about PostgreSQL? It’s battle tested, we can set up a few replicas. In fact, let’s go one step futher and think about performance. We can’t put all memory cells into one database. That would be slow. How about we put odd addresses in one and even addresses in the other? Such sharding should speed up things a little. Obviously, we need some kind of a load balancer at front. And, we are done!   You think I’m joking? Here’s a prototype - https://github.com/zserge/bfapi - and it works much like we’ve designed above. Most of the code there was copied from StackOverflow, proving that it’s correct and we can skip writing tests and save us some time. In fact, the service runs a “hello world” script in only 400ms! That’s faster than a typical page loading on the web, proving that our design is a real improvement! Enjoy, and please, think twice before adding complexity to the systems! I hope you’ve enjoyed this article. You can follow – and contribute to – on Github, Twitter or subscribe via rss.  Apr 01, 2021  See also:
A "Better C" Benchmark and more.
Scalable, Resilient Brainf∗ck
https://zserge.com/posts/bfaas/
https://www.zsiegel.com/2021/04/15/defensive-programming-crashing-early
Defensive Programming - Crashing Early
2021-04-15
Zachary Siegel
<div><div><article><header><time>April 15, 2021</time></header><div><p>I try to always fail hard during development. When errors occur I want them to be loud and stop me dead in my tracks. This forces me to deal with edge cases and unexpected issues right away since they affect the development flow. The more I encounter, the more I fix, and the more predictable my software becomes.</p>
<h2>Using assertions</h2>
<p>One way to fail loud and hard during development is to use assertions. Every language I have used has an <strong>assert</strong> function built-in. These will typically halt execution and throw an error when the assertion is false.</p>
<p>For example in <strong>Kotlin</strong> I might write something like the following.</p>
<div><pre><code>assert(count &gt; 0) { 
	"Count should be greater than 0 but is $count" 
}</code></pre></div>
<p>It is important to note that assertions in different languages may not be enabled by default. In the case of the JVM you need to enable it on the process with the <strong>-ea</strong> flag.</p>
<p>In Swift assertions are on by default and you can read a great overview of <strong>assert</strong> and the other handy assert-like functions <a href="https://andybargh.com/swift-assertions/">here</a>.</p>
<p>In Swift I often use this to check the current thread to ensure my callbacks are all running on the main thread.</p>
<div><pre><code>doSomeWork { 
	assert(Thread.isMainThread)
	//update ui
}</code></pre></div>
<p>Note that assertions in Swift are optimized out in release mode.</p>
<p>Fail early. Fail often. Fail hard. Make better software.</p></div></article></div>||||I|||| < Zac Siegel >
 Articles
 Videos
 Gear
 Github
 Twitter
 RSS

 Defensive Programming - Crashing Early

 April 15, 2021

 I try to always fail hard during development. When errors occur I want them to be loud and stop me dead in my tracks. This forces me to deal with edge cases and unexpected issues right away since they affect the development flow. The more I encounter, the more I fix, and the more predictable my software becomes.

 Using assertions

 One way to fail loud and hard during development is to use assertions. Every language I have used has an assert function built-in. These will typically halt execution and throw an error when the assertion is false.

 For example in Kotlin I might write something like the following.

 assert(count > 0) { 
 	"Count should be greater than 0 but is $count" 
 }

 It is important to note that assertions in different languages may not be enabled by default. In the case of the JVM you need to enable it on the process with the -ea flag.

 In Swift assertions are on by default and you can read a great overview of assert and the other handy assert-like functions here.

 In Swift I often use this to check the current thread to ensure my callbacks are all running on the main thread.

 doSomeWork { 
 	assert(Thread.isMainThread)
 	//update ui
 }

 Note that assertions in Swift are optimized out in release mode.

 Fail early. Fail often. Fail hard. Make better software.

April 15, 2021   I try to always fail hard during development. When errors occur I want them to be loud and stop me dead in my tracks. This forces me to deal with edge cases and unexpected issues right away since they affect the development flow. The more I encounter, the more I fix, and the more predictable my software becomes.
Using assertions
One way to fail loud and hard during development is to use assertions. Every language I have used has an assert function built-in. These will typically halt execution and throw an error when the assertion is false.
For example in Kotlin I might write something like the following.
  assert(count &gt; 0) { 
	"Count should be greater than 0 but is $count" 
}  
It is important to note that assertions in different languages may not be enabled by default. In the case of the JVM you need to enable it on the process with the -ea flag.
In Swift assertions are on by default and you can read a great overview of assert and the other handy assert-like functions here.
In Swift I often use this to check the current thread to ensure my callbacks are all running on the main thread.
  doSomeWork { 
	assert(Thread.isMainThread)
	//update ui
}  
Note that assertions in Swift are optimized out in release mode.
Fail early. Fail often. Fail hard. Make better software.
Defensive Programming - Crashing Early
https://www.zsiegel.com/2021/04/15/defensive-programming-crashing-early
https://vladmihalcea.com/high-performance-java-persistence-newsletter-issue-33/
High-Performance Java Persistence Newsletter, Issue 33 - Vlad Mihalcea
2021-12-01
Vladmihalcea
<div><div>
	
<h2>Introduction</h2>
<p>Welcome to a new issue of the High-Performance Java Persistence Newsletter in which we share articles, videos, workshops, and StackOverflow answers that are very relevant to any developer who interacts with a database system using Java.</p>
<h2>Articles</h2>
<p>The pick of this edition is <a href="https://connor-mcdonald.com/2021/11/13/when-does-a-table-become-too-big/">this article</a> written by Connor McDonald about database indexing performance. Many devs believe that having tens of millions of records in a table is a lot of data. But, that’s not really the case. In his article, Connor builds a 2 billion rows table that’s indexed for three types of queries, and, because of indexing, all those queries run very fast. For instance, fetching 200k users for a given post took on average 50ms, and in the worst case, 371ms. Not bad for a 2 billion record table and a 40-years old database, right?</p>
<p>A very interesting read is <a href="https://postgres.ai/blog/20211029-how-partial-and-covering-indexes-affect-update-performance-in-postgresql">this article</a> that shows the overhead of various indexing options when using PostgreSQL. The article goes through simple indexes, multi-column indexes, partial indexes, and covering indexes as well.</p>
<p>Talking of indexes or lack thereof, <a href="https://briananglin.me/posts/spending-5k-to-learn-how-database-indexes-work/">this article</a> explains that bad data access strategies can lead to hefty cloud bills. So, if you want to get good performance and lower your cloud bill, then you should optimize your data access layer based on your application-specific requirements.</p>
<p>If you’re using MySQL, then you should definitely read <a href="https://www.percona.com/blog/how-triggers-may-significantly-affect-the-amount-of-memory-allocated-to-your-mysql-server/">this article</a>, which explains the memory overhead of using very large triggers.</p>
<p>Now, if you are curious what technology stack I’d choose when starting a new web application, then check out <a href="https://vladmihalcea.com/revogain-software-architecture/">this article</a> which explains the <a href="https://revogain.com/">RevoGain</a> software architecture.</p>
<h2>Best Tweets</h2>
<p>Here are the best tweets I posted since the last newsletter:</p>
<blockquote><p>In software, legacy means it runs in production.</p>— Vlad Mihalcea (@vlad_mihalcea) <a href="https://twitter.com/vlad_mihalcea/status/1457233187474710530?ref_src=twsrc%5Etfw">November 7, 2021</a></blockquote>
<blockquote><p>Debugging in production <a href="https://t.co/xvZ8UN9lR5">pic.twitter.com/xvZ8UN9lR5</a></p>— Vlad Mihalcea (@vlad_mihalcea) <a href="https://twitter.com/vlad_mihalcea/status/1463225601838485504?ref_src=twsrc%5Etfw">November 23, 2021</a></blockquote>
<blockquote><p>Modern software architectures <a href="https://t.co/DSoVMGGEdS">pic.twitter.com/DSoVMGGEdS</a></p>— Vlad Mihalcea (@vlad_mihalcea) <a href="https://twitter.com/vlad_mihalcea/status/1461939789943513092?ref_src=twsrc%5Etfw">November 20, 2021</a></blockquote>
<blockquote><p>Knowing your IDE very well is more important than the IDE itself.</p>— Vlad Mihalcea (@vlad_mihalcea) <a href="https://twitter.com/vlad_mihalcea/status/1463784648488669185?ref_src=twsrc%5Etfw">November 25, 2021</a></blockquote>
<blockquote><p>In software, minimalism works best.</p>— Vlad Mihalcea (@vlad_mihalcea) <a href="https://twitter.com/vlad_mihalcea/status/1464108757168926721?ref_src=twsrc%5Etfw">November 26, 2021</a></blockquote>
<h2>Project Releases</h2>
<p>A very highly expected upgrade came from Amazon Aurora who added support for MySQL 8. Check out <a href="https://aws.amazon.com/blogs/database/amazon-aurora-mysql-3-with-mysql-8-0-compatibility-is-now-generally-available/">the release notes for more details</a> about your upgrade options.</p>
<p>PostgreSQL <a href="https://www.postgresql.org/docs/release/14.1/">14.1</a> has been released, providing several bug fixes for issues reported for the 14.0 version.</p>
<p>The H2 database has reached a very important milestone and released the <a href="http://www.h2database.com/html/changelog.html">2.0 version</a>.</p>
<p>The Hibernate team released the <a href="https://in.relation.to/2021/11/09/hibernate-reactive-1_1_0_Final/">1.1</a> version of Hibernate Reactive, which aims to provide a reactive data access alternative to the JDBC-based Hibernate ORM.</p>
<div><div><p></p></div><div><p>Get in touch with my latest articles!</p></div></div>
<h2>StackOverflow Answers</h2>
<p>The following StackOverflow answers have been trending over the past two weeks:</p>
<ul>
<li><a href="https://stackoverflow.com/a/28820952/1025118">How to inject multiple JPA EntityManager (persistence units) when using Spring</a></li>
<li><a href="https://stackoverflow.com/a/29004723/1025118">Hibernate Annotations – Which is better, field or property access?</a></li>
<li><a href="https://stackoverflow.com/a/25251602/1025118">How to clear a JPA collection and add new items</a></li>
<li><a href="https://stackoverflow.com/a/69492715/1025118?stw=2">Java encoding issues after upgrading to Java 17 using the Microsoft OpenJDK</a></li>
<li><a href="https://stackoverflow.com/a/69541938/1025118">Why use your application-level cache if the database already provides caching?</a> </li>
</ul>
<div>
	<p><a href="https://vladmihalcea.com/affiliate/">
	</a>
	</p>
	<p><a href="https://vladmihalcea.com/hypersistence-optimizer/">
	</a>
	</p>
	</div>
	
</div>||||I|||| Vlad Mihalcea

 * Home
 * Blog
 * Store
 + Books
 + Courses
 + Hypersistence Optimizer
 o Documentation
 - Installation Guide
 - User Guide
 o Examples
 o Release Notes
 o Issue Tracker
 o Trial Version
 o Full Version
 + Affiliate
 * Training
 + High-Performance Java Persistence
 + High-Performance SQL
 + Spring and Hibernate 6
 * Consulting
 + Coaching
 * Tutorials
 + Hibernate
 + SQL
 + Spring
 * Newsletter
 + Subscribe
 + Videos
 + Talks

 Leave a Comment

 Posted on December 1, 2021 by vladmihalcea

 High-Performance Java Persistence Newsletter, Issue 33

 Last modified:
 Follow @vlad_mihalcea

 Imagine having a tool that can automatically detect JPA and Hibernate performance issues. Wouldn’t that be just awesome?

 Well, Hypersistence Optimizer is that tool! And it works with Spring Boot, Spring Framework, Jakarta EE, Java EE, Quarkus, or Play Framework.

 So, enjoy spending your time on the things you love rather than fixing performance issues in your production system on a Saturday night!

 You can earn a significant passive income stream from promoting my book, courses, tools, training, or coaching subscriptions.

 If you're interested in supplementing your income, then join my affiliate program.

 Introduction

 Welcome to a new issue of the High-Performance Java Persistence Newsletter in which we share articles, videos, workshops, and StackOverflow answers that are very relevant to any developer who interacts with a database system using Java.

 Articles

 The pick of this edition is this article written by Connor McDonald about database indexing performance. Many devs believe that having tens of millions of records in a table is a lot of data. But, that’s not really the case. In his article, Connor builds a 2 billion rows table that’s indexed for three types of queries, and, because of indexing, all those queries run very fast. For instance, fetching 200k users for a given post took on average 50ms, and in the worst case, 371ms. Not bad for a 2 billion record table and a 40-years old database, right?

 A very interesting read is this article that shows the overhead of various indexing options when using PostgreSQL. The article goes through simple indexes, multi-column indexes, partial indexes, and covering indexes as well.

 Talking of indexes or lack thereof, this article explains that bad data access strategies can lead to hefty cloud bills. So, if you want to get good performance and lower your cloud bill, then you should optimize your data access layer based on your application-specific requirements.

 If you’re using MySQL, then you should definitely read this article, which explains the memory overhead of using very large triggers.

 Now, if you are curious what technology stack I’d choose when starting a new web application, then check out this article which explains the RevoGain software architecture.

 Best Tweets

 Here are the best tweets I posted since the last newsletter:

 In software, legacy means it runs in production.

 — Vlad Mihalcea (@vlad_mihalcea) November 7, 2021

 Debugging in production pic.twitter.com/xvZ8UN9lR5

 — Vlad Mihalcea (@vlad_mihalcea) November 23, 2021

 Modern software architectures pic.twitter.com/DSoVMGGEdS

 — Vlad Mihalcea (@vlad_mihalcea) November 20, 2021

 Knowing your IDE very well is more important than the IDE itself.

 — Vlad Mihalcea (@vlad_mihalcea) November 25, 2021

 In software, minimalism works best.

 — Vlad Mihalcea (@vlad_mihalcea) November 26, 2021

 Project Releases

 A very highly expected upgrade came from Amazon Aurora who added support for MySQL 8. Check out the release notes for more details about your upgrade options.

 PostgreSQL 14.1 has been released, providing several bug fixes for issues reported for the 14.0 version.

 The H2 database has reached a very important milestone and released the 2.0 version.

 The Hibernate team released the 1.1 version of Hibernate Reactive, which aims to provide a reactive data access alternative to the JDBC-based Hibernate ORM.

 Get in touch with my latest articles!

 SUBSCRIBE

 StackOverflow Answers

 The following StackOverflow answers have been trending over the past two weeks:

 * How to inject multiple JPA EntityManager (persistence units) when using Spring
 * Hibernate Annotations – Which is better, field or property access?
 * How to clear a JPA collection and add new items
 * Java encoding issues after upgrading to Java 17 using the Microsoft OpenJDK
 * Why use your application-level cache if the database already provides caching?

 Related

 Category: Newsletter Tags: hibernate, java, jpa, MySQL, PostgreSQL, SQL

 ← RevoGain Software Architecture
 The best way to log SQL statements with Spring Boot →

 Leave a Reply

 Your email address will not be published. Required fields are marked *

 Comment *

 Name *

 Email *

 Website

 Notify me of follow-up comments by email.

 Δ

 This site uses Akismet to reduce spam. Learn how your comment data is processed.

 Let’s connect

 * Twitter
 * YouTube
 * LinkedIn
 * Email
 * Facebook
 * Amazon
 * GitHub

 Find Article

 Search

 Book

 Affiliate

 Video Courses

 Hypersistence Optimizer

 Training

 Localhost Tunneling

 RevoGain

 Get it Now

 Tutorials

 * Hibernate
 * SQL
 * Spring
 * Git
 * FlexyPool

 Social Media

 * Twitter
 * Facebook
 * YouTube
 * GitHub
 * LinkedIn

 About

 * About
 * FAQ
 * Archive
 * Privacy Policy
 * Terms of Service

 Meta

 * Log in
 * Entries feed
 * Comments feed
 * WordPress.org
 Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
 To find out more, including how to control cookies, see here: Our Cookie Policy

 Vlad Mihalcea

 Powered by WordPress.com.

 * 
 * 
 * 
 * 
 * 
 * 
Introduction
Welcome to a new issue of the High-Performance Java Persistence Newsletter in which we share articles, videos, workshops, and StackOverflow answers that are very relevant to any developer who interacts with a database system using Java.
Articles
The pick of this edition is this article written by Connor McDonald about database indexing performance. Many devs believe that having tens of millions of records in a table is a lot of data. But, that’s not really the case. In his article, Connor builds a 2 billion rows table that’s indexed for three types of queries, and, because of indexing, all those queries run very fast. For instance, fetching 200k users for a given post took on average 50ms, and in the worst case, 371ms. Not bad for a 2 billion record table and a 40-years old database, right?
A very interesting read is this article that shows the overhead of various indexing options when using PostgreSQL. The article goes through simple indexes, multi-column indexes, partial indexes, and covering indexes as well.
Talking of indexes or lack thereof, this article explains that bad data access strategies can lead to hefty cloud bills. So, if you want to get good performance and lower your cloud bill, then you should optimize your data access layer based on your application-specific requirements.
If you’re using MySQL, then you should definitely read this article, which explains the memory overhead of using very large triggers.
Now, if you are curious what technology stack I’d choose when starting a new web application, then check out this article which explains the RevoGain software architecture.
Best Tweets
Here are the best tweets I posted since the last newsletter:
 In software, legacy means it runs in production.— Vlad Mihalcea (@vlad_mihalcea) November 7, 2021 
 Debugging in production pic.twitter.com/xvZ8UN9lR5 — Vlad Mihalcea (@vlad_mihalcea) November 23, 2021 
 Modern software architectures pic.twitter.com/DSoVMGGEdS — Vlad Mihalcea (@vlad_mihalcea) November 20, 2021 
 Knowing your IDE very well is more important than the IDE itself.— Vlad Mihalcea (@vlad_mihalcea) November 25, 2021 
 In software, minimalism works best.— Vlad Mihalcea (@vlad_mihalcea) November 26, 2021 
Project Releases
A very highly expected upgrade came from Amazon Aurora who added support for MySQL 8. Check out the release notes for more details about your upgrade options.
PostgreSQL 14.1 has been released, providing several bug fixes for issues reported for the 14.0 version.
The H2 database has reached a very important milestone and released the 2.0 version.
The Hibernate team released the 1.1 version of Hibernate Reactive, which aims to provide a reactive data access alternative to the JDBC-based Hibernate ORM.
      Get in touch with my latest articles!  
StackOverflow Answers
The following StackOverflow answers have been trending over the past two weeks:
 How to inject multiple JPA EntityManager (persistence units) when using Spring 
 Hibernate Annotations – Which is better, field or property access? 
 How to clear a JPA collection and add new items 
 Java encoding issues after upgrading to Java 17 using the Microsoft OpenJDK 
 Why use your application-level cache if the database already provides caching?
High-Performance Java Persistence Newsletter, Issue 33 - Vlad Mihalcea
https://vladmihalcea.com/high-performance-java-persistence-newsletter-issue-33/#more-23126
https://rootsofprogress.org/a-new-philosophy-of-progress
We need a new philosophy of progress
2021-08-23
None
<div><div>
 
 
 <p>We live in an age that has lost its optimism. Polls show that people <a href="https://ourworldindata.org/uploads/2016/12/Optimistic-about-the-future-2.png">think the world is getting worse</a>, not better. Children <a href="https://www.theguardian.com/environment/2020/feb/10/overwhelming-and-terrifying-impact-of-climate-crisis-on-mental-health">fear dying from environmental catastrophe</a> before they reach old age. Technologists are as likely to be told that they are <a href="https://www.scientificamerican.com/article/big-tech-out-of-control-capitalism-and-the-end-of-civilization/">ruining society</a> as that they are bettering it.</p>
<p>But it was not always so. Just a few centuries ago, Western thinkers were caught up in a wave of optimism for technology, humanity and the future, based on the new philosophy of the Enlightenment.</p>
<p>The Enlightenment was many things, but in large part, it was a philosophy of <em>progress.</em></p>
<p>At the end of the 18th century, the Marquis de Condorcet gave expression to this philosophy and its optimism in his <em>Sketch for a Historical Picture of the Progress of the Human Mind</em>. In it, he predicted unlimited progress, not only in science and technology, but in morality and society. He wrote of the equality of the races and the sexes, and of peace between nations.</p>
<p>His optimism was all the more remarkable given that he wrote this while hiding out from the French Revolution, which was hunting him down in order to execute him as an aristocrat. Unfortunately, he could not hide forever: he was captured, and soon died in prison. Evidently, the perfection of mankind was slow in coming.</p>
<p><em>Material</em> progress, however, was rocketing ahead. After the end of the Napoleonic Wars in Europe, and then the Civil War in America, the path was clear for technological innovation and economic growth: the railroad, the telephone, the light bulb, the internal combustion engine.</p>
<p>By the end of the 19th century, it was obvious that the world had entered a new age, and progress was its watchword. The naturalist Alfred Russel Wallace (best known for his work on evolution with Darwin) titled his book about the 1800s <em>The Wonderful Century</em>. In it, he attributed twenty-four “great inventions and discoveries” to the 19th century, as compared with only fifteen in all of human history preceding it. The boundless optimism of the early Enlightenment seemed to have been justified.</p>
<p>And if the material progress <a href="/the-baconian-program">prophesied by Francis Bacon</a> could be realized, perhaps the moral progress prophesied by Condorcet would come true as well. By the end of the 19th century, slavery had been ended in the West, and some hoped that the growth of industry and the expansion of trade would lead to and end to war and a new era of world peace.</p>
<p>They were wrong.</p>
<p>The 20th century violently shattered those naive illusions. The world wars were devastating proof that material progress does not inevitably lead to moral progress. Technology had not put an end to war—in fact, it had made war all the more terrible and deadly. In 1945, the nuclear bomb put a horrible exclamation point on this lesson: the most destructive weapon ever devised was the product of modern science, technology, and industry.</p>
<p>At the same time, other concerns were coming to the fore—including old ones, like poverty, and new ones, like the environment. By the mid-20th century, the philosophy of progress had been dealt a severe challenge. The optimism at its foundation had been shaken. In its place, we saw the rise of radical social movements based on a deep <em>distrust</em> of technology and industry. Today, progress and growth are called an “<a href="https://thischangeseverything.org/book/">addiction</a>”, a “<a href="https://www.allenandunwin.com/browse/books/general-books/current-affairs-politics/Growth-Fetish-Clive-Hamilton-9781741140781">fetish</a>”, a “<a href="https://www.chronicle.com/article/we-are-all-madoffs/">Ponzi scheme</a>”, or a “<a href="https://www.eesi.org/articles/view/the-fairy-tale-of-eternal-economic-growth">fairy tale</a>.” Some even advocate a new ideal of “<a href="https://www.vox.com/future-perfect/22408556/save-planet-shrink-economy-degrowth">degrowth</a>”.</p>
<p>It’s no wonder, then, that the last fifty years have seen relative stagnation in technological and industrial progress. <a href="/devanney-on-the-nuclear-flop">Nuclear power was stunted</a>, the Apollo program was canceled, the Concorde was grounded.</p>
<p>But now, in the 21st century, some people are starting to call attention to the problem: <a href="https://www.technologyreview.com/2014/09/18/171322/technology-stalled-in-1970/">Peter Thiel</a>, <a href="https://en.wikipedia.org/wiki/The_Great_Stagnation">Tyler Cowen</a>, <a href="https://www.theatlantic.com/science/archive/2019/07/we-need-new-science-progress/594946/">Patrick Collison</a>. There’s now a growing community that recognizes the threat of stagnation and the value of progress.</p>
<p>The 19th century philosophy of progress <em>was</em> naive. But the 20th century turn away from progress was no solution.</p>
<p><strong>We need a new philosophy of progress for the 21st century.</strong> One that teaches people not to take the modern world for granted. One that acknowledges the problems of progress, confronts them directly, and offers solutions. And one that holds up a positive vision of the future.</p>
<p>To establish that new philosophy is the mission of The Roots of Progress.</p>
<p><em>Today The Roots of Progress is transforming from a blog to a new nonprofit organization. <a href="/nonprofit-announcement">Read the announcement »</a></em></p>
 
 <p>
 
 
 
 
 
 
 Comment: <a href="https://www.lesswrong.com/posts/NSE5inD4zACqSu7zz">LessWrong</a>, <a href="https://www.reddit.com/r/rootsofprogress/comments/pa2a9h">Reddit</a>
 
 </p>
 
 <p>
 Social media link image credit:
 <a href="https://commons.wikimedia.org/wiki/File:Nicolas_de_Condorcet.JPG">Wikimedia / Caroline Léna Becker</a>
 </p>
 
 </div>||||I|||| The Roots of Progress

 About • Posts • Interviews • Bibliography • Subscribe • Community • Support

 We need a new philosophy of progress

 by Jason Crawford · August 23, 2021 · 3 min read

 We live in an age that has lost its optimism. Polls show that people think the world is getting worse, not better. Children fear dying from environmental catastrophe before they reach old age. Technologists are as likely to be told that they are ruining society as that they are bettering it.

 But it was not always so. Just a few centuries ago, Western thinkers were caught up in a wave of optimism for technology, humanity and the future, based on the new philosophy of the Enlightenment.

 The Enlightenment was many things, but in large part, it was a philosophy of progress.

 At the end of the 18th century, the Marquis de Condorcet gave expression to this philosophy and its optimism in his Sketch for a Historical Picture of the Progress of the Human Mind. In it, he predicted unlimited progress, not only in science and technology, but in morality and society. He wrote of the equality of the races and the sexes, and of peace between nations.

 His optimism was all the more remarkable given that he wrote this while hiding out from the French Revolution, which was hunting him down in order to execute him as an aristocrat. Unfortunately, he could not hide forever: he was captured, and soon died in prison. Evidently, the perfection of mankind was slow in coming.

 Material progress, however, was rocketing ahead. After the end of the Napoleonic Wars in Europe, and then the Civil War in America, the path was clear for technological innovation and economic growth: the railroad, the telephone, the light bulb, the internal combustion engine.

 By the end of the 19th century, it was obvious that the world had entered a new age, and progress was its watchword. The naturalist Alfred Russel Wallace (best known for his work on evolution with Darwin) titled his book about the 1800s The Wonderful Century. In it, he attributed twenty-four “great inventions and discoveries” to the 19th century, as compared with only fifteen in all of human history preceding it. The boundless optimism of the early Enlightenment seemed to have been justified.

 And if the material progress prophesied by Francis Bacon could be realized, perhaps the moral progress prophesied by Condorcet would come true as well. By the end of the 19th century, slavery had been ended in the West, and some hoped that the growth of industry and the expansion of trade would lead to and end to war and a new era of world peace.

 They were wrong.

 The 20th century violently shattered those naive illusions. The world wars were devastating proof that material progress does not inevitably lead to moral progress. Technology had not put an end to war—in fact, it had made war all the more terrible and deadly. In 1945, the nuclear bomb put a horrible exclamation point on this lesson: the most destructive weapon ever devised was the product of modern science, technology, and industry.

 At the same time, other concerns were coming to the fore—including old ones, like poverty, and new ones, like the environment. By the mid-20th century, the philosophy of progress had been dealt a severe challenge. The optimism at its foundation had been shaken. In its place, we saw the rise of radical social movements based on a deep distrust of technology and industry. Today, progress and growth are called an “addiction”, a “fetish”, a “Ponzi scheme”, or a “fairy tale.” Some even advocate a new ideal of “degrowth”.

 It’s no wonder, then, that the last fifty years have seen relative stagnation in technological and industrial progress. Nuclear power was stunted, the Apollo program was canceled, the Concorde was grounded.

 But now, in the 21st century, some people are starting to call attention to the problem: Peter Thiel, Tyler Cowen, Patrick Collison. There’s now a growing community that recognizes the threat of stagnation and the value of progress.

 The 19th century philosophy of progress was naive. But the 20th century turn away from progress was no solution.

 We need a new philosophy of progress for the 21st century. One that teaches people not to take the modern world for granted. One that acknowledges the problems of progress, confronts them directly, and offers solutions. And one that holds up a positive vision of the future.

 To establish that new philosophy is the mission of The Roots of Progress.

 Today The Roots of Progress is transforming from a blog to a new nonprofit organization. Read the announcement »

 Comment: LessWrong, Reddit

 Social media link image credit: Wikimedia / Caroline Léna Becker

« Interview: Ingenuism with Don Watkins The Roots of Progress is now a nonprofit organization »

 Get posts by email:

 Copyright © Jason Crawford. Some rights reserved: CC BY-ND 4.0

 Get posts by email

 ╳

 Don't show again

We live in an age that has lost its optimism. Polls show that people think the world is getting worse, not better. Children fear dying from environmental catastrophe before they reach old age. Technologists are as likely to be told that they are ruining society as that they are bettering it.
But it was not always so. Just a few centuries ago, Western thinkers were caught up in a wave of optimism for technology, humanity and the future, based on the new philosophy of the Enlightenment.
The Enlightenment was many things, but in large part, it was a philosophy of progress. 
At the end of the 18th century, the Marquis de Condorcet gave expression to this philosophy and its optimism in his Sketch for a Historical Picture of the Progress of the Human Mind. In it, he predicted unlimited progress, not only in science and technology, but in morality and society. He wrote of the equality of the races and the sexes, and of peace between nations.
His optimism was all the more remarkable given that he wrote this while hiding out from the French Revolution, which was hunting him down in order to execute him as an aristocrat. Unfortunately, he could not hide forever: he was captured, and soon died in prison. Evidently, the perfection of mankind was slow in coming.
 Material progress, however, was rocketing ahead. After the end of the Napoleonic Wars in Europe, and then the Civil War in America, the path was clear for technological innovation and economic growth: the railroad, the telephone, the light bulb, the internal combustion engine.
By the end of the 19th century, it was obvious that the world had entered a new age, and progress was its watchword. The naturalist Alfred Russel Wallace (best known for his work on evolution with Darwin) titled his book about the 1800s The Wonderful Century. In it, he attributed twenty-four “great inventions and discoveries” to the 19th century, as compared with only fifteen in all of human history preceding it. The boundless optimism of the early Enlightenment seemed to have been justified.
And if the material progress prophesied by Francis Bacon could be realized, perhaps the moral progress prophesied by Condorcet would come true as well. By the end of the 19th century, slavery had been ended in the West, and some hoped that the growth of industry and the expansion of trade would lead to and end to war and a new era of world peace.
They were wrong.
The 20th century violently shattered those naive illusions. The world wars were devastating proof that material progress does not inevitably lead to moral progress. Technology had not put an end to war—in fact, it had made war all the more terrible and deadly. In 1945, the nuclear bomb put a horrible exclamation point on this lesson: the most destructive weapon ever devised was the product of modern science, technology, and industry.
At the same time, other concerns were coming to the fore—including old ones, like poverty, and new ones, like the environment. By the mid-20th century, the philosophy of progress had been dealt a severe challenge. The optimism at its foundation had been shaken. In its place, we saw the rise of radical social movements based on a deep distrust of technology and industry. Today, progress and growth are called an “addiction”, a “fetish”, a “Ponzi scheme”, or a “fairy tale.” Some even advocate a new ideal of “degrowth”.
It’s no wonder, then, that the last fifty years have seen relative stagnation in technological and industrial progress. Nuclear power was stunted, the Apollo program was canceled, the Concorde was grounded.
But now, in the 21st century, some people are starting to call attention to the problem: Peter Thiel, Tyler Cowen, Patrick Collison. There’s now a growing community that recognizes the threat of stagnation and the value of progress.
The 19th century philosophy of progress was naive. But the 20th century turn away from progress was no solution.
 We need a new philosophy of progress for the 21st century. One that teaches people not to take the modern world for granted. One that acknowledges the problems of progress, confronts them directly, and offers solutions. And one that holds up a positive vision of the future.
To establish that new philosophy is the mission of The Roots of Progress.
 Today The Roots of Progress is transforming from a blog to a new nonprofit organization. Read the announcement »  
 
 
 
 
 
 
 
 
 Comment: LessWrong, Reddit
 
 
 
 
 Social media link image credit:
 Wikimedia / Caroline Léna Becker
We need a new philosophy of progress
https://rootsofprogress.org/a-new-philosophy-of-progress
https://medium.com/@alieckaja/mastering-rails-dates-operations-2033de8bd4cb
Mastering Rails time operations - Julija Alieckaja - Medium
2021-08-06T09:54:25Z
Julija Alieckaja
<html><body><div><div><div><a href="https://medium.com/@alieckaja?source=post_page-----2033de8bd4cb--------------------------------"><div><p></p></div></a></div><p>Rails ActiveSupport time methods are slow. Let’s take <code>beginnig_of_month</code> as an example and look at the benchmarks comparing it with a pure Ruby implementation (Rails 6.1.4, Ruby 2.6.6):</p><pre><span>time = Time.now.utc<br/>n = 1_000_000</span><span>Benchmark.bm do |x|<br/> x.report do<br/> n.times do<br/> time.beginning_of_month<br/> end<br/> end<br/> x.report do<br/> n.times do <br/> Time.utc(time.year, time.month)<br/> end<br/> end<br/>end</span><span>user system total real<br/>5.256222 0.012179 5.268401 ( 5.280565)<br/>0.614967 0.004348 0.619315 ( 0.620292)</span></pre><p>The difference in performance is quite noticeable, ActiveSupport is almost 9 times slower. The same situation appears with other ActiveSupport methods, such as <code>beginnig_of_day</code>, <code>beginning_of_quarter</code> , <code>end_of_month</code> , etc. which are not so easily replaceable with quick pure Ruby versions.</p><p>So, why does ActiveSupport take so much time?<br/>First of all, the problem persists for both <a href="https://github.com/rails/rails/blob/main/activesupport/lib/active_support/core_ext/date_time/calculations.rb">DateTime::Calculations</a> and <a href="https://api.rubyonrails.org/files/activesupport/lib/active_support/core_ext/date/calculations_rb.html">Date::Calculations</a> modules, though they are affected in varying degrees.<br/>Internally ActiveSupport <a href="https://github.com/rails/rails/blob/main/activesupport/lib/active_support/core_ext/date_and_time/calculations.rb">adds a lot of complexity</a>, checks multiple options, handles timezones and conversions, and thus causes performance degradation. Yet it can be avoided by relying on native Ruby solutions where it makes sense.</p><p>I’ve run into the ActiveSupport date methods performance issue while trying to speed up a code iterating through a huge collection of entities with dates in their attributes. The entities were aggregated and modified depending on the result of <code>beginning_of_quarter</code> and <code>end_of_quarter</code>methods. And it turned out that a significant improvement could be achieved by simply using a cache. Classic.</p><p>Here goes a class wrapping the dates cache implementation. Be careful, It’ll help only in case the date range is limited, and the dates are repeated within the given loop.</p><pre><span>class MemoizedDates<br/> attr_accessor :cache<br/> MDate = Struct.new(:beginning_of_quarter, :end_of_quarter)</span><span> def initialize<br/> @cache = {}<br/> end</span><span> # handles cache misses<br/> def find(date)<br/> cache[date] || memoize(date)<br/> end</span><span> private</span><span> def memoize(date)<br/> cache[date] = MDate.new(<br/> date.beginning_of_quarter,<br/> date.end_of_quarter<br/> )<br/> end<br/>end</span><span>### sample usage</span><span>memoized_dates = MemoizedDates.new<br/>collection.each do |entry|<br/> date = memoized_dates.find(entry.date)<br/> if date.beginning_of_quarter &gt; n<br/> # do stuff<br/> end<br/>end</span><span>### benchmark</span><span>time = Time.now.utc<br/>memoized_cache = MemoizedDates.new<br/>n = 1_000_000</span><span>Benchmark.bm do |x|<br/> x.report do<br/> n.times do<br/> rand_date = time - rand(1000).days</span><span> rand_date.beginning_of_quarter<br/> rand_date.end_of_quarter<br/> end<br/> end<br/> x.report do<br/> n.times do<br/> rand_date = time - rand(1000).days<br/> mem_date = memoized_cache.find(rand_date)</span><span> mem_date.beginning_of_quarter<br/> mem_date.end_of_quarter<br/> end<br/> end<br/>end</span><span>user system total real<br/>39.541243 0.210272 39.751515 ( 39.920949)<br/>10.769937 0.065638 10.835575 ( 10.891969)</span></pre><p>Happy coding!</p></div></div></body></html>
Rails ActiveSupport time methods are slow. Let’s take beginnig_of_month as an example and look at the benchmarks comparing it with a pure Ruby implementation (Rails 6.1.4, Ruby 2.6.6):  time = Time.now.utcn = 1_000_000 Benchmark.bm do |x| x.report do n.times do time.beginning_of_month end end x.report do n.times do  Time.utc(time.year, time.month) end endend user system total real5.256222 0.012179 5.268401 ( 5.280565)0.614967 0.004348 0.619315 ( 0.620292)  The difference in performance is quite noticeable, ActiveSupport is almost 9 times slower. The same situation appears with other ActiveSupport methods, such as beginnig_of_day, beginning_of_quarter , end_of_month , etc. which are not so easily replaceable with quick pure Ruby versions. So, why does ActiveSupport take so much time?First of all, the problem persists for both DateTime::Calculations and Date::Calculations modules, though they are affected in varying degrees.Internally ActiveSupport adds a lot of complexity, checks multiple options, handles timezones and conversions, and thus causes performance degradation. Yet it can be avoided by relying on native Ruby solutions where it makes sense. I’ve run into the ActiveSupport date methods performance issue while trying to speed up a code iterating through a huge collection of entities with dates in their attributes. The entities were aggregated and modified depending on the result of beginning_of_quarter and end_of_quartermethods. And it turned out that a significant improvement could be achieved by simply using a cache. Classic. Here goes a class wrapping the dates cache implementation. Be careful, It’ll help only in case the date range is limited, and the dates are repeated within the given loop.  class MemoizedDates attr_accessor :cache MDate = Struct.new(:beginning_of_quarter, :end_of_quarter)  def initialize @cache = {} end  # handles cache misses def find(date) cache[date] || memoize(date) end  private  def memoize(date) cache[date] = MDate.new( date.beginning_of_quarter, date.end_of_quarter ) endend ### sample usage memoized_dates = MemoizedDates.newcollection.each do |entry| date = memoized_dates.find(entry.date) if date.beginning_of_quarter &gt; n # do stuff endend ### benchmark time = Time.now.utcmemoized_cache = MemoizedDates.newn = 1_000_000 Benchmark.bm do |x| x.report do n.times do rand_date = time - rand(1000).days  rand_date.beginning_of_quarter rand_date.end_of_quarter end end x.report do n.times do rand_date = time - rand(1000).days mem_date = memoized_cache.find(rand_date)  mem_date.beginning_of_quarter mem_date.end_of_quarter end endend user system total real39.541243 0.210272 39.751515 ( 39.920949)10.769937 0.065638 10.835575 ( 10.891969)  Happy coding!
Mastering Rails time operations - Julija Alieckaja - Medium
https://medium.com/@yuliaoletskaya/mastering-rails-dates-operations-2033de8bd4cb
https://www.calnewport.com/blog/2021/11/13/the-bestselling-magic-of-the-writing-shed/
The Bestselling Magic of the Writing Shed - Study Hacks - Cal Newport
2021-11-13
Study Hacks
<div><article>	
	<small>November 13th, 2021 · <a href="https://www.calnewport.com/blog/2021/11/13/the-bestselling-magic-of-the-writing-shed/#comments">26 comments</a></small>
	<p></p>
<p>A few years ago, my family faced a housing dilemma. We lived at the time in our first house, a small cape cod near the top of an elongated cul-de-sac, situated on a bluff above Sligo Creek, a half mile outside downtown Silver Spring. We had two kids who comfortably shared an upstairs bedroom. But then my wife and I decided they needed a new brother, and we soon realized that we might not actually have anywhere to put him.</p>
<p>So we started thinking through options to gain more space. At one point, I landed on what I deemed to be an ingenious plan. We would give up my home office and instead build, in the corner of our small backyard, a custom writing shed. Inspired by <a href="https://www.amazon.com/gp/product/0143114743/ref=as_li_qf_asin_il_tl?ie=UTF8&amp;tag=stuhac-20&amp;creative=9325&amp;linkCode=as2&amp;creativeASIN=0143114743&amp;linkId=8f68f6f3a0d4e9002bfb06bf6337a706">the cabin Michael Pollan built</a> in the woods outside his home in Kent, Connecticut, I began to daydream about making that short walk from our back door to a wood-paneled oasis; heated in the winter by a marine pellet stove, and cooled by tilt-open windows in the pleasant DC spring.</p>
<p>For various reasons, including the potential illegality of cramming an outbuilding of this size into our cramped yard, we ended up instead buying a new house ten minutes down the road. But the daydream of my impractical writing shed lingered.</p>
<p>Which is all a long way of establishing the importance of what happened earlier today. I was watching an old episode of a promotional show called Disney Insider with our youngest son (now three), when we were suddenly confronted with a great example of a writer who actually acted on my impulse. The segment in question focused on Eoin Colfer, the Irish author of the massively successful <a href="https://www.amazon.com/gp/product/B002KP6DXQ/ref=as_li_qf_asin_il_tl?ie=UTF8&amp;tag=stuhac-20&amp;creative=9325&amp;linkCode=as2&amp;creativeASIN=B002KP6DXQ&amp;linkId=1f46c755565664bffb2b8727ab51ddff"><em>Artemis Fowl</em></a> book series.</p>
<p>At the time of the segment’s filming, Colfer lived in a modest row house near the town center of Wexford, Ireland. After the birth of their first child, <a href="https://www.shedworking.co.uk/2011/06/eoin-colfer-shedworker.html">he converted a shed in the back of their narrow garden into an office</a>, and, by the looks of it, he did it right.  The building is clad in red stained wood, with visible slatting and a modern slanted roof. Inside there’s room only for a desk that faces wall-to-wall windows looking back at the house. Behind him is a simple shelf holding the books crafted in the space.</p>
<p>It’s actually hard to find images of the shed online. The picture at the top of the post, <a href="https://twitter.com/eoincolfer/status/1238515243439054849">extracted from Colfer’s Twitter account</a>, is one of the better examples I could find,</p>
<p>Here’s another incomplete image that showcases the shelf behind the desk:</p>
<p></p>
<p>And one that reveals some of the front windows:</p>
<p></p>
<p>The television segment we watched did a much better job of showcasing the shed’s pleasing, depth-supporting aesthetics, both inside and outside. But the above images provide a good general sense of the space.</p>
<p>Perhaps non-surprisingly, Colfer no longer writes in this modest shed. A little internet sleuthing reveals that he eventually moved to a stately manner house, situated on 14 acres in the countryside outside Wexford proper. The property included a collection of free-standing stables and garages, one of which was converted into an elegant new office clad in light American oak. Even more recently, Colfer sold this property to move to Dublin.</p>
<p>When you sell 25 million books, you have options.</p>
<p>It was nice to see, however, that the spark that initiated this wildly lucrative and impactful writing career was a small but intentional shed built at the back of a narrow garden. Perhaps my daydream wasn’t so far-fetched after all.</p>
 
	</article>||||I|||| Cal Newport

 Cal Newport Website Navigation

 * Home
 * About
 * Books
 * Media
 * Podcast
 * Blog
 * Contact

 Study Hacks Blog

 The Bestselling Magic of the Writing Shed

 November 13th, 2021 · 26 comments

 A few years ago, my family faced a housing dilemma. We lived at the time in our first house, a small cape cod near the top of an elongated cul-de-sac, situated on a bluff above Sligo Creek, a half mile outside downtown Silver Spring. We had two kids who comfortably shared an upstairs bedroom. But then my wife and I decided they needed a new brother, and we soon realized that we might not actually have anywhere to put him.

 So we started thinking through options to gain more space. At one point, I landed on what I deemed to be an ingenious plan. We would give up my home office and instead build, in the corner of our small backyard, a custom writing shed. Inspired by the cabin Michael Pollan built in the woods outside his home in Kent, Connecticut, I began to daydream about making that short walk from our back door to a wood-paneled oasis; heated in the winter by a marine pellet stove, and cooled by tilt-open windows in the pleasant DC spring.

 For various reasons, including the potential illegality of cramming an outbuilding of this size into our cramped yard, we ended up instead buying a new house ten minutes down the road. But the daydream of my impractical writing shed lingered.

 Which is all a long way of establishing the importance of what happened earlier today. I was watching an old episode of a promotional show called Disney Insider with our youngest son (now three), when we were suddenly confronted with a great example of a writer who actually acted on my impulse. The segment in question focused on Eoin Colfer, the Irish author of the massively successful Artemis Fowl book series.

 At the time of the segment’s filming, Colfer lived in a modest row house near the town center of Wexford, Ireland. After the birth of their first child, he converted a shed in the back of their narrow garden into an office, and, by the looks of it, he did it right. The building is clad in red stained wood, with visible slatting and a modern slanted roof. Inside there’s room only for a desk that faces wall-to-wall windows looking back at the house. Behind him is a simple shelf holding the books crafted in the space.

 It’s actually hard to find images of the shed online. The picture at the top of the post, extracted from Colfer’s Twitter account, is one of the better examples I could find,

 Here’s another incomplete image that showcases the shelf behind the desk:

 And one that reveals some of the front windows:

 The television segment we watched did a much better job of showcasing the shed’s pleasing, depth-supporting aesthetics, both inside and outside. But the above images provide a good general sense of the space.

 Perhaps non-surprisingly, Colfer no longer writes in this modest shed. A little internet sleuthing reveals that he eventually moved to a stately manner house, situated on 14 acres in the countryside outside Wexford proper. The property included a collection of free-standing stables and garages, one of which was converted into an elegant new office clad in light American oak. Even more recently, Colfer sold this property to move to Dublin.

 When you sell 25 million books, you have options.

 It was nice to see, however, that the spark that initiated this wildly lucrative and impactful writing career was a small but intentional shed built at the back of a narrow garden. Perhaps my daydream wasn’t so far-fetched after all.

 26 thoughts on “ The Bestselling Magic of the Writing Shed ”

 1. Alexander Lewis says:
 November 13, 2021 at 4:34 pm

 What a beautiful little writing cove. Maybe I’ll invest in one of those someday.

 My wife and I recently moved and I used the occasion to reimagine my office. I bought a second desk and now use one for all my computer-based work. The other is used exclusively for writing freehand or reading books. My one rule for the second desk is that I’m not allowed to bring my phone or computer to it. It’s a technology-free writing zone. The new setup is working out great so far.

 Keep sharing these author office setups. They’re great inspiration!

 Reply
 2. George says:
 November 13, 2021 at 5:03 pm

 Fah! Hipster designer sheds are just an excuse to put off writing. One of the most productive nonfiction writers I’ve known worked for trade magazines – he spewed forth a prodigious output from a real shed – a Home Depot 10′ x 12′ tin garden shed where he worked year-round, teaching in his spare time. I write in a 5′ x 10′ cubby on the path to our bathroom: five nonfiction books published in the last six years plus innumerable web posts and articles. At 79, writing keeps me young and happy.

 Reply
 1. Sameer says:
 November 14, 2021 at 11:16 am

 What a respectful remark that denigrates the choice(s) of others. Done in a shed or done in a sewer, done in a skyscraper or done in a slum, a great book is a great book, if and when it is done and published. One can wonder if you feel young and happy by trampling on would-be-writers with your high horse.

 Reply
 3. Andrea says:
 November 13, 2021 at 10:45 pm

 I love that you embrace your romantic, intuitive, inspired side, sensitive to environments, as well as your logical reasonable disciplined nature. It’s a fallacy to think they cannot coexist in the same person.

 Reply
 4. Pingback: The Bestselling Magic of the Writing Shed – Study Hacks – Patricia Snodgrass
 5. Jason Chatfield says:
 November 14, 2021 at 4:24 pm

 If you love that, you’ll be enamoured of Roald Dahl’s writing cabin out back. After he sharpened his pencils, poured tea from his thermos, and settled in his writing chair with a lap desk, he would create some of the most imaginative books ever written. He was all about rituals. Look up Roald Dahl Pebble Mill on YouTube.

 Reply
 6. Alex Johnson says:
 November 15, 2021 at 10:01 am

 Nice piece. Thanks for the link back to my site Shedworking.

 Reply
 7. Steve Carol says:
 November 15, 2021 at 12:08 pm

 Although I am not a writer myself I have always been attracted to the private study/office a short walk from the madding crowds of domestic life. The book-lined offices of mansions and Mark Twain’s study in Elmira, NY come to mind. see:

 https://www.atlasobscura.com/places/mark-twain-s-study

 Reply
 8. Caroline Starr Rose says:
 November 15, 2021 at 3:55 pm

 A dream! My first “office” was a 3 x 4 closet (I’m not kidding). I had a outlet put in and wrote my first (published) novel with my laptop balanced on my knees. My printer was on the shelf above, and I had to stand on my chair to use it. That novel was a historical about a girl trapped in a blizzard in a Kansas soddy, which looking back makes the tight quarters about perfect for that writing experience.

 Reply
 9. Melissa says:
 November 16, 2021 at 2:08 am

 I need a cove of my own. Until I read this blog post, I didn’t realise it, but know I can’t stop thinking about it. I will not stop until I have achieved my new dream!

 Reply
 10. Melissa says:
 November 16, 2021 at 2:09 am

 typo: now
 I am so embarrassed.
 Sorry.

 Reply
 11. Divyanshu Bora says:
 November 17, 2021 at 8:22 am

 Hello, Sir. It’s Divyanshu Bora here from Assam, India. Currently, I’m a student of 12th standard, Science stream, who’s been highly inspired from your book titled “Deep Work”. First of all, I’d like to thank you for paving a path to understand the real meaning of “deep work” for a seventeen years old boy like me and letting him to quit one of the toxic stuffs that has been ruining humanity for a long time i.e. social media. To be very frank sir, I’ve come to know your TED Talk which has completely blown my mind by your different way of analyzing the real meaning of “deep work” and letting me to go through your books as well. Sir, right from my childhood I really have been passionate to pursue higher studies in Cosmology. How can I contact you sir? Is there any way to directly contact you? I’d be honored to receive guidance from a person like you sir.

 Yours sincerely
 Divyanshu Bora
 Assam, India

 Reply
 1. Randy Martens says:
 November 23, 2021 at 6:09 pm

 Divyanshu, I can only wish that I had learned the lesson in my earlier years that you have taken to heart, and at such a young age. Honestly, I see a very productive and meaningful future ahead for you. All the best to you, my friend!

 Reply
 2. Nitin says:
 December 3, 2021 at 5:07 am

 Divyanshu, glad to see you having stumbled upon the kind of ideas and mindset that Cal writes about. Stick to these ideas and they’ll be immensely rewarding.

 That said, you don’t need to “contact” Cal or any other mentors personally. Just follow their work, writings, videos, podcasts etc and digest the message. Society will benefit more by Cal putting content out in the public than trying to guide a handful of individuals privately. Since the impact in the former is bigger than in the latter, the mentors you’re seeking won’t have time to entertain that anyway.

 Wish you the best!

 Reply
 12. Sébastien Tarnowski says:
 November 17, 2021 at 10:29 am

 The kind of cove we would all like to own, not only for deep work but also for reading, relaxing… Well basically for everything that requires focus without any distraction…
 Thank you Cal for this inspiring article and it’s true, DC is amazing in spring.

 Reply
 13. James says:
 November 17, 2021 at 1:15 pm

 This seems to me to be re-inventing the wheel to a large extent. In most houses where I grew up there was a study–a place where the adults would go to either get work done or engage in activities that the kids would interfere with (crafts, or writing, or reading, or the like). This concept was corrupted into the “man-cave” of the perpetual teenagers that dominate society–a place where they could allow themselves to behave like children. I suppose the fact that a writing shed is detached allows us to dissociate it from the odious “man-cave” concept, but ultimately it’s a return to the idea of the study.

 It’s not necessarily a bad thing–in fact I would argue it’s a good thing, a sign that we’re maturing as a society–I just don’t get why we’re adding a bunch of cultural baggage to it. Why not call it a study?

 Reply
 14. Ryan says:
 November 18, 2021 at 5:40 pm

 I also have this dream. John Nelson, a cartographer I follow, has a rather rustic workspace in his backyard in Michigan that I envy: https://adventuresinmapping.com/2020/03/20/working-well-remotely/

 Reply
 15. Matt Vallance says:
 November 21, 2021 at 11:00 am

 Good piece, however, I cannot repress my inner sub-editor.

 Mr Colfer does not live in: “a stately manner house,” he lives in “a manor house.” Do get it right please.

 Reply
 16. Sharon Villines says:
 November 23, 2021 at 5:25 pm

 If you Google “Tiny House” you will find all manner of small houses and sheds that are prefabricated or you can buy plans. The TinyLife website has all the tips you need for saving cost, adding convenience, avoiding risks. One tip: a shed is defined differently from a residential living space. If you follow the guidelines you can avoid the codes for living spaces. And houses on wheels are considered temporary in some place and require no permits at all.
 https://affordablecohousing.com/glossary/the-tiny-life-practical-tools-for-simple-living/

 Reply
 17. Randy Martens says:
 November 23, 2021 at 6:03 pm

 In the overall spirit of withdrawing into quiet retreat for deep creative work, the best little item I have found that provides instant peace and serenity is a pair of high-quality ear protectors. Actually, I have THREE pair of them in various locations around my home environment, and they are among my most treasured of physical possessions. And that’s no exaggeration. Here’s the brand I use, $30 out on Amazon, if you cherish calm quietude. They are extremely effective, and far more comfortable than they appear:

 https://www.amazon.com/dp/B00CPCHBCQ?psc=1&ref=ppx_yo2_dt_b_product_details

 Reply
 18. Brett says:
 November 24, 2021 at 1:03 am

 One of the greatest and most prolific writers of all time, George Bernard Shaw, enhanced his productivity with his famed writing shed, which could rotate to catch heat and light (what we’d now call passive solar heating, only steps away from the home he shared with his wife, Charlotte. It provided the solitude he needed to generate so many books, plays and articles. https://www.idler.co.uk/article/inside-bernard-shaws-writing-shed/

 Reply
 19. Brett says:
 November 24, 2021 at 1:11 am

 An earlier Irish author, George Bernard Shaw, enhanced his productivity with a celebrated writing shed that he could rotate to catch heat and sunlight throughout the day — what we’d now call passive solar heating. Only steps away from the rural home he shared with his wife Charlotte, the hut provided the astonishingly prolific, Nobel Prize winning celebrity writer the solitude he needed to generate thousands of popular and influential books, plays and articles. https://www.idler.co.uk/article/inside-bernard-shaws-writing-shed/
 (Apologies if this double posts; I tried to submit something similar last week but it apparently vanished into the Ethernet.)

 Reply
 20. Nikola Bošnjak says:
 November 28, 2021 at 10:40 pm

 It’s not just writers, there are also programmers who are working from sheds, like Wes Bos: https://www.youtube.com/watch?v=HPOq3QJz_7s&t=213s

 Reply
 21. homeschooling for kids says:
 December 15, 2021 at 8:15 am

 It is fun when it comes to hands-on learning. If you are a first-year homeschooler you can start with creating the baking soda. Then you can teach your children how to use it in making a cake and what are its benefits. Or you can make a vinegar volcano. Boys usually love this activity and would love to do it more as compared to girls. But if your kids are middle and high school students then there is a little chaos that may arise.
 The children in middle and high school rarely have any activity in their books and are more concentrating on the text without any scientific research or even an experiment. But to keep them on track I researched on google to check what a mother can do to teach kids hands-on learning at this age. So I am going to share some tips in this article which will help you in a promising way.

 Reply
 22. Sanjid says:
 May 15, 2022 at 10:19 am

 Elegant stuff. I, too, as a (learning) writer daydream of a space as such. It’s can’t be that far-fetched, for sheds are truly magical.

 Reply
 23. Pingback: How to create a productive working environment when on a budget – Gio Lodi

 Leave a Reply

 Your email address will not be published. Required fields are marked *

 Comment *

 Name *

 Email *

 Website

 you MUST enable javascript to be able to comment Δ

 About Cal Newport

 I'm a computer science professor who writes about the intersection of digital technology and culture. I’m particularly interested in our struggles to deploy these tools in ways that support instead of subvert the things we care about. If you’re new to my writing, a good place to start is with my latest book, A World Without Email: Reimagining Work in an Age of Communication Overload, or my about page. You can also access over a decade's worth of posts in the blog archive.

 My Web Host

 My podcast

 MY TOP-RATED PODCAST WHERE I ANSWER QUESTIONS FROM MY READERS ABOUT WORK, TECHNOLOGY,
 AND THE DEEP LIFE.

 My Latest


 My Other Books


 Get the Latest from the Study Hacks Blog in your inbox:

 You'll receive the blog posts via email. Your email address is never sold or shared.

 My Online Courses

 A course by Cal Newport and Scott Young

 LEARN MORE

 A course by Cal Newport and Scott Young

 LEARN MORE

 Note: This site is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

 © 2023 Cal Newport •

 * Home
 * About
 * Books
 * Media
 * Podcast
 * Blog
 * Contact
 site design: Juxta prose
 Manage Cookie Consent
 We use cookies to optimize our website and our service.
 Functional cookies Functional cookies Always active
 The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
 Preferences Preferences
 The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
 Statistics Statistics
 The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
 Marketing Marketing
 The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
 Manage options Manage services Manage vendors Read more about these purposes
 Accept Functional only View preferences Save preferences View preferences
 {title} {title} {title}
 Settings
November 13th, 2021 · 26 comments 
	 
A few years ago, my family faced a housing dilemma. We lived at the time in our first house, a small cape cod near the top of an elongated cul-de-sac, situated on a bluff above Sligo Creek, a half mile outside downtown Silver Spring. We had two kids who comfortably shared an upstairs bedroom. But then my wife and I decided they needed a new brother, and we soon realized that we might not actually have anywhere to put him.
So we started thinking through options to gain more space. At one point, I landed on what I deemed to be an ingenious plan. We would give up my home office and instead build, in the corner of our small backyard, a custom writing shed. Inspired by the cabin Michael Pollan built in the woods outside his home in Kent, Connecticut, I began to daydream about making that short walk from our back door to a wood-paneled oasis; heated in the winter by a marine pellet stove, and cooled by tilt-open windows in the pleasant DC spring.
For various reasons, including the potential illegality of cramming an outbuilding of this size into our cramped yard, we ended up instead buying a new house ten minutes down the road. But the daydream of my impractical writing shed lingered.
Which is all a long way of establishing the importance of what happened earlier today. I was watching an old episode of a promotional show called Disney Insider with our youngest son (now three), when we were suddenly confronted with a great example of a writer who actually acted on my impulse. The segment in question focused on Eoin Colfer, the Irish author of the massively successful  Artemis Fowl  book series.
At the time of the segment’s filming, Colfer lived in a modest row house near the town center of Wexford, Ireland. After the birth of their first child, he converted a shed in the back of their narrow garden into an office, and, by the looks of it, he did it right.  The building is clad in red stained wood, with visible slatting and a modern slanted roof. Inside there’s room only for a desk that faces wall-to-wall windows looking back at the house. Behind him is a simple shelf holding the books crafted in the space.
It’s actually hard to find images of the shed online. The picture at the top of the post, extracted from Colfer’s Twitter account, is one of the better examples I could find,
Here’s another incomplete image that showcases the shelf behind the desk:
 
And one that reveals some of the front windows:
 
The television segment we watched did a much better job of showcasing the shed’s pleasing, depth-supporting aesthetics, both inside and outside. But the above images provide a good general sense of the space.
Perhaps non-surprisingly, Colfer no longer writes in this modest shed. A little internet sleuthing reveals that he eventually moved to a stately manner house, situated on 14 acres in the countryside outside Wexford proper. The property included a collection of free-standing stables and garages, one of which was converted into an elegant new office clad in light American oak. Even more recently, Colfer sold this property to move to Dublin.
When you sell 25 million books, you have options.
It was nice to see, however, that the spark that initiated this wildly lucrative and impactful writing career was a small but intentional shed built at the back of a narrow garden. Perhaps my daydream wasn’t so far-fetched after all.
The Bestselling Magic of the Writing Shed - Study Hacks - Cal Newport
https://www.calnewport.com/blog/2021/11/13/the-bestselling-magic-of-the-writing-shed/