[User] Please transcribe this text

[Assistant] And he meant a lot of different things by this. Part of it is the best you can ever do with any kind of representation is make a, make a kind of a map. Even when you're trying to make a map of it's, of a map.


[User] Please transcribe this text

[Assistant] You're making a lie compared to the thing you're trying to represent. But if you do it right, people can...


[User] Please transcribe this text

[Assistant] Gain some intuition about what the map is about. That's what science is all about.


[User] Please transcribe this text

[Assistant] This talk, uh, in 50 minutes has to be a lie.


[User] Please transcribe this text

[Assistant] It really is, I'm leaving out a lot of important things and, but,


[User] Please transcribe this text

[Assistant] I think the shadow of the talk or whatever is projected by the talk is pretty close to the way things actually are for doing this very different process.


[User] Please transcribe this text

[Assistant] A thing that Picasso didn't say but he knew and he meant is art is also the lie that tells the truth that wakes you up.


[User] Please transcribe this text

[Assistant] So if something wakes you up in the next two days, uh, I will have done my job.


[User] Please transcribe this text

[Assistant] The two days are really about another Picasso quote, which is, learn the rules like a pro so you can break them like an artist.


[User] Please transcribe this text

[Assistant] And not doing this is probably the greatest sin of Silicon Valley since the 80s.


[User] Please transcribe this text

[Assistant] Almost never, not, nobody has bothered becoming a pro, particularly in software and user interface design.


[User] Please transcribe this text

[Assistant] And so when they break the rules, they're breaking them like a dumb child does, by throwing rocks through windows. And it's probably the most sickening thing to see about the field. So...


[User] Please transcribe this text

[Assistant] Another way of thinking about the mantra is you have to learn everything and then find a way of forgetting it so you can have your own ideas, but what you...


[User] Please transcribe this text

[Assistant] Forget is everything except the perfume.


[User] Please transcribe this text

[Assistant] So when you have an idea, then your nose will pick up the right scent and you'll be able to make use of all the stuff that you've learned after you've had the idea.


[User] Please transcribe this text

[Assistant] I was specifically asked to talk about Xerox PARC because that was an example of making trillions instead of billions.


[User] Please transcribe this text

[Assistant] So I thought he- what I'll do here is just show you a couple of results, uh, from Xerox PARC. And I'll-


[User] Please transcribe this text

[Assistant] People, so this machine happened in 1973, which is 11 years before the Mac, and its screen was more than twice the size of the Mac, and it was more like a Mac of 1988 or 1989. So it was about


[User] Please transcribe this text

[Assistant] Maybe 15 or 16 years ahead of the commercial development and the commercial development actually


[User] Please transcribe this text

[Assistant] Was based on the stuff that was done on this machine. So this is an example of the best way to predict the future is to invent it.


[User] Please transcribe this text

[Assistant] There was nothing like this before. Once we did it, people could see, oh yeaah.


[User] Please transcribe this text

[Assistant] Uh, there could be something like this because here it is. In fact, we made 2,000 of these.


[User] Please transcribe this text

[Assistant] So there's a lot to look at. They had bitmap screens, they had pointing device, the famous GUI, which is, you're still using today.


[User] Please transcribe this text

[Assistant] Uh, wizzy wig means what you see is what you get.


[User] Please transcribe this text

[Assistant] Uhm, desktop publishing and the whole media gig. Uhm, what does that say?


[User] Please transcribe this text

[Assistant] What you almost never get on the web, but you did get in many of the apps of the 1980s, which is when you get a document, uh,


[User] Please transcribe this text

[Assistant] Much more made for consumption than it is made for authoring. They're completely decoupled and most of the authoring facilities are typing in tiny little windows and then pushing a button to see what you did. So this is a vast retrogression.


[User] Please transcribe this text

[Assistant] Uh, what we call real loop these days. We just called it object oriented programming back then. I made up that term.


[User] Please transcribe this text

[Assistant] But what's called object oriented programming today is not what we had back then. And I don't think I'll have time to really talk about the profound differences. Suffice to say that what we did at PARC became popular and everybody wanted it even if they only got the label.


[User] Please transcribe this text

[Assistant] Comparison is you can go out and buy a set of designer jeans with the label Harvard on them. And for all I know it was Stanford on them. Uh, laser printer.


[User] Please transcribe this text

[Assistant] Main difference between this and what you have today is it was, the first one was a page a second. So most people have never printed with a page a second printer.


[User] Please transcribe this text

[Assistant] Postscript, outline fonts, and so forth. Ethernet.


[User] Please transcribe this text

[Assistant] Peer, peer and client server and 50% of the, uh, the internet. Park did an internet before there was the internet.


[User] Please transcribe this text

[Assistant] And we're part of that community, so we participated in the official internet.


[User] Please transcribe this text

[Assistant] So we think of that as eight and a half inventions. How long did it take? Well, 25 researchers for five years. Think about that. Two dozen people did all of these things in about five years.


[User] Please transcribe this text

[Assistant] Costs about 10 million to 12 million bucks a year in today's money.


[User] Please transcribe this text

[Assistant] Return is about 35 trillion. This is an old number. It's probably more like 40.


[User] Please transcribe this text

[Assistant] Now, that's a pretty good return on investment, if you think about it. You like to do little spreadsheets and stuff. 15% is good, right Sam? Yeah. So.


[User] Please transcribe this text

[Assistant] And it was an industry rather than an increment. Uh, people say, well but Xerox didn't


[User] Please transcribe this text

[Assistant] Benefit from this. They didn't make any money for it. That's complete bullshit. It's made up by companies that don't want to invest in research.


[User] Please transcribe this text

[Assistant] Xerox made about a factor of 250 over their entire investment in Park. So that's 25,000% return right there.


[User] Please transcribe this text

[Assistant] Xerox's bug was they didn't understand the rest of the stuff, but they certainly understood what a laser printer was and they made billions from it.


[User] Please transcribe this text

[Assistant] And in fact, if you look at it, it was better for the world that they stonewalled us. Because no single company can handle an entire industry.


[User] Please transcribe this text

[Assistant] So this made it possible. The Japanese had to do printers slightly differently than that. And this, uh, kind of heaven on earth,


[User] Please transcribe this text

[Assistant] Well, it lasted for about 12 years. Uh, Xerox finally, uh,


[User] Please transcribe this text

[Assistant] People who are doing something different because it makes middle management and upper management extremely uncomfortable.


[User] Please transcribe this text

[Assistant] The real job of upper management in the 20 and 21st century is to learn things. Because change is the constant.


[User] Please transcribe this text

[Assistant] thing that's going on what they try to do is to maximize whatever they had when they got.


[User] Please transcribe this text

[Assistant] into the job and whatever the company was successful at. Think about it.


[User] Please transcribe this text

[Assistant] If large companies were actually rational, there'd never be a small company. There'd be no startups.


[User] Please transcribe this text

[Assistant] Because large companies have vastly more resources for doing new things than any venture capitalist. Right, Sam? You've got gazillions of money.


[User] Please transcribe this text

[Assistant] But because they refuse to get into new businesses.


[User] Please transcribe this text

[Assistant] And refuse to change their old businesses, you guys have a chance. So just pray that they don't ever wake up.


[User] Please transcribe this text

[Assistant] Now the interesting thing is if you look at that collection of inventions, you see every single one of them today. And what's interesting,


[User] Please transcribe this text

[Assistant] Is if I were to enlarge this a bit, I would show that there's hardly been anything interesting invented since this funding stopped. So it's been a nu- basically nuclear winter of, uh,


[User] Please transcribe this text

[Assistant] People cashing in on these inventions and extracting the wealth from them, but hardly any efforts of any interesting kind to go beyond them.


[User] Please transcribe this text

[Assistant] Uh, and the last point here is the lack of curiosity. Sam is a huge, I met Sam because he wanted to, he wanted to know where'd all this stuff come from. I've hardly ever been asked that question. It's the most successful generation of wealth in computing history and almost nobody wants to know. Sam did.


[User] Please transcribe this text

[Assistant] And the difficult thing about today and Thursday is the world that you grew up in. Because I don't see anybody. Is anybody here older than 35?


[User] Please transcribe this text

[Assistant] Okay, yeaah, I see a little, little more reflectivity back there.


[User] Please transcribe this text

[Assistant] Many, many things were qualitatively different in ways that are sometimes difficult to explain.


[User] Please transcribe this text

[Assistant] So here's the book to read. This is why lectures suck.


[User] Please transcribe this text

[Assistant] This is a 500 page book done very carefully about the whole story of ARPA and Xerox PARC. Anybody who's really interested, read this book. This is a tribute I wrote to this whole research community about, I don't know, 14, 15 years ago.


[User] Please transcribe this text

[Assistant] And I'm gonna, I'll hand it out at the end of the class. And the main thing that's interesting about it perhaps is the bibliography, which has a lot of references on more detailed, uh, things you can read about how this particular community operated.


[User] Please transcribe this text

[Assistant] Right? And the important thing about Xerox PARC, which is not emphasized enough, is that PARC was just another one of the ARPA research projects.


[User] Please transcribe this text

[Assistant] There was eight years of research before PARC happened.


[User] Please transcribe this text

[Assistant] And Parker only happened because of the Vietnam War. Nobody wanted to, uh, try to do this stuff inside of a company. The company quarterly cycles and everything else are really antithetical to long-range thinking.


[User] Please transcribe this text

[Assistant] So this stuff was all funded by Cold War funds in the public domain. So none of, none of the IP was kept secret. But it was done in a, in a better rhythm. And so to get the 35 trillion, it actually required not five years, but 12 to 14 years. We were the lucky researchers who got our PhDs in this process and were the right age to go to park and finish it off.


[User] Please transcribe this text

[Assistant] Right, and as we'll see, I'm not going to dwell on it much more, but the zeitgeist of this way of doing things goes back to the rate, especially the radar effort at MIT.


[User] Please transcribe this text

[Assistant] Then the air defense effort and then ARPA and then PARC.


[User] Please transcribe this text

[Assistant] So there's a long continuity here of kind of how do you work on things that are


[User] Please transcribe this text

[Assistant] But where you have to invent several generations of technology to get to the thing that's doable.


[User] Please transcribe this text

[Assistant] So most of these things are not doable with the technology that's lying around. That is the uh, the problem with these hard problems. Okay, so the general world, the normal world, the present,


[User] Please transcribe this text

[Assistant] For most of this period was either punch card accounting machines or their replacement. After IBM said famously they'll no, there's no.


[User] Please transcribe this text

[Assistant] To replace their punch card machines before other computer companies did. So they did a sort of a Trump-like reversal on their belief and wound up owning the 60s and much of the 70s. And


[User] Please transcribe this text

[Assistant] Most important thing, and this is a dumb sentence because


[User] Please transcribe this text

[Assistant] You don't know what these machines were like. So I realized after I put it in there that, why did I even put this sentence in? Right? But it, the way I look at what's going on at any given time.


[User] Please transcribe this text

[Assistant] Whether it's today, 30 or 40 years ago, is whatever's going on right now is just crap. By definition. If we know about it, uh,


[User] Please transcribe this text

[Assistant] Except in the one tenth of one percent or one thousandth of one percent. It's gotten mundane. And part of it is just because of the bell curve.


[User] Please transcribe this text

[Assistant] Of normality in humans. Whatever it is, it gets converted to something like normal. No matter how exciting it is. And so when things are widely successful, they tend to bring up the, the, the mean of the bell curve a little bit. For everybody, raises all boats. But in fact, there's also this regression to the mean on almost all of these things. So living in the present, man, you're out of, you're just out of it.


[User] Please transcribe this text

[Assistant] Think about things from what we have now. Now the, there are two processes. There's the bomb project.


[User] Please transcribe this text

[Assistant] And the radar project at MIT that had roughly the same process.


[User] Please transcribe this text

[Assistant] Roughly the same difficulty. You have to realize the U.S. was in World War II for only about two and a half years. Pearl Harbor was, uh,


[User] Please transcribe this text

[Assistant] Most interesting thing is for the first time in history really, a lot of really good scientists and a lot of really good engineers started working together and cloning each other.


[User] Please transcribe this text

[Assistant] So seven Nobel Prizes came out of the Building 20 thing because they were physicists who put on their engineering hats to make 185 different kinds of radar systems and install them in every size of building and plane and


[User] Please transcribe this text

[Assistant] And World War II was basically a war of supply and it was the ability to, uh, stave off the German submarines that actually won the war for us. Most people don't think of it that way, but that's what happened.


[User] Please transcribe this text

[Assistant] So, and the zeitgeist I'm talking about came out of this group and there's a couple of good books about how these people went about doing things. Maybe the number one thing is what I have on the right hand side is basically they said, forget about your egos, and I really meant it.


[User] Please transcribe this text

[Assistant] Doesn't matter who you are, how smart you are, how smart you think you are.


[User] Please transcribe this text

[Assistant] This, there's only one thing that counts here is making progress and we make progress through synergy. And they learned how to do this and they passed it on generation after generation.


[User] Please transcribe this text

[Assistant] Well the next round of this after World War II was the Cold War and the air defense system that was done in the 50s and again at MIT the first displays


[User] Please transcribe this text

[Assistant] Uh, that you could interact with, that was, that is,


[User] Please transcribe this text

[Assistant] Essentially a stylus in the guy's hand. It was called the light gun.


[User] Please transcribe this text

[Assistant] So he's pointing at something on the screen and squeezing the trigger like you would put a stylus down and push it down and the light gun can tell, uh,


[User] Please transcribe this text

[Assistant] Computer can tell what the light gun is looking at and so you can do all of the stuff that you're used to today. To give you an idea of what these guys did.


[User] Please transcribe this text

[Assistant] You can't really see what this is, right? It's just like, it's obviously four floors of a building. Uh, this is a football field.


[User] Please transcribe this text

[Assistant] And if you notice the second floor there, it says computer A and computer B.


[User] Please transcribe this text

[Assistant] Uhm, each one of those computers had 50,000 vacuum tubes.


[User] Please transcribe this text

[Assistant] Both running the same programs at the same time and many other interesting things I don't have time to talk about.


[User] Please transcribe this text

[Assistant] I'll just say that when one of these computers started crashing, it took it three or four days to crash.


[User] Please transcribe this text

[Assistant] And the reason is it was running diagnostics on itself and every time an instruction failed it would patch in a simulated instruction based on what instructions were still working. So what would happen is the machine would just get slower and slower and more software like, more like a Turing machine.


[User] Please transcribe this text

[Assistant] Usually they could fix it before it came all the way down. Meanwhile, the other one was still working. Uhm, but even though I shouldn't be digressing, there's a fun thing here. There are 32 of these concrete bunkers made.


[User] Please transcribe this text

[Assistant] Uhm, so the two interesting things, what happened to this stuff?


[User] Please transcribe this text

[Assistant] Anybody knows? Does this look like anything you've ever seen before?


[User] Please transcribe this text

[Assistant] Yeah, yeah, this actually was the invention of our, uh, air traffic control system. That's what it was designed to do, except it was designed to control traffic of both American and Russian bombers. And, in fact, if,


[User] Please transcribe this text

[Assistant] Many of the displays before they replaced them with flat screen displays were basically the same big round.


[User] Please transcribe this text

[Assistant] Things, uh, so this whole system lasted until 1982, when the last one was finally decommissioned, and 50,000 vacuum tubes? They're like incandescent light bulbs. They blow out, so they're always blowing out, so of course there are redundant ones there. Where do we get the vacuum tubes from?


[User] Please transcribe this text

[Assistant] Huh? No, no, this is vacuum, no transistors in these machines. No transistors, these are the...


[User] Please transcribe this text

[Assistant] But we kept it going and there were reasons why it was kept going.


[User] Please transcribe this text

[Assistant] We bought, while we're still contending in the Russian for the cold war, we're buying vacuum tubes from them, too.


[User] Please transcribe this text

[Assistant] By the way, people still keep buying backing tubes from the Russians for getting old 50s guitar amp sounds.


[User] Please transcribe this text

[Assistant] Those overdriven sounds. And this guy who...


[User] Please transcribe this text

[Assistant] Was one of the inventors of artificial intelligence and the inventor of the programming language Lisp, John McCarthy.


[User] Please transcribe this text

[Assistant] Looked at one of these in the 50s and said, everybody's gonna have one in their home someday.


[User] Please transcribe this text

[Assistant] Because... He didn't give a shit about...


[User] Please transcribe this text

[Assistant] The concrete bunkers and anything else like that. Because what he thought was, yeaah, this is like, we get, this is like an electric power generating station.


[User] Please transcribe this text

[Assistant] Nobody ever sees them, but they're out there. We got wires going into the...


[User] Please transcribe this text

[Assistant] Home for, it's like where we get our water from, it's where we get our gas from.


[User] Please transcribe this text

[Assistant] Be an information utility and it will actually be a human right, like the telephone, to have one of these things.


[User] Please transcribe this text

[Assistant] Uh, I'm gonna show you a system done on one of these supercomputers. This is done on the test computer at Lincoln Labs for this whole Sage system. So, it's, this computer was close to the size of this entire building.


[User] Please transcribe this text

[Assistant] With one guy on it at three o'clock in the morning. So take a look at this.


[User] Please transcribe this text

[Assistant] So it doesn't even really have a display. It's actually simulating a computer display here. This is just an oscilloscope.


[User] Please transcribe this text

[Assistant] So what Ivan wants to do is to draw a flange and so he says, okay, it's kind of like this.


[User] Please transcribe this text

[Assistant] Now take these guys and make them all mutually perpendicular and wow, Sketchpad just solved that problem. So it's a dynamic problem solver in there. First window that clipped. Now he wants to put a hole in the flange.


[User] Please transcribe this text

[Assistant] So these are guidelines and the first thing he does is said, okay, I want to make these parallel.


[User] Please transcribe this text

[Assistant] And now he's using them as guidelines to draw dashed lines.


[User] Please transcribe this text

[Assistant] Okay, and now he makes the guidelines invisible. They're still there.


[User] Please transcribe this text

[Assistant] And he wants some rivets to go in, go along with that flange and so again he


[User] Please transcribe this text

[Assistant] This is why this system is called Sketchpad, because you just casually draw. It's gonna use that as the center for the arc here.


[User] Please transcribe this text

[Assistant] And again, he's gonna say, take these guys and make them mutually perpendicular, and here Sketchpad solves that problem. So you wind up with a symmetric object.


[User] Please transcribe this text

[Assistant] And he'll get another solution. He could have constrained.


[User] Please transcribe this text

[Assistant] The side lengths to be uh, ratios of each other.


[User] Please transcribe this text

[Assistant] And the kind of problem solving this system could do were included non-linear problems. So there he's got a rivet. And what's cool about this, that is a master rivet. What we call a class in object oriented programming.


[User] Please transcribe this text

[Assistant] So this is not that rivet, but an instance of that master rivet.


[User] Please transcribe this text

[Assistant] You might wonder, why is the display bouncing around like that? Well, it's putting up every dot individually. And about half the power of this super computer is being used to just do that. So here's another instance of the rivet. Here's another one. Here's another one.


[User] Please transcribe this text

[Assistant] Make the crossbars invisible. They're still there but invisible. And we see the instances. So this is object oriented programming. Maybe nicer than you've seen it.


[User] Please transcribe this text

[Assistant] That construction that he made, he's made it into a master, and so he can make instances of it.


[User] Please transcribe this text

[Assistant] Yeah, we'll, we'll pick that, go along. It's, it's sort of the larger question that we have. It's worthwhile bringing up the question who's rotating three of them.


[User] Please transcribe this text

[Assistant] So this is the most shocking thing I'd ever seen when I went to graduate school in 66. This system was only three years.


[User] Please transcribe this text

[Assistant] Old at that time and I've been programming for five years and just conventional programming and


[User] Please transcribe this text

[Assistant] Seeing this, it blew my mind because it was exactly different than everything I thought about computing. As soon as I saw it, I realized, oh yea, of course you can do that. But I didn't think about that. I even did.


[User] Please transcribe this text

[Assistant] In a way we recognize today for the first time objects, masters, and instances.


[User] Please transcribe this text

[Assistant] Uh, the programming was not done by the kind of programming today, but by problem solving, which I think you can imagine is much nicer. Like, it will find a solution.


[User] Please transcribe this text

[Assistant] Ivan, you, you did all of this in one, this was his PhD thesis.


[User] Please transcribe this text

[Assistant] He said, you did all this in one year by yourself. How could you possibly do it? And he said, well, I didn't know it was hard.


[User] Please transcribe this text

[Assistant] Of absolute goodness still today. It's not just relatively good for, for being done more than 50 years, it's absolutely good.


[User] Please transcribe this text

[Assistant] And his thesis is, every other page is an apology because it doesn't do more. Because Ivan was working on the problem. He wasn't working on what you could do, he was working on the problem. Yeah.


[User] Please transcribe this text

[Assistant] So, Sketchpad was a bombshell in this research community that was about to happen. Because right away you were looking at the future. You just had to believe that a room


[User] Please transcribe this text

[Assistant] A building sized computer was going to wind up on a laptop. Or even something in a desk.


[User] Please transcribe this text

[Assistant] So the other, uh, the main player here back then was a guy by the name of Lick, Licklider, the ARP, ehm, he got given some money in 1962 by ARPA before the D and


[User] Please transcribe this text

[Assistant] Whenever anybody asks him what are you gonna do, this is what he would say. Computers are destined to become interactive intellectual amplifiers for everyone in the world, universally networked worldwide. He would not say anything more.


[User] Please transcribe this text

[Assistant] I hate bullet points, but this is a perfect class I think to put bullet points in. And I found 16.


[User] Please transcribe this text

[Assistant] An idea that's worth dedicating your life if necessary to.


[User] Please transcribe this text

[Assistant] So this stuff was human destiny fixing big human problems like we can't think very well. We need to make things to help us to think. We need to make things to help us cooperate.


[User] Please transcribe this text

[Assistant] So these were save the world kind of ideas and they were done when, you know, the Russians were...


[User] Please transcribe this text

[Assistant] Starting to test hydrogen bombs and, uh, things did not look so good. Forget about goals.


[User] Please transcribe this text

[Assistant] The problem is the goals tend to be much more idiosyncratic to individual humans.


[User] Please transcribe this text

[Assistant] Had ways of approaching the dream and some of these people didn't, uh, uh, agree and some of them hated each other. And Look didn't give a shit. He just wanted smart people working on this dream. So fun people, not projects. ARPA never decided, and we'll see, uh, a slight modification of this, but basically fun people, not projects.


[User] Please transcribe this text

[Assistant] So, if you know, MacArthur grants are for individuals, but


[User] Please transcribe this text

[Assistant] This funding was funding groups like you fund in MacArthur. Just five years. Forget it.


[User] Please transcribe this text

[Assistant] Nobody at MacArthur asks, you don't get a second MacArthur grant, here they would get


[User] Please transcribe this text

[Assistant] Another grant if they'd done something good in five years, but basically the idea of MacArthur is throwaway.


[User] Please transcribe this text

[Assistant] We've identified this person of extreme potential. Let's just give them five years of funding and we won't cry if they don't do anything.


[User] Please transcribe this text

[Assistant] Turns out most MacArthur people do do something because the people


[User] Please transcribe this text

[Assistant] That attract attention are people who are not working for money. They're doing it because they must.


[User] Please transcribe this text

[Assistant] There are people, like artists are people who do their art because they must.


[User] Please transcribe this text

[Assistant] Community not a project. You have to fund problem finding. This really drives funders crazy today.


[User] Please transcribe this text

[Assistant] Most of the time when you're working on hard problems, you don't know what the right problem is. This is why visions are really good. Visions are detached.


[User] Please transcribe this text

[Assistant] If you pick a problem too early, you might be picking it out of the current context.


[User] Please transcribe this text

[Assistant] And therefore you're going to be hampered unless you're incredibly lucky.


[User] Please transcribe this text

[Assistant] So ARPA put a lot of money into just people dicking around with stuff. Milestones not deadlines. Uh, baseball not golf.


[User] Please transcribe this text

[Assistant] Meaning, uhm, when you lose a stroke in golf, you cry. When you strike out in baseball, you better not cry. Because you're gonna do it a lot. And...


[User] Please transcribe this text

[Assistant] This is what Licklider said to them. He said, look, if you're batting 350 in baseball, you're really doing well. And, uh,


[User] Please transcribe this text

[Assistant] If you look at what we're funding, if we bat $350, we're gonna change the world. That's what happened.


[User] Please transcribe this text

[Assistant] Nobody cares about all the stuff that didn't work. And people said, well what about the 65% failure? And they said, well it's not failure in baseball, it's overhead.


[User] Please transcribe this text

[Assistant] Hitting a ball is hard. When you're doing something really, really hard,


[User] Please transcribe this text

[Assistant] Uninteresting projects that are guaranteed for success.


[User] Please transcribe this text

[Assistant] But sports is right in front of us. Sports is really hard and most sports people are not succeeding all the time.


[User] Please transcribe this text

[Assistant] The thing, what's the thing in, anybody know baseball here? What's the thing in baseball that's called an error?


[User] Please transcribe this text

[Assistant] Uh, 98.5% effective. So really good ones are like 1% error. And so an error in technical stuff is deciding to build a computer system and failing to build it.


[User] Please transcribe this text

[Assistant] Like anybody should be able, or deciding to build a software system and failing to build it. You just should be able to knock that off. But you're in this other range when you're trying to do design. And it's really two different things. So here's a memo Lick wrote uhm, in 63 shortly after he got this initial money from ARPA to members and affiliates of the Intergalactic Computing Network.


[User] Please transcribe this text

[Assistant] And they asked him, why did you call it that? And he said, well, engineers always give you the minimum.


[User] Please transcribe this text

[Assistant] And I want a network that spans the entire planet, so I'm asking for an intergalactic one. And when they scale it down, we'll still get, so that's where the internet came from, literally.


[User] Please transcribe this text

[Assistant] The original name of the internet was the intergalactic computing. Nobody knew how to do it back then. Packet, for instance, packet switching had not been invented in 63.


[User] Please transcribe this text

[Assistant] possible way, and I don't have time to really explain it. It's really interesting. People who are interested in this should write me an email.


[User] Please transcribe this text

[Assistant] Didn't mean just other people. It meant other software, other computers. It meant what does it mean to communicate when you scale things up?


[User] Please transcribe this text

[Assistant] Number nine, you can't think inside the beltway. That's of course referring to Washington DC, but this is a general principle. And they asked him why, and he said,


[User] Please transcribe this text

[Assistant] Well, because there's too much noise, there's too much politics, there's too much bullshit, and nobody does research in Washington DC.


[User] Please transcribe this text

[Assistant] So the last thing we want to do as ARPA funders is to try to think while we're there. Our job is to get money out of the government and pass it along.


[User] Please transcribe this text

[Assistant] So a solution to that was say, hey, let's not be here for longer than two years.


[User] Please transcribe this text

[Assistant] And so every two years, uh, so Lick set up the process to get his successor. Happened that Ivan Sutherland got drafted into the army at just the right time. And, uh, so they grabbed him. And so at age 26, Ivan ran this whole show, and man was he good. So he was a, he was a second lieutenant chairing meetings with generals in it. And one of the most famous things


[User] Please transcribe this text

[Assistant] Recalled of that, some general was going on and on, at some point Ivan said,


[User] Please transcribe this text

[Assistant] General, you ju- you have just two minutes to make your point if you have one.


[User] Please transcribe this text

[Assistant] That's about the simplest way of describing Ivan. Bob Taylor.


[User] Please transcribe this text

[Assistant] And Larry Roberts. Those are the four that made this happen.


[User] Please transcribe this text

[Assistant] Taylor is special because he also is the guy who set up Park.


[User] Please transcribe this text

[Assistant] And he was a student of Licklider. Licklider was intuitively wonderful. Taylor was a student of what was wonderful about Licklider. He could explain everything that Licklider, uh, could do and why it worked.


[User] Please transcribe this text

[Assistant] Uhm, so here's a couple of things I'm not going to go on about, but...


[User] Please transcribe this text

[Assistant] Basically, the idea here is it isn't like, let's look around and see what is available and what we can do with it. The idea was, stick with the vision and just make every frigging thing that's necessary. It doesn't matter what it is. We have to make a new kind of integrated circuit, we'll do it. All of these things were done.


[User] Please transcribe this text

[Assistant] So it's very much like these other cultures where nobody worried about whether you


[User] Please transcribe this text

[Assistant] Had a half centimeter radar in building 20 at MIT. They just did it.


[User] Please transcribe this text

[Assistant] And here's one that is really a bug today.


[User] Please transcribe this text

[Assistant] And if you don't have the chops, yeaah, you shouldn't make your tools. You shouldn't make your own operating system.


[User] Please transcribe this text

[Assistant] You shouldn't make your own programming language because that's not what you're trying to do. If you do have the chops,


[User] Please transcribe this text

[Assistant] And you better not do this unless you have the chops. Then you have to make your own.


[User] Please transcribe this text

[Assistant] Hardware and software and operating system and programming languages. Otherwise you're


[User] Please transcribe this text

[Assistant] Working in the past on some vendors bad idea of what computing is about. So part of this deal, remember what, what, uh, Picasso said. You have to get really much, much better than most people want to today.


[User] Please transcribe this text

[Assistant] And here's another reversal. Today, people buy hardware and, uh, put software on it basically to make the hardware look good.


[User] Please transcribe this text

[Assistant] The ARPA community in part did exactly the opposite. It started off with the software using super computers to simulate the thing and when you got the


[User] Please transcribe this text

[Assistant] Uh, and make more efficient what you're doing on a super computer. That was what Sketchpad was.


[User] Please transcribe this text

[Assistant] Nobody thought that the next graphics systems were going to be done on a computer the size of this building.


[User] Please transcribe this text

[Assistant] So almost every project in the ARPA project, they actually made enough of them so they could be used as tools. They weren't just demos.


[User] Please transcribe this text

[Assistant] And so part of this invention process had to require a kind of limited


[User] Please transcribe this text

[Assistant] Engineering. Park, for instance, had a thing where whatever you did you had to make a hundred of them.


[User] Please transcribe this text

[Assistant] Made an ethernet. It had to run a hundred machines. If you made a time sharing system, it had to run a hundred users. It made a personal computer. You had to be able to build a hundred of them.


[User] Please transcribe this text

[Assistant] Learn how to argue. This is what you learned as a graduate student there. Argue for clarity not to win. It's the biggest bug today. People are always contending with each other.


[User] Please transcribe this text

[Assistant] Trying to be winners and losers, and that's one of the problems with Washington. They've forgotten what their job is. No, what you have to do is understand these complicated things. Here's a biggie. Every researcher at Park.


[User] Please transcribe this text

[Assistant] Was a second or third generation PhD that ARPA had created. Right? Eight years.


[User] Please transcribe this text

[Assistant] So it wasn't just the stuff in the past, I'll show you a couple more of those. Wow, we're getting close to...


[User] Please transcribe this text

[Assistant] It was creating the next generation. Again, this is baseball. Baseball, you don't, you have to have formed, you have to develop talent going all the way down to Little League. And they did, and the other thing is a little difficult to, some of you will understand this readily, that the reward of doing this stuff


[User] Please transcribe this text

[Assistant] Wasn't the reward of making it happen, because a lot of times it didn't happen. It was the reward of


[User] Please transcribe this text

[Assistant] Actually being funded to work on what the actual problems were. I can't emphasize this too much. Working on what the problems actually are rather than something that's gonna get a paper, something that's gonna make you money. But taking, like, big human problems could, could be something like drinkable water for the 70% of the planet that doesn't have it.


[User] Please transcribe this text

[Assistant] Doesn't matter, there's no other goal. You know, if you pick one of those things, that's a great one worth putting a life into. Okay, so background was partly tinkering.


[User] Please transcribe this text

[Assistant] A lot of people came from New York, down where the World Trade Center was.


[User] Please transcribe this text

[Assistant] Before the World Trade Center there was about a mile across Manhattan Island that was nothing but electronic surplus stores. Mow a few lawns.


[User] Please transcribe this text

[Assistant] Put a dime into the subway and you can go down there and buy almost anything to mess around with.


[User] Please transcribe this text

[Assistant] And that group got used to everybody was broke.


[User] Please transcribe this text

[Assistant] So, but you could build your own, like there's a kit to build your own oscilloscope. And if you wanted an oscilloscope and you were broke,


[User] Please transcribe this text

[Assistant] You could get one of these on time for 30 or 40 bucks and build it.


[User] Please transcribe this text

[Assistant] Okay, so Moore's Law, this was the original thing in Moore's paper, doubling, uh, every year. He picked


[User] Please transcribe this text

[Assistant] M.O.S. Silicon, which is too slow to make things out of, but it would get faster if you made it smaller. Here's doubling every two years. What happened, the prediction was 30 years and


[User] Please transcribe this text

[Assistant] So what happened was very in line with doubling every two years to every eighteen months.


[User] Please transcribe this text

[Assistant] And there was physics behind this. This wasn't just an engineering aspiration. And so if you believe this you had something


[User] Please transcribe this text

[Assistant] I think you should see a little of this though.


[User] Please transcribe this text

[Assistant] This was done at Rand and how did the Rand people go about it? Well at midnight they would go through people's waste paper baskets to see how they worked. What were they throwing away?


[User] Please transcribe this text

[Assistant] And what people were doing when they were working is making all kinds of diagrams, little flow charts and all this stuff. So they said, okay, well let's invent the first tablet.


[User] Please transcribe this text

[Assistant] As good as most of the tablets you've ever used today. It's a hallmark of these people, is they generally did it good enough.


[User] Please transcribe this text

[Assistant] This system really, you could ask your question more even about this. This system was one of the best systems I've ever used in every, it felt so intimate, it was so different from the mouse.


[User] Please transcribe this text

[Assistant] You know, it was one of the things that we, that we actually looked at. Uhm, I guess I do have to show the next, the next thing here.


[User] Please transcribe this text

[Assistant] Just so you think that VR wasn't something done recently, but the second thing Ivan did after he came back, the thing when he came back from ARPA was to do this. I worked on this when I was a graduate student.


[User] Please transcribe this text

[Assistant] You could grab things and have a thing in your hand. You could grab things and move them around it.


[User] Please transcribe this text

[Assistant] It's a little exciting because instead of the thing that you're used to today in VR it had


[User] Please transcribe this text

[Assistant] Two CRTs with 15,000 volts right above your ears crackling away. So it's a very exciting...


[User] Please transcribe this text

[Assistant] Thing to do, and of course one of the hardest things was to do good head positioning back then. There are many ways of


[User] Please transcribe this text

[Assistant] Doing one of the things they did here, I won't explain what it is, but it was hooked to a crane.


[User] Please transcribe this text

[Assistant] So as you walked around this big room at MIT,


[User] Please transcribe this text

[Assistant] A crane would fall, would automatically follow you with a positioning thing over your head. To find out what, what this was about. And, oh man.


[User] Please transcribe this text

[Assistant] But I really have two more things to do. If you could give me two more minutes, I will do the first one here is this is something you have to think about. Let time go one way and progress go the other way.


[User] Please transcribe this text

[Assistant] Yay, boo, yay, boo. It goes up, it goes down. This is the way people tend to measure things. Problem is they never put in a threshold. Put in a threshold, then the only things above the threshold count. Like if these are reading scores,


[User] Please transcribe this text

[Assistant] What is actually needed changes over in time. You need more.


[User] Please transcribe this text

[Assistant] One of the ways of improving things or making it look like you're not doing such a bad job is to lower the baseline.


[User] Please transcribe this text

[Assistant] Completely lowered the baseline on what constitutes a reasonable user interface for iPhones and iPads.


[User] Please transcribe this text

[Assistant] Interfaces on the systems beforehand had an undo and these don't. And I can name fifteen more things. Think about it. So what they did is they decided, well we're just, we're, we're just not gonna work on all of those.


[User] Please transcribe this text

[Assistant] We're gonna condition the unsophisticated public to do work at the level of a 2 year old or a 92 year old. And we're just gonna eliminate everything in between. And the public has bought it. Because you don't want to count noses when you're looking for quality. People can be talked into anything.


[User] Please transcribe this text

[Assistant] So what you have to do on this stuff is you have to pick something that is absolutely above that line.


[User] Please transcribe this text

[Assistant] We call that the McCready sweet spot. He's the guy who did man powered flight. And once you have achieved that,


[User] Please transcribe this text

[Assistant] It opens up a whole region that you can explore, and that's what we did at Park. The Moonshot was set space travel back 50 years.


[User] Please transcribe this text

[Assistant] Everybody uses this as a metaphor. We need a moonshot. No, we don't need a moonshot. You just don't do space travel with chemical rocketry.


[User] Please transcribe this text

[Assistant] And what's interesting about the people who, like, uh, Jeff Bezos and, uh,


[User] Please transcribe this text

[Assistant] You either have to carry a shitload of reaction mass.


[User] Please transcribe this text

[Assistant] But the thing that was a tragedy was that there were very good proposals for how to get that high exhaust velocity using various forms of atomic power.


[User] Please transcribe this text

[Assistant] Because I want to end with, oh yeaah, here's, so this tablet computer I thought up in 1968, it had two initial forms. One was the familiar tablet.


[User] Please transcribe this text

[Assistant] The other one was what Ivan's head mounted display was inevitable with Moore's Law.


[User] Please transcribe this text

[Assistant] And then Nicholas Negroponte had this idea of wearing a watch and communicating with the rest of the world.


[User] Please transcribe this text

[Assistant] Okay, so here's the last little segment and then I'll let you go.


[User] Please transcribe this text

[Assistant] Uh, they asked, uh, they asked him why he took so many shots on goal. He says, well, you miss 100% of the shots you don't take. So he wasn't trying to be careful. He scored more goals than anybody in history by a thousand. So the fact that his percentage of misses, uh, was also high was irrelevant.


[User] Please transcribe this text

[Assistant] Why he was better than anybody else. And he said, well a good hockey player goes to where the puck is. A great one goes to where the puck is going to be. And he didn't mean tracking the puck.


[User] Please transcribe this text

[Assistant] So what he did is he looked at the entire configuration, saw where the future was going to be, and went to that place. So you can make a game to invent the future out of this.


[User] Please transcribe this text

[Assistant] So, you start off with a cosmic goodness intuition, like for me the tablet one was good. You identify a favorable exponential, like Moore's law.


[User] Please transcribe this text

[Assistant] You take the cosmic intuition out 30 years and you ask


[User] Please transcribe this text

[Assistant] At that point, you can just pay money, because that's also what Moore's Law and what, uh, Turing means, is...


[User] Please transcribe this text

[Assistant] If you pay a lot of money now, you can make the commodity computer of 10-15 years in the future. That's what the Alto was.


[User] Please transcribe this text

[Assistant] It was about 50 times faster than what you got out of the time sharing terminal. And you make a bunch of them. And you intertwine it with the, so it'll run the software. And then you can do two kinds of computing in the future. One kind of computing is.


[User] Please transcribe this text

[Assistant] Zillions of experiments to get a user interface that would be universal for now five billion people and deal with millions of people you've never, who are doing applications you've never seen before.


[User] Please transcribe this text

[Assistant] And the other thing you can do, the thing on the right there is Microsoft Word as it was at Xerox PARC in 1974. If you optimize the code, you can get what the applications are going to be ten years out.


[User] Please transcribe this text

[Assistant] So those are the two things you can do by doing this supercomputer thing.


[User] Please transcribe this text

[Assistant] And it cost money. In today's dollars, those altos cost about 125k apiece. And we made 2,000 of them. Think about that. Gotta be a, a funder who's serious. Xerox went batshit when we did it. Uh, they really went batshit when we wanted to cash them in.


[User] Please transcribe this text

[Assistant] But I'll just leave you this, uh, since I'm over time now. I'll leave you this as one of the ways of escaping the present. Have a glimmer of an idea. Take it so far out.


[User] Please transcribe this text

[Assistant] Innovating out from the present, what you want to do is invent the future from the future. You go out and live in the future and bring the future back. Thank you.


[User] Please transcribe this text

[Assistant] Sure, uhm, I'm Jess, uh, back in 2004, I was sitting where you guys are, uh, taking, I think I took databases in this room, but, uhm, I studied computer science and then went to Google where I was a PM, uh, for Google Maps and then co-founded a company called Polyvore, uh, and was CEO there.


[User] Please transcribe this text

[Assistant] Then there's, uhm, like you said, there's people who have demonstrated, uh,


[User] Please transcribe this text

[Assistant] You know, I think on average it takes about 8 years to exit. So, when you and your founder win the investor.


[User] Please transcribe this text

[Assistant] Yeah, I meant exit, so even just acquisition. Right. Uhm, but, you know, it's, you're partnering with a person who, you're gonna be...


[User] Please transcribe this text

[Assistant] It's a long-term relationship you're getting into, especially once you start to get to your series A and B. Uhm, so you wanna find someone who you have good fit with. Like, it's almost like you're...


[User] Please transcribe this text

[Assistant] I know it sucks because I have to step away from running my business but I'm gonna go all in and just work on this process and you just kind of think about it as a process and time it so you have conversations at the right time and I think you want to prioritize like


[User] Please transcribe this text

[Assistant] I will say though, I think the line between seed and A is kind of blurring, has been shifting over the years. So, Polyvore raised our series A in 2007, and that was a two and a half million dollar round, or 2.6 I think. Now that seems more like a seed, so I think it can bleed, I wouldn't, I would


[User] Please transcribe this text

[Assistant] Like I think the larger the fun size, but uhm, it is interesting. I've definitely noticed a huge shift in the size of rounds getting bigger and bigger and then they're starting to be called


[User] Please transcribe this text

[Assistant] If you think about all the things you could be thinking about, like the valuation and the terms, uh, the person that you're working with, the individual partner, and then maybe the reputation or the, the


[User] Please transcribe this text

[Assistant] Getting help from your peers, who are other founders and other CEOs, and rubbing shoulders with like really amazing founders who help you out but then also elevate your game, like that's really, really important. That's really something to look for.


[User] Please transcribe this text

[Assistant] You want someone who's going to be hardworking, who you feel comfortable with, who can be your first phone call, right?


[User] Please transcribe this text

[Assistant] Uh, a sparring partner, right, who like says, like, have you thought about this? Someone who pushes you and challenges you to go even, even greater. So that's, I mean, that's certainly part of our philosophy as board members at Sequoia.


[User] Please transcribe this text

[Assistant] I think the, the other model that's not great is the person who's totally uninvolved. You know, you like, you, you want your investors to work hard for you too, right? So that's another sort of, uh, bad model. What's the most another, oh, the one who's too involved, who wants to like,


[User] Please transcribe this text

[Assistant] I think if, yeaah, if it's gonna be an investor who comes over and like stands over your computer and like says like what's in that spreadsheet? Definitely, definitely not. But I think there are models where uhm,


[User] Please transcribe this text

[Assistant] And 94% of investors are male. So I couldn't rely on, you know, when I pitched an app saying, imagine an app where you could buy dresses all day. You know, 94% of investors are like, why would I want that? That sounds terrible. I hate shopping.


[User] Please transcribe this text

[Assistant] It just didn't work. So you had to come up, we had to come up with a way to communicate that really clearly. So sometimes it's an analogy. Sometimes it's, you know, you draw out a persona. This is your user. This is their problem. You use a real customer, for example. For me, what we did is, for Polyvore,


[User] Please transcribe this text

[Assistant] Yeah, I think it also says something about a skill that's actually really important in being a founder, which is can you take a big hairy complex problem and then


[User] Please transcribe this text

[Assistant] Break it down into simpler problems, like just how you'd like take up.


[User] Please transcribe this text

[Assistant] You might be better to talk about that one at the friends and family stage and that very early stage. You know, I mean, I think you don't want to, like when you, I think a startup is,


[User] Please transcribe this text

[Assistant] One meeting with the first person, a second meeting with multiple partners and then the full partner meeting, right? Not every firm requires unanimous, not every firm requires that everyone sees it.


[User] Please transcribe this text

[Assistant] Uhm, I think I'm the last person between now and lunch and now and maybe the summer, right? Summer exams. So I'll try to be quick. My name is Oli Rogani, uhm, uh, subject of my talk is how to succeed long term.


[User] Please transcribe this text

[Assistant] And, uh, I've been amazed. I was watching some of the videos that you guys saw and reading up on some of the notes. You guys have heard from some amazing people, uh, Emmet Shearer and Michael Seibel and Aaron Levy, et cetera, talking about how to build a great product and...


[User] Please transcribe this text

[Assistant] Adam DeAngelo and Stuart Butterfield about how to come up with ideas and measure success. Peter Reinhardt about product market fit. Vinod Khosla about hiring and building teams. It's an incredible list of people and now, uh, from Jess and Aaron about the tactics of fundraising.


[User] Please transcribe this text

[Assistant] I'm here to talk about a topic that's perhaps, uh, more slippery, uh, and abstract than any of the topics you guys have heard about so far. Uhm, and that topic is leadership.


[User] Please transcribe this text

[Assistant] Uh, and in particular, I wanted to answer, uh, the following question. What do you have to do to be a great leader? And hopefully it goes without saying that, uh,


[User] Please transcribe this text

[Assistant] That learning to be a great leader is pretty fundamental to long term success. And the reason's pretty simple. If you're gonna succeed in building a really successful startup,


[User] Please transcribe this text

[Assistant] There's no really no way around having to hire a lot of people and motivating and aligning them and getting to achieve something much bigger than you could ever achieve on your own. So there's really kind of no way around learning how to be a great leader if you want to succeed. And for me, uhm, the question,


[User] Please transcribe this text

[Assistant] Really starts, uh, the most important initial question to ask is the following. When you study leaders, are great leaders largely similar to one another? Or are they really different?


[User] Please transcribe this text

[Assistant] And I remember when I was sitting at Stanford, I was an undergrad and a business school student here. And I remember sitting in business school, and if you'd asked me that question, I would have said pretty clearly that


[User] Please transcribe this text

[Assistant] I think there's kind of a single personality trait. Like, leaders are all pretty much the same.


[User] Please transcribe this text

[Assistant] And either you had those genes or that sort of leadership mojo.


[User] Please transcribe this text

[Assistant] Or if you didn't, then you better kind of work on changing your personality traits around so you could exhibit some of those leadership personality traits so you could be a leader. And this question for me was largely an academic one.


[User] Please transcribe this text

[Assistant] Until after business school, I started working at Pixar, the animation studio. And had the extreme good fortune.


[User] Please transcribe this text

[Assistant] Of working with four people, four leaders that I consider extraordinary. Uh, and these four people were Steve Jobs, who is our CEO.


[User] Please transcribe this text

[Assistant] Ed Catmull, who was Pixar's founder, uh, and was the president of Pixar when I joined.


[User] Please transcribe this text

[Assistant] John Lasseter, who is Pixar's Chief Creative Officer.


[User] Please transcribe this text

[Assistant] And Bob Iger, who was the CEO of Disney, who acquired Pixar in 2006.


[User] Please transcribe this text

[Assistant] The surprising thing for me about working with these leaders was that they couldn't have been more different.


[User] Please transcribe this text

[Assistant] In terms of their personality, their temperament, their style.


[User] Please transcribe this text

[Assistant] And really, just the way they went about doing their jobs. And I want to tell you a little bit about each one of them. With the aid of a picture. That's John Lasseter.


[User] Please transcribe this text

[Assistant] He loves toys and he has a massive toy collection. You can see part of that behind him. He's emotional. He's energetic. You could always read how he felt on his face. He loves wine and good food. He had dozens of hobbies outside of work.


[User] Please transcribe this text

[Assistant] He didn't manage his time particularly well and was always over committed.


[User] Please transcribe this text

[Assistant] And he always showed people a lot of love, the people around him. He gave everyone hugs, and he wore Hawaiian shirts every day. Ed Catmull, on the other hand, was an introverted scientist. He had a PhD. He studied computer graphics.


[User] Please transcribe this text

[Assistant] He was really health conscious. He kind of lived his life the right way. He didn't drink, he didn't smoke. He ate well.


[User] Please transcribe this text

[Assistant] And his primary hobby outside of work, believe it or not, was to go on week-long silent meditation retreats.


[User] Please transcribe this text

[Assistant] You guys all know about Steve. He was a college dropout. He'd never really worked for anyone in his life.


[User] Please transcribe this text

[Assistant] He was a magnetic presence in meetings, walking around. Uh, he was intense. He was impatient. He was extraordinarily quick on his feet.


[User] Please transcribe this text

[Assistant] He was not really pro- prone to bonding with people or showing love or affection.


[User] Please transcribe this text

[Assistant] You certainly never, I never saw him give anyone a hug at work.


[User] Please transcribe this text

[Assistant] He could say really pointed things, he was aggressive, he was always in a hurry, and he was in a hurry to get to the answer. He had this, uh, energy about him.


[User] Please transcribe this text

[Assistant] Uh, and a clarity of thought and communication that was really remarkable.


[User] Please transcribe this text

[Assistant] And then finally, Bob Iger. Bob was the CEO of Disney. Uhm, he had, unlike Steve, spent more than 30 years at the same company, uh, before he eventually became the CEO. He was diplomatic. He was genteel. He was patient, wise, calm. He had gravitas.


[User] Please transcribe this text

[Assistant] He exuded a trustworthiness and honor that you wanted to be around and follow and he had really really high emotional intelligence. Here's a picture of the four of them on the day we announced Disney's acquisition of Pixar. When I look at this picture,


[User] Please transcribe this text

[Assistant] I think to myself, these four guys worked extremely well together.


[User] Please transcribe this text

[Assistant] And if they'd been each other in high school or in college, there's no way they would have ever been friends. You had an artist, and a scientist, a hippie,


[User] Please transcribe this text

[Assistant] taught me a lot and it kind of led me to my personal first insight about leadership, which is that there's no single personality type for great leaders. There's really no single model.


[User] Please transcribe this text

[Assistant] And this is actually really great news if you think about it, because it means that people of varying personality types


[User] Please transcribe this text

[Assistant] Can all strive to be great leaders. You're not, you don't, there's not a gene set or a certain personality type that, you know, allows you to be a great leader or disqualifies you from being a leader. But the other implication of this insight


[User] Please transcribe this text

[Assistant] Is that in your quest to become a leader of your startup or, or even, or even a bigger aspiration for leadership, you have to be yourself. You have to be authentic. You can't try to be someone else or copy someone else's personality or style and hope to be a great leader. You can't try to be like Steve Jobs.


[User] Please transcribe this text

[Assistant] And the reason for that is that as humans, we're actually really highly evolved at detecting inauthenticity in other people. And we don't follow people we find to be inauthentic.


[User] Please transcribe this text

[Assistant] So this kind of leads to my second insight that I wanted to share.


[User] Please transcribe this text

[Assistant] Which is despite the variability in personalities, despite the fact that really all personality types can be great leaders, all great leaders have to do three things exceptionally well. The first is,


[User] Please transcribe this text

[Assistant] Paint a compelling vision for the future that other people can understand and follow.


[User] Please transcribe this text

[Assistant] And this is the case if you're a three person startup and you're trying to motivate your co-founders or early employees. And it's even more the case when you're a hundred person company, a thousand person company, and so on.


[User] Please transcribe this text

[Assistant] And the key thing about communication as a leader is that simplicity of communication is vital.


[User] Please transcribe this text

[Assistant] I remember marveling at Steve Jobs' ability at Pixar to take the most complicated problem we were discussing.


[User] Please transcribe this text

[Assistant] And be able to simplify it to its core essence. Break something complicated apart into what really mattered and how we should approach something first, second, third.


[User] Please transcribe this text

[Assistant] It was like we were all sitting in darkness and there was one guy who had a lantern. And that was Steve.


[User] Please transcribe this text

[Assistant] And because of that, because he had clarity and could explain something in a way that you could understand and sounded right, you would follow him.


[User] Please transcribe this text

[Assistant] The other example I love on simplicity is an example from Jeff Bezos of Amazon.


[User] Please transcribe this text

[Assistant] Uh, he once said that in Amazon's retail business, which now encompasses almost every category, if not every category of products on earth, he said only three things mattered. First, low prices. Second, broad selection. And third, fast delivery.


[User] Please transcribe this text

[Assistant] These three things were the only things that consumers cared about and they would always care about those three things. So anything that an Amazon employee would do.


[User] Please transcribe this text

[Assistant] To, to improve Amazon's performance in those three areas was aligned with Amazon's strategy. And that simplicity has been there, the bedrock of Amazon's retail strategy from the beginning. That's great communication.


[User] Please transcribe this text

[Assistant] Now obviously Bezos and Jobs set a pretty high bar and I'm not here to try to teach you to be any of those guys.


[User] Please transcribe this text

[Assistant] Uhm, but I can tell you that if you want to improve as a communicator,


[User] Please transcribe this text

[Assistant] The first thing you should do is to step back and give yourself more time to think.


[User] Please transcribe this text

[Assistant] Even Steve Jobs used to tell us that when he was at his best, running both Apple and Pixar.


[User] Please transcribe this text

[Assistant] Half of his time, half of his schedule was unscheduled, meaning it wasn't filled with meetings with other people. He created time in his calendar to be able to think. And that's going to be really hard when you guys start your startups.


[User] Please transcribe this text

[Assistant] It's gonna be frenetic, your time is gonna be, you're gonna be pulled in a million directions, and it's gonna be very, very difficult to actually devote time to just thinking. I don't mean answering email. I mean time to think and plan ahead. And plan your communication. What are you gonna say? How are you gonna explain what you think in a clear way?


[User] Please transcribe this text

[Assistant] And so my best advice on this is force yourself to take as little as an hour a day. Or an hour every other day at the beginning.


[User] Please transcribe this text

[Assistant] Some people have natural gifts, but anyone with practice and with giving themselves, uh, time, uh, to think can get better.


[User] Please transcribe this text

[Assistant] Uh, and the core insight here is that clarity of thought always precedes clarity of language.


[User] Please transcribe this text

[Assistant] Second thing all great leaders do is they show great judgment about people.


[User] Please transcribe this text

[Assistant] So you heard Vinod Khosla a few weeks ago say that the first ten people you hire into your startup are absolutely critical. And he's right. And when you grow your startup,


[User] Please transcribe this text

[Assistant] You have to have great judgment, not only about the people you hire into sort of individual contributor roles, but even more so the people, uh, to whom you decide to give authority and power. The people you decide to promote and to make managers and to make into leaders. And if you show bad judgment there,


[User] Please transcribe this text

[Assistant] If you pick the wrong people, if you bring the wrong people into your company, you can never grow into a great leader.


[User] Please transcribe this text

[Assistant] And like communication, you can actually improve and develop your intuition about people. Practice and experience again help here.


[User] Please transcribe this text

[Assistant] And so as you guys start your companies, many of you will likely be hiring people for the first time. And so here's my advice. Don't rush it.


[User] Please transcribe this text

[Assistant] Take the time to get to know the people you're hiring. Ideally, start with co-founders and early employees who you already know.


[User] Please transcribe this text

[Assistant] It's important to be aligned from a values perspective.


[User] Please transcribe this text

[Assistant] It's not just can this person code or can this person get the job accomplished. You want to try to hire people for the long term. Spend time to get to know them.


[User] Please transcribe this text

[Assistant] You know, it's not unusual when people at larger startups are hiring their first members of leadership that they spend 10, 20, 30 hours with the person before they decide to hire them.


[User] Please transcribe this text

[Assistant] So spending time to get to know someone is very, very important and trying to bond with them on values. Like what do they really care about? Are you guys aligned? Do you think you can get along well in the long term? Evaluate that as well as their competence. And the second thing that's very important in developing your intuition


[User] Please transcribe this text

[Assistant] Is try to meet people who are absolutely great at what they do, even if you can't hire them.


[User] Please transcribe this text

[Assistant] In order to develop your own understanding of what great is.


[User] Please transcribe this text

[Assistant] Again, I could give you lots of examples of founders who spend a lot of time doing this, uh, but the more people you meet,


[User] Please transcribe this text

[Assistant] Uh, the more great people you meet, you fine tune your ability to detect those characteristics in other people.


[User] Please transcribe this text

[Assistant] Finally, a lot of hires don't work out no matter how well you do it.


[User] Please transcribe this text

[Assistant] You're gonna have to change people out, you're gonna have to ask people to leave. Make sure you do that quickly.


[User] Please transcribe this text

[Assistant] Make sure you're courageous about it, and make sure that you reflect on the lessons learned every time someone doesn't work out, and also every time someone works out.


[User] Please transcribe this text

[Assistant] Make sure you're reflective about your own process and how you've decided to bring people into the company and who you decide to promote into more bigger leadership roles.


[User] Please transcribe this text

[Assistant] Integrity means standing for something that's important, bigger than yourself. It means having a mission to accomplish that is


[User] Please transcribe this text

[Assistant] That would erode trust in all of us. Things like favoritism, or conflicts of interest, or using inappropriate language, or having inappropriate work relationships.


[User] Please transcribe this text

[Assistant] The key test for me is if you consider if all of your communication


[User] Please transcribe this text

[Assistant] All of your emails, all of your verbal communications that you have with those around you, if they were all transparent to everyone, would you be embarrassed by anything you've done or said? And really you should try to hold yourself to that bar.


[User] Please transcribe this text

[Assistant] Commitment means making your work, making your startup into a life mission in a way that inspires other people. It means giving it your all. That's inspiring. When people see


[User] Please transcribe this text

[Assistant] Others, when people see leaders motivated by a big purpose and giving it their all to accomplish that purpose it's motivating in and of itself. And when you think of the opposite, when you see someone who is motivated by selfish ends


[User] Please transcribe this text

[Assistant] Who is cutting corners, who is slacking off, who is showing ethical weakness.


[User] Please transcribe this text

[Assistant] Those are not people you want to follow. So if you want to be a great leader, it's really important to hold yourself to the spark.


[User] Please transcribe this text

[Assistant] Leave you guys with the last insight I have and a piece of parting advice.


[User] Please transcribe this text

[Assistant] Ultimately, if you want to measure a great leader by one thing, I think the most important success metric for a leader is trust. And trust is a squishy concept.


[User] Please transcribe this text

[Assistant] And in the way I'm using it, I mean it almost in a 360 degree sense.


[User] Please transcribe this text

[Assistant] The job of every leader is to build trust. Is to build trust in employees and investors and customers and users.


[User] Please transcribe this text

[Assistant] And building trust is both an art and a science. The science part of it is fairly easy. It means


[User] Please transcribe this text

[Assistant] When there is an empirical question to answer, when you have to think through what is the technology we should build, or the product direction we should go in, or the, or uhm, what sort of partnership we should strike in the marketplace, like you've gotta be right about those, those things a lot, right? If you, it's, it's a question of competence.


[User] Please transcribe this text

[Assistant] And it's also a question of competence when it comes to your people judgment. Are you good at the people choices you make, both hiring and promotion?


[User] Please transcribe this text

[Assistant] It's about showing empathy, having good timing, choosing the right words, dealing with the right, different people in the right ways.


[User] Please transcribe this text

[Assistant] And it's about the integrity and commitment that I mentioned. Striving for something bigger than yourself, rather than being selfish or self-centered.


[User] Please transcribe this text

[Assistant] When you see a leader who has incredible trust, think about it in your personal life, the people you follow in your life, whether it's your friends or family members, etc. Think about who you have the deepest level of trust for in your life. Uh, outside of, maybe outside of your parents.


[User] Please transcribe this text

[Assistant] Uh, and it's likely that those same people are people you would listen to and follow. Because trust is such a strong bond. So here's my parting, parting advice.


[User] Please transcribe this text

[Assistant] At every step as you guys move forward, build your startup, startup may succeed, it may fail, you'll try it again, etc.


[User] Please transcribe this text

[Assistant] You're gonna have hard times and, and hopefully good times. Always try to optimize for trust. You're gonna have a lot of hard decisions to make.


[User] Please transcribe this text

[Assistant] You're gonna, you're gonna, if you scale a startup, you're gonna have to fire people, you're gonna have to admit mistakes to your employees and to your customers, you're gonna have to say no to people because you disagree with their ideas.


[User] Please transcribe this text

[Assistant] Try to view every challenge you guys face as leaders as an opportunity to increase the trust that people have in you.


[User] Please transcribe this text

[Assistant] And always as you're evaluating one course of action versus another.


[User] Please transcribe this text

[Assistant] My best advice is to ask yourself, which path will generate more trust in you as a leader? And always try to choose that path.


[User] Please transcribe this text

[Assistant] You know, uh, Steve, as you guys know, started off very early as an entrepreneur. And, uh, you know, there was a great interview of him, if you guys haven't watched it, with Steve Jobs and Bill Gates together.


[User] Please transcribe this text

[Assistant] And one of the things that Steve talks about, he said one of the things that Bill and I have in common is we were always very early on the youngest people in the room. And uhm, how, how hard that was.


[User] Please transcribe this text

[Assistant] Uh, and sort of what a trial by fire that was to try to, as the youngest person in the room, convince other people that you're right.


[User] Please transcribe this text

[Assistant] Uh, so I think in his case he had years and years of practice before, you know, I ever met him.


[User] Please transcribe this text

[Assistant] I'm not, I don't know if he had any particular techniques. I mean, I'm sure a lot of these gifts came naturally. Uhm, but uhm,


[User] Please transcribe this text

[Assistant] Practice definitely helps. And, and being around people who are great at it and observing them and holding yourself to a high bar, uhm, really helps.


[User] Please transcribe this text

[Assistant] You know, uhm, brushing under the carpet or hoping that people won't notice or people will forget, uh, is a natural human instinct, but it's...


[User] Please transcribe this text

[Assistant] You're like them. You have some humanity. And that you can accept when you've made a mistake. That you're open to feedback. All these things are really actually important. So don't feel like if you've made a mistake or goofed something up. Talking about it and admitting it.


[User] Please transcribe this text

[Assistant] Makes you weaker or that shows lack of leadership. It's actually usually the opposite. It usually builds trust to see people be honest about mistakes that they've made and to see people have the self-awareness to realize that they've made a mistake.


[User] Please transcribe this text

[Assistant] I'll just talk about John and Ed, uhm, because I know them the best.


[User] Please transcribe this text

[Assistant] John was motivated, I think, first and foremost by


[User] Please transcribe this text

[Assistant] Making great films that brought families together and, you know, would last forever. You know, he used to say that what other film other than Snow White can anyone name that was made in 1937? I hope I got the year right.


[User] Please transcribe this text

[Assistant] Like, having fun doing what he did. I mean, he, that, that guy really, you know, squeezed all the joy he could out of life. Uhm, and I think those were probably his, from my observation, some of his biggest motivators.


[User] Please transcribe this text

[Assistant] Ed is a great learner and a thinker. I think what motivated him was trying to create an organization, uh, that could repeatedly


[User] Please transcribe this text

[Assistant] Uh, be successful, repeatedly produce great films. And he kind of became an organizational thinker. Uhm, he of course also wanted to make great films. And, you know, he's also a technologist, so he cared a lot about the, you know, the software we were building. Uhm, but I think fundamentally he wanted to try to figure out


[User] Please transcribe this text

[Assistant] Almost impossible, continue it's almost impossible streak of creative success. Build an organization that's able to repeatedly be successful and understand why and how that was.


[User] Please transcribe this text

[Assistant] So at least the, but yeaah, the motivations did differ. I think there was some, some definitely some shared values, some shared aspirations, but each of them had a different, uh, a different specific motivator. Yeaah.


[User] Please transcribe this text

[Assistant] Uhm, the point about their diversity had to do with, for all of you in the room, I'm sure there are many personality types here. All of you in the room, your personality type, whether you're introverted or extroverted, whether you have a more silly personality, a more serious personality, whether, whatever you've studied, whatever your background is, I don't think personality type disqualifies you.


[User] Please transcribe this text

[Assistant] From being a, being a great leader or becoming a great leader. That was really the point.


[User] Please transcribe this text

[Assistant] Not that every company should strive to hire four people as diverse as them as their first employees. That would probably be right. That would probably be a mistake.


[User] Please transcribe this text

[Assistant] Hi, my name is Jen. Uh, I was an early employee at Lever. We build hiring software. So if you've applied for a job or an internship in the last year or two, you've probably, uh, used Lever before. Uh, as for my role, I've had various roles building and leading teams in our product, customer success, recruiting, and employee experience functions. Oh, and we're at over a hundred.


[User] Please transcribe this text

[Assistant] So why was diversity and inclusion important for us? You know, even though we are an HR tech startup.


[User] Please transcribe this text

[Assistant] It really just started as very organically, like what do we want to do as people and as coworkers? How can we build an inclusive, uh, inclusive place to work? And actually once we started telling stories, uh, writing blog posts about it, we realized there was a lot of resonating with both our applicants and our customers. They wanted to know what are you doing differently? What can we learn from?


[User] Please transcribe this text

[Assistant] And to be honest, what's been really cool is that it makes a lot of difference in also, uh, sales and growth so that we're able to show


[User] Please transcribe this text

[Assistant] You know, this is how we're approaching culture and inclusion, uh, with this much intention. It allows prospective customers to look at, well, if they are building culture, uh, their internal company culture that way, I want to know what their software looks like. I want to know what their product is like. So it's been a really huge benefit for us that it gets to be an ongoing conversation.


[User] Please transcribe this text

[Assistant] Yeah, the particular problem we were trying to solve for at that time, uh, about a year and a half ago was this question of how do you make inclusion truly inclusive? To make inclusion and culture building a company-wide activity? It's not just, oh that's like the women's issue or that's like the racial minorities issue.


[User] Please transcribe this text

[Assistant] So something we realized was when we had events and different discussions about inclusion, you know, there was a smaller subset of the company that was very active, very passionate about it, and actually a larger subset


[User] Please transcribe this text

[Assistant] That considered themselves kind of like a little bit on the sidelines. So don't get me wrong, they thought it was a good idea, they were very supportive, but for whatever reason they were a little bit hesitant to like jump really, uh, deep in.


[User] Please transcribe this text

[Assistant] Mistakes, missteps, uhm, here's something I just had no idea about. I said something and I offended someone and I was so mortified. But, you know, by having leaders and by having, you know, their colleagues.


[User] Please transcribe this text

[Assistant] And then second, again, uh, really designing it for with that audience in mind.


[User] Please transcribe this text

[Assistant] Uh, the subtitle to that, uh, seminar was D&I for Dummies. So it's like, if you're ever confused or uncertain and you just don't want to say the wrong thing, so you've been holding yourself back, this is the space to come for you.


[User] Please transcribe this text

[Assistant] And it ended up being really successful and a lot of people who previously weren't actively involved came and shared really personal stories and opened up. And, uh, just to wrap this up, I think, uhm,


[User] Please transcribe this text

[Assistant] It can feel really uncomfortable to speak out on behalf, even if you really, really want to. Because again, that fear of saying the wrong thing, being misinterpreted, being attacked even. So it really is a deliberate effort you have to make for, you know, your future employees, your culture, to say that this is okay, we're going to navigate through these things.


[User] Please transcribe this text

[Assistant] So we have an employee resource group called Leverettes, and that's what we call female employees of Lever. It's a Slack channel and a dinner, I think like two or three times a year, to talk about issues, you know, resources, mentorship.


[User] Please transcribe this text

[Assistant] And uhm, so this is a little bit of gray area, and I think you'll find the pattern in that, you know,


[User] Please transcribe this text

[Assistant] There was nothing, like, super wrong with it, but there was this tone, uh, that could be interpreted as a little bit of, uh, mocking, right? So, it was said, you know, something like, well, they have a Leverettes channel, so it's only fair. And there was some jokes that, again, nothing super crazy, but it didn't make people who were in it, actually, other men, didn't feel good about it.


[User] Please transcribe this text

[Assistant] So when it was brought up to me, we talked about how one of our standards and principles for DNI is this phrase, uhm, the standard that you, sorry, sorry, uh, if you see something, say something. The standard that you walk past is a standard you accept.


[User] Please transcribe this text

[Assistant] And even though it's comfortable, it was our job as startup employees, as leaders, to do something about it. So we reached, uh, reached out to one of our founders, Randall.


[User] Please transcribe this text

[Assistant] Super professionally, like not trying to call him out, just saying that like, hey, this is not okay, so I'd like to talk about what happened. Again, really trying to understand and listen, but then also draw the lines.


[User] Please transcribe this text

[Assistant] And then this is, uh, I think the next part, what he did was really stepping up as a leader. And again, you know, like, Stripe, we decided to address it at our next All Hands.


[User] Please transcribe this text

[Assistant] And I think that's been one of our key learnings, like you have to address things. Um, it's not fun. Nobody goes up to all hands in front of your entire company saying like this happened and it was super uncomfortable and awkward, but you have to do it. Otherwise it becomes an elephant in the room. So.


[User] Please transcribe this text

[Assistant] Uh, he made a joke about like, well maybe I'll grab some of the engineers and go and get pizza or something like the same night. And it wasn't until later he realized that wasn't actually very, you know, uh, that wasn't what he meant to say.


[User] Please transcribe this text

[Assistant] Because again, if you think about why that group exists in the first place, because women maybe may not be minorities, but they are minorities in tech and therefore they have experiences, you know, burdens, hardships that you might not even know about. So the reason the group exists is not to have fun and go out and go to drinks and


[User] Please transcribe this text

[Assistant] Talk about, I don't know, nail polish. It really is about, you know, the work and what can we do to support ourselves, each other, and just, uh, for future employees and future inclusion.


[User] Please transcribe this text

[Assistant] And just being really honest and authentic as a leader, I think made a huge difference. Uh, later that day in Slack, multiple employees, both uhm, and also in person went up to Randall to thank him.


[User] Please transcribe this text

[Assistant] To say things like, you know, my previous startup founders would never do anything like that. I feel really grateful and lucky to work for a place where the leaders really care about the people and it's very human and authentic. So I think that was a lesson that we learned. Uhm, it's not just to address it one on one. You want to prevent it from happening in the future.


[User] Please transcribe this text

[Assistant] Uh, you know, very easily could a new hire could have been joined, you know, six months later and the same thing happens over again. So how do you address it? Uh, head on, really step up as a leader. It's not going to be fun sometimes. It's not going to be comfortable, but that is your responsibility. But the thing is people will remember.


[User] Please transcribe this text

[Assistant] The tone you set, what you say, uh, your employees remember it and it gets embedded into the culture as you grow.


[User] Please transcribe this text

[Assistant] You know, figuring out what works for your company that might not be for every other company. Uh, for us, we are a SaaS organization, uh, in that we have a quite a large sales part of the, uh, company.


[User] Please transcribe this text

[Assistant] But that doesn't mean that, you know, if you weren't an athlete, you're going to be not good enough, right? So it really took to examining some of these biases and beliefs that we have in the Valley and workplaces in general and really looking at is that actually true and is that going to work for us at Lever? So we've been able to have


[User] Please transcribe this text

[Assistant] Uhm, yeaah, just make, you know, the previous, uh, athletic, you know, previous college athletic experience required, like taking that out of job descriptions and making sure people are not being punished or feeling like they're being left out because they didn't have this certain background.


[User] Please transcribe this text

[Assistant] Anyway, thank you very much for coming to speak to us. Thank you.


[User] Please transcribe this text

[Assistant] Uhm, I did not pay Sam for that introduction, uh, and I would note that Javi is my boss, so hi Javi. Uhm, but no, Javi, Javi, and Naomi, Danny, uhm, there are these people who have been on the growth team for four years, and then there are people like Blake and Shamoth and...


[User] Please transcribe this text

[Assistant] And, uh, and Ray, uhm, and James Wang who set the, who set the team up and, and uhm, uh, it's very nice, but I am a VP on growth. I am not the VP of growth.


[User] Please transcribe this text

[Assistant] Uhm, so you've had an amazing, amazing lecture series. I've just watched all of them yesterday to make sure I didn't say anything without actually having seen what everybody else said and it was totally awesome. I, uh, I loved what D'Angelo had to say.


[User] Please transcribe this text

[Assistant] So that lecture, if you weren't here, if you didn't watch it, I strongly recommend you go back and see what Adam D'Angelo said because I could not, I could not emphasize his points on metrics more than enough.


[User] Please transcribe this text

[Assistant] And so what I'm gonna try and do for the first half is go over some of the points that have been made already and emphasize what I think is most important when you're thinking about growth and what my take is and our take is on the, on the Facebook growth team on those. And then in the second half try and talk to you a little bit more about tactics that have been particularly, particularly useful. Uhm.


[User] Please transcribe this text

[Assistant] The caveat that I think you should take on board is what Stuart from Slack and Flickr said, which is to some extent you do have a hundred million monkeys bashing at typewriters, and I am very, very lucky to be on the growth team at Facebook.


[User] Please transcribe this text

[Assistant] To get to work on a product like Facebook that has the retention that Facebook does that has everyone in the world wanting to use the product and to say that like I drove it or any of us drove it beyond there being great product market fit is perhaps not true.


[User] Please transcribe this text

[Assistant] I can't tell you what the A-B test is if we didn't even have a growth team at Facebook. So what matters most to growth?


[User] Please transcribe this text

[Assistant] The single most important thing to growth is retention.


[User] Please transcribe this text

[Assistant] Adam commented on this in his talk and he showed you cohorts from some real company. But here is a stylized cohort of how you should look at retention.


[User] Please transcribe this text

[Assistant] The number of days from acquisition is on the X axis. On the Y axis is the percent of people who are still monthly active.


[User] Please transcribe this text

[Assistant] I care a lot about using monthly active, not daily active, not weekly active. There are really good arguments for using daily active that people say all the time because it's about intensity of usage.


[User] Please transcribe this text

[Assistant] But what really, really matters is that you've got skin in the game and people come back to your product at least once a month. And then you build on top of that engagement, revenue, whatever your key metrics are, which we'll get to in a minute. But firstly, I believe monthly active is the most important thing.


[User] Please transcribe this text

[Assistant] So I've got a real graph for revenue in a few slides from Facebook from the first, uh, 250 days of our ads product which I'll show you. But I think what's really, really important is to say that it's very easy to calculate this curve even if you have a thousand or a hundred users.


[User] Please transcribe this text

[Assistant] And what you do is you look at all of your users on the first day after they signed up. Were they monthly active?


[User] Please transcribe this text

[Assistant] Yes, clearly, right? The first 30 days, everyone's gonna be monthly active. I'd prefer if you defined a month as 28 days, by the way, because multiples of 7 is really, really good for these kind of metrics, because of weeks.


[User] Please transcribe this text

[Assistant] But for the first 30 days, everyone has been active on their first day, on their second day, on their third day as a monthly active user, whether they came back or not. And then what you do is you look at day 31. And obviously only some of your users will have been active for 31 days. Everyone who signed up a day ago was not active for 31 days.


[User] Please transcribe this text

[Assistant] So what percentage of them were active on day 31? What percentage of them were active on day 32? And as you go further and further out in the number of days since registration, the number of people who have been on your service that long declines, which means that your metric is going to get noisier, right? As you get out to the far right hand side of that curve.


[User] Please transcribe this text

[Assistant] But that doesn't mean you can't get a handle of where things flatten out very, very early on with very small numbers of users.


[User] Please transcribe this text

[Assistant] The number one criticism people give me when I talk about retention as a metric or even when you look at what Adam presented in his presentation earlier is you need lots and lots of users.


[User] Please transcribe this text

[Assistant] So you need millions of users to calculate that. That is not true.


[User] Please transcribe this text

[Assistant] We did it with thousands of users in the early days of advertisers for Facebook for example.


[User] Please transcribe this text

[Assistant] Or you need to have had all of those users on for a very long time. Look at Adam's graph. It was multiple years with months and months and months. No, you can do it day by day and you can get to a really good place super, super fast using this methodology. So what are you looking for in a retention curve?


[User] Please transcribe this text

[Assistant] You're looking for it to flatline. Above zero. Not flatline like a heart monitor. You want it to flatline at some line that asymptotes with the x-axis. There are three types of these curves that I've seen.


[User] Please transcribe this text

[Assistant] The first type is where it asymptotes to the x-axis. Like really, really close, goes down to almost zero.


[User] Please transcribe this text

[Assistant] You hear about a lot of those companies, you think about most games, right? Most games are smash hits. How many people use FarmVille now?


[User] Please transcribe this text

[Assistant] Then there's other ones which look like this, where they stabilize out. So you think about Minecraft, right? Or World of Warcraft, these games that somehow have network effects and people stay in them for a long time.


[User] Please transcribe this text

[Assistant] The other version of this curve I've seen is the ones where they go down, they look like they're gonna hit the x-axis, and then they re-accelerate and they go up.


[User] Please transcribe this text

[Assistant] There's a few different things that can cause that, right? So you have this, this overwhelming thing that's driving you down to have zero retention, and somehow the numbers go up again. So things that drive that for businesses, and you actually, I think, mentioned this in, in your intro. One. Network effects.


[User] Please transcribe this text

[Assistant] So when all your friends get on Facebook, you're much more likely to use Facebook. When all your friends get on Twitter, you're more likely to use Twitter, WhatsApp, Messenger, whichever social service you're talking about. There are network effects that really matter. Network effects also count in marketplaces.


[User] Please transcribe this text

[Assistant] When you have enough sellers, you can have enough buyers. When you have enough traffic, you get more sellers. And so as marketplaces go, often what you find is these old cohorts that signed up resurrect and come back to the service. So network effects can drive the curve up after it's flatlined somewhere or is heading towards zero.


[User] Please transcribe this text

[Assistant] The second thing, uh, that can do this is opening up a new interface.


[User] Please transcribe this text

[Assistant] So think about it, a lot of people release their apps first on iOS. I actually loved Yan's talk last week for this by the point that he made of starting with Nokia focusing on Android because it's what most of the world use but in Silicon Valley we typically build our apps first on iOS more often than not and