[User] Please transcribe this text

[Assistant] Suppose I tell you no lie. Suppose I don't tell you anything.


[User] Please transcribe this text

[Assistant] You and your men are covered by Sergeant Harry Hall. Sergeant is an excellent marksman. You will find out if you don't believe me.


[User] Please transcribe this text

[Assistant] They are. They are sick. Come on. This is, look at this. Glamour.


[User] Please transcribe this text

[Assistant] Hey, this seems as good as any. Why not set fire to this coal bugger? He'd die enough to be seen, alright. And a military objective, too.


[User] Please transcribe this text

[Assistant] Editing your course videos is an important step in providing a great experience for your students. In this video, I'll teach you how to edit your videos with the simple and free video editing app iMovie. Let's get into it.


[User] Please transcribe this text

[Assistant] Hello creators, I'm Ben Tulson from Podio where we empower creators like you to make a living doing what you love. iMovie is a great video editing tool if you're just starting out and comes free with iPhone, iPad, and Mac devices. Today's video is going to focus on editing in this app on a Mac.


[User] Please transcribe this text

[Assistant] But if you have Windows, there are a couple of iMovie alternatives you can use. Movavi is free to download as a trial.


[User] Please transcribe this text

[Assistant] And like iMovie features a simple user interface. After the trial period, you can purchase the basic version for about $39. You can find it at Movavi.com.


[User] Please transcribe this text

[Assistant] It comes pre-installed on most devices and to check for it, you can either type command space bar and type iMovie or you can navigate to your applications folder and finder and search for it there. If you plan to use iMovie quite a bit, I recommend moving it to your dock.


[User] Please transcribe this text

[Assistant] If you don't have it already, you can download it at apple.com slash iMovie or from the app store. When you first open iMovie, click on the big plus sign above create new and select movie. I'm going to give you a quick tour of what you'll see on the next screen. Here we have three basic sections.


[User] Please transcribe this text

[Assistant] A preview monitor where you can see a preview of your video and the timeline where you'll be editing your video.


[User] Please transcribe this text

[Assistant] In this video, we won't get into each of the individual features and tools, but only touch on what we need for editing and saving your lesson video.


[User] Please transcribe this text

[Assistant] Adding your footage. To add your footage, you can either click on the import media button and find your file, or you can drag your file directly into the timeline from the finder window. If you use the import media button, just be sure to drag your clip down into the timeline. I'm going to use a clip from one of our Podia videos as an example.


[User] Please transcribe this text

[Assistant] Before moving on to the next step, you'll want to click settings at the upper right of the timeline and make sure show waveforms is checked. You can zoom in and out of your timeline with command plus or minus.


[User] Please transcribe this text

[Assistant] Zooming in will make it easier to make cuts, and zooming out will make it easier to navigate to different sections of your video.


[User] Please transcribe this text

[Assistant] Editing your footage. The first thing we'll do is right click the clip in your timeline and select add cross dissolve. This will add a fade transition to the beginning and to the end of your video. The previous video in this series covered recording your lesson slides and I encouraged you to clap whenever you made a mistake so it would be easy to find during the editing process.


[User] Please transcribe this text

[Assistant] If you've done that, you should be able to see a spike in your audio waveform and can navigate to those points in your clip. To edit out your mistake,


[User] Please transcribe this text

[Assistant] Click the point on the clip just after the clap and type command B. This will split the clip. Now bring the cursor to the edge of the clip and you should see a vertical line with an arrow pointing away on either side.


[User] Please transcribe this text

[Assistant] If you click, you can drag that clip to just before the beginning of the bad take.


[User] Please transcribe this text

[Assistant] The remaining clips will automatically join together and make a jump cut. That's a jump from one clip to the next with no transition. If you haven't used the clap method or if you just want to watch through the whole thing and edit as you go, you can follow the same process. You may want to do this anyway so you can remove any unwanted pauses or excessive dead air.


[User] Please transcribe this text

[Assistant] From here you can edit the title, add a video description, and even add tags. You should title your videos with the name of your lesson. For the other settings I recommend the following. Resolution, 1080p.


[User] Please transcribe this text

[Assistant] Unless your course is meant to be viewed on a larger screen, then you might want to export it in 4K.


[User] Please transcribe this text

[Assistant] For quality, choose high quality and for compression, choose better quality. Then click next. Now you're going to choose where you want to save your video file. If you don't see your folder, click the down arrow so you can navigate to the module folder that corresponds with your lesson and click save.


[User] Please transcribe this text

[Assistant] You'll see a circle appear in the upper right hand of your screen that will show you the saved progress of your video. iMovie will notify you once your video is finished rendering. And you're done!


[User] Please transcribe this text

[Assistant] I want to hear from you. What video editing software do you prefer and why? Let me know in the comments. If you found this video helpful and you want more content like this, subscribe and hit the notification bell so you don't miss any future videos. Thank you for watching and I'll see you next time.


[User] Please transcribe this text

[Assistant] Hi. Good morning. Good morning. Haven't we met somewhere before? I don't think so. I've got a wonderful memory for faces. Well, I haven't, but I think I remember yours. I mean... My name's Trenton. Yes, you're Mr. Barrington's assistant, aren't you? Yes, I am. That is laboratory. Yes, it is.


[User] Please transcribe this text

[Assistant] Will Chamberlain's size, strength, and athleticism were unimaginable. He made large men appear small. And when he played, it seemed all too easy, almost unfair.


[User] Please transcribe this text

[Assistant] A professional basketball game? In his entire career?


[User] Please transcribe this text

[Assistant] If he had been perceived as unstoppable on the hardwood, Chamberlain would also challenge the values of middle American society.


[User] Please transcribe this text

[Assistant] After being ordained a campus cult figure, Chamberlain hosted a late night radio show called Flippin' with the Dipper.


[User] Please transcribe this text

[Assistant] When he wasn't spinning 45s on the air, Wilt found refuge in the brotherhood of sports, and it wasn't always basketball.


[User] Please transcribe this text

[Assistant] Anchored by the Big Center, who KU's former coach Fogg Allen once whimsically remarked could win the national title with two cheerleaders and two Phi Beta Kappas, the Jayhawks were seven point favorites over top ranked North Carolina.


[User] Please transcribe this text

[Assistant] In 1962, the Warriors moved to San Francisco, where for the next two and a half seasons, Chamberlain State had topped the scoring and rebounding charts.


[User] Please transcribe this text

[Assistant] Meanwhile, the Syracuse Nationals moved to Philadelphia and were renamed the 76ers. In January of 1965, Wilt would resume his on-again, off-again relationship with the city of Brother


[User] Please transcribe this text

[Assistant] The germ of Wilt Chamberlain's foul-line affliction had been planted back in Kansas. A 62% free-throw shooter with the Jayhawks, Wilt, by 1968, was missing more than six of ten from the strike.


[User] Please transcribe this text

[Assistant] Against the Celtics in the 1968 Eastern Division Finals, the 76ers blew a three games to one lead. In their four point loss in game seven, Chamberlain was less than dominant, shooting four for nine from the field and six of 15 from the foul line. It was his last game as a 76er.


[User] Please transcribe this text

[Assistant] With Russell gone in 1970, the path to the NBA title was rerouted through Madison Square Garden. When Knicks center Willis Reed went down with a knee injury in Game 5, the way to glory seemed open for Wilt and his talented supporting cast.


[User] Please transcribe this text

[Assistant] Content to average just 15 points, Wilt led the league in rebounds at 19 again.


[User] Please transcribe this text

[Assistant] The Lakers soared, mounting a record 33 game winning streak and finishing with 69 victories.


[User] Please transcribe this text

[Assistant] While Wilt seemed to stay young, an old foe was closing in on a record that had seemed unreachable when he retired.


[User] Please transcribe this text

[Assistant] In 1984, Kareem Abdul-Jabbar eclipsed Chamberlain's mark to become the NBA's all time leading scorer.


[User] Please transcribe this text

[Assistant] If Chamberlain's scoring record was overtaken by Abdul-Jabbar, Wilt went one better. In his 1991 book, A View From Above, The Lifelong Bachelor claims statistical superiority in another activity.


[User] Please transcribe this text

[Assistant] The New Jersey Nets offered him about $360,000 for the final few games of the 86th season. Chamberlain had been retired for 13 years, and here was someone else agreeing with him that yes, Wilt still had game. For Sports Century, I'm Chris Fowler.


[User] Please transcribe this text

[Assistant] Will Chamberlain's reputation coming out of high school was that he was so big and so overwhelming that, uh,


[User] Please transcribe this text

[Assistant] Nobody could ever deal with him on a basketball court.


[User] Please transcribe this text

[Assistant] Wilt has always said that he had been promised a piece of ownership of the team. And if he was not going to get it, he wanted to be traded.


[User] Please transcribe this text

[Assistant] My brother joined the escaping crowd of Londoners.


[User] Please transcribe this text

[Assistant] The Martians had come from out of their pit and walking to renew their attacks on our civilization.


[User] Please transcribe this text

[Assistant] The whole day they had apparently been working on something inside the pit, sending large billows of green smoke mysteriously into the air. Finally, at about nine o'clock,


[User] Please transcribe this text

[Assistant] They finished their work and began to follow the earlier line of attackers in their horrible killing machines.


[User] Please transcribe this text

[Assistant] They attacked in a wide line, maybe a mile and a half from one another.


[User] Please transcribe this text

[Assistant] And their means of communication was a loud cry that often changed pitch.


[User] Please transcribe this text

[Assistant] The first defensive line that the machines met was made up of inexperienced volunteers who should not have been placed at the front of the battle.


[User] Please transcribe this text

[Assistant] None of the men in the unit had ever fired a gun under the threat of also being fired upon.


[User] Please transcribe this text

[Assistant] Seeing the approaching machines, they became terribly frightened and began to shoot at them too early, missing them completely.


[User] Please transcribe this text

[Assistant] The Martians now knew exactly where the unit had been hiding and turned their weapons in their direction. The entire unit began running away.


[User] Please transcribe this text

[Assistant] However, the Martians moved quickly and crushed every man under their feet.


[User] Please transcribe this text

[Assistant] This easy win allowed the Martians to surprise the next line of defense and quickly defeat them in battle as well.


[User] Please transcribe this text

[Assistant] The soldiers at St. George's Hill had a bit more success.


[User] Please transcribe this text

[Assistant] They took one Martian by surprise and blew off one of its legs, causing the machine to fall hard to the ground.


[User] Please transcribe this text

[Assistant] This brought two other machines into the area to help.


[User] Please transcribe this text

[Assistant] The soldiers tried to take them down as well, but missed.


[User] Please transcribe this text

[Assistant] This of course gave away their hiding place, and the two Martian machines attacked with incredible violence.


[User] Please transcribe this text

[Assistant] They fired their heat rays upon the poor unit and killed most of them within just a few seconds.


[User] Please transcribe this text

[Assistant] That the two fighting machines then returned to the injured one and stood on either side of it standing guard.


[User] Please transcribe this text

[Assistant] Moments later the Martian which had been operating the fallen machine came out slowly and painfully from the head.


[User] Please transcribe this text

[Assistant] It carried in its arms several tools that it then used to repair the damaged leg.


[User] Please transcribe this text

[Assistant] Within a half an hour, the fallen machine was back on its feet, walking onward with its two companions.


[User] Please transcribe this text

[Assistant] Not long after, they met up with four other Martian machines that were carrying long black tubes.


[User] Please transcribe this text

[Assistant] Soon, all seven Martian machines had tubes and they then spread out in a 12 mile long half circle.


[User] Please transcribe this text

[Assistant] Two of the machines came very near to the place where the curate and I were hiding.


[User] Please transcribe this text

[Assistant] Knowing the uselessness of running from the Martians,


[User] Please transcribe this text

[Assistant] Ran into the tall grass at the side of the road and hid in a ditch.


[User] Please transcribe this text

[Assistant] The curate noticed this and decided that my idea might be the better one. He ran back and joined me in the ditch. The two machines stood there silently.


[User] Please transcribe this text

[Assistant] One facing stains, one facing sunbury, for what seem like hours.


[User] Please transcribe this text

[Assistant] The Martians were no longer communicating with their strange musical cries.


[User] Please transcribe this text

[Assistant] Now everything was quiet as the Martian stared toward London.


[User] Please transcribe this text

[Assistant] And the military sat hiding somewhere out there with their guns at the ready. Looking at the Martians standing there it seemed that they had already won.


[User] Please transcribe this text

[Assistant] And that everything under the moon and sun was theirs.


[User] Please transcribe this text

[Assistant] They looked unbeatable in their giant metal machines.


[User] Please transcribe this text

[Assistant] I began to feel no bigger or stronger than an insect.


[User] Please transcribe this text

[Assistant] Then the sound of a large army gun broke the silence.


[User] Please transcribe this text

[Assistant] The Martian facing stains had been shot at, but was missed.


[User] Please transcribe this text

[Assistant] It then raised its long, tube-like gun,


[User] Please transcribe this text

[Assistant] And fired an incredibly loud shot into the distance. The one facing stains then did the same.


[User] Please transcribe this text

[Assistant] Each time they shot their guns, the ground shook violently. But no smoke or fire came from the tube.


[User] Please transcribe this text

[Assistant] We sat waiting for the military to respond, but all that we heard was some shouting followed by dead silence.


[User] Please transcribe this text

[Assistant] We both climbed up out of our ditch to have a look around.


[User] Please transcribe this text

[Assistant] Over the trees and to the north and east, mysterious black clouds seem to have formed.


[User] Please transcribe this text

[Assistant] The only thing we could hear were the cries of the Martians in the southeast as they communicated with one another. I learned, later on,


[User] Please transcribe this text

[Assistant] That what had happened was that the Martians fired their guns into every possible hiding place in the surrounding area.


[User] Please transcribe this text

[Assistant] Groups of trees, hilly land, and closely built buildings.


[User] Please transcribe this text

[Assistant] Their guns shot large containers that broke open when they hit the ground and released a thick black gas, which then spread quickly through the air and immediately killed anything that breathed it in.


[User] Please transcribe this text

[Assistant] Fortunately, the gas was very heavy and could only rise to a certain height before starting to fall slowly to the ground where it would lay in thick dusty clumps.


[User] Please transcribe this text

[Assistant] I say fortunately because there were some who had hidden high up in the tops of trees or on the highest floor of their home where the gas could not climb.


[User] Please transcribe this text

[Assistant] They waited there for days until the gas fell completely to the ground.


[User] Please transcribe this text

[Assistant] And the Martians came through and cleaned it up by shooting steam upon the ground with a special machine. I suppose that I should not call it cleaning up, for the black dust remained. It, however, lost its poisonous effect.


[User] Please transcribe this text

[Assistant] For the rest of the night, the Martians continued to move forward toward London in their ever-widening half-circle, shooting their containers into every possible hiding place in their path.


[User] Please transcribe this text

[Assistant] Not a single soldier or citizen that had sat waiting for the Martians between St. George's Hill and London survived that night.


[User] Please transcribe this text

[Assistant] Before a sunrise, the black gas had already arrived in Richmond.


[User] Please transcribe this text

[Assistant] The leaders in London responded by ordering its citizens to leave the city as quickly as they could, for the Martians would soon be there.


[User] Please transcribe this text

[Assistant] Who elected you leader of this outfit? Well, Pete, I figured it should be the one with the capacity for abstract thought, but if that ain't the consensus view, then hell, let's put her to a vote. Suits me. I'm voting for yours truly. Well, I'm voting for yours truly too.


[User] Please transcribe this text

[Assistant] It's fun to be, uh, in a cross-disciplinary group. So basically, I mostly talk to economists and sociologists, and then to talk to people that often are subjects of our studies is actually really, really fun. Um, uh, Stanford, this is a personal, Stanford is near and dear to my heart.


[User] Please transcribe this text

[Assistant] My mother in the 70s left Pakistan to come to Stanford to do a master's degree, uh, and that enabled us to immigrate in the early 80s to Canada, uh, and it was really the Stanford education that really set us up for where we are today as a family, um, and it's always had a soft spot in our


[User] Please transcribe this text

[Assistant] Uh, and of course, uh, when I was considering graduate school, Stanford was at the top of my list.


[User] Please transcribe this text

[Assistant] Uh, and somehow MIT kept looting over every single time. But I'm super happy, uh, to be here and to present, uh, this work.


[User] Please transcribe this text

[Assistant] Uhm, this work is, uh, in collaboration with Kevin Boudreau, uh, my good friend and co-author at the London Business School. Uhm, we went to MIT together as, as PhD students, uh, and have, uh, started to work together on a bunch of these kinds of questions.


[User] Please transcribe this text

[Assistant] Uhm, and so what I'm going to talk about, you saw the title, I keep changing the title around to make it more simpler. Uh, so this is the new title. Uh, you know, how disclosure policies impact search and open innovation. I'm going to explain to you all of this as we move forward. Uh, I just want to give you a sense of my research and how it all fits in.


[User] Please transcribe this text

[Assistant] I've been really been thinking a lot about uhm, you know, how crowds get organized and how we can get productive work from crowds.


[User] Please transcribe this text

[Assistant] Uhm, and there are really two major institutions that we see out in the economy that enable us to work with crowds. The first setup is competitions and contests.


[User] Please transcribe this text

[Assistant] Uhm, and in this, in this work, I've done a bunch of work to sort of explain


[User] Please transcribe this text

[Assistant] Who wins and why, you know, what are the basis for, uh, for performance in these settings, why people participate, uhm, and behavior in contests. And I've done both research using, uhm, archival data, uh, looking at InnoCentive and TopCoder, uh, to pull out these results, uh, and also field experiments. And we'll walk you through a field experiment, uh, soon.


[User] Please transcribe this text

[Assistant] Uh, the other way to set this up is actually through a community. So everybody's aware of the open source software world and what that's done.


[User] Please transcribe this text

[Assistant] Uhm, and I've in fact started my research first there to say, you know, how does open source work and why does that actually produce such robust quality software? Uhm, and again, I've looked at issues about motivation, self-selection and participation and, uh, and now doing increasingly more work.


[User] Please transcribe this text

[Assistant] Are diametrically opposed if you sort of think about it. In the one case, you're basically running a winner-take-all contest where people are competing and kind of rushing to solve a problem.


[User] Please transcribe this text

[Assistant] Uhm, and the other setting, uhm, you have communities. People are sharing knowledge, you know, they're learning from each other and they're collectively improving, uh, the, uh, uh, the, the overall perspective of the code being produced and the overall performance of the code being produced.


[User] Please transcribe this text

[Assistant] And for me, this has always been a puzzle in terms of what's the better way to organize ourselves? Should we, when we have a creative task, when we have a software programming task, when we have a scientific task, is it better for us to set up ourselves in communities and collaborations, or is it better for us to set up contests and competitions?


[User] Please transcribe this text

[Assistant] In many ways, I started here, this I was actually kind of allergic to, uh, when I first started paying attention to this, uh, during my dissertation time, because I just, you know, I drank the open source cool, and I just felt like that was just a great way to organize work.


[User] Please transcribe this text

[Assistant] But as these became more and more popular and powerful, it required attention to say, like, what's going on here and why are these equally as effective as this? And that's what the puzzle I'm gonna try to solve over the next, uh, time that, time we have.


[User] Please transcribe this text

[Assistant] The work I do is part of what's called the NASA Tournament Lab. It's a research lab funded by NASA to help us, uh, uh, understand how these types of open innovation perspectives can be put to use for government agencies but for innovation in general.


[User] Please transcribe this text

[Assistant] Uhm, and so what we do is we work with NASA, we take their toughest computational challenges, and then we crowdsource them through platforms like TopCoder.


[User] Please transcribe this text

[Assistant] I've also done a bunch of work with our medical school, uh, in the computational biology side of things. We're again saying that, in fact, we can take, uh, computational biology questions and also get them solved through contests in communities.


[User] Please transcribe this text

[Assistant] And in addition, uhm, a bunch of work. Let me just, hang on a second.


[User] Please transcribe this text

[Assistant] Uh, and, and a bunch of work, um, around how we actually organize collaborations in the medical school setting. So if you look at the Harvard Medical School, you know, we have, uh, 20,000 researchers, about 1.5 billion dollars in funding from the NIH, and, you know, three campuses that are far apart from each other.


[User] Please transcribe this text

[Assistant] How we find collaborators and partners is actually a big, big question here. And that's part of our research agenda that we also, uh, pursue. So let me give you an example first of a, of a contest we recently ran, uh, for NASA, uh, uh, for the space station. So let's have a look.


[User] Please transcribe this text

[Assistant] This ran in March, uh, you know, 30k, uh, pool of cash available, uh, to solve this problem. It was a three week long contest. Uhm, here's what we got.


[User] Please transcribe this text

[Assistant] Uh, we had broad engagement, uh, you know, four hundred and fifty-nine competitors submitting about, about two thousand code submissions. We had actually had two thousand sign-ups. Uh, it was quite incredible. NASA can be a big draw for people to participate. Uh, the winners are here, uh, so Italian, Chinese, Chinese, Polish, uh, Chinese, uh,


[User] Please transcribe this text

[Assistant] Of course NASA cares about, like, how good is the code, right? Does it actually do anything at all? Uhm, and what we have here for you is the final output of the code submissions that they had for how much power output you can get based on the configurations, uh, that they predicted.


[User] Please transcribe this text

[Assistant] Uh, and this is sort of, you know, solar power output, and then this is every single person that had submitted in their final submissions.


[User] Please transcribe this text

[Assistant] Now, some things you'll see immediately is that there's actually quite a bit of bunching, right? People have done really well. Uh, and in many ways, sort of the, the differences between the first and second are not that great. But then, here's an answer solution.


[User] Please transcribe this text

[Assistant] Okay, and depending upon how you model the NASA solution versus our solutions, you know, we either exceed or meet what they've done.


[User] Please transcribe this text

[Assistant] Alright, so in what was, um, you know, multi-year, you know, multi-X doll, millions of dollars of projects that they do to get this kind of information out, uhm, we were able to basically get it done in three weeks for 30k.


[User] Please transcribe this text

[Assistant] Uhm, and this becomes a very compelling way for us to think about organizing creative work if we can actually take


[User] Please transcribe this text

[Assistant] These very tough software problems that require lots of people, uh, and lots of attention and say, can we actually get it done this way?


[User] Please transcribe this text

[Assistant] So, so in fact, that's a good question. Uh, the price size is in fact, uh, uh,


[User] Please transcribe this text

[Assistant] What we're finding is that too high a prize, and that will actually deter entry. Because people think the problem is too hard.


[User] Please transcribe this text

[Assistant] Uh, and, uh, but you want to actually reward people for, uh, for their, their efforts as well. So we've been typically in these types of projects in the 10 to 50K depending upon the problem and what we're trying to get done on the, on the backend side.


[User] Please transcribe this text

[Assistant] In terms of the reward size, that's on the high end for top coder. Top coder projects are typically much smaller than that. They'll be like $500 or $1500. But those are for more software components. These are longer term projects, 3 weeks, 4 weeks, and so on.


[User] Please transcribe this text

[Assistant] And, uh, of course, as we'll discuss later, uh, you know, beyond just the prize money here, you know, having Get Your Code assessed by NASA, having your name on the wall.


[User] Please transcribe this text

[Assistant] We hear, you know, there's a deadline effect, so people, you know, like everybody here, you know, works through the deadline, so right, you know, the night before, there's a ton of stuff going on. Uhm, but, uhm, but there's also, uh,


[User] Please transcribe this text

[Assistant] So I think that's an open question for us. Uhm, you know, there's a company called Local Motors which is building cars through crowds. It's a car design.


[User] Please transcribe this text

[Assistant] And so on. So the design that you saw of the space station, they're actually sourcing from


[User] Please transcribe this text

[Assistant] Cars from the crowd as well. So I think, I think we're.


[User] Please transcribe this text

[Assistant] For sure, when it comes to information products, we have seen crowds perform amazingly well. From Wikipedia to open source to this kind of stuff.


[User] Please transcribe this text

[Assistant] What's happening, what's interesting is that the whole world is going digital, the whole world is becoming an information product, right? Any physical object can be represented as, uh, you know, with this information shadow, uh, and then we can then put it on the computer and let all you guys go crazy at it.


[User] Please transcribe this text

[Assistant] This is actually what's so interesting that in fact as we, as the entire world gets digital and cyberspace is actual real space, then these approaches are becoming more and more, uh, effective I think.


[User] Please transcribe this text

[Assistant] In this contest, we had those ten winners, uh, but again, sometimes it's two or five, depending upon what we're trying to encourage here as well.


[User] Please transcribe this text

[Assistant] Yeah, I think, I think there's a, there's some work we're doing now and saying, what happens if we actually limited the transparency of the signal being generated? Because right now, this is a public signal, right? Everybody sees your performance, everybody sees how good you are, right? What happens if you actually limited that signal, so only you had it, and then you could then keep, and you had the option to turn that on or off?


[User] Please transcribe this text

[Assistant] Uh, and that, that may in fact have a, have some very interesting consequences. Absolutely. Great questions.


[User] Please transcribe this text

[Assistant] Right, so let me move on, uhm, and walk you through the paper in a quick nutshell and then we'll get into all the gory details. Uhm, so uhm,


[User] Please transcribe this text

[Assistant] I'm going to take a very broad view. When we think about innovation systems, you know, we think about the patent system, academic science, we think of open source. These are all what society has invented over time. And these are all systems designed to both encourage innovation, we want innovation to happen, so we create incentives.


[User] Please transcribe this text

[Assistant] So these two components are part and parcel of how we think about innovation.


[User] Please transcribe this text

[Assistant] And what we ascertain and we'll talk about is that what makes them different is really the timing of disclosure. So all innovation systems are in fact open. So even if you think about patents,


[User] Please transcribe this text

[Assistant] Patents are open because you have to publicly disclose your invention and how it got done. You just have monopoly rights over what happens to its use later on, right? Similarly, in science, open science, right, you publish a paper.


[User] Please transcribe this text

[Assistant] That's what gets disclosed and that's what gives you accolades. And so the timing is what, what, what, what makes these things different instead of just, uh, sort of this battle is about open versus closed.


[User] Please transcribe this text

[Assistant] And what we show in this paper is that a system like open source where there's lots of intermediate disclosures, we're sharing code snippets, we're sharing


[User] Please transcribe this text

[Assistant] Intermediate advances with each other actually dappens incentives and participation. So we get kind of what I expected, get lower participation, lower effort in these systems, yet we get higher performance. And we'll try to explain why that is going on.


[User] Please transcribe this text

[Assistant] Uh, and we in fact see more experimentation, right, and more exploration in closed systems. And we'll explain why that may be going on.


[User] Please transcribe this text

[Assistant] And the argument that we make is that there's a real convergence of technical paths that people follow in open systems in intermediate disclosure systems that enables them to get high performance. But this has implications for how we think about the knowledge stocks that are available to people and who's participating in the, in these systems. You're talking here about a competitive environment?


[User] Please transcribe this text

[Assistant] What differentiates open source from open science per se or from closed development is the regime of intermediate disclosure as one element, right? That sort of says we can share what we've done right away with everybody else.


[User] Please transcribe this text

[Assistant] And, and, and other people can build on that. So we use that as our, sort of our treatment. Say, does that, how does that in fact happen to, uh, work out in a system where people are, are able to use anybody else's code and still retain some kind of rewards at the end.


[User] Please transcribe this text

[Assistant] Yeah, absolutely, because it's continuing to be built upon people are disclosing different things. Although it's not an innovation task per se, you know, they're just creating content, but that would be the same line. Absolutely. Okay. So uhm,


[User] Please transcribe this text

[Assistant] Uhm, so uhm, let's just talk about these dual objectives for innovation systems. So the first is you want to offer, uhm, incentives for inventive effort.


[User] Please transcribe this text

[Assistant] So patents allow monopoly rights or use rights. Academia is based on funding, promotion, awards, and honor. Uhm, and open source, there's use benefits, there's reputation, intrinsic rewards, and so on that come into it. All these elements are basically ways for society to sort of say, we want to reward inventive effort. At the same time, right, we want to ensure


[User] Please transcribe this text

[Assistant] Inventions like the Google Glass get disclosed, right, that in fact people know how these things work and that other people can build upon that.


[User] Please transcribe this text

[Assistant] And that's the, that's the, that's the essential trade-off that, that society makes. We want incentives for invention, but we also want disclosure.


[User] Please transcribe this text

[Assistant] Uh, so, you know, patents, of course, you make knowledge, uh, in public domain. In academia, you're publishing or perishing. You've got to publish, right? If you don't publish, you don't exist as an academia. You want to disclose what you've discovered. And open source is sort of enshrined in the process. People are always.


[User] Please transcribe this text

[Assistant] Making disclosures, and it can also be in the license itself.


[User] Please transcribe this text

[Assistant] So these are the dual goals of, uh, of, uh, of all systems. And there's, we think there's a false debate between sort of closed and open innovation. Uhm, it's really, all innovation systems are geared to be open in the end.


[User] Please transcribe this text

[Assistant] Right, it's just a matter of when you enforce disclosures and what timing it is and what form they take.


[User] Please transcribe this text

[Assistant] But everything that society does beyond, I think, trade secrets is geared towards openness.


[User] Please transcribe this text

[Assistant] One format is sort of this final disclosure, right? So if you think about patents, a completed invention is put out there, right, for other people to consume later on. In academia, it's a published paper.


[User] Please transcribe this text

[Assistant] Right? We submit a paper, we do the experiment, we do the work, and we publish a paper. And in prize contests, some technology is developed that becomes available for other people to use. So we call that sort of final disclosure, which later allows people to then build upon and so forth. But the process is, is secret, uh, and not available for the people to participate within.


[User] Please transcribe this text

[Assistant] Uhm, in the world of, uhm, in immediate disclosure, we actually have the setting where, uh, where we're constantly making available our advances to everybody else.


[User] Please transcribe this text

[Assistant] And a great example actually was the Human Genome Project. Uh, the Human Genome Project in the 90s, basically, uh, once they, once they were under attack from Solera actually, in the race to sequence the human genome, they actually came up with this radical approach called the Bermuda Principles. They were in Bermuda for a conference, you know, great place. Uhm, and they said,


[User] Please transcribe this text

[Assistant] Any discovery that's made by any lab, uh, as they're sequencing the human genome will be made immediately available to everybody else.


[User] Please transcribe this text

[Assistant] Publicly disclosed, right? This violates the most principles of academic science because we, we keep our data secret, we keep our experiments secret, and we have a race for priority to publish our papers. And here they said, no, in fact, we're going to make everything available very quickly so people can then actually coordinate their activities and go after things that haven't been done before.


[User] Please transcribe this text

[Assistant] Uh, of course in open source and the world that we live in, you know, there's code fragments, advice and tips on problem solving, complete code, everything is being continuously being disclosed.


[User] Please transcribe this text

[Assistant] And you know, for me, it's, I was talking to some, uh, some of the students here before, uh,


[User] Please transcribe this text

[Assistant] You know, having been at the start of the open source movement when this stuff was taking hold, uhm,


[User] Please transcribe this text

[Assistant] So our, our experience now of going to Stack Exchange and asking somebody for help and, and reading their solutions.


[User] Please transcribe this text

[Assistant] Was not the default, right? Before people didn't have those things available. We might have Usenet available, if you guys know what Usenet is, as a solution. Uhm, but that was not common. So all these things are in fact new in terms of behavior change in society in terms of us being able to live with this.


[User] Please transcribe this text

[Assistant] Uhm, and certainly in the world of biology, there's biological repositories. You can actually get microorganisms and mice, uh, you know, mice have code in them that, that people can share and use for their own experiments as well. Uh, but here we're actually again making intermediate products available for people to reuse.


[User] Please transcribe this text

[Assistant] You know what's interesting is that this is also a historically important process. So lots of studies now in the history of technology showing that, you know, the airplane, uh, Bessemer steel, uh, you know, uh, blast furnace technology, all those


[User] Please transcribe this text

[Assistant] were based upon systems of collective invention where lots of people were sharing knowledge throughout.


[User] Please transcribe this text

[Assistant] Teams of engineers, teams, Allah, distributed teams of people were sharing knowledge back and forth.


[User] Please transcribe this text

[Assistant] And in fact, uh, some work by Meyer in fact shows that all the, uh, the U.S. system was actually very open and open to sharing lots of knowledge flows happening in the U.S.


[User] Please transcribe this text

[Assistant] Once, uh, the Wright brothers came up with their invention, they were parts of the, uh, they were participants in the system. Once they came up with their invention, they actually started to patent.


[User] Please transcribe this text

[Assistant] Uh, and the, and the, uh, the, the, the, the, the, the, the locus of activity for aircraft in mission then moved over to France where there were no patents on aircraft at that time.


[User] Please transcribe this text

[Assistant] So if you think about words like aérolion or fuselage, those are all French words because that's where all those developments were being done. Uh, and, and in the U.S. actually had a, had a, actually a dip in, in, uh, in inventive output for aircraft for a while. Uh, but, but, but this is the world that, that, that exists.


[User] Please transcribe this text

[Assistant] Uh, and so we have, within this setup, this incentives versus reuse trade-off.


[User] Please transcribe this text

[Assistant] Uh, within, uh, our, our invention systems.


[User] Please transcribe this text

[Assistant] Uh, the, the worry is, in the economics literature, is that as you bring in greater disclosures, right, you will degrade incentives. People will not.


[User] Please transcribe this text

[Assistant] And what's happened, uh, in our society is that we create compensating mechanisms for greater disclosure. So we have things like priority and citations in academia that are worth something that we can use.


[User] Please transcribe this text

[Assistant] And signing an authorship in, in, in, in open source which says that there will be other rewards for participation other than direct use rights or monopoly rights that could be useful as well. And then also we may actually attract different people with different ethos for sharing, uh, and so forth as well.


[User] Please transcribe this text

[Assistant] And so that's how society sort of dealt with this, uh, with this perspective.


[User] Please transcribe this text

[Assistant] Uhm, but incentives matter a lot. So we're in the Gates building, so I thought I'd use a quote from Bill Gates. So, you know, uh, in 1976, right, he writes basically a letter to hackers, right, saying incentives matter.


[User] Please transcribe this text

[Assistant] Incentives matter a lot, and he built Microsoft, right, on this comp, uh, on this basis that, that proprietary incentives matter a lot.


[User] Please transcribe this text

[Assistant] And, you know, we're the benefactories of that, of that, of this letter and what he was able to do in terms of innovation. But there's also the Torvalds perspective.


[User] Please transcribe this text

[Assistant] I like feedback on things. This is a program for hackers by hackers. I enjoy doing it. Somebody might enjoy looking at it. Drop me a line if you want to let me use your code.


[User] Please transcribe this text

[Assistant] Right, so he's definitely on the, you know, compensating mechanisms for incentives and about sharing and reuse.


[User] Please transcribe this text

[Assistant] And we are living now in this world of both of these institutions, at least in the world of software, existing, coexisting, and people going back and forth often times between them.


[User] Please transcribe this text

[Assistant] Alright, so this debate is not just theoretical, this is real. People make choices, make investments based on where they end up on this, on this debate.


[User] Please transcribe this text

[Assistant] Uhm, what we know from research and economics is that more open, more access equals increased reuse. So the whole world of user innovation.


[User] Please transcribe this text

[Assistant] Have shown that, that, that people, once you make technology available, people will reuse them and share them and keep, keep using them over and over again. Uh, of course platform encourages reuse of components, you disclose those components and then platforms grow and become really powerful because we've got those components available.


[User] Please transcribe this text

[Assistant] Uh, and academic science has shown that basically, uh, future citation rates are a function of reuse rights. The more reuse rights you provide to people, the higher their reuse becomes. So it's a very obvious kind of a finding that we would sort of expect. Uhm, but what's missing is the counterfactual. What, what we don't know is what happens when both the systems are in play and how do people actually work out.


[User] Please transcribe this text

[Assistant] So the perspective I'm going to bring is that while we have a view about innovation in terms of incentives,


[User] Please transcribe this text

[Assistant] And reuse, we also need to think about innovation as a search process. You're searching for the best solution.


[User] Please transcribe this text

[Assistant] And it's not just a production function thing where we think about incentives and outputs and efforts, but we actually want to think about this as a search process. And there's a, there's a broad literature, uh, in economics, in management that thinks about innovation as a search process.


[User] Please transcribe this text

[Assistant] And if you think about the search process, uhm, what, what you want to think about is you're, there's some kind of a landscape on which you're searching for the best solution.


[User] Please transcribe this text

[Assistant] And that landscape may have peaks, uh, and valleys and you want to find the highest peak along the way. Uhm, yes?


[User] Please transcribe this text

[Assistant] Yeah, that's a deeply philosophical question. So the question is do we invent the solution landscapes as we do the problem solving or do they already exist?


[User] Please transcribe this text

[Assistant] Invention, sort of indulgence invention of.


[User] Please transcribe this text

[Assistant] I think, I think it happens for sure, but I don't think we have an apparatus yet.


[User] Please transcribe this text

[Assistant] And the innovation literature to actually deal with that, uh, and to think about how that may come through.


[User] Please transcribe this text

[Assistant] Exactly. No, and I think, you know, in my other work on contests, we actually think that defining the problem is actually a big part of what makes, what drives success.


[User] Please transcribe this text

[Assistant] Uh, and so then there's some, some effort in saying we actually need problem designers, right, not problem solvers, right, because if, if crowds can solve any problem, right, then we need actually really good problems to give them.


[User] Please transcribe this text

[Assistant] And that, that's a whole different scale. And I think the D schools are the ones that are probably pushing the boundaries on that. I don't think, you know, management schools are anywhere near, near that perspective. Absolutely.


[User] Please transcribe this text

[Assistant] Yeah, so I'm gonna show you a field experiment where we made it super easy, but the practicality is that it's actually really, there are transactions cost in sharing. Alright, we have tools available now for us to share more easily and more broadly, but it still is effort. So I'm gonna try to take care of that to the experimental side, but, you know, that's, that's what, that's what experiments allow us to do.


[User] Please transcribe this text

[Assistant] Alright, so uhm, and so the, the, the element to sort of think about when we think about innovation as search is that the direction and pathway of research activity is as important as


[User] Please transcribe this text

[Assistant] So this is exactly the point you made, which is like, you know.


[User] Please transcribe this text

[Assistant] Uh, which, which direction are we gonna go after? Are we gonna go after sort of a low-end cell phone that lasts for 30 days, right? Or one that has nine hours of life and then you're like scrambling to find the power. Those are choices that people make and those choices can either be correlated or they can be independent. And that's the point we're gonna try to address here, uh, going forward.


[User] Please transcribe this text

[Assistant] And, you know, there's again a big literature about innovation happening along pathways, trajectories, and paradigms.


[User] Please transcribe this text

[Assistant] And, uh, the thing to keep in mind is that


[User] Please transcribe this text

[Assistant] As we think about disclosure, right, we may in fact get either parallel independent paths or we may get convergent paths. And this is the big thing that we're going to try to unpack in our, in our work.


[User] Please transcribe this text

[Assistant] Uh, so there's been a whole bunch of formal mathematical models of search based on Simon's work and Kaufman's NK models. Uhm, and what we have shown, uh, the literature has shown is that in fact, you know, multiple independent search trajectories parallel search is important to find high value solutions to problems.


[User] Please transcribe this text

[Assistant] Uhm, at the same time, we know from this literature that communities of innovators tend to converge on patterns.


[User] Please transcribe this text

[Assistant] Alright, so we need to be able to ascertain between them and we want to think about concerns about local versus global optimal. Alright, so uhm, what do we do?


[User] Please transcribe this text

[Assistant] I've got a bunch of predictions that are kind of wordy. I just want to show you what we thought about in the paper and then we'll get in the data. The first prediction we made is that in fact, in a, in a, in a,


[User] Please transcribe this text

[Assistant] In a, in a regime where you have intimate disclosure, we'll actually see incentives drop. People will not work as hard, will not participate as much.


[User] Please transcribe this text

[Assistant] As compared to the regime where there's no intermediate disclosure. Uhm, we, our second prediction from the literature is that in fact we see, uh, you know, much more experimentation in closed regimes than in open regimes.


[User] Please transcribe this text

[Assistant] And the third prediction is that in the open regimes, we see a lot of convergence.


[User] Please transcribe this text

[Assistant] And the convergence and the performance will depend upon the stock of knowledge that already exists.


[User] Please transcribe this text

[Assistant] Alright, and so we'll try to show how these all three elements work for you.


[User] Please transcribe this text

[Assistant] It's actually tough for us to, uh, uh, to do this in the natural world because it's hard for us to compare open source to open science. They're both open in wording, but very different sets of people, very different sets of outputs. It's hard to compare across them. So going to a field experimental setting is better because then we can control a bunch of things along the way. Uh, and what we try to see is,


[User] Please transcribe this text

[Assistant] We offered like 6,000 bucks, uh, and two weeks, uhm.


[User] Please transcribe this text

[Assistant] And, uh, we took a problem from Harvard Medical School, uh, uh, sort of a genomics problem, uh, to solve this, uh, to make this challenge happen.


[User] Please transcribe this text

[Assistant] And what we can do is we can, uh, distinguish between technical approaches, uh, and we have lots of micro data on entry, effort, their ability to peak on other people's solutions and individual level data about who, who these people are.


[User] Please transcribe this text

[Assistant] So what's the problem? So basically, uhm, you know, in genomics, one big issue is sequence alignment. You get lots of reads of sequences and then you want to be able to...


[User] Please transcribe this text

[Assistant] Uh, uh, and you saw the details of the contest. So what did we set up?


[User] Please transcribe this text

[Assistant] We had, in the, uh, we had analysis on Topcoder, uhm, and we have 733 people sign up, say, this looks like an interesting challenge for me to work on, I'm gonna participate.


[User] Please transcribe this text

[Assistant] And what we do then is we, uh, we have great data on skills. I'll show you in a minute what the skills data we have. Because skills are actually a really important control you want to have in any kind of analysis of crowdsourcing. Because it could just be that we're attracting the really top skilled guys and that explains the performance dimensions right away.


[User] Please transcribe this text

[Assistant] So we work really hard to make sure that we can act in fact control for skills in all of our analysis.


[User] Please transcribe this text

[Assistant] We took these individuals, right, we rank-ordered them by skill and then we took triplets of closely tied folks and assigned them randomly to, you know, an intermediate disclosure regime, a mixed regime, and a final disclosure regime, which I'll explain to you, right. And so basically we had about 244, 245 people assigned to these regimes and then we had prized amounts here as well for getting the problem solved in week one and week two. In the intermediate disclosure regime,


[User] Please transcribe this text

[Assistant] What happened was, you got the problem statement and.


[User] Please transcribe this text

[Assistant] Basically, in order for you to know how good your solution is, you have to submit the solution back to the system.


[User] Please transcribe this text

[Assistant] And as soon as you submitted that solution back to the system, it made that code available to everybody else to use.


[User] Please transcribe this text

[Assistant] Okay, so you got a score, but then your code, anybody else can look at and basically, uh, reuse as they want it to. We built in a citation system to allow for sharing of rewards, but we actually basically had that set up.


[User] Please transcribe this text

[Assistant] Right, you submitted your code, you got a score, people saw your score, and that's it, uh, and then you can keep moving on. And then we set up a mixed regime as a control to allow us to do one week of closed and one week of open to allow us to see robustness testing.


[User] Please transcribe this text

[Assistant] Alright, uhm, and the, the, the disclosure regime was really one of click. So we just made it super easy, right? You just click on the thing and you got the entire code base. Uhm, and you could then see it's evolution.


[User] Please transcribe this text

[Assistant] And then we, and what we do is, you know, we hold constant the problem, the assignments, the interface, the production technology, the time, the market signal, the top total person, the top pipe.


[User] Please transcribe this text

[Assistant] Uhm, in the, uh, in the disclosure regime they could, they could chat, uh, in the other regime they couldn't.


[User] Please transcribe this text

[Assistant] So, so you can imagine, right, what's going on. This is exactly the underlying question we have. Like if you implement a disclosure regime like this, where it's available right away to everybody else, what are the incentives going to be?


[User] Please transcribe this text

[Assistant] So uhm, just in terms of the overall performance of the system, it actually rocked quite good. So let me just show you. So again, so we had 733 signups, 122 people actually submitted code.


[User] Please transcribe this text

[Assistant] We uncovered 89 different approaches to solve the problem. Okay, so not only do we get high numbers, but we get diversity, which is actually really important for innovation. Uhm, winners from Russia, France, Egypt, Belgium, and US, right? People that, you know, the medical school didn't know existed and could never even hire.


[User] Please transcribe this text

[Assistant] So pretty awesome results. But let me show you the results graphically. So our, our, our scoring function had two, two elements. One was time. How fast were you able to do this? And the second was accuracy.


[User] Please transcribe this text

[Assistant] 0.8 is the directly accurate, uhm, the directly maximum accuracy you can get from this data set. Alright. Uh, these are all the submissions that we got from our, uh, from our folks, the green, the green dots, and this is a log scale. Alright. So, you know, we want to be here basically. Uh, this is the NIH Megablast solution.


[User] Please transcribe this text

[Assistant] And this is the code from, uh, the Harvard researcher who volunteered to give us a problem. So, you know, he's no slouch, he's a good colleague of mine, you know, undergrad from MIT, PhD from Oxford, you know, is in, at Beth Israel as a pathologist and a computational biologist and has a lab working on this problem. Right?


[User] Please transcribe this text

[Assistant] And oh, by the way, none of these guys have any background in competition biology. Right? Because what we did is we took the problem.


[User] Please transcribe this text

[Assistant] Okay, so who signed up? So again, we had all these people, uh, uh, 733 people show up from 69 different countries. Okay.


[User] Please transcribe this text

[Assistant] Uh, 44% of them were professionals, and 55% were students, other PhD, or masters, or undergraduate programs. So a lot of these guys here maybe show up. What's that?


[User] Please transcribe this text

[Assistant] And looks at the history of competition and based on that gives them, uh, a metric of skill. I think I might even have it. Yeah, right here. Uh, so we can see here this guy, uh, he's from Brazil, um, uh, his, he's a red-rated coder and you can see his performance in algorithmic matches over time and he's, he's, he's here.


[User] Please transcribe this text

[Assistant] So really on the high end of the distribution in terms of performance.


[User] Please transcribe this text

[Assistant] So this, this, this is generated, uhm, ELO is a way that chess grandmasters get evaluated against each other, and they've implemented the same system for, uhm, for, for programmers.


[User] Please transcribe this text

[Assistant] What, what do we see first? I'm just gonna show you some pictures to sort of drive home the point. There's no fancy econometrics here at all, alright? So one is lower participation overall, right, in the intermediate disclosure regime, right? So you can see both in terms of number of active entrants and the cumulative number of entrants, you know, who, who shows up.


[User] Please transcribe this text

[Assistant] Right? And there's a, and I'll show you some more results in a second. I'll show you some p-values. But just from this you can start to see.


[User] Please transcribe this text

[Assistant] You know, that incentives matter, like your question, and as soon as you're in a regime where, uh, for the same sets of people where you change,


[User] Please transcribe this text

[Assistant] When you bring in more disclosure, you will have actually less entry. And again, this was contrary to my bias. My bias was that open more participation. But in fact, we're seeing the incentive story kick in here.


[User] Please transcribe this text

[Assistant] The second thing is, hours work as well, uh, uh, and this is by skill, uh, you know, big difference, uh, pretty much across the distribution in terms of, uh, of people, uh, uh, working on the problems. Alright? So, few people enter, they exert less effort.


[User] Please transcribe this text

[Assistant] Uhm, and overall what we find is, you know, participation rates are about 26% lower in an intermediate, number of submissions are 56% lower, right, and hours work are about 30% lower. Okay, overall.


[User] Please transcribe this text

[Assistant] So, people are putting in time, but still, it's quite, it's quite, quite, uh, quite important to sort of see at these, at these regimes. So when we see open source and this great mobilization of open source, there's gotta be other things going on.


[User] Please transcribe this text

[Assistant] Beyond just the real incentive effect of participation, yeaah.


[User] Please transcribe this text

[Assistant] Right? In two ways. One, you allow for reuse. Right? And secondly, you actually, because we have this dropout. Right?


[User] Please transcribe this text

[Assistant] People are like, are being allocated in the economy better, right? Because I see you participating and go, he's a hot shot.


[User] Please transcribe this text

[Assistant] I'm not even going to show up because there's no way I'm ever going to be able to, you know, uh, to, to improve upon what you've done. So that may also be going on.


[User] Please transcribe this text

[Assistant] Alright, uhm, and then also the problem solving performance, uh, was higher in intermediate.


[User] Please transcribe this text

[Assistant] Because of this reuse effect, which I'll show you, then, uh, then in the final. And average score was 1.6 points higher, controlling for skill and effort. So we basically have a whole lot of controls here in the regression that allows us to basically show that 1.6 actually makes a difference between being top or not.


[User] Please transcribe this text

[Assistant] Uh, uh, uh, overall. So this is, this is quite, uh, quite, quite interesting for us to see as well. Alright.


[User] Please transcribe this text

[Assistant] So you see a story coming together about what's happening in a world of disclosure. And the question you have, when you're in a world of being able to look at other people's code,


[User] Please transcribe this text

[Assistant] People are peeking a lot. People are looking at code and learning from it and in fact doing it. So we have about, about, uh, uh,


[User] Please transcribe this text

[Assistant] I believe, uh, close to, uh, a thousand, uh, uh, thousand incidences of people looking at, uh, because we could see when they clicked on the, on the, on the code here and then in the mixed regime,


[User] Please transcribe this text

[Assistant] You know, it's flat, of course in the first week and the second week it again takes off.


[User] Please transcribe this text

[Assistant] Most of the peaking occurred in the first week where people were looking at the code bases and trying to understand what the best paths were.


[User] Please transcribe this text

[Assistant] Okay, so what explains these results? How do we actually make sense of these results?


[User] Please transcribe this text

[Assistant] So the first thing we want to understand is how do we actually discern the solution paths, right? How do we actually figure this out?


[User] Please transcribe this text

[Assistant] So what we had was we got a top coder participant.


[User] Please transcribe this text

[Assistant] Look through the code bases and see what were the elemental techniques being used by the programmers themselves. So he made an inventory of basically sampling a whole bunch of submissions across the distribution and saying what's being used. And he came up with all these different approaches. So basically 10 distinct elemental approaches being used.


[User] Please transcribe this text

[Assistant] Uhm, and then we had other top coder PhDs, uh, you know, one German and two Chinese to go through every submission.


[User] Please transcribe this text

[Assistant] Right, and say, you know, are they using this or not along the way?


[User] Please transcribe this text

[Assistant] And then what we do is we do, solutions are a combination of techniques, right? So it's basically a bit, bit switch 1 0 1 1 1 of the ten different ways to do that. And you can then look at how unique are those, are those approaches being put together.


[User] Please transcribe this text

[Assistant] So it's a manual effort, we weren't smart enough to sort of somehow computerize this effort, uh, uh, but it, it got us a really high level of fidelity, uh, to, to, to, to, to do this. So basically you, you sort of see,


[User] Please transcribe this text

[Assistant] You know, you see here, this was our, uh, uh, sequence alignment, uh, uh, pro-project. This is the coder. This is a submission, and you can see his code, and then you can see what's, what's going on in this code. Alright. So, uh, paths are consequential.


[User] Please transcribe this text

[Assistant] So what I'm showing you are two graphics. This is performance versus combination of approaches. And we're just basically rank order them.


[User] Please transcribe this text

[Assistant] The second thing we see is that the number of elemental techniques you use to solve a problem are in fact consequential as well. So the more techniques you use, the higher the average score. Okay? So both things matter.


[User] Please transcribe this text

[Assistant] And what's interesting as a counterfactual for us is that when we look at what Megablast had done or what, uh, uhm, you know, uh, the Harvard research had done, we were re- really like orders of magnitude more than what, what they had done in terms of the approaches they had considered. And here we see the entire landscape, basically.


[User] Please transcribe this text

[Assistant] The second thing, I just want you to sort of look at this, sort of this picture and say, the search patterns are very different. Right?


[User] Please transcribe this text

[Assistant] So in the, in the world of no-intermediate disclosures, you basically have this gyration going on. People are going up and down, searching up and down, up and down, up and down, right? Uh, versus here, it's very much sort of a smooth progression to the top. Alright?


[User] Please transcribe this text

[Assistant] Exactly, and I'll show you that in a second, that basically the, the, the intermediate guys were able to basically find the high performing code, you had access to those code, and then you could just


[User] Please transcribe this text

[Assistant] Boom, go crazy at it. Versus these guys who are blindly searching.


[User] Please transcribe this text

[Assistant] Alright, they had a score function that they could compare themselves against, but that's it. But they didn't have anybody access to anybody else's knowledge or do anything with it. Okay.


[User] Please transcribe this text

[Assistant] Uhm, and so what we see is that there were 30 percent, what's interesting is that there were 30 percent fewer novel techniques in an intermediate disclosure.


[User] Please transcribe this text

[Assistant] Simply because, you know, they just found the cool stuff and they went right after it, versus these guys are exploring much broader. And these points, these lines, if you put them on the same graph, never cross.


[User] Please transcribe this text

[Assistant] Right, they're always, the, they're, the, the, the, the closed, no intermediate disclosure regime is typically just, just heads above in terms of experimentation that's going on. And we have the herpedile index of the submission that you can sort of see.


[User] Please transcribe this text

[Assistant] That there's much more homogeneity concentration here than here.


[User] Please transcribe this text

[Assistant] Okay, and then what we can also look at is number of high potential null approaches versus low potential null approaches considered. And again, what you see is that the, that the closed guys do much more exploration on the low end of the potential approaches.


[User] Please transcribe this text

[Assistant] And the high, and the open guys are pretty much on the high end. Okay?


[User] Please transcribe this text

[Assistant] So, so, so what's happening, right, is that in the, in the world of intermediate disclosure, people get access to other people's knowledge?


[User] Please transcribe this text

[Assistant] Right? And if you have a performance metric available, you can then take that and go, go crazy and improve upon that. Right? But that's at a cost of


[User] Please transcribe this text

[Assistant] Less experimentation, right? Less exploration, right? And, and less participation. And so we have to be able to balance between those things.


[User] Please transcribe this text

[Assistant] Uhm, so, you know, again, in individual disclosure, fewer participants, less effort, yet higher average and maximum performance.


[User] Please transcribe this text

[Assistant] Uh, vast majority of people will peak and reuse. So of the 32 people that were in our open disclosure regime, right, 30 of them actually did the peaking and reused.


[User] Please transcribe this text

[Assistant] Uh, it's available. Why wouldn't you use it? Uhm, search behavior, more experimentation, more exploration in the final disclosure regime, uhm, and convergence and coordination over the best paths, uh, in intermediate disclosure. But this has consequences for us. Yes?


[User] Please transcribe this text

[Assistant] So what we're doing is in our experiment we're trying to make the difference really sharp to see in fact what's going on. And as I'll discuss next, what we have in other systems are compensating mechanisms, right, allowing you to actually get rewards for credit and so forth along the way as well.


[User] Please transcribe this text

[Assistant] The thing we need to sort of think about is that increased reuse, right, comes at a cost of lower incentives. And it's highly dependent on the stock of knowledge being created, right. Because the stock of knowledge, if the stock of knowledge created is not that great, we're going to be in trouble.


[User] Please transcribe this text

[Assistant] Conversion and coordinate behavior, you know, relies on the stock of knowledge available. If that's not good, then we won't get the best solutions.


[User] Please transcribe this text

[Assistant] You find the solution and you go boom, you go after it. Alright? And everybody can share it. But if we're in a world of rugged landscapes with lots of peaks and valleys, right? We may be in a different world, right?


[User] Please transcribe this text

[Assistant] This requires broad experimentation. This requires broad search.


[User] Please transcribe this text

[Assistant] And so, you know, one of the consequences of the study is that in fact the choice of problem you have


[User] Please transcribe this text

[Assistant] You know, uh, can actually, uh, should then determine the disclosure policy you're actually following, right?


[User] Please transcribe this text

[Assistant] You would use an image disclosure when there is a broad stock of knowledge already available for who we use. Right? So, one of the things that open source has shown is that Linux did well because we understood the Unix framework. Right? That was not a mystery.


[User] Please transcribe this text

[Assistant] And that allowed people to then take that knowledge and then build upon that and elaborate amazingly well. Same thing with Wikipedia. But if we were starting a brand new operating system, you know, herd didn't go anywhere for some reason, uh, if you guys didn't know what that is, uh, you know, uhm, but, but,


[User] Please transcribe this text

[Assistant] But a brand new completely novel conceptualization may have problems, uh, because of, of, of these effects.


[User] Please transcribe this text

[Assistant] The diversity of participants we argue may...


[User] Please transcribe this text

[Assistant] So, you know, we, in our, in our study, we can control for skill and, and sort of actually take away the diversity that exists. But in fact, in the real world,


[User] Please transcribe this text

[Assistant] There may be different perspectives and open, open source and open communities may in fact attract also diverse people and they may not get locked into these perspectives.


[User] Please transcribe this text

[Assistant] Again, empirical question for us to go after. Uhm, and of course, this compensating incentives, right? That there may be other benefits to participation above and beyond the final outcome reward that we set up and that may drive participation as well.


[User] Please transcribe this text

[Assistant] Alright, uhm, in the final disclosure world, uh, useful when you want broad-based experimentation, right? And this is really the startup economy.


[User] Please transcribe this text

[Assistant] The 998 of them died and went away because they were in the low part and you know, two or three did really well and did amazingly well. So our economy is set up to actually do this parallel independent search depending upon the problem that we have in front of us.


[User] Please transcribe this text

[Assistant] And again, this is the insights that you want to start thinking about here. Uhm, and, and even here in terms of file disclosure, uh, in some of the work we're doing on contests, uhm,


[User] Please transcribe this text

[Assistant] The emphasis that we have, uh, can be on implicit incentives, participation incentives, and not just on explicit incentives, and that can also drive behavior. So like when we do these contests where, you know, if you have a thousand people participating,


[User] Please transcribe this text

[Assistant] And most of them are losing, right, there must be other benefits to participation. And we again, uhm, see that in the, in the setup.


[User] Please transcribe this text

[Assistant] So I'm going to end with, you know, every HBS professor has to show a 2x2 that's in our contract, you know, uh, so I'm just going to show you like a...


[User] Please transcribe this text

[Assistant] Two thirds by two by two or something like that. One, one is a missing element. So what you want to think about, right, are two dimensions, right? Emphasis on disclosure of individual advances, right, versus emphasis on disclosure of final innovation.


[User] Please transcribe this text

[Assistant] That's one dimension. And the other is property rights and exclusion versus disclosure, right? What, what are you putting together? So our industrial innovation system market-based systems have a, have a process that says we give you property rights and exclusion if you disclose your final products. That's where we're at.


[User] Please transcribe this text

[Assistant] Academic science and public innovation contests are really geared towards emphasizing disclosure, but only of final products. Okay?


[User] Please transcribe this text

[Assistant] There's a lot of controversy in science right now that, you know, people should list out all their data, right? There's a, there's a study out recently that apparently,


[User] Please transcribe this text

[Assistant] 70% of biomedical research can't be replicated.


[User] Please transcribe this text

[Assistant] Which is kind of crazy, right? And so that's, but we're in a system where, you know, you never get the underlying data set. You basically get the, the tables and the graphics.


[User] Please transcribe this text

[Assistant] Alright, so this is the world we live in. And then what we've seen, you know, so clearly in the last decades has been the Human Genome Project, open source and user innovation communities, open data, open APIs, biological repositories, all these guys pushing for disclosure of intermediate advances.


[User] Please transcribe this text

[Assistant] And I think what's, and so that's how we're set up right now as sort of our major institutions for innovation. And what's, what's interesting for me is to sort of say, as a, as a, as a B-school product,


[User] Please transcribe this text

[Assistant] What are firms doing? And if you think about a company like IBM or like Google, they're actually playing in all three dimensions.


[User] Please transcribe this text

[Assistant] Right? So IBM, the biggest patent in the world, right, also does a lot of scientific research that they publish, also is the biggest contributor to open source. Right? Google, the contributor to open source, scientific research, also patenting. Right? So, so in the economy we have,


[User] Please transcribe this text

[Assistant] Firm's figuring out, uh, either through some master plan or just trial and error living in these three, three regimes. And I think the question for us as, as academics and scholars is to sort of say, you know, is this the right way for this particular problem? Right? Or should we be thinking about alternative ways for us to put together?


[User] Please transcribe this text

[Assistant] So the, the, what's amazing as a researcher for us right now is that we can now run these counterfactuals on real problems with real people and then get these types of results.


[User] Please transcribe this text

[Assistant] So uhm, I think I'm almost out of time too. So uhm, questions, thoughts, comments? Yes.


[User] Please transcribe this text

[Assistant] We require some of your smarts to help us along the way.


[User] Please transcribe this text

[Assistant] Certainly, uhm, you know, uhm, some of the things, uh, a colleague of mine at MathWorks, uh,


[User] Please transcribe this text

[Assistant] At MATLAB, they actually run this sort of intermediate disclosure contest all the time. So basically they have two days of closed and then five days of open. It's just for, you know, tchotchkes. There's no cash involved.


[User] Please transcribe this text

[Assistant] And uh, Ned there has actually said that he's tried to build systems where you could signal to other people saying, here's a novel bit of code in here that hasn't been tried before. And get people to sort of raise


[User] Please transcribe this text

[Assistant] So even though it may score really poorly, there might be an olive bit of insight that somebody can grab and do that. Because, you know, to be a programmer you need to understand both algorithmic things, but also need to think about efficiency and how fast you are and that kind of stuff. And some people may not be good at both.


[User] Please transcribe this text

[Assistant] Uh, and, uh, and certainly when we've done an analysis of, of the math force code bases, we actually see sort of, you can think of the code is in a cassette, right, and there's some little gems inside of it, and the question is can somebody find those gems, grab them out there, and then go with it. Uh, but we haven't built up a system of that. I think that would be, would be very meaningful.


[User] Please transcribe this text

[Assistant] Yeah, so that's a great, so, you know, what you've hit upon is, in doing this work, uhm, I think the, the,


[User] Please transcribe this text

[Assistant] Solutions are easy, it's the problems that are now the constraint. Like how we define the problems. And there's no, I think maybe the d.schools have some insights, but at least from my perspective,


[User] Please transcribe this text

[Assistant] There's really no signs of problem formulation that exists right now that we can actually take and replicate. We know that that matters.


[User] Please transcribe this text

[Assistant] Uh, you know, I've got a doctoral student graduating right now, Hila Lifshitz, who spent a year at NASA looking at problems, not these competition ones, but really hard ones like, you know, uhm, you know, how do you, how do you, uh, reduce microbes on the space station?


[User] Please transcribe this text

[Assistant] Uh, and how do you formulate that problem, right? Because right now they send, you know, uh, liquids up to the space station and it's $10,000 per kilogram per up mass. So it matters a lot. Uh, and how do they actually do the problem formulation? So I think problem formulation is actually the, the next big thing that those of us that are doing crowd research are going to be focusing on.


[User] Please transcribe this text

[Assistant] Uhm, we've done, we did this interesting thing with the medical school where uhm,


[User] Please transcribe this text

[Assistant] We said, uh, and I have a little, little practitioner paper out on this where we said, if you think about research, it's completely vertically integrated, right? The PI and the lab decides on the interesting hypothesis in the question, comes up with a proposal, sends it to the NIH.


[User] Please transcribe this text

[Assistant] Their peers in the same community evaluate and then there's some possibility of you getting funding and working away.


[User] Please transcribe this text

[Assistant] Uh, what happens if we actually disaggregate that process? What happens if we actually open up the hypothesis generation?


[User] Please transcribe this text

[Assistant] So we did that with type 1 diabetes where we basically said, you know, we want, you don't need ability to solve a problem, you need to come up with a really good question.


[User] Please transcribe this text

[Assistant] Uh, and we, we, we have a paper on that trying to understand how that, how those processes work as well. And then we did the same thing on evaluation. We said, not just the, the, the close experts, but in fact a whole range of people in the medical sciences can be evaluating. And then we actually did more work where we said, let's take


[User] Please transcribe this text

[Assistant] These hypotheses generate new teams of people that have never worked together before as well.


[User] Please transcribe this text

[Assistant] So that's the, that's the bleeding edge. Uh, and we're trying to understand that. But I think, I think the insight about problem formulation is actually, is actually critical. Because, you know, we can now, with a credit card, get answers.


[User] Please transcribe this text

[Assistant] Really good answers. Really fast. And then it's a matter of, like, have you asked the right question? And, and the finding we have is...


[User] Please transcribe this text

[Assistant] The question you ask is the answer you'll get. Like, it's, it's, it's so, these guys are so, you know, so like, there's nothing else but, oh, this is the question, this is the value of the metric, great, I'm gonna give you the answer.


[User] Please transcribe this text

[Assistant] That's a great question. I haven't, not in this context I haven't looked at that. There's another paper I'm working on right now where we look at, uh,


[User] Please transcribe this text

[Assistant] Okay, so invention capability and copying capability. So, and what we're finding is that, uh, you become a better inventor as you, as you copy more.


[User] Please transcribe this text

[Assistant] So the quality of your inventions are better once you've learned. And this effect runs both ways. So as you invent more, you're better at copying as well. You find better stuff to copy as well.


[User] Please transcribe this text

[Assistant] Again, from an econometric point of view, this is a hard study for us to do because you've got all this endogeneity going on and we have a bunch of


[User] Please transcribe this text

[Assistant] Controls and fixed effects and so forth to deal with that but we feel pretty comfortable about this finding that in fact there's a virtuous circle between invention and copying.


[User] Please transcribe this text

[Assistant] And that's, that's the society, right? People, you know, there's no sort of lone geniuses. You're in, you're embedded in a world of other ideas and from that you formulate something coming out of it. Uh, and that we use the math books data for to actually get at.


[User] Please transcribe this text

[Assistant] Uh, and the, the, and so that in itself is a good outcome for innovation because we want diverse experiments to take place.


[User] Please transcribe this text

[Assistant] So, so the answer would be, we see more, we don't see correlation in, in search strategies when, once you're blind. Which sort of makes sense, right? You sort of have your own idyssocratic knowledge. And given the types of people we attracted, they weren't all from Stanford, they were from around the world.


[User] Please transcribe this text

[Assistant] Different backgrounds, different training, and that's what probably drove it.


[User] Please transcribe this text

[Assistant] Yes, I think the learning question is a really important question and I don't have good answers for that because I mean I think, you know, it's hard for us to be able to judge what they might have learned. Uh, you know, they looked at somebody else's code, at the end of the contest all the code was available to everybody else to use.


[User] Please transcribe this text

[Assistant] He remained locked in a room for many days, seeing nobody between early morning and midnight.


[User] Please transcribe this text

[Assistant] He spent his time thinking sadly about his friends in Pentonville.


[User] Please transcribe this text

[Assistant] After a week he was free to wander around the house during the day.


[User] Please transcribe this text

[Assistant] It was a dirty place, full of rats and insects but no other living thing.


[User] Please transcribe this text

[Assistant] All the windows were closed and covered with wood and metal bars that kept out the light.


[User] Please transcribe this text

[Assistant] One afternoon, the Dodger and Charlie Bates were at home, and they started telling Oliver about their lives as thieves.


[User] Please transcribe this text

[Assistant] It's like watching The Wizard of Oz, and then Dorothy dies at the end.


[User] Please transcribe this text

[Assistant] The Yankees started crawling up at the standings. The Red Sox were creeping down.


[User] Please transcribe this text

[Assistant] He's hopping around like the three students were all laughing at him. I mean, just waiting for the guy to get put away.


[User] Please transcribe this text

[Assistant] Finally, it's our turn. Finally, we have a storybook finish pulling out of the coffin and crush some other teams hard.


[User] Please transcribe this text

[Assistant] I can still see him sliding into second, kind of sitting up on the couch. Game's not over yet. Keep him alive. Keep him alive.


[User] Please transcribe this text

[Assistant] So on Red Sox like to see these moments unfold the same way in our favor. It was remarkable.


[User] Please transcribe this text

[Assistant] I have to admit I still, I was still waiting for us to drop the ball.


[User] Please transcribe this text

[Assistant] From being clinically dead against the Yankees, to coming back and winning the World Series, I just...


[User] Please transcribe this text

[Assistant] I was in a state of shock for, well I'm probably still in it.


[User] Please transcribe this text

[Assistant] Tension eases somewhat here at Fenway. With two out in the shortstop.


[User] Please transcribe this text

[Assistant] And they're gonna call him out.


[User] Please transcribe this text

[Assistant] But don't let's think of troubles tonight. Christmas is a time for joy.


[User] Please transcribe this text

[Assistant] Emily's silly dream. Wherever you may go, you can't escape from yourself, and that's the truth. Mother.


[User] Please transcribe this text

[Assistant] Well, if you don't want to eat him, you know what will happen? The whiskered witch will come and eat you.


[User] Please transcribe this text

[Assistant] And remember, make sure to apologize to your mother.


[User] Please transcribe this text

[Assistant] Now you go to school tomorrow and you explain to comrade teacher why you haven't made the flowers. Oh, it's how the flowers are made, sweetheart. Right. You care more for Adam and his ineffectual underground friends than you do for us. You're putting the whole family in danger.


[User] Please transcribe this text

[Assistant] You know I have a weak heart.


[User] Please transcribe this text

[Assistant] May God forgive you. God sees and hears everything.


[User] Please transcribe this text

[Assistant] Thou wouldst not suffer me to kiss thy mouth, but I will kiss it now.


[User] Please transcribe this text

[Assistant] But wherefore dost thou not look at me? Thine eyes that were so terrible, so full of rage and scorn, are shut now.


[User] Please transcribe this text

[Assistant] Where have you been? You had me worried to death. I was walking with Andrea. Oh, you were walking with Andrea.


[User] Please transcribe this text

[Assistant] Meanwhile, your grandpa is dying, your uncle is missing, and what do you care? That girl is a bad influence.


[User] Please transcribe this text

[Assistant] Are you answering me back? On top of everything else? You know what happens to bad girls.


[User] Please transcribe this text

[Assistant] Life is painful enough without you. I... You know I have a weak heart.


[User] Please transcribe this text

[Assistant] And now, you must practice the piano. And I don't care if you have to stay up till midnight. Time is running out.


[User] Please transcribe this text

[Assistant] Hello my little sweetheart. I'm afraid grandma couldn't come and fetch you. Poor grandpa's not very well.


[User] Please transcribe this text

[Assistant] Nobody said anything about a child. I'm not risking a kid's life.


[User] Please transcribe this text

[Assistant] You had to have kind of a mass curse removal festival. Kiss the ground that babe is being grown in. That's what we were trying to provide was a place where people could come in.


[User] Please transcribe this text

[Assistant] Up through his nose and his broad shoulders and you know maybe by giving the babe a little walking over you could help lift the spirits.


[User] Please transcribe this text

[Assistant] Why are the baseball guns so fickle? Why are we treated this way?


[User] Please transcribe this text

[Assistant] I was down in the bowels of Shea Stadium. I watched them roll those carts of champagne into that locker room. I watched them put the world championship t-shirts on everybody's seat. It was sewn up. It was going to be the ultimate purging of everything. I stood on the bar.


[User] Please transcribe this text

[Assistant] I watched them wheel that champagne back out and grab those t-shirts and stuff them back in the bag. It was really like a horror.


[User] Please transcribe this text

[Assistant] It just couldn't have really happened. It had to have been a movie or somebody's idea of a cruel joke.


[User] Please transcribe this text

[Assistant] And then, in three boatloads, Caspian and his people, and Bern with a few of his, put out for Narrowhaven. The king's flag flew in the stern of his boat, and his trumpeter was with him.


[User] Please transcribe this text

[Assistant] When they reached the jetty at Narrow Haven, Caspian found a considerable crowd assembled to meet them. This is what I sent word about last night, said Byrne. They are all friends of mine, and honest people.


[User] Please transcribe this text

[Assistant] So that the street shook, and their armour shone, but it was a sunny morning, so that one could hardly look at it steadily.


[User] Please transcribe this text

[Assistant] But then, all the children joined in, because they liked a procession, and had seen very few. And then, all the school boys joined in, because they also liked processions.


[User] Please transcribe this text

[Assistant] The less likely they would be to have any school that morning.


[User] Please transcribe this text

[Assistant] And then all the old women put their heads out of doors and windows and began chattering and cheering because it was a king. And what is a governor compared with that? And all the young women joined in for the same reason. And also because Caspian and Drinion and the rest were so handsome.


[User] Please transcribe this text

[Assistant] Open for the King of Narnia! Come to visit his trusty and well-beloved servant, the Governor of the Lone Islands! In those days...


[User] Please transcribe this text

[Assistant] And out came a tousle fellow with a dirty old hat on his head instead of a helmet, and a rusty old pike in his hand. He blinked at the flashing figures before him. God sees efficiency, he mumbled, which was his way of saying, you can't see his sufficiency.


[User] Please transcribe this text

[Assistant] No interviews without appointments except between nine, ten p.m. Second Saturday every month. Uncover before Narnia, you dog! Thundered the Lord Byrne.


[User] Please transcribe this text

[Assistant] Began the doorkeeper, but no one took any notice of him. Two of Caspian's men stepped through the postum, and after some struggling with bars and bolts, for everything was rusty, flung both wings of the gate wide open. Then the king and his followers strode into the courtyard.


[User] Please transcribe this text

[Assistant] These were fellows who might have fought if they had been led, or had known what was happening. So this was the dangerous moment. Caspian gave them no time to think. Where is the captain? he asked.


[User] Please transcribe this text

[Assistant] Said a languid and rather damnedified young person without any armour at all. It is our wish, said Caspian.


[User] Please transcribe this text

[Assistant] That our royal visitation to our realm of the Lone Islands should, if possible, be an occasion of joy and not a terror to our loyal subjects.


[User] Please transcribe this text

[Assistant] If it were not for that, I should have something to say about the state of your men's armor and weapons. As it is, you are pardoned.


[User] Please transcribe this text

[Assistant] Command a cask of wine to be opened, that your men may drink our health.


[User] Please transcribe this text

[Assistant] But at noon tomorrow, I wish to see them here in this courtyard, looking like men at arms and not like vagabonds. See to it, on pain of our extreme displeasure. The captain gaped, but Burn immediately cried, Three cheers for the King!


[User] Please transcribe this text

[Assistant] And the soldiers who understood about the cost of wine, even if they understood nothing else, joined in.


[User] Please transcribe this text

[Assistant] Caspian then ordered most of his own men to remain in the courtyard. He, with Byrne and Drinion and four others, went into the hall.


[User] Please transcribe this text

[Assistant] Behind a table at the far end with various secretaries about him sat


[User] Please transcribe this text

[Assistant] Now interviews without appointments except between 9 and 10pm on a certain Saturdays. Caspian nodded to burn and then stood aside.


[User] Please transcribe this text

[Assistant] But an Andrinian took a step forward and he teased one end of the table. They lifted it.


[User] Please transcribe this text

[Assistant] And flung it on one side of the hall when it rolled over.


[User] Please transcribe this text

[Assistant] Scattering a cascade of letters, dossiers, ink pots, pens, ceiling wax and documents.


[User] Please transcribe this text

[Assistant] Then, not roughly, but as firmly as if their hands were pincers of steel, they plucked Gumpus out of his chair and deposited him, facing it, about four feet away. Caspian at once sat down in the chair and laid his naked sword across his knees. My lord, said he, fixing his eyes on Gumpus, you have not given us quite the welcome we expected. I am the King of Narnia.


[User] Please transcribe this text

[Assistant] Nothing about it in the correspondence, said the governor. Nothing in the minutes. We have not been notified of any such thing. All irregular. Happy to consider any applications.


[User] Please transcribe this text

[Assistant] And we are come to inquire into your sufficiency's conduct of your office, continued Caspian.


[User] Please transcribe this text

[Assistant] There are two points especially on which I require an explanation.


[User] Please transcribe this text

[Assistant] Firstly, I find no record that the tribute due from these islands to the Crown of Narnia has been received for about 150 years. That will be a question to raise at the Council next month, said Gampas.


[User] Please transcribe this text

[Assistant] If anyone moves, let the mission of inquiry be set up to report on the financial history of the islands at the first meeting next year. Why then...


[User] Please transcribe this text

[Assistant] The Caspian had only one ship and one ship's company with him. He would have spoken soft words for the moment.


[User] Please transcribe this text

[Assistant] And hope to have them all surrounded and killed during the night. But he had seen the ship of war!


[User] Please transcribe this text

[Assistant] He sailed down the straits yesterday and seen it signalling, as he supposed, to its consort. He had not then known it was the king's ship.


[User] Please transcribe this text

[Assistant] It would never have occurred to Gumpers that anyone would walk into Narrow Haven to take the islands with fewer than 50 men.


[User] Please transcribe this text

[Assistant] It was certainly not at all the kind of thing he could imagine doing himself.


[User] Please transcribe this text

[Assistant] I want to know why you have permitted this abominable and unnatural traffic in slaves to grow up here, contrary to the ancient custom and usage of our dominions. Necessary, unavoidable, said his sufficiency.


[User] Please transcribe this text

[Assistant] An essential part of the economic development of the islands, I assure you. Our present burst of prosperity depends on it. What need have you of slaves? For export, Your Majesty. Sell'em to Calamen, mostly.


[User] Please transcribe this text

[Assistant] And we have other markets. We are a great center of the trade. In other words, said Caspian, you don't need them.


[User] Please transcribe this text

[Assistant] Tell me what purpose they serve, except to put money into the pockets of such as Pug. Your Majesty's tender years, said Gumpus, with what was meant to be a fatherly smile.


[User] Please transcribe this text

[Assistant] Tender as my ears may be, said Caspian, I believe I understand the slave trade from within quite as well as your sufficiency. And I do not see that it brings into the islands meat.


[User] Please transcribe this text

[Assistant] And before Gumpus quite realised what was happening, Bern was kneeling with his hands between the King's hands.