Debugging your mind
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Hunt, Andy
We make important decisions and try and solve critical problems everyday. But our decisions and problem solving is based on faulty memory and our emotional state at the time. Join Andy Hunt and explore common cognitive biases which can dramatically affect your decision making and problem solving skills. You'll learn why most predictions are wrong from the start. Together we'll look at aspects of context which can subtly affect you, including your own brain's legacy hardware, and how to recognize and stop that when it happens.
 Alright, hello. How was everyone? Thinking about lunch already? Yeah, okay. Dynamite. You having fun so far? Okay, that's good, that's cool. Alright, well, so if you're anything like me, by now you've been to a whole bunch of talks and you've heard stuff about C-Sharp and functional programming and Agile and this and that and your head's about to explode, right? Good, cool. Well, this will help. This will really make it just burst out. So I love this, I love this background picture here. Alright, what if my clicker worked? Come on clicker, you can do it. There we go. Do you know what this is? This is in fact the first bug. You've heard the story about when they were debugging the, what was it, the Mark II relay calculator, September 1945 and they had this problem, this was a relay calculator, they're going through running a sign and cosine unit test, bless their hearts, they're running it and there was a bug and they found there was a moth trapped in the relay. They took the moth out, taped it to the log book and that was indeed the first incident of debugging a computer. 1945, an old joke. So we've had bugs in computers for a long time, sometimes literally so, but as it turns out, we've got bugs in the very way that our brains work. We've got quite a few of them in fact. So I want to go through today and just show some of the more interesting or amusing or common bugs that affect our decision making, how we write software, how we do agile processes, all these sorts of things. The first thing I want to talk about is how we're creatures of habit. So here we've got the policeman out to give tickets behind the sign that says, slow down, the cop hides behind this sign. Clearly he's still hiding there, just creature of habit. But it's kind of funny, they do a lot of studies on selective awareness and gradual change and it turns out we're really not nearly as observant as you might think we are. And I've got a couple samples here just to show how that works. The first thing I want to show is a fairly difficult counting task. So what's going to happen is you're going to see a bunch of kids passing a basketball, wearing some wearing white shirts, some wearing black shirts. And the idea is you've got to really focus on the players in the white shirts and count how many times they pass the basketball. Now it's not impossible, this isn't like one of those trick, you know, sight questions. You can actually do it but you just have to focus very closely and count the total number of passes. Alright, you ready? Okay, Twitter down, email down, here we go. We're going to count the number of passes. Count how many passes? The correct answer is 16 passes. Did you spot the gorilla? For people who haven't seen or heard about a video like this before, about half missed the gorilla. If you knew about the gorilla, you probably saw it. But did you notice the curtain changing color or the player on the black team leaving the game? Let's rewind and watch it again. Here comes the gorilla and there goes a player and the curtain is changing from red to gold. When you're looking for a gorilla, you often miss other unexpected events. And that's the monkey business illusion. Learn more about this illusion and the original gorilla experiment at theinvisiblegorilla.com. So I used to show the original version of this, just the regular gorilla experiment where you've got the kids passing the basketball and the gorilla walks in and they're like, I've got 15, I've got 17. It's like, great, did you see the gorilla? The what? I need to show it again. And how can you miss it? You see it the second time and it's like, how do you possibly miss this? But we do. So this has been around the interwebs, people got used to it. It's like, I know this one, I know about the gorilla. Yeah, great, did you catch the curtain changing color? Did you catch the player leaving the scene entirely? Right out the edge. But that's okay. I mean, these are, this is a very busy video. You know, there's a lot to keep track of, a lot of motion, you're trying to keep track of the passes. So let's try something a little simpler, perhaps. I'm just going to show a couple of single still photographs. Nothing's moving, but something's changing. Raise your hand when you spot the change. How'd you do? Oh, oh dear. That whole bit of greenery went away. Okay, let's try the next one then. Now, now, now you got warmed up now. Now you know what to look for. We'll try the next one. Okay, let's try the next one. Okay, you got warmed up now. Now, you know what to look for. Okay, a little better. Okay, you got warmed up now. Now, you got warmed up. Now, okay, only a little better. Okay, this is an easy one. You'll definitely get this. Yeah, the entire side goes away. So, it's interesting, you know, we're really just not well equipped to notice slow moving things. You know, gradual change creeps up on us and we just don't notice it. This is a real problem. If you're on a software project, what's changing? Everything. Right? The requirements are changing, the technology is changing, there's stuff changing all around us and we're not really well equipped to see that or to experience it. So, that's kind of the first interesting thing. But I like to go on, I mean that's just sort of fun. You know, we're blind as bats. Okay, fine. But I want to talk about some actual sort of processing errors, if you will, in how our brains operate. If you look on, say, Wikipedia and you look up cognitive biases, there are some 90 common known cognitive biases. I've certainly met people who can exceed that number by a good bit, but there's 90 common ones. And I'm just going to go through some of my favorites here and just talk about them for a little bit. The first is relying on memory. Our memory is so bad. You know, we tend to think of memory as like a hard drive. You know, something happened and it gets written to disk and we've got it. Maybe we'll forget it, but at least what we remember is accurate. Right? Now, it doesn't work that way. In fact, the way that memory works, every time you access a memory, you change it just a little bit. You access it enough over the course of a lifetime, it can change quite a bit to the point where that incident that you think you remember when you were six years old probably didn't happen that way. Maybe it didn't even happen at all. It's very easy to implant false memories into people and to suggest things were very suggestible. And every time you access a memory, that rewriting takes place in your current context, how you currently understand the world, your current age. You know, so every time you're accessing it, it changes a little bit. This can get really difficult on a software project. You're sitting there, it's months and months into it, and it's like, why did we decide to do XYZ again? Oh, well, because, and you're going to get 15 different answers because everyone will remember it differently. This is why it used to be good practice to keep like an engineering log or something where you just scribble some notes to yourself. You know, when we had this meeting, we decided to use this component because XYZ. That's a great thing to do because then months later you can go back and look, why did we decide to do that? Oh, yeah. We thought that at the time. Okay, makes sense. This is different now. Now we can do this, whatever and ever. But if you rely on memory alone, even communal memory in the team, it's very foggy, not reliable at all. Some other things that can throw us off the track a bit. There's this great experiment related to a phenomenon called anchoring. The way this works is our brain is always seeking patterns, trying to find stuff to compare, but it's not always so picky about what it compares, and it can get confused really easily. So for instance, if I started talking about the fact that my publishing company, our publishing company, the Pragmatic Bookshelf, that we have hundreds of books for you to peruse. We've got hundreds of authors, and I keep tossing out this word hundreds in various contexts, and then I say, here's a nice leather-bound edition of Programming Ruby for $85. Your brain somewhere says, oh, that's less. That's a bargain. It has nothing to do with it. They're completely unrelated. Doesn't matter. Your brain seizes on that. So they did an experiment with undergrads doing bids in an auction. And before the experiment, they prepped the group and said, okay, if your Social Security number, your national identification number, your driver's license number, whatever you have, take your number out and write down just the last two digits. That's all. And now we're going to go ahead and bid on this auction. And what they found was people in the audience who had relatively high last two digits tended to outbid the other students by anywhere between 60 and 120 percent. They bid twice as much on the objects. They perceived their value to be twice as much solely because the last two digits of their ID number happened to be high. We do this kind of thing, unfortunately, all the time. Marketing people know this. The politicians know this. And they use this to their advantage. You anchor on something and then you make the wrong decision. We'll talk more about that in a bit. Another interesting thing is the fundamental attribution error. And I just love the picture. The problem here is lack of context. We have an innate system. Maybe it's some kind of ego prevention. But if I've done something terrible to the team or something bad to the code or whatnot, it's because I had a bad day. I missed breakfast. I didn't get a good night's sleep, whatever. You have kind of an excuse for it. If you do something bad, it's because you were born that way. It's because you are from this part of the country or that or from this country or the other or this political party or this race or this species, whatever it is. It's because you are you that this happened. Whereas if I did it, that was just context. That was just whatever. So we have this mistake of attributing something to the fundamental behavior of a person instead of the context that a person is in. Something to bear in mind. You can see this a lot on internet troll fests when the comments start flying. It's not that your position is wrong. It's you are an idiot and your mother has had sex with farm animals or whatever. It degrades pretty rapidly. So just something to bear in mind. It's context. It's all about context. Now the next one is particularly bad for agile projects and agile adoption. And that is a built in need for closure. And Dilbert sums it up quite nicely. Dilbert says, I didn't have any accurate numbers so I just made this one up. Studies have shown that accurate numbers aren't any more useful than the ones you make up. How many studies showed that? 87. You laugh. The problem is and this one is really pernicious because right every project starts off and some being counter in accounting says, okay, I got there's a spot in the spreadsheet I've got to put a number in. How much is this going to cost me? And you give the agile answer, well, we don't know yet. It's exploratory. We'll find out. No, no, no, no. I need a number to put in the box. I don't have a number. I need 87. Good. Doesn't matter. Doesn't matter. And this is a curious thing. All parties concerned, including the requester, are well aware the number you just pulled out is rubbish. It's garbage. They know that. It's okay. It's a number. I have satisfied my need for closure. I have a number. I know it's wrong. I don't care it's wrong. I have it. I can get my hands on it. I'm happy. This happens unfortunately all the time. You know, you say, well, you know, what technology are you going to use here in this project? Well, we haven't decided yet. I got to fill that out. I have to know. Okay, fine. We'll use SQL Server. We're not really. But fine. You know, I write it down. Oh, yeah, we changed our mind. We're using amnesia instead. We're using MySQL or we're using, you know, what have you, Mongo. But you've got to have that need for closure. This flies in the face of agile development where we want to leave all our decisions open as long as possible, defer decisions, and make them late in the game. This is a really hard one to get by. Some folks have an easier time with this than others. A lot of folks, this is really hard. They have to have closure even knowing that the number is wrong. Confirmation bias, another popular one. You see this a lot on the TV news cycle. Any facts or events that don't fit your worldview, you just throw out. So you just amplify the stuff that you agree with. And of course, the whole world thinks this way because look, everybody agrees with it and you throw out the rest of it. Entire books. You know, every author who writes a book with any opinion in it, you could argue that's just a big example of confirmation bias because they're going to throw out the other stuff that disagrees with them. That's just how we work. The exposure effect. This is a great one. The more often you see something, the more acceptable it becomes to you. The more familiar it is. Politicians know this very, very well. And that's why they will do everything to get their face out there. They will play sacks in the bar. They'll get their picture on the poster. They make sure you know their name. Even if they're a scumbag, you know the name. So when the time comes, you're like, okay, here's a name I know versus some name I've never heard of, you're going to go with the familiar. It's the way it works. You see something over and over and over again, even if it's horrific, even if it's horrible, even if it is the intellectual equivalent of broken glass on a toilet seat, it's familiar. And you will stick with it to your detriment. Even if the plane is crashing, you will stick with it. Familiar is not good, but we conflate the two. Just like in that anchoring experiment, we just get kind of muddled with it and we confuse them. If something is damaging your project, if it's not good, dump it. Get rid of it. It doesn't matter how familiar it might be. There's the Hawthorne effect. This is a lovely one. This is the deal where consultants come in. They're studying your team, studying your organization, and suddenly everything's working great. They've got some new giga they're trying to sell, some new process, some new tool. Everyone's using it. Productivity's up. Numbers are great. Everyone's happy. What happens when they leave and stop watching? Everything goes right back to the way it was before. We work differently if we know somebody's watching, at least for a little while. Relativity. So when, what company was it made, the first bread maker? I've lost track. And the font's too small. William Sonoma. They made the first automatic bread maker. This thing was a marvel. You open up this little R2-D2-like box, stick some flour and water in, hit the timer, and some hours later, a baked loaf of fresh bread comes popping out of it. This was a marvel in what, the early 90s, I suppose it was? An absolute marvel. The problem is, they were the first to market with it. So how do you price the thing? Oh gosh, I don't know. So they picked a price, set it, put it out there, you know. Harold did this technology. This was incredible. Fresh baked bread and all you do is hit the button. Nobody bought them. Sales were modest at best. And they're scratching their head. They didn't know why. So they fished around, talked to some psychologist types and said, okay, I know what we're going to do. We're going to keep this at about the same price it was, $250. Doesn't matter what units. Pick a unit that's comfortable to you. But we're going to come out with the luxury model, the high-end model that does many more things and price it at twice the amount and put those two out there. Suddenly, no one ever bought the big one. But now they bought the middling one because it was perceived to be a bargain. Because you start off with the question, well, okay, is this a good value? Well, I don't know. What does a bread maker cost? I have no idea. They just invented the thing. But now, what does a bread maker cost? Oh, well, it either costs $500 or $250. Oh, look, this $250 one is a bargain. It's half the price. Okay, right? I mean, they made these numbers up completely. These aren't even the real ones. But you make it up completely because now you've got this false relativity to concern with. It's like, oh, this is less. I will go for it. We fall for it every time. We fall for free. Free is not the same as cheap, however. Free is immensely valuable. Free is awesome. Free is better than one cent or one franc or one kroner or what have you. In fact, when Amazon.com first rolled out their Amazon Prime option, you could pay a flat rate and then shipping was free. Right? And they're like, this is going to be dynamite. They rolled it out in the US, sales took off. They rolled it out in different other countries, Asia, Pacific, Europe. Sales took off. Worked great everywhere except France. They rolled the offer out in France. Nothing happened. What was up with that? So they looked into it a bit and they discovered for whatever reason the French division, when they did this, the free shipping wasn't actually free. It was cheap. It was like one franc or Euro, whatever they had at the time. It wasn't free. It was very cheap. Didn't count. People did not fall for it. They changed it to free. Their sales then matched everyone else. Popped right back up again. Why do we do that? Well, that's a good question. We're cheap. We like free. We have a very strong sense of loss aversion. We don't want to lose anything that we already have, which is kind of funny. There's an example. I'm not sure if it's in this talk or not, so I'll just give it now anyway. Have you ever heard of the South Indian monkey trap? Wonderful thing. Apparently, there's a lot of monkeys running around Southern India and you need to trap them. Okay? So what you do is you dig a hole in the ground with sort of a narrow entry and then a hollowed out part and you put bananas or bait, whatever down in it. Monkey comes along, sticks his hands in, grabs the bananas, can't get back out again. The entrance is too narrow. Simple thing to do, right? Just let go of the bananas and you'd be free of the trap. What does the monkey do? Right? You sit in there, chattering away, stuck because they will not let go of the bait. They are so loss averse, they will not let go of the bait and they get trapped. Happens to us in a similar manner. Sometimes you just got to let go of it. Reasons we fall into some of these traps. I want to talk about symbolic reduction. So what happens here is we tend to focus even on the wrong questions a lot of the time. I had to move houses recently and as part of that I had to clean up the back of my office. So this was a very terrifying prospect. I'm sitting there going through this mountain of cables, right? And in the middle of this bundle of cables I found an old 19.2 modem. Wasn't hooked up to anything, just this ancient modem in this ball of cables. It's like, wow, that was, it's like archaeology, right in the office. It was awesome. But I also found a bunch of old magazines from sort of mid 1990s kind of time period. Objects something or other, Java something or other. I, you know, a bunch of these kind of popular magazines at the time. And the headlines were the same on all these magazines. The huge questions of the day. Who will win the desktop wars? Will it be motif or open look? Wrong question entirely. Missed the whole web thing entirely. And right after that big debate and this went on like a year, two years, these were the headlines. Another set of headlines. Who will win the middleware wars? What will be the triumphant technology? Will it be Corba or RMI? Neither, right? Dead, missed the web, missed it completely, you know, missed the dominance of the PC. Totally the wrong question even to ask. And all of these, you know, you can go through other periods of history and find the same thing. Who's going to win A or B? And the answer is almost always E, none of the above. It's none of the contenders you think it is. So we make predictions. This is kind of part and parcel as how our brain works is to go in and try to, you know, see something and predict what's going to happen on a daily basis. Even just reaching out, grabbing a thing of water, walking. We're constantly making predictions. But to predict anything longer range than, you know, say even a couple of minutes, we're really bad at it. Largely because some unexpected event changes the game. You know, in that case the introduction of the IBM PC, changing the face of the desktop, later the Mac, advent of the web, all these kinds of things. These were more or less unforeseen events. So there's a great book out there whose name just went out of my head. Talib wrote it. Black Swan's in the title somewhere. Google will find it for you. But the idea of a black swan is that some very high impact, hard to predict event, will change the shape of what you're trying to predict. And in Talib's book, he's like, okay, anything significant in history, this is what happens. It's always the big unexpected change that comes in. The title of the book comes from the idea that, you know, for many years scientists thought there was no such thing as a black swan. There was only white ones because they'd never seen one. And one comes trotting by and they're like, oh, would you look at that? Whoops. I've got to change that a bit. So why does this happen to us? Well, one of the ways the brain works is to reduce much of your input to a kind of symbolic representation. So this is coincidentally why none of us probably can draw very well. Right? So you're all programmers, tech people, right? Raise your hand if you're a great artist. Yeah, don't be shy. Go ahead. Come on. Yeah, right. No, it's not happening. Because we're so very good at manipulating and dealing with symbols. What happens is we're very good at symbolic reduction. So when you say, you know, draw a hand or even mention a hand, somewhere in your brain it's like, oh, hand, I know that. It's a stick with five lines. And you go to try to draw it and some part of your brain is shouting, what do you do with all these curves and shades and curves? It's a stick with five lines. And there you go. So of course what happens is in this reduced state in our brain, you lose all the detail. You lose all the nuances, all the context. And that's part of why we fail to see some of these big changes coming on the horizon. We just shrink the details up and then we don't have them to access. So, okay, that's all kind of bad. But these are structural things. These are processing things. There's not much you can do about it except kind of be aware of it. But there's worse things, I think, perhaps. There's things that affect the very core of our judgment and decision making. And this is largely because we still cling to this idea that this is the representation of human decision making, that we're logical, we're rational, we're thoughtful, we think things through, we make checklists. Unfortunately, that's really not a good model of human decision making. This is a lot closer to what's really going on. You've got the gibbering monkey brain inside of you going, wow, squirrel! And it's, you know, if you really want to down or read the book, Predictably Irrational, which goes through and catalogs an absurd number of ways that we just think the wrong thing, totally, all the time. But I want to talk here about something that's maybe less touched upon is things that influence our decision making. First one's kind of easy. You know, what's your basic personality? Are you the kind that values realism and common sense, probably? Or the kind that values imagination and innovation? They're different. Folks can come down on either end of that spectrum. Some folks will insist on fairness at all costs, even if it hurts them personally. Others are more into empathy and harmony. Some folks have to be quiet when they're thinking about it. You think about it in their head until the thought is fully formed, then it comes out at the meeting. Some folks like us, the jaw just flaps. It just comes out while I'm trying to form the thought. I sound like a gibbering moron in a meeting. Because it's well, you know, all this stuff's coming out. And the person sitting at the end just waiting, they won't say anything for an hour, then come out with some fully fledged thought that's like, well, that's really bright. Of course, they haven't said a bloody word for an hour, so it took some time to cook. Some folks genuinely just want to be told what to do. Just give me the step by step, I'll go through it, I'll go home, play World of Warcraft, life is good. Some of us think that's horrid. I don't want to follow a bloody list of steps. I want to figure it out for myself. All fine, all valid. It's baked in differently for each person. This is going to affect how you approach decision making. So along this line, you've got the Myers-Briggs personality types. And this is kind of an interesting look. You can be one of these four values. I'm not going to go through them in any detail, for reasons I'll explain in just a second. But if you look at the combination of values, you see some things that they put in these sort of classifications. You're a planner, you're a developer, you're an analyzer, a composer, you may be your promoter, motivator, explorer, strategist, this sort of stuff. Really kind of interesting different angles on how you might approach life and what your baked in philosophy is. There's only one problem with this. It's a bit like a horoscope in that you really can't fit human behavior into four bits. It's just kind of a difficult proposition. There is some value to it. And it's worth it. If you haven't taken one of those Myers-Briggs tests, it's always kind of interesting, but you have to realize this is a self-assessment. It's going to depend on your mood that day. Depends whether you answer with your work persona or your home persona. Maybe they're the same for you, maybe they're different. So it's a little bit fudgy, but it's kind of interesting to look at to see where your preferences may lie. But deeper than that comes with sort of what you value. Do you characterize yourself as a liberal, a conservative, an anarchist? Do you know how hard it was to find an anarchist logo? Just saying. That was less obvious than I thought it would be. But the problem is, okay, so you identify yourself one of these ways, right, conservative, liberal, what have you. But why? Were those your parents' values? Was that directly opposed to your parents' values? Were you just born that way? Yes. All the above. This can all happen. But what's interesting is, do you know what makes the biggest difference as to what kind of values you hold and how this works? It depends on who else was born the same time as you. Your cohort, your peers, your birthmates. The researchers who look into this kind of stuff say that you have more in common with somebody born at the same time who lives in Mexico City or Tokyo than you do with somebody right next door who is 10, 20 years older or younger. So the power of the cohort is really quite strong because of the sort of shared memories, habits, styles, significant events that happen. Depending on your age, you might remember where you were when the Vietnam War ended. I remember that. When Elvis died, when Reagan was shot, 9-11, all these sorts of major events that happen that your cohort will remember the same way. Because what happens is, if you're 20 and one of these events happens, you'll have one outlook on it. If you're 40 or 60, you're going to perceive that same event quite differently just because you had a different point in your life. So you and your cohort tend to view events in a very similar way. And that sort of drives us a bit. I love this poster here because this is the one where the young ones are like, wait, which Star Wars was that? There's no number. So what kind of differences can this bring about? Well, different generational groups, some are going to be much more prone to taking risks. Some will be very risk averse and not want to hear about it. Some will really value individualism. Others not so much. They favor more of a teamwork approach. Some value stability. Others put their faith in freedom and will accept a little chaos for that. Some focus on family. Other generations more on those to the grindstone work. So what's interesting is, according to such researchers, there's actually only four different kinds of generations and they repeat. So they did their research starting in Europe around the Renaissance and followed it over to the Americas. And in 20 generations of their study, there was only one gap in the pattern and that was in the American Civil War in the 1800s because we killed everybody. So there wasn't anyone there to kind of take their place in society as they would have normally. Other than that gap, they found this very regular repeating pattern of four archetypes. You have a profit generation, a nomad, a hero, and an artist. And in general, I'll give the caveat in generalities in just a second, the profit generation values values, values vision. The nomad, not so much into that, they value their liberty, their survival, their honor. Or if you'd in Klingon, their honor. The hero generation, much more into community, into affluence. Let's share all this together. The artist, pluralism, expertise, do process, do it by the numbers. So they mapped this out onto the last couple generations and tacked cute little names on them because that makes for good press. So you have the GI generation, which was a hero generation, people born back around the turn of the previous century. The silent generation after that, an artist generation. The baby boomers, the boom generation, a profit. Generation X, the nomads, millennials, back to hero again, and then the newest homeland generation back to the artist. So a couple things here. First of all, a caveat. When you talk about a generation of people having this particular characteristic, it does not mean that everyone born at this time thinks this way. What it means is that if you take on the aggregate a whole bunch of people, the folks born in this bucket will tend to share these values and these outlooks, more than people on any other adjacent one. The dates are not firm. They're kind of, yeah, about then. So you can be born on a cuspier and you might feel partly this way, partly that way. These things happen. That's okay. But here's a question. Why does the cycle only go for four generations? Why does it then repeat? Well, that's kind of an interesting thing to ponder about. You probably knew your parents pretty well in most cases. You probably knew your grandparents, at least to some degree. What do you know about your great-grandparents? The names, where they lived? What about their parents? Anything? Anything? If you're lucky, maybe you've got a family tree, you can pluck some names out, but you really have no experience of what their lives were like. So what happens after four generations kind of passes out of memory? Now, it'll be interesting to see if this changes. Now that you've got Facebook recording everything and NSA making a backup copy. Great. For archival, it's wonderful. Really, we've got archival copies of everything. I don't have to worry now. But this kind of thing kind of falls out of human memory. Once we start keeping all that and you can go back and look at great grandma's Facebook posts, is that going to change the cycle any? I don't know. It'll be interesting to see. So given the last couple generations that have happened here, you start off, you've got the GI and silent generations very hip on a very military metaphor. Business and military was conducted on command and control. The focus was more on process rather than necessarily on action. The idea is that, oh, you just have, you know, as long as you've got a good process, as long as you follow it, it'll just turn out okay. Because the faith is in the process, not necessarily the outcomes. These folks are now in what? They're 60s, they're 80s. These would be the elder statesmen of any industry, any generation. And this is where they're coming from. Next, you've got up the baby boomers. This was a profit generation. They wanted to save the world, collect the whole set. Again, less interested in outcome than approach a tendency to what some other generations might call moralizing or preachy. Because they're all about their values and do this and do that. It's like, well, yeah, okay. But then you get generation X. What some say raised by wolves. This was the greatest entrepreneurial generation in history. But rejects labeling. They have no logo. I want no stinkin' logo. When interviewed in these kinds of studies, they don't identify themselves as Gen X. They identify themselves as not a boomer and not a millennial. Which is kind of interesting. If there's a problem, they'll quit and move on. I ain't fooling with this. Folks from other generations will take different views of it. Well, someone should fix this. Well, I ought to fix this. Whatever. There's different viewpoints. This group will be like, saw it. Move on. Next. Fiercely. Fiercely individualistic. Then you get down to those kids today, get off my lawn. The millennials. They plan ahead, bless their little hearts. Much more upbeat, much more team and community oriented. They're not going to save the world so much. They've got this more of an attitude of, well, somebody in charge should fix this. Kids, I got news for you. You'll find out. So these things kind of go through and cycle and repeat. Now, what I find interesting just from a software development point of view is if you look at the older generations, you see a much greater emphasis on rigid hierarchies and a just hell bent for leather focus on process. You've got to do these steps in this order. Then that moves on to the more GeneXus, oh, just do it. Individualism. I'm okay. Bad luck to you, but just do it. Then a move into the more recent generations. Well, let's get everybody involved. Let's get the team involved. This is more of a community thing. Could it be that one of the reasons that open source is exploding so much is because a different generation is coming up to do it, where they treasure that kind of interaction more than us old fogies? Perhaps. Interesting to think about. But there's a definite parallel here to methodologies as they come down and how we develop software as reflecting the values of whatever generation is kind of moving through. Now, this makes it particularly interesting because at the moment, we've got more generations in play in the workforce than ever before. You got everybody's got their hand at. The poor old folks can't retire because they're not enough money and the young folks are desperately trying to get money for college. You got everybody in the mix, which makes it really quite interesting. So these are things that can subtly affect your approach and your decision making that you're probably not consciously aware of. One more topic I want to go into. That's about skills themselves. Back in the 80s, the brothers Dreyfus, Stuart and Hubert, wanted to research artificial intelligence because they wanted to build an artificial intelligence that would learn skills the same way that humans did. I thought this would be a dynamite idea. Only one problem. They had no idea how humans learned skills. So they had to research that first. They never got around to the AI part particularly. But they did come up with this idea of the Dreyfus model of skill acquisition. What happens is you start off anything as a novice with no experience and work your way up toward expert. And along the way, a lot of things change. The biggest thing they discovered was that as an expert, you're not just a really smart beginner. You fundamentally perceive the world differently. You approach problem solving differently. You value different things. Your decision making process is different. The whole thing is different and gets that way as you climb up the ladder. Let's take a quick look at the stages in the Dreyfus model. Stage one, predictable. You have no experience. You don't really want to learn. You just want to get it done. You don't know how to respond to mistakes and so are vulnerable to confusion. If I'm learning a new programming language, I'm there. This is it. I just want to get this done. What do you mean there's no loops in this language? You have to use recursion. Fine. Okay. Then the compiler spits out, blah, blah, blah. I have no idea what that means. It could mean a permission problem on my installation. It could mean I left off a bloody semicolon. No idea. I'm at a novice level at this particular skill. The only way you can succeed as a novice is if you're given context free rules. When this happens, do that. This is how call centers are able to operate with people answering the phones who really have no domain knowledge. They're novices in the field, but there's a decision tree. My computer doesn't work. Is it plugged in? Yes, no. They go down the tree. You've got rules. Note on the word experience here. Experience in this context means I've done something and I've learned from it. So by that definition, I've filed my taxes for 30 years now. I have no practical experience at it. I'm still a novice. I still don't care. I just want to accomplish a goal. I don't know how to respond to mistakes. The tax service sends me this big yellow letter yelling at me and I have no idea what they're on about. I send it to the accountant. You have to give me context free rules like take all your money, send it in. Okay. That seems about right. So novices need recipes. They need to know, okay, I do this, then I do that. And then you can go on from there. Stage two, you grow up a little bit. You can start trying things on your own that are off the recipe card. There's still difficulty troubleshooting because you've got no sense of the big picture yet, no mental models to work with. You want information faster now. So you're learning a new language. It's like, oh, okay, I want this thing out of their standard library. Where is it? Where is it? That one. Yeah, okay. Wrong one. Okay. It's frustrating because you kind of want to suck the information in very fast. You can begin to follow someone else's advice a little bit but not great yet. Again, you don't have that overall understanding. And worse than that, you don't want the big picture. The big picture at this point in time would just be confusing to you. And you see this problem sometimes in like large corporate meetings where they have an all hands meeting and the sales guy gets up and he's talking about sales things and you're like, oh, just gnaw my leg off and get me out of here. I don't care about this because you are a novice or advanced beginner at that particular topic. So it really doesn't make sense to you. It's just frustrating and annoying. Then we get up to competent. All right, now we're starting to get some traction. At this level, and now for the first time, you begin to develop conceptual models of the topic area. You begin to understand that programming language. When a syntax error comes up, it's like, oh, that's because it's parsing the line like this and it didn't see my comma, semicolon, whatever it was. You can start to troubleshoot problems now that you've got a bit of a model. So the big leap when you get to step three is that you can start to troubleshoot. And that's important. Now we start getting into the real fun. You get up to proficiency. Now you really have to understand the big model. You want to understand the larger framework. You get very frustrated with oversimplified information. You call up that helpline and say, I'm getting a blue screen at death because of this new device driver I just compiled and they say, is it plugged in, sir? Punch him right through the phone. You can self-correct previous poor task performance. Now this is important. This is only at level four. And one of the things that an agile process says is you should learn from your mistakes and correct them. That's dynamite if you're a level four practitioner. Less than that, you need help. You need a coach. You need a mentor. You need somebody who knows what they're doing because you don't have the skill to do it yet. Level four, you do. You can learn from the experience of others. You can understand and apply maxims, proverbs, fundamental pithy sayings that make sense to you in this context. For instance, an XP says test everything that could possibly break. Dynamite, great advice if you know what it means. If you're a beginner, what can break? Hell, everything. Getters and setters could break. Log statements could break. I wrote it. Everything could possibly break. Up the chain a bit, it's like, okay, you're not going to bother with the stupid stuff, but this big hairy calculation where there's an even odd and a divide. I know I'm not going to get that right. Okay. I know my strengths and weaknesses. I know what's likely to break and what's not from my experience. If you're less experienced, you just don't know. And this is where you get into trouble with things like design patterns. For example, a gang of four book comes out and documents the, what was it, 24, 27, 23 design patterns. And experienced people are like, okay, I recognize this is kind of cool. These are patterns that make up for deficiencies in the C++ language that was popular at the time. If you had a real language, you don't need half of these. But that's fine. And everyone else is like, wow, recipes. So true story, I knew this guy on a project, he was writing this little bit of report writer code. He just read the design patterns book. And in this hapless little piece of code, he wedged in something like 19 out of the 23 design patterns because he could. This also, by the way, is why you really want to do code reviews or pair programming. So proficient practitioners can self-correct. That's important. Now we get to the end of the line, we get to the expert. These become the primary sources of knowledge and information looking for new and better methods. Interestingly, experts work from intuition far more than reason. You know, you look at this design, is my design good? No, it sucks. Why? It just does. They know it. They're correct. But they're often very inarticulate about how they arrived at decisions. The doctor looks at you and is like, oh, you've got such and such syndrome. And they'll run the tests and you're right. How did you know? I don't know. He looked wrong. It's just, it's so ingrained. It's so baked in, I can't really articulate it. Also, if you force the expert to follow the rules that you set down for the novices, you drag their performance down to the novice. And they actually did this experiment with airline pilots and showed very conclusively that you wreck expertise by making it follow the rules because experts need to rely on their intuition. So you've got this overall movement from rules to intuition, from considering everything to being able to narrow down to relevant focus, and even such things as a novice will consider themselves over here and the project over there. It's sort of detached from it. The expert understands systems thinking and systems and realizes that they are a part of the system. Everyone on the team is a part of the system. That's a part of the company. It's all interrelated and reacted. So they're much more aware of that kind of viewpoint. Whereas the beginner, not so much. So how you approach problem solving, how you view these things, the decisions you make here, changes depending on your skill level. So even if you've figured out, okay, I'm a left leaning liberal who likes dogs and cats and whatever else from all that other stuff and my personality wants me to do this and whatnot, that's dynamite. That's all going to change. It changes as you gain skill, changes as you get older. So I leave you in a bit of a quandary then. Your brain's really messed up. What are you going to do about it? Oh, my, look at the time. Okay. Well, so we'll just go now. You want to, where possible, and as you get up the expertise ladder in some particular skill, you want to be able to start off with intuition. Once you have enough skill where you can rely on that, you want to start with that, but you can't stop there. You have to verify it. And this is where some experts get into trouble. They get so used to relying on their intuition, facts change. Things change underneath them. You experience that change blindness that we saw with the gradual change studies and the gorilla and the curtain and all of that. And so they begin to rely on their intuition and forget to actually verify it and run into trouble. So we want to actually, where we can, place our trust in actual outcomes. Did the project succeed? Did the code work? Are we actually doing the right thing? And not trust to process alone or metrics or wishful thinking, which is our most popular way. We get the wonderful four side cartoon where they realize, it's time we face reality, my friends, we're not exactly rocket scientists. They look at, they've got the lab coats, they got the little clipboards, but look at the outcome. The outcome didn't make it. It is, he says with a pun, critically important to apply critical thinking. And I think in general, we're lazy and we just don't do this enough. We jump to conclusions, we make assumptions and we tend to ignore actual facts. You know, actual facts become, well, I read it on the internet, you know, it was on Reddit, it must be true. And this is a problem. We need to be better at observing something without bias, actually research statements. You know, I wish, I honestly wish that every time somebody hit the post button on Facebook, it would automatically like search through snopes.com and if there's a match, say, no, you just can't post that. It's stupid, it's a rumor, it's not real, stop posting it. You know, this kind of thing. Look for credible sources if they be any of these days and ask why. This is the favorite consultants trick, right? You get into a situation and it's like, you know, there's oil on the floor in the factory. Why? Well, something must be leaking. Okay, why? Now they're stumped. They go back, you get to five whys later. Well, purchasing had to buy this kind of seal because of this corporate directive and because of that and this and this, you go back and you find out the entire factory's got the wrong size seals in the ductwork because of some stupid PO memo that went through two years ago. Alright? Never stop at one why. Keep going and going and going like a frustrated three-year-old, but why, daddy, until you get to the end and actually get to the root of the problem? And ask yourself when you find yourself pontificating that, you know, Java is dead or Oracle is dead or everybody's programming in Ruby or nobody's programming in Ruby or Elixir is the next hot thing or it's not, whatever your position, first ask yourself, okay, great. How do you know? You know, says who? Are they credible? What are they basing it on? How specifically? What is dead dead? No adoption, less adoption? Kick to the rubbish bin of history? How specifically? So on and so on. And there's some really interesting penetrating questions you can ask yourself. Like what would happen if you did do something you didn't think you should do? What would happen if you didn't do something you thought you were supposed to do? Well, my personal favorite, what stops you from? And this was an actual exchange I had with someone a number of years ago. Said, ask that question. What's stopping you? Nothing, I answered. That gets in my way a lot too. With that in a few minutes for questions, I want to thank you all for coming. My email address, my Twitter handle, link to my book, Pragmatic Thinking and Learning that covers a lot of this kind of material. Thank you all very much. And before I let you go or entertain any questions, I am bade to remind you to put your little cards in. This is a yellow card. It's a bad example. Green cards. You want to put the green cards in the bucket outside to evaluate the talk. Feedback or comments may be written on the cards if you've got really tiny handwriting. So please do that for our good hosts. And are there any questions? Good. All right. Go get cookies and coffee. Thanks again.
 Alright, hello. How was everyone? Thinking about lunch already? Yeah, okay. Dynamite.  You having fun so far? Okay, that's good, that's cool. Alright, well, so if you're anything  like me, by now you've been to a whole bunch of talks and you've heard stuff about C-Sharp  and functional programming and Agile and this and that and your head's about to explode,  right? Good, cool. Well, this will help. This will really make it just burst out. So I  love this, I love this background picture here. Alright, what if my clicker worked?  Come on clicker, you can do it. There we go. Do you know what this is? This is in fact  the first bug. You've heard the story about when they were debugging the, what was it,  the Mark II relay calculator, September 1945 and they had this problem, this was a relay  calculator, they're going through running a sign and cosine unit test, bless their hearts,  they're running it and there was a bug and they found there was a moth trapped in the  relay. They took the moth out, taped it to the log book and that was indeed the first  incident of debugging a computer. 1945, an old joke. So we've had bugs in computers for  a long time, sometimes literally so, but as it turns out, we've got bugs in the very way  that our brains work. We've got quite a few of them in fact. So I want to go through today  and just show some of the more interesting or amusing or common bugs that affect our  decision making, how we write software, how we do agile processes, all these sorts of  things. The first thing I want to talk about is how we're creatures of habit. So here we've  got the policeman out to give tickets behind the sign that says, slow down, the cop hides  behind this sign. Clearly he's still hiding there, just creature of habit. But it's kind  of funny, they do a lot of studies on selective awareness and gradual change and it turns  out we're really not nearly as observant as you might think we are. And I've got a couple  samples here just to show how that works. The first thing I want to show is a fairly  difficult counting task. So what's going to happen is you're going to see a bunch of kids  passing a basketball, wearing some wearing white shirts, some wearing black shirts. And  the idea is you've got to really focus on the players in the white shirts and count  how many times they pass the basketball. Now it's not impossible, this isn't like one of  those trick, you know, sight questions. You can actually do it but you just have to focus  very closely and count the total number of passes. Alright, you ready? Okay, Twitter  down, email down, here we go. We're going to count the number of passes. Count how many  passes? The correct answer is 16 passes. Did you spot the gorilla? For people who haven't  seen or heard about a video like this before, about half missed the gorilla. If you knew  about the gorilla, you probably saw it. But did you notice the curtain changing color or  the player on the black team leaving the game? Let's rewind and watch it again. Here comes  the gorilla and there goes a player and the curtain is changing from red to gold. When  you're looking for a gorilla, you often miss other unexpected events. And that's the monkey  business illusion. Learn more about this illusion and the original gorilla experiment  at theinvisiblegorilla.com. So I used to show the original version of this, just the regular  gorilla experiment where you've got the kids passing the basketball and the gorilla walks  in and they're like, I've got 15, I've got 17. It's like, great, did you see the gorilla?  The what? I need to show it again. And how can you miss it? You see it the second time  and it's like, how do you possibly miss this? But we do. So this has been around the interwebs,  people got used to it. It's like, I know this one, I know about the gorilla. Yeah, great,  did you catch the curtain changing color? Did you catch the player leaving the scene entirely?  Right out the edge. But that's okay. I mean, these are, this is a very busy video. You  know, there's a lot to keep track of, a lot of motion, you're trying to keep track of  the passes. So let's try something a little simpler, perhaps. I'm just going to show a  couple of single still photographs. Nothing's moving, but something's changing. Raise your  hand when you spot the change. How'd you do? Oh, oh dear.  That whole bit of greenery went away. Okay, let's try the next one then. Now, now, now  you got warmed up now. Now you know what to look for. We'll try the next one.  Okay, let's try the next one. Okay, you got warmed up now. Now, you know what to look  for. Okay, a little better. Okay, you got warmed up now. Now, you got warmed up. Now,  okay, only a little better. Okay, this is an easy one. You'll definitely get this.  Yeah, the entire side goes away. So, it's interesting, you know, we're really just not well equipped  to notice slow moving things. You know, gradual change creeps up on us and we just don't notice  it. This is a real problem. If you're on a software project, what's changing? Everything.  Right? The requirements are changing, the technology is changing, there's stuff changing  all around us and we're not really well equipped to see that or to experience it. So, that's  kind of the first interesting thing. But I like to go on, I mean that's just sort of  fun. You know, we're blind as bats. Okay, fine. But I want to talk about some actual  sort of processing errors, if you will, in how our brains operate. If you look on, say,  Wikipedia and you look up cognitive biases, there are some 90 common known cognitive biases.  I've certainly met people who can exceed that number by a good bit, but there's 90 common  ones. And I'm just going to go through some of my favorites here and just talk about them  for a little bit. The first is relying on memory. Our memory is so bad. You know, we  tend to think of memory as like a hard drive. You know, something happened and it gets written  to disk and we've got it. Maybe we'll forget it, but at least what we remember is accurate.  Right? Now, it doesn't work that way. In fact, the way that memory works, every time you  access a memory, you change it just a little bit. You access it enough over the course  of a lifetime, it can change quite a bit to the point where that incident that you think  you remember when you were six years old probably didn't happen that way. Maybe it didn't even  happen at all. It's very easy to implant false memories into people and to suggest things  were very suggestible. And every time you access a memory, that rewriting takes place  in your current context, how you currently understand the world, your current age. You  know, so every time you're accessing it, it changes a little bit. This can get really  difficult on a software project. You're sitting there, it's months and months into it, and  it's like, why did we decide to do XYZ again? Oh, well, because, and you're going to get  15 different answers because everyone will remember it differently. This is why it used  to be good practice to keep like an engineering log or something where you just scribble some  notes to yourself. You know, when we had this meeting, we decided to use this component  because XYZ. That's a great thing to do because then months later you can go back and look,  why did we decide to do that? Oh, yeah. We thought that at the time. Okay, makes sense.  This is different now. Now we can do this, whatever and ever. But if you rely on memory  alone, even communal memory in the team, it's very foggy, not reliable at all. Some other  things that can throw us off the track a bit. There's this great experiment related to a  phenomenon called anchoring. The way this works is our brain is always seeking patterns,  trying to find stuff to compare, but it's not always so picky about what it compares,  and it can get confused really easily. So for instance, if I started talking about the  fact that my publishing company, our publishing company, the Pragmatic Bookshelf, that we  have hundreds of books for you to peruse. We've got hundreds of authors, and I keep  tossing out this word hundreds in various contexts, and then I say, here's a nice leather-bound  edition of Programming Ruby for $85. Your brain somewhere says, oh, that's less. That's  a bargain. It has nothing to do with it. They're completely unrelated. Doesn't matter. Your  brain seizes on that. So they did an experiment with undergrads doing bids in an auction.  And before the experiment, they prepped the group and said, okay, if your Social Security  number, your national identification number, your driver's license number, whatever you  have, take your number out and write down just the last two digits. That's all. And  now we're going to go ahead and bid on this auction. And what they found was people in  the audience who had relatively high last two digits tended to outbid the other students  by anywhere between 60 and 120 percent. They bid twice as much on the objects. They perceived  their value to be twice as much solely because the last two digits of their ID number happened  to be high. We do this kind of thing, unfortunately, all the time. Marketing people know this.  The politicians know this. And they use this to their advantage. You anchor on something  and then you make the wrong decision. We'll talk more about that in a bit. Another interesting  thing is the fundamental attribution error. And I just love the picture. The problem here  is lack of context. We have an innate system. Maybe it's some kind of ego prevention. But  if I've done something terrible to the team or something bad to the code or whatnot, it's  because I had a bad day. I missed breakfast. I didn't get a good night's sleep, whatever.  You have kind of an excuse for it. If you do something bad, it's because you were born  that way. It's because you are from this part of the country or that or from this country  or the other or this political party or this race or this species, whatever it is. It's  because you are you that this happened. Whereas if I did it, that was just context. That was  just whatever. So we have this mistake of attributing something to the fundamental behavior of a  person instead of the context that a person is in. Something to bear in mind. You can  see this a lot on internet troll fests when the comments start flying. It's not that your  position is wrong. It's you are an idiot and your mother has had sex with farm animals  or whatever. It degrades pretty rapidly. So just something to bear in mind. It's context.  It's all about context. Now the next one is particularly bad for agile projects and agile  adoption. And that is a built in need for closure. And Dilbert sums it up quite nicely. Dilbert  says, I didn't have any accurate numbers so I just made this one up. Studies have shown  that accurate numbers aren't any more useful than the ones you make up. How many studies  showed that? 87. You laugh. The problem is and this one is really pernicious because  right every project starts off and some being counter in accounting says, okay, I got there's  a spot in the spreadsheet I've got to put a number in. How much is this going to cost  me? And you give the agile answer, well, we don't know yet. It's exploratory. We'll find  out. No, no, no, no. I need a number to put in the box. I don't have a number. I need  87. Good. Doesn't matter. Doesn't matter. And this is a curious thing. All parties concerned,  including the requester, are well aware the number you just pulled out is rubbish. It's  garbage. They know that. It's okay. It's a number. I have satisfied my need for closure.  I have a number. I know it's wrong. I don't care it's wrong. I have it. I can get my hands  on it. I'm happy. This happens unfortunately all the time. You know, you say, well, you  know, what technology are you going to use here in this project? Well, we haven't decided  yet. I got to fill that out. I have to know. Okay, fine. We'll use SQL Server. We're not  really. But fine. You know, I write it down. Oh, yeah, we changed our mind. We're using  amnesia instead. We're using MySQL or we're using, you know, what have you, Mongo. But  you've got to have that need for closure. This flies in the face of agile development  where we want to leave all our decisions open as long as possible, defer decisions, and  make them late in the game. This is a really hard one to get by. Some folks have an easier  time with this than others. A lot of folks, this is really hard. They have to have closure  even knowing that the number is wrong. Confirmation bias, another popular one. You see this a lot  on the TV news cycle. Any facts or events that don't fit your worldview, you just throw  out. So you just amplify the stuff that you agree with. And of course, the whole world  thinks this way because look, everybody agrees with it and you throw out the rest of it.  Entire books. You know, every author who writes a book with any opinion in it, you could argue  that's just a big example of confirmation bias because they're going to throw out the  other stuff that disagrees with them. That's just how we work.  The exposure effect. This is a great one. The more often you see something, the more  acceptable it becomes to you. The more familiar it is. Politicians know this very, very well.  And that's why they will do everything to get their face out there. They will play sacks  in the bar. They'll get their picture on the poster. They make sure you know their name.  Even if they're a scumbag, you know the name. So when the time comes, you're like, okay,  here's a name I know versus some name I've never heard of, you're going to go with the  familiar. It's the way it works. You see something over and over and over again, even if it's  horrific, even if it's horrible, even if it is the intellectual equivalent of broken glass  on a toilet seat, it's familiar. And you will stick with it to your detriment. Even if the  plane is crashing, you will stick with it. Familiar is not good, but we conflate the  two. Just like in that anchoring experiment, we just get kind of muddled with it and we  confuse them. If something is damaging your project, if it's not good, dump it. Get rid  of it. It doesn't matter how familiar it might be. There's the Hawthorne effect. This is  a lovely one. This is the deal where consultants come in. They're studying your team, studying  your organization, and suddenly everything's working great. They've got some new giga they're  trying to sell, some new process, some new tool. Everyone's using it. Productivity's up.  Numbers are great. Everyone's happy. What happens when they leave and stop watching?  Everything goes right back to the way it was before. We work differently if we know somebody's  watching, at least for a little while. Relativity. So when, what company was it made, the first  bread maker? I've lost track. And the font's too small. William Sonoma. They made the first  automatic bread maker. This thing was a marvel. You open up this little R2-D2-like box, stick  some flour and water in, hit the timer, and some hours later, a baked loaf of fresh bread  comes popping out of it. This was a marvel in what, the early 90s, I suppose it was?  An absolute marvel. The problem is, they were the first to market with it. So how do you  price the thing? Oh gosh, I don't know. So they picked a price, set it, put it out there,  you know. Harold did this technology. This was incredible. Fresh baked bread and all you  do is hit the button. Nobody bought them. Sales were modest at best. And they're scratching  their head. They didn't know why. So they fished around, talked to some psychologist types  and said, okay, I know what we're going to do. We're going to keep this at about the  same price it was, $250. Doesn't matter what units. Pick a unit that's comfortable to you.  But we're going to come out with the luxury model, the high-end model that does many more  things and price it at twice the amount and put those two out there. Suddenly, no one  ever bought the big one. But now they bought the middling one because it was perceived  to be a bargain. Because you start off with the question, well, okay, is this a good value?  Well, I don't know. What does a bread maker cost? I have no idea. They just invented the  thing. But now, what does a bread maker cost? Oh, well, it either costs $500 or $250. Oh,  look, this $250 one is a bargain. It's half the price. Okay, right? I mean, they made  these numbers up completely. These aren't even the real ones. But you make it up completely  because now you've got this false relativity to concern with. It's like, oh, this is less.  I will go for it. We fall for it every time. We fall for free. Free is not the same as  cheap, however. Free is immensely valuable. Free is awesome. Free is better than one cent  or one franc or one kroner or what have you. In fact, when Amazon.com first rolled out  their Amazon Prime option, you could pay a flat rate and then shipping was free. Right?  And they're like, this is going to be dynamite. They rolled it out in the US, sales took off.  They rolled it out in different other countries, Asia, Pacific, Europe. Sales took off. Worked  great everywhere except France. They rolled the offer out in France. Nothing happened.  What was up with that? So they looked into it a bit and they discovered for whatever  reason the French division, when they did this, the free shipping wasn't actually free.  It was cheap. It was like one franc or Euro, whatever they had at the time. It wasn't free.  It was very cheap. Didn't count. People did not fall for it. They changed it to free.  Their sales then matched everyone else. Popped right back up again. Why do we do that? Well,  that's a good question. We're cheap. We like free. We have a very strong sense of loss  aversion. We don't want to lose anything that we already have, which is kind of funny. There's  an example. I'm not sure if it's in this talk or not, so I'll just give it now anyway. Have  you ever heard of the South Indian monkey trap? Wonderful thing. Apparently, there's  a lot of monkeys running around Southern India and you need to trap them. Okay? So what you  do is you dig a hole in the ground with sort of a narrow entry and then a hollowed out part  and you put bananas or bait, whatever down in it. Monkey comes along, sticks his hands  in, grabs the bananas, can't get back out again. The entrance is too narrow. Simple thing  to do, right? Just let go of the bananas and you'd be free of the trap. What does the monkey  do? Right? You sit in there, chattering away, stuck because they will not let go of the  bait. They are so loss averse, they will not let go of the bait and they get trapped. Happens  to us in a similar manner. Sometimes you just got to let go of it. Reasons we fall into  some of these traps. I want to talk about symbolic reduction. So what happens here is  we tend to focus even on the wrong questions a lot of the time. I had to move houses recently  and as part of that I had to clean up the back of my office. So this was a very terrifying  prospect. I'm sitting there going through this mountain of cables, right? And in the  middle of this bundle of cables I found an old 19.2 modem. Wasn't hooked up to anything,  just this ancient modem in this ball of cables. It's like, wow, that was, it's like archaeology,  right in the office. It was awesome. But I also found a bunch of old magazines from sort  of mid 1990s kind of time period. Objects something or other, Java something or other.  I, you know, a bunch of these kind of popular magazines at the time. And the headlines were  the same on all these magazines. The huge questions of the day. Who will win the desktop  wars? Will it be motif or open look? Wrong question entirely. Missed the whole web  thing entirely. And right after that big debate and this went on like a year, two years, these  were the headlines. Another set of headlines. Who will win the middleware wars? What will  be the triumphant technology? Will it be Corba or RMI? Neither, right? Dead, missed the web,  missed it completely, you know, missed the dominance of the PC. Totally the wrong question  even to ask. And all of these, you know, you can go through other periods of history and  find the same thing. Who's going to win A or B? And the answer is almost always E, none  of the above. It's none of the contenders you think it is.  So we make predictions. This is kind of part and parcel as how our brain works is to go  in and try to, you know, see something and predict what's going to happen on a daily  basis. Even just reaching out, grabbing a thing of water, walking. We're constantly  making predictions. But to predict anything longer range than, you know, say even a couple  of minutes, we're really bad at it. Largely because some unexpected event changes the  game. You know, in that case the introduction of the IBM PC, changing the face of the desktop,  later the Mac, advent of the web, all these kinds of things. These were more or less unforeseen  events. So there's a great book out there whose name just went out of my head. Talib  wrote it. Black Swan's in the title somewhere. Google will find it for you. But the idea  of a black swan is that some very high impact, hard to predict event, will change the shape  of what you're trying to predict. And in Talib's book, he's like, okay, anything significant  in history, this is what happens. It's always the big unexpected change that comes in. The  title of the book comes from the idea that, you know, for many years scientists thought  there was no such thing as a black swan. There was only white ones because they'd never seen  one. And one comes trotting by and they're like, oh, would you look at that? Whoops.  I've got to change that a bit. So why does this happen to us? Well, one of the ways the  brain works is to reduce much of your input to a kind of symbolic representation. So this  is coincidentally why none of us probably can draw very well. Right? So you're all programmers,  tech people, right? Raise your hand if you're a great artist. Yeah, don't be shy. Go ahead.  Come on. Yeah, right. No, it's not happening. Because we're so very good at manipulating  and dealing with symbols. What happens is we're very good at symbolic reduction. So when  you say, you know, draw a hand or even mention a hand, somewhere in your brain it's like,  oh, hand, I know that. It's a stick with five lines. And you go to try to draw it and some  part of your brain is shouting, what do you do with all these curves and shades and curves?  It's a stick with five lines. And there you go. So of course what happens is in this reduced  state in our brain, you lose all the detail. You lose all the nuances, all the context.  And that's part of why we fail to see some of these big changes coming on the horizon.  We just shrink the details up and then we don't have them to access. So, okay, that's  all kind of bad. But these are structural things. These are processing things. There's  not much you can do about it except kind of be aware of it. But there's worse things,  I think, perhaps. There's things that affect the very core of our judgment and decision  making. And this is largely because we still cling to this idea that this is the representation  of human decision making, that we're logical, we're rational, we're thoughtful, we think  things through, we make checklists. Unfortunately, that's really not a good model of human decision  making. This is a lot closer to what's really going on. You've got the gibbering monkey  brain inside of you going, wow, squirrel! And it's, you know, if you really want to  down or read the book, Predictably Irrational, which goes through and catalogs an absurd  number of ways that we just think the wrong thing, totally, all the time. But I want to  talk here about something that's maybe less touched upon is things that influence our  decision making. First one's kind of easy. You know, what's your basic personality? Are  you the kind that values realism and common sense, probably? Or the kind that values imagination  and innovation? They're different. Folks can come down on either end of that spectrum.  Some folks will insist on fairness at all costs, even if it hurts them personally. Others  are more into empathy and harmony. Some folks have to be quiet when they're thinking about  it. You think about it in their head until the thought is fully formed, then it comes  out at the meeting. Some folks like us, the jaw just flaps. It just comes out while I'm  trying to form the thought. I sound like a gibbering moron in a meeting. Because it's  well, you know, all this stuff's coming out. And the person sitting at the end just waiting,  they won't say anything for an hour, then come out with some fully fledged thought that's  like, well, that's really bright. Of course, they haven't said a bloody word for an hour,  so it took some time to cook. Some folks genuinely just want to be told what to do. Just give  me the step by step, I'll go through it, I'll go home, play World of Warcraft, life is good.  Some of us think that's horrid. I don't want to follow a bloody list of steps. I want to  figure it out for myself. All fine, all valid. It's baked in differently for each person.  This is going to affect how you approach decision making. So along this line, you've got the  Myers-Briggs personality types. And this is kind of an interesting look. You can be one  of these four values. I'm not going to go through them in any detail, for reasons I'll  explain in just a second. But if you look at the combination of values, you see some  things that they put in these sort of classifications. You're a planner, you're a developer, you're  an analyzer, a composer, you may be your promoter, motivator, explorer, strategist, this sort  of stuff. Really kind of interesting different angles on how you might approach life and  what your baked in philosophy is. There's only one problem with this. It's a bit like  a horoscope in that you really can't fit human behavior into four bits. It's just kind of  a difficult proposition. There is some value to it. And it's worth it. If you haven't taken  one of those Myers-Briggs tests, it's always kind of interesting, but you have to realize  this is a self-assessment. It's going to depend on your mood that day. Depends whether you  answer with your work persona or your home persona. Maybe they're the same for you, maybe  they're different. So it's a little bit fudgy, but it's kind of interesting to look at to  see where your preferences may lie. But deeper than that comes with sort of what you value.  Do you characterize yourself as a liberal, a conservative, an anarchist? Do you know  how hard it was to find an anarchist logo? Just saying. That was less obvious than I  thought it would be. But the problem is, okay, so you identify yourself one of these ways,  right, conservative, liberal, what have you. But why? Were those your parents' values?  Was that directly opposed to your parents' values? Were you just born that way? Yes.  All the above. This can all happen. But what's interesting is, do you know what makes the  biggest difference as to what kind of values you hold and how this works? It depends on  who else was born the same time as you. Your cohort, your peers, your birthmates. The researchers  who look into this kind of stuff say that you have more in common with somebody born  at the same time who lives in Mexico City or Tokyo than you do with somebody right next  door who is 10, 20 years older or younger. So the power of the cohort is really quite  strong because of the sort of shared memories, habits, styles, significant events that happen.  Depending on your age, you might remember where you were when the Vietnam War ended.  I remember that. When Elvis died, when Reagan was shot, 9-11, all these sorts of major events  that happen that your cohort will remember the same way. Because what happens is, if  you're 20 and one of these events happens, you'll have one outlook on it. If you're 40  or 60, you're going to perceive that same event quite differently just because you had  a different point in your life. So you and your cohort tend to view events in a very  similar way. And that sort of drives us a bit. I love this poster here because this  is the one where the young ones are like, wait, which Star Wars was that? There's no  number. So what kind of differences can this bring about? Well, different generational groups,  some are going to be much more prone to taking risks. Some will be very risk averse and not  want to hear about it. Some will really value individualism. Others not so much. They favor  more of a teamwork approach. Some value stability. Others put their faith in freedom and will  accept a little chaos for that. Some focus on family. Other generations more on those  to the grindstone work. So what's interesting is, according to such researchers, there's  actually only four different kinds of generations and they repeat. So they did their research  starting in Europe around the Renaissance and followed it over to the Americas. And  in 20 generations of their study, there was only one gap in the pattern and that was in  the American Civil War in the 1800s because we killed everybody. So there wasn't anyone  there to kind of take their place in society as they would have normally. Other than that  gap, they found this very regular repeating pattern of four archetypes. You have a profit  generation, a nomad, a hero, and an artist. And in general, I'll give the caveat in generalities  in just a second, the profit generation values values, values vision. The nomad, not so much  into that, they value their liberty, their survival, their honor. Or if you'd in Klingon,  their honor. The hero generation, much more into community, into affluence. Let's share  all this together. The artist, pluralism, expertise, do process, do it by the numbers.  So they mapped this out onto the last couple generations and tacked cute little names on  them because that makes for good press. So you have the GI generation, which was a hero  generation, people born back around the turn of the previous century. The silent generation  after that, an artist generation. The baby boomers, the boom generation, a profit. Generation  X, the nomads, millennials, back to hero again, and then the newest homeland generation back  to the artist. So a couple things here. First of all, a caveat. When you talk about a generation  of people having this particular characteristic, it does not mean that everyone born at this  time thinks this way. What it means is that if you take on the aggregate a whole bunch  of people, the folks born in this bucket will tend to share these values and these outlooks,  more than people on any other adjacent one. The dates are not firm. They're kind of, yeah,  about then. So you can be born on a cuspier and you might feel partly this way, partly  that way. These things happen. That's okay. But here's a question. Why does the cycle  only go for four generations? Why does it then repeat? Well, that's kind of an interesting  thing to ponder about. You probably knew your parents pretty well in most cases. You probably  knew your grandparents, at least to some degree. What do you know about your great-grandparents?  The names, where they lived? What about their parents? Anything? Anything? If you're lucky,  maybe you've got a family tree, you can pluck some names out, but you really have no experience  of what their lives were like. So what happens after four generations kind of passes out  of memory? Now, it'll be interesting to see if this changes. Now that you've got Facebook  recording everything and NSA making a backup copy. Great. For archival, it's wonderful.  Really, we've got archival copies of everything. I don't have to worry now. But this kind of  thing kind of falls out of human memory. Once we start keeping all that and you can go back  and look at great grandma's Facebook posts, is that going to change the cycle any? I don't  know. It'll be interesting to see. So given the last couple generations that have happened  here, you start off, you've got the GI and silent generations very hip on a very military  metaphor. Business and military was conducted on command and control. The focus was more  on process rather than necessarily on action. The idea is that, oh, you just have, you know,  as long as you've got a good process, as long as you follow it, it'll just turn out okay.  Because the faith is in the process, not necessarily the outcomes. These folks are now in what?  They're 60s, they're 80s. These would be the elder statesmen of any industry, any generation.  And this is where they're coming from. Next, you've got up the baby boomers. This was a  profit generation. They wanted to save the world, collect the whole set. Again, less  interested in outcome than approach a tendency to what some other generations might call  moralizing or preachy. Because they're all about their values and do this and do that.  It's like, well, yeah, okay. But then you get generation X. What some say raised by wolves.  This was the greatest entrepreneurial generation in history. But rejects labeling. They have  no logo. I want no stinkin' logo. When interviewed in these kinds of studies, they don't identify  themselves as Gen X. They identify themselves as not a boomer and not a millennial. Which  is kind of interesting. If there's a problem, they'll quit and move on. I ain't fooling  with this. Folks from other generations will take different views of it. Well, someone  should fix this. Well, I ought to fix this. Whatever. There's different viewpoints. This  group will be like, saw it. Move on. Next. Fiercely. Fiercely individualistic. Then you  get down to those kids today, get off my lawn. The millennials. They plan ahead, bless their  little hearts. Much more upbeat, much more team and community oriented. They're not going  to save the world so much. They've got this more of an attitude of, well, somebody in  charge should fix this. Kids, I got news for you. You'll find out. So these things kind  of go through and cycle and repeat. Now, what I find interesting just from a software development  point of view is if you look at the older generations, you see a much greater emphasis  on rigid hierarchies and a just hell bent for leather focus on process. You've got to  do these steps in this order. Then that moves on to the more GeneXus, oh, just do it. Individualism.  I'm okay. Bad luck to you, but just do it. Then a move into the more recent generations.  Well, let's get everybody involved. Let's get the team involved. This is more of a community  thing. Could it be that one of the reasons that open source is exploding so much is because  a different generation is coming up to do it, where they treasure that kind of interaction  more than us old fogies? Perhaps. Interesting to think about. But there's a definite parallel  here to methodologies as they come down and how we develop software as reflecting the  values of whatever generation is kind of moving through. Now, this makes it particularly interesting  because at the moment, we've got more generations in play in the workforce than ever before.  You got everybody's got their hand at. The poor old folks can't retire because they're  not enough money and the young folks are desperately trying to get money for college. You got everybody  in the mix, which makes it really quite interesting. So these are things that can subtly affect  your approach and your decision making that you're probably not consciously aware of.  One more topic I want to go into. That's about skills themselves. Back in the 80s, the brothers  Dreyfus, Stuart and Hubert, wanted to research artificial intelligence because they wanted  to build an artificial intelligence that would learn skills the same way that humans did.  I thought this would be a dynamite idea. Only one problem. They had no idea how humans  learned skills. So they had to research that first. They never got around to the AI part  particularly. But they did come up with this idea of the Dreyfus model of skill acquisition.  What happens is you start off anything as a novice with no experience and work your way  up toward expert. And along the way, a lot of things change. The biggest thing they discovered  was that as an expert, you're not just a really smart beginner. You fundamentally perceive  the world differently. You approach problem solving differently. You value different things.  Your decision making process is different. The whole thing is different and gets that  way as you climb up the ladder. Let's take a quick look at the stages in the Dreyfus  model. Stage one, predictable. You have no experience. You don't really want to learn.  You just want to get it done. You don't know how to respond to mistakes and so are vulnerable  to confusion. If I'm learning a new programming language, I'm there. This is it. I just want  to get this done. What do you mean there's no loops in this language? You have to use  recursion. Fine. Okay. Then the compiler spits out, blah, blah, blah. I have no idea what  that means. It could mean a permission problem on my installation. It could mean I left off  a bloody semicolon. No idea. I'm at a novice level at this particular skill. The only way  you can succeed as a novice is if you're given context free rules. When this happens, do that.  This is how call centers are able to operate with people answering the phones who really  have no domain knowledge. They're novices in the field, but there's a decision tree.  My computer doesn't work. Is it plugged in? Yes, no. They go down the tree. You've got  rules. Note on the word experience here. Experience in this context means I've done something  and I've learned from it. So by that definition, I've filed my taxes for 30 years now. I have  no practical experience at it. I'm still a novice. I still don't care. I just want to  accomplish a goal. I don't know how to respond to mistakes. The tax service sends me this  big yellow letter yelling at me and I have no idea what they're on about. I send it to  the accountant. You have to give me context free rules like take all your money, send  it in. Okay. That seems about right. So novices need recipes. They need to know, okay, I do  this, then I do that. And then you can go on from there. Stage two, you grow up a little  bit. You can start trying things on your own that are off the recipe card. There's still  difficulty troubleshooting because you've got no sense of the big picture yet, no mental  models to work with. You want information faster now. So you're learning a new language.  It's like, oh, okay, I want this thing out of their standard library. Where is it? Where  is it? That one. Yeah, okay. Wrong one. Okay. It's frustrating because you kind of want  to suck the information in very fast. You can begin to follow someone else's advice  a little bit but not great yet. Again, you don't have that overall understanding. And  worse than that, you don't want the big picture. The big picture at this point in time would  just be confusing to you. And you see this problem sometimes in like large corporate  meetings where they have an all hands meeting and the sales guy gets up and he's talking  about sales things and you're like, oh, just gnaw my leg off and get me out of here. I  don't care about this because you are a novice or advanced beginner at that particular topic.  So it really doesn't make sense to you. It's just frustrating and annoying. Then we get  up to competent. All right, now we're starting to get some traction. At this level, and now  for the first time, you begin to develop conceptual models of the topic area. You begin to understand  that programming language. When a syntax error comes up, it's like, oh, that's because it's  parsing the line like this and it didn't see my comma, semicolon, whatever it was. You  can start to troubleshoot problems now that you've got a bit of a model. So the big leap  when you get to step three is that you can start to troubleshoot. And that's important.  Now we start getting into the real fun. You get up to proficiency. Now you really have  to understand the big model. You want to understand the larger framework. You get very frustrated  with oversimplified information. You call up that helpline and say, I'm getting a blue  screen at death because of this new device driver I just compiled and they say, is it  plugged in, sir? Punch him right through the phone. You can self-correct previous poor  task performance. Now this is important. This is only at level four. And one of the things  that an agile process says is you should learn from your mistakes and correct them. That's  dynamite if you're a level four practitioner. Less than that, you need help. You need a  coach. You need a mentor. You need somebody who knows what they're doing because you don't  have the skill to do it yet. Level four, you do. You can learn from the experience of others.  You can understand and apply maxims, proverbs, fundamental pithy sayings that make sense  to you in this context. For instance, an XP says test everything that could possibly break.  Dynamite, great advice if you know what it means. If you're a beginner, what can break?  Hell, everything. Getters and setters could break. Log statements could break. I wrote  it. Everything could possibly break. Up the chain a bit, it's like, okay, you're not  going to bother with the stupid stuff, but this big hairy calculation where there's an  even odd and a divide. I know I'm not going to get that right. Okay. I know my strengths  and weaknesses. I know what's likely to break and what's not from my experience. If you're  less experienced, you just don't know. And this is where you get into trouble with things  like design patterns. For example, a gang of four book comes out and documents the, what  was it, 24, 27, 23 design patterns. And experienced people are like, okay, I recognize this is  kind of cool. These are patterns that make up for deficiencies in the C++ language that  was popular at the time. If you had a real language, you don't need half of these. But  that's fine. And everyone else is like, wow, recipes. So true story, I knew this guy on  a project, he was writing this little bit of report writer code. He just read the design  patterns book. And in this hapless little piece of code, he wedged in something like  19 out of the 23 design patterns because he could. This also, by the way, is why you really  want to do code reviews or pair programming. So proficient practitioners can self-correct.  That's important. Now we get to the end of the line, we get to the expert. These become  the primary sources of knowledge and information looking for new and better methods. Interestingly,  experts work from intuition far more than reason. You know, you look at this design,  is my design good? No, it sucks. Why? It just does. They know it. They're correct. But they're  often very inarticulate about how they arrived at decisions. The doctor looks at you and  is like, oh, you've got such and such syndrome. And they'll run the tests and you're right.  How did you know? I don't know. He looked wrong. It's just, it's so ingrained. It's so  baked in, I can't really articulate it. Also, if you force the expert to follow the rules  that you set down for the novices, you drag their performance down to the novice. And they  actually did this experiment with airline pilots and showed very conclusively that you  wreck expertise by making it follow the rules because experts need to rely on their intuition.  So you've got this overall movement from rules to intuition, from considering everything  to being able to narrow down to relevant focus, and even such things as a novice will consider  themselves over here and the project over there. It's sort of detached from it. The  expert understands systems thinking and systems and realizes that they are a part of the system.  Everyone on the team is a part of the system. That's a part of the company. It's all interrelated  and reacted. So they're much more aware of that kind of viewpoint. Whereas the beginner,  not so much. So how you approach problem solving, how you view these things, the decisions  you make here, changes depending on your skill level. So even if you've figured out, okay,  I'm a left leaning liberal who likes dogs and cats and whatever else from all that other  stuff and my personality wants me to do this and whatnot, that's dynamite. That's all  going to change. It changes as you gain skill, changes as you get older. So I leave you in  a bit of a quandary then. Your brain's really messed up. What are you going to do about  it? Oh, my, look at the time. Okay. Well, so we'll just go now. You want to, where possible,  and as you get up the expertise ladder in some particular skill, you want to be able  to start off with intuition. Once you have enough skill where you can rely on that, you  want to start with that, but you can't stop there. You have to verify it. And this is  where some experts get into trouble. They get so used to relying on their intuition,  facts change. Things change underneath them. You experience that change blindness that  we saw with the gradual change studies and the gorilla and the curtain and all of that.  And so they begin to rely on their intuition and forget to actually verify it and run into  trouble. So we want to actually, where we can, place our trust in actual outcomes. Did  the project succeed? Did the code work? Are we actually doing the right thing? And not  trust to process alone or metrics or wishful thinking, which is our most popular way.  We get the wonderful four side cartoon where they realize, it's time we face reality, my  friends, we're not exactly rocket scientists. They look at, they've got the lab coats, they  got the little clipboards, but look at the outcome. The outcome didn't make it. It is,  he says with a pun, critically important to apply critical thinking. And I think in general,  we're lazy and we just don't do this enough. We jump to conclusions, we make assumptions  and we tend to ignore actual facts. You know, actual facts become, well, I read it on the  internet, you know, it was on Reddit, it must be true. And this is a problem. We need to  be better at observing something without bias, actually research statements. You know, I  wish, I honestly wish that every time somebody hit the post button on Facebook, it would  automatically like search through snopes.com and if there's a match, say, no, you just  can't post that. It's stupid, it's a rumor, it's not real, stop posting it. You know,  this kind of thing. Look for credible sources if they be any of these days and ask why.  This is the favorite consultants trick, right? You get into a situation and it's like, you  know, there's oil on the floor in the factory. Why? Well, something must be leaking. Okay,  why? Now they're stumped. They go back, you get to five whys later. Well, purchasing had  to buy this kind of seal because of this corporate directive and because of that and this and  this, you go back and you find out the entire factory's got the wrong size seals in the  ductwork because of some stupid PO memo that went through two years ago. Alright? Never  stop at one why. Keep going and going and going like a frustrated three-year-old, but  why, daddy, until you get to the end and actually get to the root of the problem? And ask yourself  when you find yourself pontificating that, you know, Java is dead or Oracle is dead or  everybody's programming in Ruby or nobody's programming in Ruby or Elixir is the next  hot thing or it's not, whatever your position, first ask yourself, okay, great. How do you  know? You know, says who? Are they credible? What are they basing it on? How specifically?  What is dead dead? No adoption, less adoption? Kick to the rubbish bin of history? How specifically?  So on and so on. And there's some really interesting penetrating questions you can ask yourself.  Like what would happen if you did do something you didn't think you should do? What would  happen if you didn't do something you thought you were supposed to do? Well, my personal  favorite, what stops you from? And this was an actual exchange I had with someone a number  of years ago. Said, ask that question. What's stopping you? Nothing, I answered. That gets  in my way a lot too. With that in a few minutes for questions, I want to thank you all for  coming. My email address, my Twitter handle, link to my book, Pragmatic Thinking and Learning  that covers a lot of this kind of material. Thank you all very much. And before I let  you go or entertain any questions, I am bade to remind you to put your little cards in.  This is a yellow card. It's a bad example. Green cards. You want to put the green cards  in the bucket outside to evaluate the talk. Feedback or comments may be written on the  cards if you've got really tiny handwriting. So please do that for our good hosts. And  are there any questions? Good. All right. Go get cookies and coffee. Thanks again.
Andy_Hunt_-_Debugging_your_mind-68378949_0_59.png Andy_Hunt_-_Debugging_your_mind-68378949_60_107.png Andy_Hunt_-_Debugging_your_mind-68378949_108_1163.png Andy_Hunt_-_Debugging_your_mind-68378949_1164_3023.png Andy_Hunt_-_Debugging_your_mind-68378949_3024_3899.png Andy_Hunt_-_Debugging_your_mind-68378949_3900_4955.png Andy_Hunt_-_Debugging_your_mind-68378949_4956_5243.png Andy_Hunt_-_Debugging_your_mind-68378949_5244_5387.png Andy_Hunt_-_Debugging_your_mind-68378949_5388_5471.png Andy_Hunt_-_Debugging_your_mind-68378949_5472_5531.png Andy_Hunt_-_Debugging_your_mind-68378949_5532_5615.png Andy_Hunt_-_Debugging_your_mind-68378949_5616_5891.png Andy_Hunt_-_Debugging_your_mind-68378949_5892_6095.png Andy_Hunt_-_Debugging_your_mind-68378949_6096_6287.png Andy_Hunt_-_Debugging_your_mind-68378949_6288_6527.png Andy_Hunt_-_Debugging_your_mind-68378949_6528_6635.png Andy_Hunt_-_Debugging_your_mind-68378949_6636_6707.png Andy_Hunt_-_Debugging_your_mind-68378949_6708_6995.png Andy_Hunt_-_Debugging_your_mind-68378949_6996_7067.png Andy_Hunt_-_Debugging_your_mind-68378949_7068_7247.png Andy_Hunt_-_Debugging_your_mind-68378949_7248_8747.png Andy_Hunt_-_Debugging_your_mind-68378949_8748_9071.png Andy_Hunt_-_Debugging_your_mind-68378949_9072_9191.png Andy_Hunt_-_Debugging_your_mind-68378949_9192_9551.png Andy_Hunt_-_Debugging_your_mind-68378949_9552_9707.png Andy_Hunt_-_Debugging_your_mind-68378949_9708_9839.png Andy_Hunt_-_Debugging_your_mind-68378949_9840_10055.png Andy_Hunt_-_Debugging_your_mind-68378949_10056_10403.png Andy_Hunt_-_Debugging_your_mind-68378949_10404_10547.png Andy_Hunt_-_Debugging_your_mind-68378949_10548_10679.png Andy_Hunt_-_Debugging_your_mind-68378949_10680_10895.png Andy_Hunt_-_Debugging_your_mind-68378949_10896_11243.png Andy_Hunt_-_Debugging_your_mind-68378949_11244_11375.png Andy_Hunt_-_Debugging_your_mind-68378949_11376_11519.png Andy_Hunt_-_Debugging_your_mind-68378949_11520_12935.png Andy_Hunt_-_Debugging_your_mind-68378949_12936_13379.png Andy_Hunt_-_Debugging_your_mind-68378949_13380_14339.png Andy_Hunt_-_Debugging_your_mind-68378949_14340_16403.png Andy_Hunt_-_Debugging_your_mind-68378949_16404_21851.png Andy_Hunt_-_Debugging_your_mind-68378949_21852_24911.png Andy_Hunt_-_Debugging_your_mind-68378949_24912_25751.png Andy_Hunt_-_Debugging_your_mind-68378949_25752_27527.png Andy_Hunt_-_Debugging_your_mind-68378949_27528_28379.png Andy_Hunt_-_Debugging_your_mind-68378949_28380_31475.png Andy_Hunt_-_Debugging_your_mind-68378949_31476_38123.png Andy_Hunt_-_Debugging_your_mind-68378949_38124_38171.png Andy_Hunt_-_Debugging_your_mind-68378949_38172_39119.png Andy_Hunt_-_Debugging_your_mind-68378949_39120_40427.png Andy_Hunt_-_Debugging_your_mind-68378949_40428_42971.png Andy_Hunt_-_Debugging_your_mind-68378949_42972_43835.png Andy_Hunt_-_Debugging_your_mind-68378949_43836_46679.png Andy_Hunt_-_Debugging_your_mind-68378949_46680_48155.png Andy_Hunt_-_Debugging_your_mind-68378949_48156_48731.png Andy_Hunt_-_Debugging_your_mind-68378949_48732_49247.png Andy_Hunt_-_Debugging_your_mind-68378949_49248_51287.png Andy_Hunt_-_Debugging_your_mind-68378949_51288_52307.png Andy_Hunt_-_Debugging_your_mind-68378949_52308_53075.png Andy_Hunt_-_Debugging_your_mind-68378949_53076_54119.png Andy_Hunt_-_Debugging_your_mind-68378949_54120_57731.png Andy_Hunt_-_Debugging_your_mind-68378949_57732_58775.png Andy_Hunt_-_Debugging_your_mind-68378949_58776_59411.png Andy_Hunt_-_Debugging_your_mind-68378949_59412_60695.png Andy_Hunt_-_Debugging_your_mind-68378949_60696_61439.png Andy_Hunt_-_Debugging_your_mind-68378949_61440_65315.png Andy_Hunt_-_Debugging_your_mind-68378949_65316_68207.png Andy_Hunt_-_Debugging_your_mind-68378949_68208_69143.png Andy_Hunt_-_Debugging_your_mind-68378949_69144_69959.png Andy_Hunt_-_Debugging_your_mind-68378949_69960_70703.png Andy_Hunt_-_Debugging_your_mind-68378949_70704_70871.png Andy_Hunt_-_Debugging_your_mind-68378949_70872_74843.png Andy_Hunt_-_Debugging_your_mind-68378949_74844_74999.png Andy_Hunt_-_Debugging_your_mind-68378949_75000_76535.png Andy_Hunt_-_Debugging_your_mind-68378949_76536_78323.png Andy_Hunt_-_Debugging_your_mind-68378949_78324_80063.png Andy_Hunt_-_Debugging_your_mind-68378949_80064_80639.png Andy_Hunt_-_Debugging_your_mind-68378949_80640_83075.png Andy_Hunt_-_Debugging_your_mind-68378949_83076_84395.png Andy_Hunt_-_Debugging_your_mind-68378949_84396_84779.png Andy_Hunt_-_Debugging_your_mind-68378949_84780_86039.png Andy_Hunt_-_Debugging_your_mind-68378949_86040_86099.png Andy_Hunt_-_Debugging_your_mind-68378949_86100_86140.png