Monday, July 08, 2019

The Jist

Hey all, happy Sunday!

Tomorrow is my first day working at Google. This has been a dream of mine for a long time and I'm very excited about it. I'm grateful for the support and coaching of my wife. She has taught me how to be brave and pursue the things I want in life. I'm stoked to give this whole thing a shot.

Given that I start tomorrow, and that I have no idea what the intellectual property rules are going to be like, I wanted to vomit out the jist of the software dev book I've been thinking about writing.

I really don't expect it to be good. It's just a rehash of the things I've learned in the first five years of doing this professionally. My desire is that it can stand as a replacement of me to my old team at Xima. It's an opinionated approach to how you should write software. Ready? Let's do this thing.

You suck at programming. And it's not your fault, really. But you suck at it. And you need to understand how you suck at it so you can compensate for your inherent suckiness. One of the core values we're going to try to live by is "Don't Suck".

Why do you suck at programming? There are three reasons.

1. You suck at understanding the rest of your code. 

Your brain can only hold a finite amount of data before it starts losing things. You can think of it as a stack. You can pop stuff on there repeatedly, but you're going to start losing things off the back of it as soon as you pop too much. This is just how you're built. It's not your fault. There's a decent amount of research into this. You can only handle so much cognitive load before things start sucking. Your physical condition affects this too. Less sleep leads to less cognitive capacity. Some people will have less cognitive capacity (smaller stack size) because of a traumatic brain injury or a urinary tract infection. You might be amazing and have a relatively large stack size. But suffice it to say: There exists a program that is too large for you to understand all at once. You don't have enough working memory to keep it all available.

Essentially: You are limited in how much you can understand of your software at one point. You have a moving window of comprehension that precludes you from writing your software well. We're going to refer to this problem as "You suck at understanding the rest of your code". In this usage, "the rest of your code" means all the code that is not currently loaded into your moving window of comprehension.

2. You suck at knowing what the code you wrote does. 

Thought experiment. Indulge me on this one. You're in a high stakes programming situation. There's a beaver dam that will be blown to shreds in five minutes unless you can write a specific method correctly. You know what the input looks like, and you know what the output should look like. You have to read data in from a flat text tile, and write out a new file with the solution in it.

You write code for four minutes straight. This is in your wheelhouse. These are not things you've never done before.

At five minutes it's time to see if you saved the beavers or if they all died and became hats. How confident are you that your dam survived? Based on your normal work history, what is the probability that they aren't hats?

If you're anything like me it's pretty fetching low. Nobody doesn't have this problem. No matter how good we think we are, we always make mistakes. Or maybe we don't make mistakes but our environment is janky. We rarely understand the real details of what we're asking the computer to do for us. We're frequently surprised.

I hope that an inspection of your history will convince you that this is accurate. I have so little confidence in the code that I write. Fun fact: I have even less confidence in the code that you write. The process of translating what we want the computer to do into the language that it understands is incredibly leaky. Even though you wrote the code, you just don't have high certainty about what it does. Really.

3. You suck at knowing what your code should do.

You're going to write software and then it is going to execute. It's going to execute in foreign environments with user-generated input. There is virtually no way that you can predict or test for all of these variables.

That software, assuming it works at all, is going to give users some level of satisfaction. Based on that satisfaction they are going to award you some amount of money. In this scenario, both satisfaction and awarded money might be less than zero. Lol.

How do you maximize satisfaction? How do you minimize disasters based on environmental things you don't control? One solution is to never give your software to users. That provides a very solid lower-bound for satisfaction and requests for refunds. Unfortunately, it also provides a frustratingly immutable upper-bound as well.

Here's the real kicker: You have no idea what your users want. And you have no idea how your software is going to behave when it's in the real world. Will it crash given very specific input? Will it offend your customers? Will it be a smash hit? You're a programmer. Who fetching knows? Not you.


So, you suck at writing software. My condolences.

Let's briefly talk about what humanity has learned about overcoming these problems. Please note that we can't completely negate them. They will always be with us. But we can use strategies to make them less painful for us. At the end of the day, you're never going to be a good programmer that doesn't have these weaknesses. You're just going to put practices in place that negate some of their suckiness.

Problem 1 was that you don't understand the rest of your code. There are a few solutions to this.

First: You minimize the cognitive cost of understanding any one unit of code. As you write a new {module, class, method}, you make sure you write it cleanly a la Uncle Bob. Except without the racism. You name things well. You refactor for clarity. In fact, you make clarity the most important freakin' thing about that code. That's all you actually care about. It does the job, and it's incredibly clear. By reducing the cognitive cost you allow yourself and others to fit more code in their brains at once. This leads to fewer window errors.

Second: You leverage abstraction appropriately. You know what abstraction is? Abstraction is "only caring about the parts of something that you really need to care about". Let me give you an example.

Suppose your code uses a Myles. Myles, in the real world, is a person with glasses, hair, and an incredibly diverse array of bacteria in his gut. But representing that in code would be very difficult. What your code cares about is Myles' ability to writeCode().

So your class DevTeam has a Set of Myles in it. But that's just stupid, because your DevTeam does not need to know about Myles' digestive system. So you make Myles implement an interface called CanFetchingWriteCode. And now your DevTeam has a Set of CanFetchingWriteCode. 

You've taken Myles, a very complicated idea, and abstracted it down into what you really care about. Ignore the other parts. They are neat, but not important to your DevTeam.

Third thing here: Write modular code. Reduce coupling. Specifically: Make sure your code in DevTeam knows as little as reasonably possible about your code over in CafeteriaLine. If understanding DevTeam requires understanding CafeteriaLine, your cognitive cost here goes through the roof. Do what you can to decouple them.


Problem 2 is that you don't really know what your code does until it executes. This one is stupidly simple to solve. You just execute your code all the dang time.

Write automated tests that execute the code you just wrote. That's it. That's the whole tweet. Just write tests. Are you thinking about committing some code to the repo? You should execute it first. And the only responsible way to execute it is to wrap a freakin' test around it.

"But Chris, the code I just wrote is resistant to testing!" Well dang, you just wrote code that sucks.

But in a more generous tone; Take whatever "change" that you just made to the existing code and isolate it into something that can be tested. And then test that!

The second solution to this problem is to run your full software frequently. This would likely involve releasing it to customers far more frequently that you're used to.

"But Chris, I can't release this software! It will break something!". You're absolutely right. Your job is to reduce the pain of failure. Here's something that you should really just accept: You're going to break things. I don't care how much QA you have. I don't care how carefully you test. If you're releasing software you are going to freaking break things.

Given that breakages are going to occur, let's just accept that they will happen and let's do what we can to make those breakages less costly. This is called harm reduction. It's the same principle behind needle exchanges, btw. We're going to release our new software to a small percentage of customers first. We're going to have a way to roll customers back to a safe version without a disruption of service. We're going to be able to toggle on and off new features at runtime to get things into a working state.

Can we talk about needle exchanges? The idea is that people are going to do drugs. If they don't have clean needles some percentage of them will use dirty needles and get Hep C or something and cost the state more money. In order to reduce the money the state spends on Hep C, the state sets up a needle exchange booth where people can trade dirty needles for new needles. This reduces the incidence of Hep C and saves money (and lives, maybe). It also creates a safe place for people using drugs to get in contact with people who could offer help. I'm a big proponent of needle exchanges.

There are a lot of less-progressive voices that think needle exchanges are stupid. They dislike the idea of helping anyone do drugs. Offering needles equates to assisting someone in doing drugs, and that feels wrong. We should not be promoting this behavior.

I'ma punch that straw man down, if you'll allow me. The drugs are going to happen either way. That is not something we can stop. Would that we could. But we just can't Nemo. We can accept this truth and try to reduce harm, or we can complain about it and let people get Hep C and then pay for their medical bills. Harm reduction is a cool idea.

We're going to get our software out to the real world as soon as possible and we're going to gather data about how it behaves. We are going to release bad software. But that was going to happen anyways! The key here is to release software that is carefully bad. Specifically: we're going to focus on reducing our mean time to recovery instead of reducing our number of incidents. Releasing 100 bugs and recovering from all of them in 10 seconds each (1000 seconds of downtime) is so much better than releasing exactly one bug with 5000 seconds of downtime.

We're going to stop trying to never release a bug. But we're going to make the cost of bugs much lower.


Problem 3 is that you don't know what your code should do. Or, you don't know what the customer wants. Or, you don't know what the market is going to do in three weeks so you might be building the wrong thing anyways.

First solution here is to release frequently. Yeah, you know how frequently you're thinking of releasing? Consider releasing faster than that.

Release and get feedback. Keep your code in an always deployable state.

How do you keep your code in a deployable state? You use trunk based development (aka continuous integration) and you take measures to commit safe code. That means you put risky code behind run-time toggler (look up the article(s) about feature toggles on Martin Fowler's site already...) and you protect your butt with a nice deployment pipeline.

When you're tempted to put your code in a non-deployable state, ask yourself: "Am I willing to throw this code away if priority changes in one week?" If the answer is no, then take measures to maintain releasability.


K, that was a fast push through the jist of it. I hope that doesn't suck. Good luck and have fun!

Please hit me up with feedback in the comments section. Thanks.