Based in London, united kingdom. working as a software engineer for palantir technologies. thoughts and ideas expressed are my own.

The top 1%

The top 1%

How does a software engineer make it to the 99th percentile bracket?

— Dijksrtra

My first inclination was to disclaim, “Not top 1%, maybe top N%” where N is some slightly higher integer, because false precision makes great humble-brags. In reality, I have a hard time identifying “top 1%” (where that line is, which side of it a software engineer would be on) because it depends on who’s counted in the population as “software engineers” and who isn’t. Do we count the data scientists using R? The business analysts writing Excel? And for that extend, I will first answer the question of how to become an elite engineer (2.0 on a scale I will soon introduce). 

Programmers tend to separate into three very broad (and overlapping) categories:

  • Adders (“level 1”, 0.0 to 1.4) who can solve most programming problems, given enough time. These tend to be the ones who do line-of-business work, write scripts to fetch data and answer questions that executives or traders care about, etc. The good ones want to progress beyond this level, remove themselves from parochial details and work on more general “core” problems.

  • Multipliers (“level 2”; 1.5 to 2.4) who can select technologies, make architectural decisions, and write or maintain useful libraries. They tend to have open-source contributions to their name, speak at conferences, and are often executive-equivalent in importance to their companies.

  • Global Multipliers (“level 3”; 2.5+) who build communities, create platforms and come up with concepts like Google’s MapReduce. Their ambitions tend to out-scope most companies so they’re often in academia, consulting independently, or highly paid in large firms’ R&D labs.

These three abstraction levels don’t tell the whole story, and of course some “adder” tasks are very hard (like building a production trading system) and could not be done by most “level 1” programmers. Then there’s the altogether different issue of unqualified people being given “level 2” (or 3) work, such as architecture of a complex software system. Perhaps a hard L1 task is really 1.3 or 1.5 or even higher. This is just the theory.

To assess programmer level and give more of a continuum (from 0.0 to 3.0) we add a decimal point and assume that, e.g., a 2.0 programmer will succeed at 95% of level-2 tasks (and likewise for L1 and L3). At 1.5, it’s 50%. At 1.0, it’s 5% (relative to L2) and 95% of L1 tasks. In building the model, I assumed a logistic S-curve — so a 1.6 programmer will succeed at 64% of L2 tasks; 1.7, 76%; 1.8, 85%; 1.9, 91% — but I don’t think it can be taken that precisely. 

At 0.7, you’re able to do most straight coding tasks (although it may take a long time, and you’re still not sure what this “compiler” thing is really doing). 1.0 is the level at which people are typically considered “software engineers” and 1.2 is the probably the median employed engineer, but at top tech companies it’s closer to 1.6. A typical SWE 3 at Google is 1.6–1.8 but assigned 1.2–1.4 level work. A typical Sr. SWE at Google is 1.8–2.0 but assigned 1.4–1.7 level work. You might sense a pattern. The bad news is that you’ll often be under-leveled at that sort of place. The good news is that the quality of most of the code at Google is quite high. On the other hand, there are plenty of companies (usually non-technical ones that still rely on software) out there that don’t have a single 1.5+ engineer (or, at least, not one that they listen to) and they still need architects… so you often see those L2 decisions made by people at the 1.2–1.4 level (or by non-technical managers, which is worse) and the results are what you’d expect. 

All of this might seem like a lot of verbiage, but it’s useful in answering the question. The definition of “top 1%” depends on whom you include in the “programmer” population; I’m looking for something more objective. I’d guess that there are about 200,000 programmers alive at or above the 2.0 level. That’s the level I’m going to focus on. (2.5, 3.0, I can’t even answer.) So how do you get to the 2.0 level?

  • Budget 7 to 14 years. Yes, it’s going to take time. (I’ve been programming, sadly not continuously, for 8 years; I’d guess that I’m 1.9–2.0 and it took work.) You have to write code, read code, experiment with new technologies, see a few architectural failures and (one hopes, but let’s not get ahead of ourselves because most “software architects” are politically empowered shit-heads) see an architectural success or two. I think you can probably get up to 0.7–0.8 (basic code fluency) in a dedicated summer slinging Javascript, and 1.0–1.1 in another year if you really dive in. Beyond that, 0.1 point per year is a pretty aggressive clip. Most employers will assign you work below your frontier of ability, so you may have to steal an education from your boss (reading papers on work time, experimenting with new technologies). Remember: it’s always better to ask forgiveness than permission.

  • Study voraciously. Computer science is a deep field, and to be good at it, you need at least a working understanding of everything. If you think assembly code or linear algebra or strong static typing is “scary” or “too deep” you’ll never reach 2.0. That doesn’t mean you need to be an expert in everything because you can’t. You can’t have “I won’t go there” areas. You probably don’t want to be hand-writing assembly code very often, but if you take the attitude that it’s “black magic” or “grunt work” or “unclean”, then it will hinder your learning. (You see some Java programmers take this attitude toward manual-memory languages like C.) I would probably use Haskell (a high-level language with strong static typing) given a clean slate; but I’ve exposed myself to Clojure, C, and even Python (data science libraries) because they cover important topics from computer science. You can’t have the “Will this be on the test?” mentality. You have to be curious about everything CS-related. (Mathematical curiosity helps.) You also need to learn about the industry itself. Why do so many software projects fail? What mistakes (technical and nontechnical) lead to that and how might they be prevented? What makes a good startup CTO? What things are worth building, and what patterns betray a death march?

  • Build things when you don’t know that you’ll succeed. How do you become a competent programmer? Or architect? Or Linux kernel hacker? Practice. If you know that you can do something, then you won’t learn as much as if there’s some chance of failure. Employers want you at a difficulty level where you’ll succeed 95% of the time. Your best learning is in the 65–75% “stretch” range, I’d say. That means that you’re eventually getting successful outcomes, but faltering a bit. There’s progress, but it’s not smooth and you definitely have to work.

  • Network, not for jobs but for ideas. Don’t think of “networking” as something you do after you get laid off. We’re social creatures and programmers are no different. The best way to become a great programmer is to go talk to other great programmers, and then get a sense of what problems they’ve solved and how. I’d be cautious at AI/machine learning talks in 2014 (some are very good, but there plenty of poseurs in “data science”; that will probably change in 3 years) but Haskell and Clojure user groups both draw a great crowd.

  • Job hop when you stop learning. Most programmers plateau around 1.2. Why? Because they aren’t pushed, and they don’t get opportunities to work on hard things. Or, the “hard” stuff they get pertains to difficult/ugly legacy code, not computer science itself. It’s useful to spend ~3 months on a bad legacy module or two to learn about architectural mistakes, but not more. Ultimately, you’ll want to work on successful projects, and you’ll want to be surrounded by strong people. If you ever stop growing, it’s time to move on. You know how people end up stagnant? One day at a time. That’s also how you become a great programmer. One day at a time (for about 4,000 days).

Serverless &amp; Function-as-a-Service – A Brief History of Time

Serverless & Function-as-a-Service – A Brief History of Time

Software Engineering Bottlenecks

Software Engineering Bottlenecks