John Carmack discusses the art and science of software engineering
















31 thoughts on “John Carmack discusses the art and science of software engineering

  1. I recently moved into a Team Lead position at my company, so Ive been thinking a lot about this. There’s a book called Peopleware that has some similar ideas. “Maybe . . . the major problems of systems work are not so much technological as sociological.”

  2. Wow – the referenced video is over 3.5 hours long. I would have liked to have watched the 10 minutes you were referencing, perhaps you could list which point to watch (for example 1:23:45 – 1:33:00).

  3. I recently had the same feeling about the last three paragraphs. we have some good engineers and designers constantly fighting for a good “design”, everyone has their own ideas and methodologies, but no one has a way of measuring it’s “correctness” or whether it’s future proven. Even though this post won’t help us to solve these “social conflicts” in software development, i think you made a valid point of view and pointed out some directions. Could be a nice piece of research project.

    • While there aren’t great ways of measuring the benefits of a methodology yet, this is an active area of software engineering research. In fact, there are probably over 50 studies done that evaluate different methodologies published every year, largely funded by the National Science Foundation in the U.S. Some of my colleagues have a nice blog that summarizes some of their findings.

  4. What about goal-directed programming?
    I really like Unicon, which is (sadly) so underrated!
    Unicon, however, supports other paradigms too.

  5. I’d really enjoy observing the process that he’s trying to manage and reading through some of the code. Part of what makes the process different, of course, is that they produce entertainment, which means the end result is only constrained by the perceptual and conceptual adaptability of the players. That gives them the freedom to “try things” in a way that would cause unacceptable risk of major economic disruptions if attempted at Amex or GE.
    His coaching process is interesting. I wonder if he maintains an organized defect wiki.

  6. I’m really surprised that he doesn’t see functional programming as a solution to all of this. I mean, yes, when it comes down to it, it really is just assembly, however, what’s really important is all of the higher level abstractions you can create and modify with ease. I think that functional programming allows you to explain the solution in the terms of it’s context. And I’m not saying that it’s something that declarative programming can’t do. I’m simply saying that declarative programming causes you to repeat yourself more often, thus causing more bugs.

    • By “declarative” programming do you mean, “imperative”?

      Functional programming can make it quite difficult to describe operational semantics. Yet, controlling operational semantics is really a big deal in writing games.

      • Agreed. And I agree with the sentiments in your keynote more generally: the purpose of most programming language features are to prevent mistakes while trying to preserve other qualities such as expressiveness, readability, learnability, etc. Functional languages are excellent at preventing mistakes, but in trade, they can increase the difficulty of both reading and authoring many types of behavior. Defects aren’t the only priority in engineering complex software.

    • Agreed, I mean with all due respect, cos he’s JC, and I’m just some peon, but couldn’t you dismiss the whole of software engineering practice by saying “it’s all just assembly, evrnrually”. The point is not what it becomes, but how we get it there. It’d like he does a talk about the mysterious social factors while they are staring him in the face.

      Presumsbly this is really evidence of how my worldview is messed up. Because his wv seems to be working out ok. But what if he’s actually wrong on stuff like language choice, but he’s just so damn smart that it hasn’t held him back enough for him to realise it?

      I wonder, sometimes, if the very brightest aren’t attracted to things that are slightly masochistic, just because they can, and then the rest if us are lead down the garden path by their highly intelligent justifications. Which are, ultimately, actuelly wrong. Or at least not remotely as widely applicable as supposed.

      But what do I know? Plus, I’m drunk. You don’t wanna listen to me. Go follow the shiny.

  7. I can’t help but think that a good ‘programmer’s helper’ AI will solve a lot of these problems, but this would require an expert system that can develop a syntactic bridge between ‘what I want the code to do’ and ‘what the code needs to look like to accomplish my goals’. Maybe not a system that takes input in plain english, but something like a drag-and-drop UI, or a visually abstract representation of a program’s flow.

    If you can write static tests that analyze code and detect bugs, you could theoretically write code that ‘fixes’ those bugs – you just need to know what you want the code to do, and what the ‘correct’ fix is.

    I think this is essentially what Mr. Carmack is talking about when he says he wants more ‘restrictive’ languages. Imagine an IDE where you literally can’t write a bug – perhaps all of NASA’s strict bug checking is automatically implemented as you write code, or perhaps you don’t actually ‘code’ in the traditional sense, you just symbolically link ‘program flow control’ blocks together like in the Unreal Engine SDK.

    • There are a lot of possibilities in this space. Although, as a researcher who’s well aware of the past 40 years of research on this topic, I’m not optimistic. Most of these approaches only work well in a small number of situations and are completely useless in other situations. The reason for this is that the intended, specified behavior is never really known by a computer, even after an implementation is laid down to achieve it, and the task of specifying the intended behavior is almost always more work then just implementing the behavior itself. And even then, specifications change and code doesn’t always reflect these changes.

        • I wrote one a few years ago that covers some of these techniques for small programs.

          Ko, A.J., Abraham, R., Beckwith, L., Blackwell, A., Burnett, M.M., Erwig, M., Scaffidi, C., Lawrance, J., Lieberman, H., Myers, B.A., Rosson, M.B., Rothermel, G., Shaw, M., and Wiedenbeck, S. (2011). The State of the Art in End-User Software Engineering. ACM Computing Surveys 43, 3, Article 21 (April 2011), 44 pages.

          Most of the systems covered in that paper are interactive tools to help developers find and fix defects.

    • It’s not possible to have a general purpose language in which it’s not possible to write a bug.

      Turing’s halting problem says that you can’t have a program that can determine (in all cases) even whether the program will halt or not.

      So, programming tools cannot calculate all properties of programming problems that require exiting for correctness.

      But it is possible to have restricted languages that are potentially bug free, with respect to a specification; but it’s also not necessarily possible to prove the specification bug-free either!

  8. Thanks for doing this, it’s nice to read again after watching the talk a while ago.

    One small edit – it has been a while since I heard the talk but I’m fairly sure “cache list” should be “cache miss”. It seems to work in context as well.

  9. That’s the voice of an older programmer, someone who’s been through a lot of work and whose classic younger-programmer self assurance has gone. It’s comforting to know that it happens even to Carmack.

    • Doing programming for ten years, i am starting to lose self assurance. i am ever more putting “software engineering” and defenses and all of these “crap”, that is slowing me down, or just is the age (i am 25). In the end, code is to be delivered, and only on real world it has value.
      Perhaps we need only to embrace that bugs is normal, as long as we are human. Now i am thinking i need to make something just for fun, just coding, forgeting all these forty years “software engineering” stuff, but i am losing interest on programming.
      Perhaps its some sort of confort to know that this can happen to “gods” like Carmack, or not, that even he cant escape from destiny.
      Its all sad.
      ps: sorry my poor writing skills.

      • Errors, mistakes, and unexpected side-effects have nothing to do with being “human”. Ada Lovelace wrote the first program only about 170 years ago, and the first “bug” was found by Grace Hopper about 70 years ago. Nature’s been doing it for over 2 billion years, and there are still plenty of genetic bugs, which can have unexpected *benefits*. If “flawless” AIs wrote programs without bugs, they would have no value. Bugs aren’t something to be avoided, but embraced. Separating what’s a bug and what’s a feature is what’s important, if you try so hard to prevent the former you also inhabit the latter.

Leave a Reply to Jason Dusek Cancel reply
