DevOps distilled, Part 5: Why everyone needs DevOps now

In this series of articles, learn about DevOps and how it can: create a collaborative relationship between development and IT operations; enable high deployment rates; and increase the reliability, resilience, and security of your production environment. In this final article, learn why DevOps is the key to unlocking tremendous economic value.


Gene Kim (, Author, IT Revolution Press


22 January 2013


DevOps for Dummies


Download "DevOps for Dummies."

In this series about DevOps, Part 1 introduced the underlying principles of DevOps. Part 2 discussed creating environments early in the development process for more deterministic, predictable, and secure releases. Part 3 profiled a Twitter effort to integrate information security into the development process and amplify the feedback loop. Part 4 talks about standardizing the work of IT operations to increase project predictability, accuracy, and throughput. In this last part of the series, I'll discuss why everyone needs DevOps now.

Since 1999, my passion has been studying high-performing IT organizations. At the beginning of this journey, I started keeping "Gene's list of people with good kung fu." The people on the list talked differently about IT operations, acted differently, and, most importantly, had profoundly different results than your typical IT organization.

I studied these high performers and benchmarked over 1,500 organizations. The goal was to understand how they did what most organizations could only dream of. In 2004 the findings went into the book The Visible Ops Handbook, which describes how the organizations made their "good to great" transformation.

I couldn't have predicted how this journey would take me straight into the heart of the DevOps movement. As my friend John Willis said after I dismissed DevOps as just another marketing fad, "DevOps is the best chance at relevance that IT operations has had in thirty years." I immediately realized he was right.

DevOps typically refers to the emerging professional movement that advocates a collaborative working relationship between development and IT operations. DevOps results in the fast flow of planned work (for example, high deployment rates) while simultaneously increasing the reliability, stability, resilience, and security of the production environment.

By putting DevOps patterns into practice, organizations such as Etsy, Netflix, Facebook, Amazon, Twitter, and Google are achieving levels of performance that were unthinkable even five years ago. They have tens or even hundreds of code deploymentss per day while delivering world class stability, reliability, and security.

Why everyone needs DevOps

There's currently a chronic conflict in almost every IT organization. It is so powerful that it practically preordains horrible outcomes—if not abject failure. It happens in both large and small organizations, for-profit and non-profit, and across every type of industry. This destructive pattern, or conflict, is the root cause of one of the biggest problems you face. But, if you can beat it, you'll have the potential to generate more economic value than anything you've seen in the previous 30 years.

The following sections outline this destructive pattern in three acts that will likely be familiar to you. (You can get the whole story in my book, The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win).

Act 1. It begins…

It begins with IT operations, where you're supporting a large, complex revenue-generating application. The problem is that everyone knows that the application and supporting infrastructure is… fragile.

How do you know? Because every time anyone touches it, it breaks horrifically and causes an epic amount of unplanned work for everyone.

The shameful part is how you find out about the outage. It's not from an internal monitoring tool, but when a salesperson calls and says, "Hey, Gene, something strange is happening. Our revenue pipeline stopped for two hours." Or, "The banner ads in my market are being served upside down and in Spanish."

There are so many moving parts nowadays that it takes way too long to figure out what caused the problem du jour. You're spending more and more time on unplanned work and increasingly are unable to get your planned work done.

Eventually, your ability to support the most important applications and business initiatives goes down. When that happens, the members of the organization suddenly find themselves unable to achieve the commitments they promised the outside world, whether it's customers, analysts, or Wall Street. Promised features aren't available, market share isn't going up, average order sizes are going down, specific revenue goals are being missed… And that's when something really terrible happens.

Act 2. Your life gets worse

Life gets worse when the business starts making even bigger commitments to Wall Street—commitments often dreamed up by others. Often not developers, but people who don't have the best grasp on what technology can and can't do.

And yet, they start dreaming up new features that are sure to dazzle the marketplace, write the business requirements, and make promises to the outside world.

Enter the developers. They start seeing more and more urgent date-driven projects in the queue. The projects often require things that the organization has never done before. Because the date can't be moved (because of all those external promises made), everyone has to start cutting corners.

Development must focus on producing the features, so the corners that get cut are all the non-functional requirements (manageability, scalability, reliability, security, and so forth). Consequently, technical debt starts to increase, and that means increasingly fragile infrastructure in production.

It is called technical debt for a reason: technical debt, like financial debt, compounds.

Act 3. How DevOps breaks you out of your downward spiral

We all know that there must be a better way, right? DevOps is proof that it's possible to break the core, chronic conflict and deliver a fast flow of features without causing chaos and disruption to the production environment.

When John Allspaw and Paul Hammond gave their seminal presentation, 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr, at the 2009 Velocity Conference, people were shocked and amazed at the audaciousness of their achievement. But it wasn't a fluke. Other organizations such as Facebook, Amazon, Netflix and the ever-growing DevOps community have replicated the performance and are doing hundreds, and even thousands, of deployments a day.

But DevOps isn't just for large companies. It's for any company where the organizational goals have a high reliance on IT. These days, that means almost every company, large or small, for-profit or not-for-profit, and so on. We all need to put DevOps-like practices into place, so Kevin Behr, George Spafford, and I wrote a new book called The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.

A novel, you might ask? How is a novel going to solve my IT problems?

As a friend once told me, "Before you can solve a complex problem, you must first have empathy for the other stakeholders. And story-telling is the most effective means of creating a shared understanding of the problem."

Dr. Eliyahu Goldratt demonstrated the power of a novel as a teaching tool with his book, The Goal: A Process of Ongoing Improvement. Written in the 1980s, it's about a plant manager who has 90 days to fix his cost and due date issues or his plant will be shut down. When I read this book years ago, I knew the story was important and that there was much I needed to learn (even though I never managed or worked in a manufacturing plant).

It isn't an overstatement to say that The Goal and Dr. Goldratt's Theory of Constraints changed my life and were an enormous influence on my professional thinking. For eight years, my co-authors and I wanted to write The Phoenix Project because we believe that IT is not just a department but a strategic capability that every business must have.

As you can imagine, I was incredibly honored and thrilled when Jez Humble, author of the award-winning book Continuous Delivery recently told me, "This book is a gripping tale that captures brilliantly the dilemmas that face companies which depend on IT. The Phoenix Project will have a profound effect on IT, just as The Goal did for manufacturing."

What economic value could be unlocked by a DevOps novel, you might ask? How is a novel going to solve my IT problems?

The Shingo Prize winner Mike Orzen and I have long discussed how the enormous waste in the IT value stream—long lead times, poor hand-offs, unplanned work, and rework—could be diminished by DevOps. We did a calculation to estimate how much value we could recapture by applying DevOps-like principles. In our calculation, if we could just halve the amount of IT waste and redeploy those dollars in a way that could return five times what was invested, we would generate $3 trillion dollars of value per year.

That's a staggering amount of value and opportunity that we're letting slip through our fingers. $3 trillion dollars is 4.7 percent of annual global GDP, or more than the entire economic output of Germany.

I think the potential savings are very important, especially when I consider the world my three children will inherit. The potential economic impact to productivity, standards of living, and prosperity almost makes DevOps a moral imperative.


In this series, I've covered a high-level version of some of my favorite lessons learned in DevOps. These are also the underpinning theories that I cover more extensively in my new book The Phoenix Project: A Novel About IT, DevOps and Helping Your Business Win. It is the first book out from our new publishing venture, IT Revolution, where our mission is to improve the livelihoods of one million IT professionals by 2017.



Get products and technologies





developerWorks: Sign in

Required fields are indicated with an asterisk (*).

Need an IBM ID?
Forgot your IBM ID?

Forgot your password?
Change your password




All information submitted is secure.

Choose your display name



Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)



All information submitted is secure.

Dig deeper into Security on developerWorks

Zone=Security, Web development, DevOps
ArticleTitle=DevOps distilled, Part 5: Why everyone needs DevOps now