Take the 2-minute tour ×

When I was reading the summary of previous work in Dogsa T, Batic D. The effectiveness of test-driven development: an industrial case study. Software Quality Journal. 2011;19(4):643-661. it struck me that the measurements used in a lot of studies around TDD are based on things like lines of code, defects and time spent on development.

Are there any studies out there that focus on the total cost of ownership for products that have been developed using TDD vs ex traditional development or test-last?

I am especially interested in the total cost of acquisition and operating costs.

share|improve this question

2 Answers 2

up vote 3 down vote accepted

There are some studies about the implications and benefits of doing TDD, but the results are contradictory. Some projects (this is in my experience) have a lower bug rate and cost to ownership as a result of using TDD, as the cost of changing a feature reduces drastically. Some others are halted to a stop.

Some studies (here is one - check slide n50) show that the number of bugs increased with the coverage. I'm assuming that greater coverage implies TDD and that a higher number of bugs implies a higher cost of ownership.

From my point of view, no metric or practice on its own can be related to better quality or lower cost of ownership. There's a combination of factors that can lead to some correlation. And those factors change between teams and projects.

I think we've all heard stories of teams, which just started doing TDD, writing 100-line test methods, which (in my view) increases the cost of ownership, as updating that test will be expensive.

My pragmatic rule is that people who care and are eager to learn, working in an environment that supports them and their ideas have better quality and cost of ownership.

share|improve this answer
slide n50 is extremely misleading. "The more coverage the more bugs" most likely means "the more coverage the more bugs... you'll find." It's possible, but I doubt that more coverage will lead to more injected defects. This is simply stating that the more coverage the higher the defect yield out of the development phase. And yes, there are plenty of metrics that can measure quality and cost of ownership -- # injected defects by phase, defect yield by phase, and rework are all measurable things with a direct impact on quality and cost. See PSP/TSP for some great examples of these metrics. –  Michael May 21 '12 at 14:05
Michael, in the context of that slide, the presenter show what correlates to a higher bug density. One of the metrics was test cases, so the more tests cases a class has, the higher there will be a bug in the class. What the presenter tries to say is that no metric on its own correlates to a smaller bug density. –  Augusto May 22 '12 at 8:27

I don't have any specific studies but I can tell you from personal experience, and from the experiences of other developers I know that when applied correctly for medium and larger projects TDD reduces time to market, reduces bugs and defects, and improves code quality.

Having said that their are no silver bullets, can you write good code with out TDD? yes, can you write bad code using TDD yes. Also depending on your project TDD can greatly increase your cost of ownership for the code, a good example is NASA, where the cost per line of code is huge, but the cost of ownership isn't the focus, it's lack of defects.

When applied correctly TDD will increase your starting costs, and your code base, but you get the long term benefits of regression testing, early bug detection, and better code design which should reduce defects and testing costs and maintenance time, thus reducing overall cost of ownership.

share|improve this answer

Your Answer


