Test Driven Development Pitfalls

Test Driven Development (TDD) has been one of the most adopted agile practice. It has bring back the importance of unit testing, but also mostly remind that people have to think about what their code really want to achieve before writing it.

Test-driven development that relies on the repetition of a very short development cycle: first the developer writes an (initially failing) automated test case, then produces the minimum amount of code to pass that test, and finally refactors the new code to acceptable standards. Kent Beck, who is credited with having developed or 'rediscovered' the technique, stated in 2003 that TDD encourages simple designs and inspires confidence. However, how simple this might appears, it is not easy to perform this approach successfully on the long term.

Two interesting articles published in the Methods & Tools free software development magazine might help you  to do this. Test Driven Development (TDD) Traps by Jakub Nabrdalik presents all the issues that a agile software developer might face when trying to apply TDD and explains how to overcome them. Writing Testable Code by Isa Goksu explain how to write a code that will be easy to test... and then easy to refactor. This two articles are a good read for every software developer that believes in the power of unit testing to achieve quality in software code.

Unit Testing Still Widely Informal

A recent Methods & Tools poll examined how organizations perform unit testing. Is it an informal activity that is done before integration if there is some time left after programming or is it the key element of the development effort? The question was: How is unit testing performed at your location?

Proportion	2008	2006
Unit testing is not performed	17%	13%
Unit testing is informal	40%	46%
Unit tests cases are documented	9%	11%
Unit tests cases and their executions are documented	14%	16%
We use a Test Driven Development approach	20%	14%

Participants: 384 (2006:460)

Ending date: October 2008 (February 2006)

Despite the fact that the number of TDD adopters has grown nicely since the previous survey, you can notice that unit testing is still widely conducted in a informal manner, when it is not simply ignored by developers. This could sound weird when many people announced a general adoption of the agile approaches, but the results of our survey are similar to many other polls on the same topic.

Comparing the two surveys, it seems that people that were already doing unit testing formally have switched towards a TDD approach. People that don’t do unit testing have different reasons. Some will consider simply that they don’t add value to their development process, which is sometimes difficult to believe. For others, it is the lack of time, a reason more easier to understand ;o) Many complains that unit test are hard to write, but creating a good unit test is a proof that you understand what your code should do. I agree however, that it could be difficult to maintain large libraries of unit testing scripts if requirements are changing constantly. In the “good” reasons not to perform unit testing, some thinks for instance that the client side of Web application is not suited for this kind of tests. There are also some organizations that have separate testing teams. Their developers will rely entirely on the QA guys to test their application. You can also consider that when the software has a very limited life expectancy, it is not worth making unit tests.

The Three Rules of Test Driven Development

Over the years I have come to describe Test Driven Development in terms of three simple rules. They are:
1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

You must begin by writing a unit test for the functionality that you intend to write. But by rule 2, you can’t write very much of that unit test. As soon as the unit test code fails to compile, or fails an assertion, you must stop and write production code. But by rule 3 you can only write the production code that makes the test compile or pass, and no more.

If you think about this you will realize that you simply cannot write very much code at all without compiling and executing something. Indeed, this is really the point. In everything we do, whether writing tests, writing production code, or refactoring, we keep the system executing at all times. The time between running tests is on the order of seconds, or minutes. Even 10 minutes is too long.

Robert Martin

http://butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd

Besides these rules and independently of an agile approach, I always like as a developer being able to verify quickly my code, because it makes much more easier to find all the errors I do ;o)

Agile Software Development: True Adoption or Just a Label?

At what stage is the agile approach (XP, Scrum, TDD, …) adoption at your location? (2005 results)

Not aware 13% (26%)
Not using 13% (16%)
Investigating 14% (14%)
Analysed and rejected 4% (3%)
Pilot projects 8%(4%)
Partial implementation (adoption of some agile practices) 17% (17%)
Partial deployment (some projects are using this approach) 14% (12%)
Deployed (all new projects are using this approach) 17% (8%)

Participants: 512 (232)

Ending date: February 2008

Source: Methods & Tools

Comparing the 2008 and 2005 results, we could notice that the level of ignorance of the agile movement has decreased, as only 13% of the organizations are ignorant of it. Full deployment numbers have doubled in the recent years to reach 17% and total rate of various adoption levels is now 56% compared to 41% in 2005. This growing adoption rate has also been confirmed by other surveys on this topic. Some critics will naturally discuss the “scientific” nature of this kind of surveys, but I believe that they are useful to give us an idea of the evolution of the software development world. The increased popularity of agile is also visible on the Indeed job request trend.

The recent improvement in agile software development process visibility and acceptance should however make us cautious with the current results on agile adoption surveys. When software development practices are more widely accepted, the number of adopting organizations increases, but the substance of valid usage of this practices decreases. The answers to surveys tend then to be biased towards what should be the correct answer instead of reflecting the reality of the software development context. The tendency of upper management to “push” new approaches to people who are more reluctant to change their old habits does usually not produce good results. This is even truer when projects will not be given additional resources to support the transition.

We are therefore transitioning from a period where agile adoption level may have been underestimated to another where it could be overestimated. Previously, some developers would not define their practices openly as agile or extreme programming because manager would have considered it a “cow-boy process”. A 2006 Forrester’s report found that agile adoption in the US and Europe was 17% according to a survey of 1078 IT decision-maker. They however precised that ” because Agile processes are often adopted at the grassroots level, they frequently fly below the radar. This makes it unlikely that a single IT decision-maker knows what methodology every development team is using. For this reason, Forrester believes that Agile adoption is actually higher than reported in this study.” Today, some companies will pretend that they are agile, but without implementing the essence of the approach.

It is also interesting to notice that the rate of rejection is similar in both surveys. This means that the percentage of companies completely opposed to agile practices is still very low. We could also see that the proportion of companies that are in a partial implementation state is still superior to those who are in partial deployment. This confirms that organizations prefer to adopt agile practices by gradually shifting their process towards agility instead of choosing a big bang implementation. Agility is about iteration and incrementation and this should be also applied to process changes.

When compared to other surveys performed in 2007 on agile adoption, the results of the Methods & Tools poll are equivalent. The adoption rate of participants is slightly inferior with 56% (48% without pilot projects) compared with 73% for the VersionOne survey and 69% for the Scott Ambler survey. Our percentage of organizations having deployed agile approaches in all projects stands at 17%, which is very close to the 15% rate that VersionOne presents for 100% agile adoption.

The importance of the agile approach is surely growing in the software development community. It remains to be determined, with an unbiased perspective, how this evolution will translate in the rate of software development project successes, because at the end, this is what matters the most.

Other agile deployment surveys and agile adoption material

Indeed.com Job Trends for: agile, scrum, “extreme programming”, “test driven”

VersionOne 2nd Annual “State of Agile Development” Survey (2007)

Scott Ambler Surveys (2007)

Scott Ambler Surveys Presented in Dr. Dobb’s (2007)

Scrum Alliance Membership Survey Shows Growing Scrum Adoption and Project Success (2007)

TDD Survey Results by Stelligent (2007)

Boston Roundtable Survey Results by Stelligent (2007)

Agile Project Management Tooling Survey by Trail Ridge Consulting (2006)

Corporate IT Leads The Second Wave Of Agile Adoption by Forrester (2005)

Waterfall Manifesto Agile Software Development Adoption Poll (ongoing)

Slowdown in ScrumMaster (CSM) Certifications?

Low Scrum Adoption Rate in German-speaking world?

Adopting an Agile Method

Agile Delivery at British Telecom - A case study on Agile process adoption

New Agile Software Development Portal

The Agile Software Development Web site is a repository for resources concerning agile software development approaches (Extreme Programming (XP), Scrum, Test Driven Development (TDD), Feature Driven Development (FDD), DSDM, Lean Software) and practices (refactoring, pair programming, continuous integration, user stories). The content consists of articles, news, press releases, quotations, books reviews and links to articles, Web sites, tools, blogs, conferences and other elements concerning agile software development. Feel free to contribute with your own articles, links or press releases.

http://www.devagile.com/