W. Edwards Deming famously said, “If you can’t describe what you are doing as a process, you don’t know what you’re doing.” I haven’t run into any consultants that don’t know what they’re doing, but there are many who shun process and are stuck in what I call the process dilemma (see More Lessons In Ground Floor Project Management).
“Our dilemma is that we hate change and love it at the same time; what we really want is for things to remain the same but get better.” – Sydney J. Harris
The process dilemma occurs when a consultant, a team or a company believes that “process” will slow down project delivery, increase overhead, raises costs, and lead to client dissatisfaction. In this context, process is usually defined as Project Management or SDLC rigor. Process is seen as inflexible, overly strict, and stifling to creativity. In fact, as studies and surveys have shown, it’s process that enables project success and allows the creative team the time to create.
The Standish Group has published the Chaos Report on IT project success (or really on project failure) since 1994. The 2011 report stated that only 37% of IT projects are successful. This would appear to be an indictment of IT Project Management that confirms the Process Dilemma. However, the Chaos Report only considers projects successful if they deliver the defined features on time and on budget.
The 2012 Chaos Manifesto (same thing) looked at project delivery success from 2002 through 2012 (as reported by Mike Cohn and others). The report says only 14% of traditional, waterfall projects were successful. The report also said that 42% of agile projects were successful during the same time period.
Dr. Dobb’s published the 2010 IT Project Success Rates based on three surveys conducted by their editorial staff. The results are significantly different than the Chaos Report, but so was the survey. Dr. Dobb’s looked at four categories of project delivery methods and found:
- Ad-hoc projects: 49% are successful, 37% are challenged, and 14% are failures.
- Iterative projects: 61% are successful, 28% are challenged, and 11% are failures.
- Agile projects: 60% are successful, 28% are challenged, and 12% are failures.
- Traditional projects: 47% are successful, 36% are challenged, and 17% are failures.
What does this mean in relation to the process dilemma? If you use a rigid, highly prescriptive methodology, you are likely to get the same results as if you had no process at all (47% success vs. 49%). I think ad hoc projects may be slightly more successful, because large projects are more apt to be managed using traditional methods. Large projects require large teams, and large teams are more prone to failure. Dr. Dobb’s State of the IT Union survey, a component of the 2010 IT Project Success Rates
, showed that team size does matter. Looking only at successful projects, the survey reported:
- Ad-hoc projects: 74% for small teams, 58% for medium-sized teams, and 40% for large teams.
- Iterative projects: 80% for small teams, 68% for medium-sized teams, and 55% for large teams.
- Agile projects: 83% for small teams, 70% for medium-sized teams, and 55% for large teams.
- Traditional projects: : 69% for small teams, 61% for medium-sized teams, and 50% for large teams.
Obviously, the probability of success for small teams is much greater. The really interesting thing is that size has the least effect on traditional project teams.
Solution delivery, like project management, can be reduced to the essentials, using agile methods. A “just enough” process approach enriches the capabilities of the team without adding overhead. It can keep costs down by managing customer expectations within the product backlog instead of negotiating frequent change orders. Most importantly, an agile approach fosters creativity, trust and teamwork by putting development decisions in the hands of the team and the client product owner.
“We realize our dilemma goes deeper than shortage of time; it is basically a problem of priorities. We confess, We have left undone those things that ought to have done; and we have done those things which we ought not to have done.” – Charles E. Hummel
Solving the Process Dilemma is in itself a process. It can take a change in attitudes or a complete culture shift. The key is on the ground floor. Build a solid foundation on practical process that ensures you are doing the right things at the right time, and you will get the right results.