Print this page
Sunday, 15 November 2009 00:00

Project Failure Bunk

Rate this item
(1 Vote)

Are project success rates getting better or worse? What is the cost? What are the controlling factors? How does someone calculate these numbers? The answers are elusive. Lately, Roger Sessions has taken exception to one source—The Standish Group. He has many valid points. However, I doubt there are any statistics giving us a complete picture.

This twitter banter prompted me to dust off some old reports, dig through my library and search my online files to pull some meaningful data together. I was wondering about the headline sentence of this year's Standish Chaos Report, which contains "[2008's] results show a decrease in project success rates, with 32% of all projects succeeding." A pretty alarming statement.

What Does the Standish Data Show?

Graph of Standish numbers since 1994
Figure 1 - Standish Project Success Data

As it turned out, I needed only two sources of data from Standish—The 2009 Chaos Report and the book My Life is Failure. I plotted the combined data of Cancelled, Challenged and Successful project to get a picture of the changes since the famed report was first published in 1994. Figure 1 shows a graph of this data.

There is some very good news here. The Success line is trending up quite nicely. It has had a few setbacks, but it is improving. It starts at about 20% Success in 1994 going to the mid-thirties in 2008. In fact, it looks relatively linear (flattening a little in the last few years).

Graph to compare Cancelled to Challenged
Figure 2 - Comparison of Cancelled
and Challenged Projects

The bad sign is on Cancelled projects. They appear to be trending up over the last three years. However, a closer look shows the Challenged curve seems to be a mirror image of Cancelled curve. Plotting the Challenged to Not Cancelled (100%-Cancelled) percentages (Figure 2) confirms the strong correlation hinted in the previous graph.

Is this simply that people are realizing that they should stop throwing good money at bad projects and cancelling them sooner? Is the improvement in success rates because companies are running more smaller, less complex projects? Both would show good fiscal responsibility. Unfortunately, Standish does not publish that data, at least in the material I have. With this, we are back to Mr. Sessions' complaint with the Standish Group's data. In addition, the headline above, that the numbers are getting worse, does not reflect the whole picture. The trend appears to be an overall improvement in handling projects. Standish has trouble saying that since they live off of failure.

The question remains, "Are Standish's numbers valid?" But, does it matter? As long as they are consistent in the way they collect data, trends should be valid. Changing the method may move an entire line up or down, but it is unlikely to change the trends.

Graph of PMI membership numbers
Figure 3 - PMI's Membership
Count Since 1964

Where is the Improvement Coming From?

Curiosity abounds. What is causing the increased project Success? I looked in the obvious place for comments on project improvement—the Project Management Institute (PMI). There is a variety of data to choose from on their website; I chose what could be easily referenced—the 2008 Annual Report. This report indicates the ranks of PMI have grown from 10,000 members in 1994 to 287,438 at the end of 2008. As they point out in their report "That's literally exponential growth." This is evident from Figure 3 (2008 Annual Report, page 5). One might expect this explosion in the number of professionals to spur an equal improvement in the project success numbers. If this were a factor, the improvement surely would look nonlinear. However, this exponential growth in the PMI numbers seems to have had little effect on the linear improvement in project success.

I guess I am going to have to continue to dig to see why there is an improvement in the success rate. Keep checking back.

Read 16002 times

Related items