What Does the Standish Data Show?
Figure 1 - Standish Project Success Data |
As it turned out, I needed only two sources of data from Standish—The 2009 Chaos Report and the book My Life is Failure. I plotted the combined data of Cancelled, Challenged and Successful project to get a picture of the changes since the famed report was first published in 1994. Figure 1 shows a graph of this data.
There is some very good news here. The Success line is trending up quite nicely. It has had a few setbacks, but it is improving. It starts at about 20% Success in 1994 going to the mid-thirties in 2008. In fact, it looks relatively linear (flattening a little in the last few years).
Figure 2 - Comparison of Cancelled and Challenged Projects |
The bad sign is on Cancelled projects. They appear to be trending up over the last three years. However, a closer look shows the Challenged curve seems to be a mirror image of Cancelled curve. Plotting the Challenged to Not Cancelled (100%-Cancelled) percentages (Figure 2) confirms the strong correlation hinted in the previous graph.
Is this simply that people are realizing that they should stop throwing good money at bad projects and cancelling them sooner? Is the improvement in success rates because companies are running more smaller, less complex projects? Both would show good fiscal responsibility. Unfortunately, Standish does not publish that data, at least in the material I have. With this, we are back to Mr. Sessions' complaint with the Standish Group's data. In addition, the headline above, that the numbers are getting worse, does not reflect the whole picture. The trend appears to be an overall improvement in handling projects. Standish has trouble saying that since they live off of failure.
The question remains, "Are Standish's numbers valid?" But, does it matter? As long as they are consistent in the way they collect data, trends should be valid. Changing the method may move an entire line up or down, but it is unlikely to change the trends.
Figure 3 - PMI's Membership Count Since 1964 |
Where is the Improvement Coming From?
Curiosity abounds. What is causing the increased project Success? I looked in the obvious place for comments on project improvement—the Project Management Institute (PMI). There is a variety of data to choose from on their website; I chose what could be easily referenced—the 2008 Annual Report. This report indicates the ranks of PMI have grown from 10,000 members in 1994 to 287,438 at the end of 2008. As they point out in their report "That's literally exponential growth." This is evident from Figure 3 (2008 Annual Report, page 5). One might expect this explosion in the number of professionals to spur an equal improvement in the project success numbers. If this were a factor, the improvement surely would look nonlinear. However, this exponential growth in the PMI numbers seems to have had little effect on the linear improvement in project success.
I guess I am going to have to continue to dig to see why there is an improvement in the success rate. Keep checking back.