Can you trust software capability evaluations?

In the early 1989s, the US Department of Defense suffered many monetary, schedule, and performance costs because it misjudged the ability of its contractors to develop software. Recognizing that this was less a reflection on the organizations themselves than on the general immaturity of the software industry, the DoD helped create the Software Engineering Institute with the aim of studying ways to help the software industry grow responsibly. In 1987, the SEI established the software capability evaluation (SCE) as a formal way to determine an organization´s software process maturity framework and generally measure its software development competence. SCEs are widely used to determine an organization´s Capability Maturity Model rating, but their results can be misleading. The authors reveal how certain practices distort results and waste time and money, through intentional contractor vagueness and detail, inappropriate sample projects, and staff coaching. They go on to suggest ways to improve the process