Expert Judgment in Software Estimation During the Bid Phase of a Project -- An Exploratory Survey

This article presents the results of an industrial study about the reliability of expert judgment in cost estimation in a medium-sized software company. The purpose of the study was to assess current practices within the company, and to use the results as a catalyst for improving the company´s cost estimation processes. The study included the analysis of two aspects: variability and calibration. First, the degree of variability can indicate the absence of a sound and repeatable process for making estimations. Second, measuring the estimators´ calibration allowed us to establish their awareness about their lack of specific knowledge when estimating, as well as their ability to properly acknowledge and represent uncertainty. The results clearly show that, in the context of the organization studied, cost estimation based on unstructured expert judgment is unreliable.