Identification of more risks can lead to increased over-optimism of and over-confidence in software development effort estimates

Software professionals are, on average, over-optimistic about the required effort usage and over-confident about the accuracy of their effort estimates. A better understanding of the mechanisms leading to the over-optimism and over-confidence may enable better estimation processes and, as a consequence, better managed software development projects. We hypothesize that there are situations where more work on risk identification leads to increased over-optimism and over-confidence in software development effort estimates, instead of the intended improvement of realism. Four experiments with software professionals are conducted to test the hypothesis. All four experiments provide results in support of the hypothesis. Possible explanations of the counter-intuitive finding relate to results from cognitive science on “illusion-of-control”, “cognitive accessibility”, “the peak-end rule” and “risk as feeling.” Thorough work on risk identification is essential for many purposes and our results should not lead to less emphasis on this activity. Our results do, however, suggest that it matters how risk identification and judgment-based effort estimation processes are combined. A simple approach for better combination of risk identification work and effort estimation is suggested.