• DocumentCode
    3561751
  • Title

    How (Much) Do Developers Test?

  • Author

    Beller, Moritz ; Gousios, Georgios ; Zaidman, Andy

  • Author_Institution
    Delft Univ. of Technol., Delft, Netherlands
  • Volume
    2
  • fYear
    2015
  • Firstpage
    559
  • Lastpage
    562
  • Abstract
    What do we know about software testing in the real world? It seems we know from Fred Brooks\´ seminal work "The Mythical Man-Month" that 50% of project effort is spent on testing. However, due to the enormous advances in software engineering in the past 40 years, the question stands: Is this observation still true? In fact, was it ever true? The vision for our research is to settle the discussion about Brooks\´ estimation once and for all: How much do developers test? Does developers\´ estimation on how much they test match reality? How frequently do they execute their tests, and is there a relationship between test runtime and execution frequency? What are the typical reactions to failing tests? Do developers solve actual defects in the production code, or do they merely relax their test assertions? Emerging results from 40 software engineering students show that students overestimate their testing time threefold, and 50% of them test as little as 4% of their time, or less. Having proven the scalability of our infrastructure, we are now extending our case study with professional software engineers from open-source and industrial organizations.
  • Keywords
    program testing; execution frequency; industrial organizations; open-source organizations; production code; professional software engineers; software testing; test runtime; Estimation; Java; Open source software; Production; Software engineering; Testing;
  • fLanguage
    English
  • Publisher
    ieee
  • Conference_Titel
    Software Engineering (ICSE), 2015 IEEE/ACM 37th IEEE International Conference on
  • Type

    conf

  • DOI
    10.1109/ICSE.2015.193
  • Filename
    7203012