An empirical assessment of a systematic search process for systematic reviews

Background: Systematic Literature Reviews (SLRs) have been gaining significant attention from Software Engineering (SE) researchers. Several researchers are also working on improving SLR methodology for SE. Objective: The study reported in this paper aims to validate the QGS-based search process for SLR, i.e. whether a more effective (sensitive) and/or productive (precise) search can be achieved by following the QGS-based process. Method: We used a dual-case study, in which each case includes two observations of SE literature search for the same SLR but using and not using the QGS-based approach. The overall sensitivity and precision were calculated for each observation that implemented different search design. Results: The use of QGS-based search process resulted in higher sensitivity and precision in this dual-case study. Conclusions: A systematic search process can help capture more relevant studies as well as save researchers´ time spent on literature search activities. Our observations also support that an integrated search strategy is recommended for SLRs in SE to avoid the possible limitations of applying a single (manual or automated) search method.