Investigating the construct validity associated with microworld research: A comparison of performance under different management structures across expert and non-expert naturalistic decision-making groups

The use of computer-simulated microworlds has become increasingly popular to test concepts related to naturalistic decision-making (NDM) in a controlled laboratory environment. However, the construct validity for such methods is unclear. The current study followed previous microworld-based studies that compared indirect (macromanagement) methods of management with direct methods (micromanagement). To explore the construct validity of microworld research, the current study compared performance scores generated by participants with experience in a prototypical NDM environment, with those without such experience. Using a networked computer simulation for firefighting, 10 Army officers and 10 civilians played the role of Fire Chief within three-person command and control teams. The two subordinates were confederates. Comparison of management structures supported previous results indicating that indirect control produces significantly better NDM performance. However, no difference was found between the experienced and inexperienced participant groups, questioning the construct validity of results produced using microworlds.