Choosing Among Alternative Complex Systems When Input Characteristics Are Uncertain

Planning for economic and social purposes often requires choosing among alternative proposed systems to fulfill particular needs. In order to make such choices, planners or decision makers must be able to predict and evaluate the performance of each alternative system. For complex systems, methods are often available for doing this when system inputs, such as system user characteristics, can be specified deterministically. The more general situation, of course, is that input characteristics are not known with certainty, but may be described by probability distributions. It is the latter case that is discussed here. A Bayesian method for choosing among alternative systems when input characteristics are uncertain is presented. The method involves generating ``sample best choices´´ among alternative systems. This is accomplished by selecting random input values, converting these to sample outputs for each system, and determining the sample best choice on the basis of the sample output values. The random input values are drawn from a probability distribution that encodes the uncertain state of information on inputs. The sample best choices are viewed as random samples of a multinomial random process whose parameters are also not known with certainty. The generation of random sample choices improves the state of knowledge of the uncertain multinomial parameters and permits a better decision. An optimal sampling policy may be found by executing a dynamic programming computation that balances the cost of sampling against the expected gains from improving the state of information.