Abstract :
While much research has focused on methods for evaluating and maximizing the accuracy of classifiers either individually or in ensembles, little effort has been devoted to analyzing how classifiers are typically deployed in practice. In many domains, classifiers are used as part of a multi-stage process that increases accuracy at the expense of more data collection and/or more processing resources as the likelihood of a positive class label increases. This paper systematically explores the tradeoffs inherent in constructing these multi-stage classifiers from a series of increasingly accurate and expensive individual classifiers, considering a variety of metrics such as accuracy, cost/benefit ratio, and lift. It suggests architectures appropriate for both independent instances and for highly linked data.
