Strategies for inference robustness in focused modelling

Advances in computation mean that it is now possible to fit a wide range of complex models to data, but there remains the problem of selecting a model on which to base reported inferences. Following an early suggestion of Box & Tiao, it seems reasonable to seek ‘inference robustness’ in reported models, so that alternative assumptions that are reasonably well supported would not lead to substantially different conclusions. We propose a four-stage modelling strategy in which we iteratively assess and elaborate an initial model, measure the support for each of the resulting family of models, assess the influence of adopting alternative models on the conclusions of primary interest, and identify whether an approximate model can be reported. The influence-support plot is then introduced as a tool to aid model comparison. The strategy is semi-formal, in that it could be embedded in a decision-theoretic framework but requires substantive input for any specific application. The one restriction of the strategy is that the quantity of interest, or ‘focus’, must retain its interpretation across all candidate models. It is, therefore, applicable to analyses whose goal is prediction, or where a set of common model parameters are of interest and candidate models make alternative distributional assumptions. The ideas are illustrated by two examples. Technical issues include the calibration of the Kullback– Leibler divergence between marginal distributions, and the use of alternative measures of support for the range of models fitted