Analysis of results in complex-valued intercomparisons

In the analysis of complex-valued intercomparisons (i.e. comparisons in which the participants provide complex quantities, together with an associated uncertainty region) the visualisation of results plays a determinant role. In a general case the region of uncertainty is an ellipse, which is defined by two standard deviations in both axes, together with the correlation factor between x and y. Besides, the problem arises regarding the assignment of a given figure of merit to each participant, which may help analysing the individual results. Traditional approaches rely on the definition of the so-called spatial median among the participants, as well as the median of absolute deviation (MAD). This allows us to define a circle encompassing all non-outliers, the definition of an outlier thus being to lie outside the circle of 3.75-MAD from the spatial median. However, both the spatial median and the MAD rely only on the measured values as obtained from each participant, and the whole approach provides no information, either visual or analytical, about the associated region of uncertainty estimated by each laboratory. Our first goal in this paper is to graphically represent the measured values, together with their uncertainty, against the computed "3.75-MAD-circle". Second objective is to define a figure of merit for each participant, from which an impartial observer can deduce the level of agreement between the individual laboratories and the reference value of the intercomparison. This figure of merit will be based on probabilities or levels of confidence, making use of the Monte Carlo method for simulation of the assumed probability density function of results. Previous approaches are based on the graphical representation of complex-valued quantities in the traditional magnitude and phase plots. This has the advantage of presenting the results in a similar way to the one-dimensional problem, as well as minor disadvantages such as the existence of asymmetric uncertainty budgets. Our feeling, however, is that we should be moving onto the full two-dimensional approach if we eventually want to get rid of old pre-conceived ideas from the one-dimensional world, e.g. the truncation of magnitude distributions as a result of the magnitude being non-negative. With this i- n mind, we will be presenting some theoretical examples of intercomparison results which show basically the same dispersion between participants, but different regions of uncertainty for each measured value. Our goal is to evaluate the change in the proposed figure of merit: (1) as the measured value changes; (2) as the uncertainty region is increased or reduced; and (3) as the uncertainty region is rotated. Finally, a similar approach as the one here presented can be applied to the computation of a generalised "normalised error" between any two complex-valued quantities and their associated regions of uncertainty. This can be of some help in topics like the assessment of measurement results when comparing between different methods or different laboratories. With this aim, several examples will be shown which try to demonstrate that, without the introduction of complex normalised errors, similarly measured values with differently-looking regions of uncertainty may be wrongly evaluated when the assessment is made in terms of magnitude or phase exclusively.