Two classes of relative measures of estimation performance

The ability to meaningfully assess performance is crucial for understanding, developing and comparing estimators. The optimality of an estimator relies on estimation criterion and there exists a significant gap between estimation criterion and application requirements, so the estimation criterion is not good for evaluating or comparing algorithms. Different viewpoints for performance comparison can help practitioners gain better insight and choose proper estimators for their applications. In this paper, two classes of relative measures of performance are investigated. First, to characterize the application requirements we propose the use of a desired error PDF. The concentration and deviation measures w.r.t. the desired one are developed to quantify the estimation performance of each algorithm. Second, we examine Pitman´s closeness as an estimation performance measure. We then propose the relative loss and relative gain as performance measures, which utilize the joint information of both estimators in all possible cases. Illustrative examples are given for these measures.