Analytical metrological characterization of the three-parameter sine fit algorithm

The three-parameter sine fit is a well known algorithm used in IEEE standard 1241-2000 for ADC characterization to estimate sine wave parameters from recorded data. An analytical metrological characterization of that algorithm is carried out in terms of its design parameters, namely the sampling rate and the number of samples. Optimal and sub-optimal parameters’ sets are found. Focusing on the amplitude estimation, an approximated statistical characterization of the second order is given in the most general case of zero mean additive noise, whereas the exact probability density function is found for the optimal set of algorithm parameters in the case of additive white Gaussian noise. In this particular case, the frequency behavior of the algorithm is fully analyzed and the exact frequency response of the amplitude is also found. An approximated expression that is easier to use is then presented and discussed. Simulation results for a single set of parameters are presented in order to graphically illustrate the analytical results.