Novel robust optimal filter design method and new transition band analysis

The optimal finite impulse response (FIR) filter design has been prevalent in science and engineering over many decades due to its guaranteed stability and a wide variety of applications, especially for telecommunication transmission systems. The equiripple-filter and eigen-filter designs are two commonly-used signal processing paradigms. However, when the filter length gets large or the required transition bandwidth becomes very small, the ill-behaved spectra of the equiripple and eigen filters emerge, including excess ripples in the passband/stopband and overshoot in the transition band. In this paper, we introduce a new filter design criterion based on the smoothness measure and present several new metrics to quantify the robustness of an arbitrary filter design method. According to our new analysis, our proposed new robust optimal filter design scheme outperforms other existing filter design techniques.