Optimal Filters for Binary Detectors with Comparator Hysteresis

Methods that increase the speed of digital detection comparators can also produce hysteresis, wherein the effective threshold in a given signaling interval is influenced by the previous output. If the comparator is fed by a peak-limited IF detector output followed by a detection filter with unity dc gain, the hysteresis can be a significant fraction of the peak comparator input. This concise paper derives the detection filter impulse response for which the resulting eye-opening-to-rms-noise ratio is maximized. The impulse response is taken to be either unconstrained in time or confined to a signaling interval (as in filter-and-dump detection). The results for each case are applied to 2- and 4-level coherent phase shift keying (CPSK). Expressions are obtained for the maximized eye-opening-to-rms-noise ratio as a function of comparator hysteresis and phase modulation pulse shape. These expressions are normalized to represent the excess in required power above the theoretical minimum, and graphical results are given. They show the suboptimality of filter-and-dump detection when hysteresis is high, the insensitivity of performance to hysteresis when unconstrained optimal filters are used, and the relative effects of phase modulation pulse shape on required power.