Two-Dimensional Maximum-Likelihood Sequence Detection Is NP Hard

A 2-D version of the classical maximum-likelihood sequence detection (MLSD) problem is considered for a binary antipodal signal that is corrupted by linear intersymbol interference (ISI) and then passed through a memoryless channel. For 1-D signals and fixed ISI, this detection problem is well-known to be solved using the Viterbi algorithm in time complexity that is linear in the sequence length. It is shown here that, in contrast, the 2-D MLSD problem is NP hard. Specifically, a decision formulation of the problem is shown to be NP complete for a particular 2-D ISI cascaded with either of two memoryless channels: one involving errors and erasures and the other corresponding to additive white Gaussian noise. The proof for the latter case is obtained through a reduction from a still NP complete restricted version of the former. These results are applied to proving the NP completeness of multi user detection under a Toeplitz constraint-a problem known to be equivalent to a variant of 1-D MLSD with growing ISI. This proves a conjecture posed by Verdú in 1989.