Precoder Optimization for Nonlinear MIMO Transceiver Based on Arbitrary Cost Function

Assuming full channel state information (CSI) at both transmitter (CSIT) and receiver (CSIR), we consider optimizing a nonlinear MIMO transceiver with decision feedback equalizer (DFE) with respect to some global cost function f₀. Setting the receiver to be a minimum mean-squared error (MMSE) DFE, the MIMO transceiver optimization problem reduces to optimizing a linear precoder. Based on the generalized triangular decomposition (GTD) and majorization theory, we prove that for any cost function f₀ the optimum precoder is of the same special structure and hence the original complicated matrix optimization problem can be significantly simplified to an optimization problem with scalar-valued variables. Furthermore, if the cost function is specialized to the cases where the composite function f₀ o exp is either Schur-convex or Schur-concave, then the nonlinear transceiver design becomes exceedingly simple. In particular, when f₀ o oexp is Schur-convex, the optimum nonlinear transceiver design turns out to be the uniform channel decomposition (UCD) scheme; when f₀ o exp is Schur-concave, the optimum nonlinear design degenerates to linear diagonal transmission.