Joint Stream-Wise THP Transceiver Design for the Multiuser MIMO Downlink

This paper addresses the problem of joint transceiver design for Tomlinson-Harashima Preceding (THP) in the multiuser multiple-input-multiple-output (MIMO) downlink. Different from previous work, we perform stream-wise (both inter-user and intra-user) interference pre-cancelation at the transmitter. A minimum total mean square error (MT-MSE) criterion is used to formulate our optimization problem. By some convex analysis of the problem, we obtain the necessary conditions for the optimal solution. Then an iterative algorithm is proposed to handle this problem and its convergence is proved. Simulation results are given to verify the efficacy of our proposed scheme and to show its superiority over existing THP-relevant schemes.