Sum Rate Maximization of an MIMO Two-Way Relay System Using MSE Duality

In this letter, we propose a novel joint beamforming scheme using mean-square error (MSE) duality to maximize the sum rate of a multiple-input multiple-output (MIMO) two-way relay system. It is difficult to solve the sum rate maximization problem directly due to its non-convexity. To manage this difficulty, the optimization problem is reformulated as a problem that minimizes the product of the minimum mean-square errors. Then, we show that the per stream MSE duality holds in a two-way relay system under cross power constraints. Based on the per stream MSE duality, an alternate algorithm is proposed to jointly optimize the linear transceivers. The simulation results demonstrate that the proposed scheme outperforms the existing linear schemes in view of the sum rate maximization.