Linear Precoding Designs for Amplify-and-Forward Multiuser Two-Way Relay Systems

We investigate the linear precoding designs for multiuser two-way relay system (MU-TWRS) where a multi-antenna base-station (BS) communicates with multiple single-antenna mobile stations (MSs) via a multi-antenna relay station (RS). The amplify- and-forward (AF) relay protocol is employed. The design goal is to optimize the precodings at BS, RS or both so as to minimize the total mean-square error (MSE) of the uplink messages while maintaining the individual signal-to-interference-plus-noise ratio (SINR) requirement for each downlink signal. We show that the BS precoding design problem can be converted to a standard second order cone programming (SOCP), while the RS precoding is non- convex for which a local optimal solution is obtained using an iterative algorithm. A joint BS-RS precoding is also obtained by alternating optimization of BS precoding and RS precoding with guaranteed convergence. Numerical results show that RS-precoding is superior to BS-precoding. Furthermore, the joint BS-RS precoding can significantly outperform the two individual precoding schemes. The implementation issues including complexity and feedback overhead are also discussed.