Self-interference aware MMSE filter design for a cellular multi-antenna two-way relaying scenario

A single cell scenario with different multi-antenna nodes, i.e., a base station and several mobile stations, is considered. An intermediate multi-antenna relay station is used to support the bidirectional communications. In such a scenario, the required data rates for up - and downlink are typically different which is considered by introducing asymmetric rate requirements. Non-regenerative two-way relaying is applied and the nodes can subtract the back-propagated self-interference. To fully utilize this capability of the nodes for the transceive filter design at the relay station, an MMSE based transceive filter is derived which does not suppress self-interference. Thus, the required number of antennas at the relay station is reduced. Additionally, the transceive filter is optimized for asymmetric rate requirements by introducing a weighting parameter. Furthermore, an alternating optimization between the transceive filter at the relay station and the transmit and receive filters at the nodes is introduced to fulfill the asymmetric rate requirements and to increase the achievable sum rates. Simulation results show that the proposed filter design at the nodes and at the relay station requires less antennas and achieves higher sum rates compared to conventional approaches.