Optimized receiver design for decode-and-forward relays using hierarchical modulation

We consider a wireless relay network with a single source, a single destination, and multiple relays. The relays are half-duplex and use the decode-and-forward protocol. The transmit source is a layered bitstream, which can be partitioned into two layers. The source broadcasts both layers to all the relays and the destination using hierarchical 16-QAM. Each relay detects and transmits successfully decoded layers to the destination using either hierarchical 16-QAM or QPSK using orthogonal channels. We derive the optimal linear-combining receiver at the destination, where the uncoded bit error rate is minimized. We also present a suboptimal combining method with a closed-form solution, which performs very close to the optimal. Numerical results show that the two-layer scheme outperforms the classical one-layer scheme using conventional modulation.