A new detection method for MIMO systems with differently modulated layers

It is well known that the proper ordering is crucial for overall performance in OSIC method. However, when the layers or streams are modulated differently such as multi-user MIMO case, exact ordering is not possible. It is called different modulation effect which causes the performance degradation. In this paper, we propose the novel detection method which can eliminate the different modulation effect. The simulation result shows that the proposed method successfully eliminates the different modulation effect and can obtain the about 2 dB gain. Further, the proposed method can achieve the ML performance.