Comparability problems in the analysis of multiway data

Almost all methods for the analysis of multiway data assume that the comparison of any two entries in the data array under study reflects or represents meaningful content-specific information. This is especially the case if one wants the data analysis to yield insight into the real mechanisms underlying the data. Violation of this assumption may imply data-analytic results that are of doubtful quality at best and worthless in the worst-case scenario. In the present paper, we first clarify why comparability is a key assumption in most methods for multiway data analysis. Next, we list a number of reasons why this assumption is very often violated in practice. We then review a few possible approaches that have been advanced to deal with problems of comparability, and discuss their advantages and shortcomings. We conclude by clarifying that any satisfactory solution to comparability problems requires a very careful reflection about the data collection and the ultimate goal of the data analysis.