Random embedding machines for low-complexity pattern recognition

It is shown that, under a local clustering condition, a set of points of a given class, embedded in binary space by a set of randomly parameterized surfaces, is linearly separable from other classes, with arbitrarily high probability. We call such a data set a local relative cluster. The size of the embedding set is linear in the input dimension and inversely proportional to the squared local clustering degree. A simple parameterization by embedding hyperplanes leads to the separation of multi-cluster data by a network with two internal layers. The computational complexity is linear in the number of relative clusters in the data. This represents a considerable reduction of the learning problem with respect to known techniques, resolving a long-standing question on the complexity of random embedding. Numerical tests show that the proposed method performs as well as state-of the-art methods, in a small fraction of the time