Boltzmann Machines Reduction by High-Order Decimation

Decimation is a common technique in statistical physics that is used in the context of Boltzmann machines (BMs) to drastically reduce the computational cost at the learning stage. Decimation allows to analytically evaluate quantities that should otherwise be statistically estimated by means of Monte Carlo (MC) simulations. However, in its original formulation, this method could only be applied to restricted topologies corresponding to sparsely connected neural networks. In this brief, we present a generalization of the decimation process and prove that it can be used on any BM, regardless of its topology and connectivity. We solve the Monk problem with this algorithm and show that it performs as well as the best classification methods currently available.