Training a network of mobile neurons

We introduce a new paradigm of neural networks where neurons autonomously search for the best reciprocal position in a topological space so as to exchange information more profitably. The idea that elementary processors move within a network to get a proper position is borne out by biological neurons in brain morphogenesis. The basic rule we state for this dynamics is that a neuron is attracted by the mates which are most informative and repelled by ones which are most similar to it. By embedding this rule into a Newtonian dynamics, we obtain a network which autonomously organizes its layout. Thanks to this further adaptation, the network proves to be robustly trainable through an extended version of the back-propagation algorithm even in the case of deep architectures. We test this network on two classic benchmarks and thereby get many insights on how the network behaves, and when and why it succeeds.