A cartpole experiment benchmark for trainable controllers

The inverted pendulum problem, i.e., the cartpole, which is often used for demonstrating the success of neural network learning methods, is addressed. It is shown that a random search in weight space can quickly uncover coefficients (weights) for controllers that work over a wide range of initial conditions. This result indicates that success in finding a satisfactory neural controller is not sufficient proof for the effectiveness of unsupervised training methods. By analyzing the dynamics of the linear controller, the cartpole problem is reformulated to make it a more stringent test for neural training methods. A review of the literature on unsupervised training methods for cartpole controllers shows that the published results are difficult to compare and that for most of the methods there is not clear evidence of better performance than the random search method.<>