Fusion Algorithm for Locally Arranged Linear Models

As an extension to a recently proposed local linear approximation method we present an algorithm that generates more compact solutions for supervised-learning problems. Given a network of linear models each trained to approximate the target function in a local region of the input space, the algorithm reduces the number of the models significantly without diminishing the accuracy of the approximation. It fuses linear models by combining their local regions of validity to more complex, non-symmetrically shaped ones. A neighborhood graph introducing edges in a purely data-driven manner between adjacent linear models is used to determine which models should be fused. The also extended model for a region of validity allows to detect automatically data which is novel to a trained network and should be regarded as an outlier. The effectiveness of the proposed methods is shown with a benchmark test achieving a five times smaller RMSE than the best competitors