Estimation with two hidden layer neural nets

We deal with function estimation by neural networks. Mean square error bounds are given for the case when the target function is in the convex hull of ellipsoids multiplied by a scalar constant. When the target function is not in this class but is bounded, we bound the difference between the mean square prediction error compared to the best approximation error of the target function (the expected regret). We also give a general theorem that gives the convergence rate of the expected regret when the functions are estimated by penalized least squares criteria