The OI, OS, OMNI, and OSMAN networks as best approximations of nonlinear systems under training data constraints

A fundamental theory of artificial neural networks based on a Generalized Fock Space (GFS) framework is presented. First, the construction of a GFS F_ρ(X) from a sequence of tensor products of a Hilbert Space X is briefly recapitulated. Then the structures of four abstract canonical nets are derived as solutions of a nonparametric optimization (best approximation of a generic nonlinear system) problem in F_ρ(X) under input-output training or design data constraints. These nets are the OI (Optimal Interpolating), OS (Optimal Smoothing), OMNI (Optimal Multilayer Neural Interpolating), OSMAN (Optimal Smoothing Multilayer Artificial Neural) nets. Depending on the specification of the underlying spaces, these nets can take the form of appropriate functional or conventional neural nets. Their capability of adaptation, learning with and without supervision, and evolution have been discussed and demonstrated in our other publications