Theoretical and statistical evaluation for approximate solution of large, over-determined, dense linear systems

The solution of linear least squares system requires the solution of over-determined system of equations. For a large dense systems that requires prohibitive number of operations. We developed a novel numerical approach for finding an approximate solution of this problem if the system matrix is of a dense type. The method is based on Fourier or Hartley transform although any unitary, orthogonal transform which concentrates power in a small number of coefficients can be used. This is the strategy borrowed from digital signal processing where pruning off redundant information from spectra or filtering of selected information in frequency domain is the usual practice. For the least squares problem the procedure is to transform the linear system along the column to the frequency domain, generating a transformed system. The least significant portions in the transformed system are deleted as the whole rows, yielding a smaller, pruned system. The pruned system is solved in transform domain, yielding the approximate solution. The quality of approximate solution is compared against full system solution and differences are found to be on the level of numerical noise. Theoretical evaluation of the method relates the quality of approximation to the perturbation of eigenvalues of the system matrix. Numerical experiments illustrating feasibility of the method and quality of the approximation at different noise levels, together with operations count are presented.