Constructing a sparse convolution matrix for shift varying image restoration problems

Convolution operator is a linear operator characterized by a point spread functions (PSF). In classical image restoration problems, the blur is usually shift invariant and so the convolution operator can be characterized by one single PSF. This assumption allows one to use fast operations such as Fast Fourier Transform (FFT) to perform a matrix-vector computation efficiently. However, as in most of the video motion deblurring problems, the blur is shift variant and so the matrix-vector multiplication can be difficult to perform. In this paper, we propose an efficient method to construct the convolution matrix explicitly. We exploit the submatrix structure of the convolution matrix and systematically assigning values to the nonzero locations. For small to medium sized images, the convolution matrix gives superior speed than some state-of-art convolution operators.