Video packet priority assignment based on spatio-temporal perceptual importance

We present a novel perceptually motivated two-stage algorithm for assigning priority to video packet data to be transmitted over the internet. Priority assignment is based on temporal and spatial features that are derived from low-level vision concepts. In the first stage, the effect on temporal fluidity due to a packet being dropped is used to estimate its temporal importance which is computationally very efficient and can be directly used in low-delay applications with limited computational resources. In the second stage, saliency-weighted structural similarity is used to estimate the spatial importance of video packets. Subsequently, a non-linear combination of the temporal and spatial importance estimates is used to assign packet priority. The efficacy of the proposed algorithm (both stages) is demonstrated using an intelligent packet drop application where it is compared with cumulative mean squared error (cMSE) based priority assignment and random packet dropping.