Regularized high temporal resolution fish motion synthesis

This paper tackles the fish motion synthesis problem with the aim to increase the temporal resolution by generating a high frame rate video from a given video captured at a lower frame rate. The proposed approach reconstructs the virtual frames via estimating motion-vector fields that represent the correspondences between consecutive frames. To be more specific, the proposed approach utilizes two regularization functions: (i) an intra regularization function that is proposed to impose a local smoothness constraint on the estimated motion vector fields; and (ii) an inter regularization function that considers both intensity consistency and kinematic consistency between the reconstructed frame and its neighboring frames. Real-world fish motion video dataset is used to demonstrate the superior performance of the proposed approach in high temporal resolution fish motion synthesis, as well as its application in fish motion analysis.