Video object segmentation and tracking in stereo sequences using adaptable neural networks

In this paper, an adaptive neural network architecture is proposed for efficient video object segmentation and tracking of stereoscopic sequences. The scheme includes (a) a retraining algorithm for adapting network weights to current conditions, (b) a semantically meaningful object extraction module for creating a retraining set and (c) a decision mechanism, which detects the time instances that a new network retraining is required. The retraining algorithm optimally adapts network weights by exploiting information of the current condition with a minimal deviation of the network weights. Description of the current conditions is provided by a segmentation fusion scheme, which appropriately combines color and depth information.