Detecting assembly actions by scene observation

We present a fast and reliable method to analyse an image sequence where two human hands perform assembly actions. Our classifier to detect the initial skin coloured regions is based on a polynomial classifier of sixth degree. For each region a judgement is calculated using two cues. The first cue is motion information obtained from a difference image. The second cue is obtained from confidence mapping performed on the output of the polynomial classifier. All resulting regions with a high judgement indicating movement are tracked using Kalman filters. Based on the trajectories of the Kalman filters action hypotheses are generated. The hypotheses are verified through a detailed analysis using a Fourier transformation on the derivatives of the trajectories