3D hand posture estimation and task semantic monitoring technique for human-robot collaboration

We develop a monitoring system for human-robot collaboration (HRC) of hybrid automation. The system enables the production line to avoid product defects as well as guarantee a real-time warning system in a cellular manufacturing (CM). A task semantic monitoring system includes the three-dimensional (3D) hand posture estimation and the motion trajectory monitoring. The coordinate of hand, the motion of hand, and the semantic of hand are identified by two integrated modules. The verification is executed by applying the system to the assembly lines for power protector.