Viewing and reviewing how humanoids sensed, planned and behaved with Mixed Reality technology

How can we see how humanoids sensed, planned, and behaved in the actual environment? We propose a mixed reality environment for viewing and reviewing of internal parameters computed by the humanoids against the actual environment. In the environment, the parameters snapped on each sampling time are treated as data streams, and stored to the distributed log servers in real time. 3-D graphical representation of the parameters helps us to observe plural multi-dimensional parameters. A video see-through head mounted display is used for viewing the representation. The stored parameters can be projected on the current actual scene from arbitrary viewpoints. By the viewing and reviewing functions, the mixed reality environment becomes a powerful tool for the autonomous behavior developer in order to debug the actual behavior by contrast with the actual environment. This paper describes the implementation of the system with a full size humanoid, HRP-2, and shows some experimental examples.