Colour based off-road environment and terrain type classification

Terrain classification is an important problem that still remains to be solved for off-road autonomous robot vehicle guidance. Often, obstacle detection systems are used which cannot distinguish between solid obstacles such as rocks or soft obstacles such as tall patches of grass. Terrain classification is needed to prevent that the robot is stopped needlessly by the obstacle detection system. It can also be used to recognize sand roads or other drivable areas. In this paper we present a colour based method to classify typical terrain coverings such as sand, grass or foliage. Using colour recognition outdoors is difficult, because the observed colour of a material is heavily influenced by environment conditions such as the scene composition and illumination. A new approach to colour based classification is presented. It is based on the assumption that images with large similarities in environment related properties such as illumination, materials and geometry also have similar pixel distributions in a colour space. Classification based on a maximum likelihood method with Gaussian mixture models (GMMs) is improved by first distinguishing image sets in the training set that share the same environment state. Because the terrain type colours are modelled separately for each found image set, the influence of changing environment conditions is reduced. Terrain types in a new image are classified with the GMMs of the environment state that is the most similar to it. The results show that our approach is able to classify terrain types in real images with large differences in illumination.