Tests of localization errors of tomatoes based on binocular stereo vision caused by occlusion

The accuracy of the three dimensional localization of the vision systems of harvesting robots is likely to be effected by the occlusion of leaves and branches. To quantitatively analyze the influence to the localization for tomatoes based on binocular stereo vision caused by the occlusion, we analyzed the three dimensional localization errors of occluded tomatoes in three kinds of occlusion degree based on three stereo matching methods. The occlusion degrees were classified to slight occlusion, moderate occlusion and serious occlusion based on the rate of the occluded area of a tomato to the total area of this tomato. Tomato image segmentation was realized using the fixed threshold image segmentation method based on the normalized color difference between the red and green component. Stereo matching methods included centroid-based matching, area-based matching and combination matching. After stereo matching, three dimensional coordinates of tomatoes were computed using the triangle ranging principle. Tests based on 240 stereo images of five tomatoes in three different occlusion degrees captured at distances from 300 to 1000 mm showed that there were little difference for the widths of the error intervals of x coordinates acquired based on three stereo matching methods, but the widths of the error intervals of y coordinates and depth values acquired based on area-based matching and combination matching were smaller. Three dimensional coordinates were all fluctuated with different occluded part and different occlusion degrees of occluded tomatoes. The measurement values of x and y coordinates were more likely to be affected by the occlusion than those of depth.