Segmentation-based joint classification of SAR and optical images

The authors devise a new data fusion technique for classification that exploits the information contained in single-channel SAR and optical images optimally. In fact, even though good classification performance can be achieved by using multiple optical channels, it is shown that classification results from a single optical channel are not appreciably better than those obtained with a single SAR image. The impact of fusion of SAR and optical images is investigated quantitatively. To characterise the limits that can be achieved, both lower and upper bounds to classification performance are introduced, corresponding respectively to single-pixel classification and joint classification of all pixels in the regions defined by the ground truth. First, an optimised technique for single-channel image segmentation followed by maximum likelihood (ML) classification is proposed. A significant performance improvement is demonstrated by classifying the homogeneous regions identified by segmentation, instead of single pixels or even small windows of 3×3 pixels. Indeed, the result consistently approaches the upper bound. Next, the proposed ML segmentation technique is extended to exploit the information available in SAR and optical images jointly to define the best set of segments. The segmented SAR and optical images are then used together to obtain the best possible classification of the segments, yielding a significant performance improvement with respect to using either sensor alone. Classification performance is again significantly better than with single pixels, or small windows, and approaches the upper bound.