Watershed-based segmentation of cell nuclei boundaries in Pap smear images

In this work we present a fully automated method for the accurate detection of cell nuclei boundaries in conventional Pap smear images, based on the watershed transform. For the extraction of nuclei and cytoplasm markers, which are used as starting points for the flooding process, a morphological reconstruction step is initially performed in each image. The watershed transform is then applied in the color morphological gradient image, which shows the boundaries of the more pronounced nuclei. For the elimination of false positive findings, salient features of shape and intensity of the detected regions were calculated and a clustering step is performed. The method was evaluated with a data set of 19 images containing 3616 recognized cells nuclei. The performance of the method was evaluated in terms of the correct detection of the positions of the nuclei. Comparisons with the segmentation results of the gradient vector flow (GVF) deformable model showed that the segmentation of the watershed transform captures more accurately the boundaries of nuclei, leading to a better performance of the clustering algorithm.