Unsupervised Segmentation of Overlapped Nuclei Using Bayesian Classification

In a fully automatic cell extraction process, one of the main issues to overcome is the problem related to extracting overlapped nuclei since such nuclei will often affect the quantitative analysis of cell images. In this paper, we present an unsupervised Bayesian classification scheme for separating overlapped nuclei. The proposed approach first involves applying the distance transform to overlapped nuclei. The topographic surface generated by distance transform is viewed as a mixture of Gaussians in the proposed algorithm. In order to learn the distribution of the topographic surface, the parametric expectation-maximization (EM) algorithm is employed. Cluster validation is performed to determine how many nuclei are overlapped. Our segmentation approach incorporates a priori knowledge about the regular shape of clumped nuclei to yield more accurate segmentation results. Experimental results show that the proposed method yields superior segmentation performance, compared to those produced by conventional schemes.