Cell nuclei segmentation by learning a physically based deformable model

In this work, we present an efficient framework for the training of active shape models (ASM), representing smooth shapes, which is based on the representation of a shape by the vibrations of a spring-mass system. A deformable model whose behavior is driven by physical principles is used on a training set of shapes. The boundary of the regions of interest of the elements of the training set is detected with the convergence of the physics-based deformable model and attributes of the shapes of interest are expressed in terms of modal analysis. Based on the estimated modal distribution, we develop a framework, similar in spirit to ASM, to detect and describe an unknown new shape. The main difference with respect to standard ASM is that the modal amplitudes of the learnt model are used instead of the 2D landmark points and the cost function to be minimized is accordingly modified. The proposed method is evaluated using cytological images of conventional Pap smears, which contain 44 nuclei of squamous epithelial cells.