(A-27) Unsupervised Nuclei and Cell Segmentation in Multiplexed Images of Complex Tissue Samples

Introduction Recent achievements in highly multiplexed fluorescence imaging have substantially increased the range of proteins that can be analyzed in a tissue section up to 70 [1, 2]. In particular, it opens up new opportunities in understanding the spatial context of heterocellular interactions in tumor microenvironments. The spatial characterization of expressions for multiple proteins requires the localization of all cells within a tissue image that eventually leads to the cell segmentation problem. Recently, various deep neural networks capable segmenting cells and nuclei have been developed. In particular, the sophisticated adaptation of the U-Net architecture for tracking of diffused gradient flows led to the development of a deep learning segmentation method called Cellpose [3]. Another significant deep learning algorithm for automatic nuclei and cell segmentation, called Mesmer, was developed on the base of ResNet50 architecture [4]. Despite Cellpose and Mesmer demonstrating high segmentation accuracy, it remains unclear how to proceed when they give incorrect segmentation results. Retraining of the deep neural networks to improve their generalization usually requires a substantial amount of annotated data, without the guarantee of performance improvement. We have recently developed an UNsupervised SEGmentation (UNSEG) algorithm for nuclei and cell segmentation that does not require training, can be adapted to individual tissues, and demonstrates performance at par with Cellpose and Mesmer, when tested on a new dataset that we have also curated. The dataset comprises of 75 images, annotated by expert pathologists, and includes both normal and malignant tissue samples from 8 organs of the human gastrointestinal system.

Materials and Methods We generated the images by adapting cyclic immunofluorescence imaging to our dataset [2]. In this data set nuclei were stained with Hoechst and cell membranes with Na⁺K⁺ATPase. The imaging was performed using a 40X objective with numerical aperture of 0.95. The dimensions of every image are 1000×1000 pixels. We developed a Bayesian-like framework to generate global and local a posteriori probability maps of cell nuclei and membranes and used them to estimate their unsupervised segmentation masks.

Results, Conclusions, and Discussions An illustrative example demonstrating UNSEG’s high nuclei and cell segmentation performance is shown in Figure 1. Figure 1(a) shows one tissue image from our 75-image dataset with nuclei shown in blue and cell membranes in red. Figure 1(b) shows the ground truth contours of nuclei (white lines) and cells (green lines) manually annotated by the expert pathologist. On the other hand, Figures 1(c), (d), and (e) show the contours of nuclei (white lines) and cell (green lines) segmentation labels predicted by UNSEG, Cellpose, and Mesmer, respectively. Finally, Figure 1(f) (Figure (g)) demonstrates the objective comparison of nuclei (cell) segmentations predicted by UNSEG, Cellpose, and Mesmer with the ground truth annotation. The comparison is based on the computing of F₁ score (Dice coefficient) curves as functions of intersection over union threshold between annotated and segmented labels. The segmentation performance of UNSEG, Cellpose, and Mesmer overall 75-image dataset is presented in Figure 2, where for each of three algorithms the median F₁ score curve is plotted as a function of the intersection over union threshold between the segmented and annotated nuclei (Figure 2(a)) and cells (Figure 2(b)).
We have developed UNSEG, a novel unsupervised algorithm for automatic segmentation of nuclei and cells in complex tissues whose performance matches that of current state-of-the-art deep learning algorithms trained on millions of cells. UNSEG does not require any training and has a small set of parameters allowing it to be adapted to different types of images. Conceptually, it demonstrates that compartment specific labeling is highly rich in information that can be exploited to achieve human level performance equivalent to that of deep learning model, but with the flexibility to tunability that does not require generation of new models and does not require training. As part of this method development, we have also curated a new dataset with 16201 labeled nuclei and 16217 labeled cells for training and testing segmentation algorithms.

Acknowledgements