(D-131) Deep-learning triage of 3D pathology datasets for sensitive and time-efficient detection of esophageal neoplasia

Introduction:: Early detection of esophageal neoplasia (defined here as dysplasia or esophageal adenocarcinoma) in patients with Barrett’s esophagus is key to maximizing patient survival^1,2. Unfortunately, histologic evaluation of biopsies obtained during regular surveillance endoscopy of Barrett’s esophagus patients is not ideal for detecting neoplasia since only a limited number of thin tissue sections from each biopsy are visualized in 2D. Comprehensive evaluation of whole biopsies with non-destructive 3D pathology may improve the sensitivity of detecting neoplasia^3,4. However, the large datasets generated by 3D pathology can be tedious for pathologists to manually review and analyze. Therefore, we have developed a weakly supervised deep-learning-based triage method to automatically identify 2D image sections that are most likely to contain neoplasia within 3D pathology datasets for time-efficient pathologist review.


Materials and Methods:: Esophageal specimens (n=60) from 21 patients were labeled with a fluorescent analog of H&E, optically cleared with ethyl cinnamate, and imaged with an open-top light-sheet (OTLS) microscope⁵ at ~0.6 micron lateral resolution. En face 2D image sections (n=247) were extracted from these 3D pathology datasets and rapidly annotated by an expert pathologist as either containing neoplasia (anywhere in the 2D image) or entirely benign. Based on the 2D image-level annotations, we designed a weakly-supervised deep-learning pipeline to predict the neoplastic risk for 2D image sections. First, our method divided tissue regions into image patches and encoded them with two models: CTransPath⁶, which was pre-trained in a self-supervised manner on large histopathology datasets, and ResNet, which was pre-trained in a supervised manner on ImageNet. Data augmentation was applied to the patches to enhance image diversity. Subsequently, we trained an attention-based network⁷ to aggregate patch-level features and generate image-level predictions.


Results, Conclusions, and Discussions:: To evaluate the algorithm’s performance, we employed 10-fold cross-validation over all annotated image sections. Our method achieves an area under the curve (AUC) of 87% (averaged over all cross-validation sets), which outperforms the previous clustering-constrained-attention multiple-instance learning (CLAM)⁷ method by 8%. Attention heatmaps overlaid on example images demonstrate that the model’s highly-attended regions roughly agree with pathologist’s pixel-level annotations of neoplasia. As future work, additional training is being performed to further improve the performance of our model. We are comparing the performance of the weakly supervised method to a previously reported fully-supervised method⁸, which was trained on 43 image sections with tedious pixel-level annotations. Our objective is to show that our weakly supervised model achieves superior performance, due to training on a larger and more varied set of clinical samples (expected n=100), while requiring less overall pathologist annotation time since generating image-level labels is much faster than generating pixel-level labels. In conclusion, our weakly supervised deep-learning triaged 3D pathology workflow has the potential to improve the detection of esophageal neoplasia while reducing pathologist workloads both in terms of generating training data (annotations) and in clinical practice.


Acknowledgements (Optional): :

References (Optional): : 1 Berry, M. F. Esophageal cancer: staging system and guidelines for staging and treatment. Journal of thoracic disease 6 Suppl 3, S289-297 (2014). https://doi.org:10.3978/j.issn.2072-1439.2014.03.11

2 Horvath, B. et al. Risk for esophageal neoplasia in Barrett's esophagus patients with mucosal changes indefinite for dysplasia. Journal of Gastroenterology and Hepatology 30, 262-267 (2015). https://doi.org:https://doi.org/10.1111/jgh.12696

3 Liu, J. T. C. et al. Harnessing non-destructive 3D pathology. Nature Biomedical Engineering (2021). https://doi.org:10.1038/s41551-020-00681-x

4 Reddi, D. M. et al. Nondestructive 3D Pathology Image Atlas of Barrett Esophagus With Open-Top Light-Sheet Microscopy. Archives of Pathology & Laboratory Medicine (2023). https://doi.org:10.5858/arpa.2022-0133-OA

5 Barner, L. A., Glaser, A. K., Huang, H., True, L. D. & Liu, J. T. C. Multi-resolution open-top light-sheet microscopy to enable efficient 3D pathology workflows. Biomed Opt Express, 1-15 (2020). https://doi.org:10.1364/OA_License_v1

6 Wang, X. et al. Transformer-based unsupervised contrastive learning for histopathological image classification. Medical Image Analysis 81, 102559 (2022). https://doi.org:https://doi.org/10.1016/j.media.2022.102559

7 Lu, M. Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering 5, 555-570 (2021). https://doi.org:10.1038/s41551-020-00682-w

8 Barner, L. A. et al. in Multiscale Imaging and Spectroscopy IV. PC1236302 (SPIE).