Skip to main content
2023 BMES Annual Meeting Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.

    Data Analysis and Deep Learning

    (G-249) Leveraging the Prognostic Potential of Histopathology Data: Deep Convolutional Neural Networks for Novel Biomarker Discovery in Colorectal Polyp Subtype Classification

    Thursday, October 12, 2023
    9:30 AM - 10:30 AM PDT
    Location: Exhibit Hall - Row G - Poster # 249
    Introduction::



    Polyps are clusters of cells that form on the colon lining and affect 25% of individuals over the age of 45. These seemingly innocuous clumps can grow abnormally to form malignant tumors that severely impact the quality of life. Histopathology analysis, a common diagnostic approach, stains extracted polyp biopsies for subtyping by expert pathologists. However, the cost, limited clinical expertise, and considerable inter-pathologist variability of polyp subtyping contribute to a high rate of misdiagnosis, leading to inaccurate and overly aggressive treatment plans. The promising field of deep learning has shown remarkable potential in enhancing disease diagnostics. Yet, there remains an underexplored opportunity to leverage deep learning models with histopathology images — a diagnostic medium rich in prognostically-relevant proteomic and genomic markers of disease. Moreover, dependencies on large datasets impede translational progress in medical AI. To address this gap, we present HistoPolyp, a low-cost, efficient, and accurate deep convolutional neural network pipeline for robust colorectal polyp subtyping of hyperplastic polyps (HP) and precancerous sessile serrated adenomas (SSA) in histopathology images. HistoPolyp addresses one of precision medicine's primary limitations in that it attains high performance after training on a minimalist dataset of only a few thousand images (n = 2,522) – compared to state-of-the-art models that require training on millions of images to become remotely useful in the clinical setting. Furthermore, we employ gradient-weighted class activation mapping, a computational visualization tool, to identify prognostically-relevant imaging biomarkers for both HP and SSA subtypes. HistoPolyp's technology has applications in diagnosing other diseases as well.







    Materials and Methods::



    We employed the MHIST dataset from the Dartmouth-Hitchcock Medical Center (DHMC), comprising 3,152 deidentified H&E-stained images of colorectal polyp subtypes. Our task was to differentiate between benign HPs and precancerous SSAs, which exhibit notable inter-pathologist variability and whose presence informs patient prognosis and risk of cancer. The dataset, stored in *.png format, was divided into training (n = 2,522) and testing (n = 630) subsets. To mitigate the impact of stain variance on model performance, images underwent stain normalization and were subsequently put through a random augmentation data pipeline for increased generalizability. We trained three deep convolutional neural networks (DenseNet-121, Resnet-34, and MobileNetV3) on the preprocessed data for 100 epochs. Each network used different respective learning rates, with a constant batch size of 34. Subsequently, we evaluated the models individually on a holdout test set of 630 images, balanced between HP and SSA. Performance assessment included accuracy, precision, recall, F1 score, and AUROC to determine clinical viability. Additionally, each model underwent a gradient-weighted class activation mapping pipeline, generating heatmaps for potential imaging biomarker identification and structure-based histological features corresponding to subtype classification. All models were developed and trained using PyTorch 2.0, on a Tesla T4 GPU. The model was evaluated on a standard CPU and time for diagnosis was measured.







    Results, Conclusions, and Discussions:: Results (measured in metrics):




































    AccuracyPrecisionRecallF1AUROC
    DenseNet-1210.8690.8690.8680.8680.86
    ResNet-340.8660.8660.8650.8630.840
    MobileNetV30.8680.8690.8680.8650.841


    * Gradient-Weighted Class-Activation Mapping Data viewable in the attached figure


    The DenseNet-121 convolutional neural network demonstrated exceptional performance with only 100 training epochs, establishing it as the optimal clinical choice for subtyping. It outperformed all other models, achieving the highest AUROC, F1, and accuracy values. AUROC measures the model's classification abilities in differentiating between positive and negative classes (HP vs. SSA), accounting for class imbalance in the test dataset. The F1 score measures the model's ability to correctly classify positive instances, referencing both false positives and false negatives. Accuracy measures the overall correctness of predictions made by a model. It calculates the ratio of correctly classified instances (both true positives and true negatives) to the total number of instances. The impressive results across multiple metrics confirm DenseNet's robustness and clinical viability for accurate subtyping. DenseNet was integrated into the HistoPolyp pipeline and made publicly accessible. Other metrics, such as precision and recall, measure accuracy across positive predictions and the completeness of positive predictions, respectively.

    Conclusion and Discussion: Our study presents HistoPolyp, an efficient and robust deep convolutional neural network pipeline for accurate subtyping of HP and SSA. By leveraging deep learning with stain-normalized histopathology images, we address the limitations of cost, limited expertise, and inter-pathologist variability in traditional analysis. Despite a minimalist dataset of only 2,522 images, our pipeline surpasses the performance of state-of-the-art models trained on millions of images. We successfully differentiate between HP and SSA subtypes, a crucial task for prognosis and cancer risk assessment. Stain normalization mitigates variance due to batch effects across institutions, and DenseNet-121 emerges as the best model for subtyping. Grad-CAM aids in visualizing diagnostically-relevant features that predict accurate diagnosis. HistoPolyp improves diagnostic accuracy, reduces misdiagnosis, enables personalized treatment plans for colorectal polyps well in advance, and addresses one of precision medicine's primary limitations in data dependency. The approach extends to diverse diseases and other diagnostic/prognostic tasks, including survival prediction, recurrence prediction, and segmentation.


    Acknowledgements (Optional): : No conflicts to disclose. Some diagrams were created using Biorender.

    References (Optional): : 1. Y. Zhu and S. Newsam, "DenseNet for dense flow," 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017, pp. 790-794, doi: 10.1109/ICIP.2017.8296389.

    2. Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778

    3. Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V. Le, Hartwig Adam. Searching for MobileNetV3. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 1314-1324

    4. Chung-Jyi Tsai; Lu, David K. Small Colorectal Polyps: Histopathology and Clinical Significance. American Journal of Gastroenterology (Springer Nature) . Jun1995, Vol. 90 Issue 6, p988-994.

    5. Thomas K.L. Lui, Chuan-Guo Guo, Wai K. Leung. Accuracy of artificial intelligence on histology prediction and detection of colorectal polyps: a systematic review and meta-analysis. Gastrointestinal Endoscopy, Volume 92, Issue 1, 2020, Pages 11-22.e6. https://doi.org/10.1016/j.gie.2020.02.033.

    6. du Toit, R., Locker, A., Ellis, I. et al. Invasive lobular carcinomas of the breast – the prognosis of histopathological subtypes. Br J Cancer 60, 605–609 (1989). https://doi.org/10.1038/bjc.1989.323

    7. Marzouk, Ola, and John Schofield. 2011. "Review of Histopathological and Molecular Prognostic Features in Colorectal Cancer" Cancers 3, no. 2: 2767-2810. https://doi.org/10.3390/cancers3022767

    8. Morson BC, Whiteway JE, Jones EA, et al. Histopathology and prognosis of malignant colorectal polyps treated by endoscopic polypectomy. Gut 1984;25:437-444.

    9. Wu, Yawen, Michael Cheng, Shuo Huang, Zongxiang Pei, Yingli Zuo, Jianxin Liu, Kai Yang, Qi Zhu, Jie Zhang, Honghai Hong, Daoqiang Zhang, Kun Huang, Liang Cheng, and Wei Shao. 2022. "Recent Advances of Deep Learning for Computational Histopathology: Principles and Applications" Cancers 14, no. 5: 1199. https://doi.org/10.3390/cancers14051199

    10. Rastogi, Archana. “Changing role of histopathology in the diagnosis and management of hepatocellular carcinoma.” World journal of gastroenterology vol. 24,35 (2018): 4000-4013. doi:10.3748/wjg.v24.i35.4000

    11. Jadvar, H. Imaging evaluation of prostate cancer with 18F-fluorodeoxyglucose PET/CT: utility and limitation

    12. Echle, A., Rindtorff, N.T., Brinker, T.J. et al. Deep learning in cancer pathology: a new generation of clinical biomarkers. Br J Cancer 124, 686–696 (2021). https://doi.org/10.1038/s41416-020-01122-x

    13. Shuo Wang, Zhenyu Liu, Yu Rong, Bin Zhou, Yan Bai, Wei Wei, Wei Wei, Meiyun Wang, Yingkun Guo, Jie Tian. Deep learning provides a new computed tomography-based prognostic biomarker for recurrence prediction in high-grade serous ovarian cancer. Radiotherapy and Oncology, Volume 132, 2019, Pages 171-177. https://doi.org/10.1016/j.radonc.2018.10.019.

    Some graphical abstract diagrams were made using BioRender.