(F-238) ChatGPT Interprets Deep Learning Model for Alzheimer’s Disease Classification

Introduction:: Alzheimer's disease (AD) is the most prevalent neurodegenerative disease; yet current treatments are limited to preventing disease progression [1]. Moreover, the effectiveness of treatment is not guaranteed due to the heterogeneity of the disease. Therefore, it is essential to identify the disease at a very early stage. Deep learning models have been developed for AD diagnosis. However, these models lack explainability. Even though various explanation methods have been proposed, the results from those explanations are hard to interpret; thus, limiting their usefulness to clinicians and patients. Interpreting the multi-modal learning models is even more challenging. On the other hand, large language models have recently reached a point of offering highly coherent answers to prompts, resembling a very knowledgeable entity that is familiar with all content on the internet. Their great versatility to answer questions from multiple domains offers an excellent opportunity to bridge the gap between deep learning models and the medical field by providing explanations for the model predictions. We investigate using ChatGPT to interpret the result of our deep learning models. We build a prompt for each patient based on the top-weighted features for ChatGPT to generate interpretable reports for the model prediction per subject. The generated report includes clear descriptions for each feature of interest and their relationship toward AD development and how jointly they impact the deep learning model’s assessment. This approach provides not only high explainability to the model assessment but also key insights into cross-modal feature associations supported by known biological mechanisms.


Materials and Methods:: Cognitive and behavioral assessments, brain magnetic resonance imaging (MRI) scans, and genotype data were obtained from 494 participants of the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Our in-house deep learning model, named “tri-modal co-attention”, was used to classify each subject as AD, mild cognitive impairment (MCI), or healthy control (HC). As part of the classification, the deep learning model computes a probability score for each subject belonging to each outcome category. Moreover, it computes an attribution score for every input feature for a given patient implementing the integrated gradients method. This attribution score describes how relevant each feature is toward classifying patients. The input features are ranked using the attribution scores and the top K features are selected for each subject. In order to describe how each subject differs from the rest of the group based on the analyzed biomarkers, the standard deviation for every feature is calculated. Descriptions of each patient case (prompts) were built using the top K features and their corresponding standard deviations. In detail, the prompts were structured as a question and a case describing the patient biomarker measurements. The question asks to explain the model prediction and the case contains the selected features’ contribution and corresponding standard deviation. The prompts were analyzed by a large language model, namely ChatGPT (GPT-3.5 March 23, 2023 version). A final report for each subject that interprets the model’s AD assessment was obtained as the output from ChatGPT.

Results, Conclusions, and Discussions:: As seen in Table 1, the prompt describes the numerical contribution of each feature to the model classification and how different the patient-measured trait is from the rest of the group via the standard deviation. We used the top 10 features in our experiments. In turn, ChatGPT describes the biological significance of each feature and how it relates to AD development. ChatGPT is able to explain the classification based on the measured trait of each patient. For example, ChatGPT describes the biological functions of the right insula and its connection with AD. Next, it explains that the significantly reduced volume in this brain structure suggests a loss of function and a rapidly progressing AD. Similarly, it is promising to see that even when the measured trait deviates from the group in the opposite direction as expected for an AD patient, ChatGPT identifies the value as unexpected and provides an explanation for it. For example, for the abnormally higher cortical thickness of the right cuneus, it explains that this could indicate a compensatory mechanism , which has been previously proposed in the literature [2].
This study demonstrates the potential of language models to bridge the gap between deep learning models and the medical field, providing interpretable and actionable insights for clinicians and patients. There are a few interesting future directions to explore. First, selecting the value of K for the top features could lead to more personalized results. Similarly, further exploration of the integration of attention mechanisms and feature attribution analysis warrants further investigation. Finally, it would be important and beneficial to establish standardized evaluation metrics for the reports generated through GPT.

Acknowledgements (Optional): :

References (Optional): : [1] Alzheimer's Association Report. Alzheimer’s. Dement., 2022, 18, 780–789.

[2] Niskanen, E. et al. New Insights into Alzheimer’s Disease Progression: A Combined TMS and Structural MRI Study. PLOS ONE 6, e26113 (2011).