(E-191) Assessing the Necessity of Refined Segmentation Annotations in Cardiovascular Risk Assessment

Introduction:: Cardiovascular disease (CVD) remains a significant health concern, causing 32% of global deaths, with 17.9 million fatalities annually, per the World Health Organization [1]. In the U.S., nearly half of all adults were impacted by CVD between 2015 and 2018; in 2019, CVDs accounted for 874,613 deaths [2]. Many risk factors, including age, cholesterol, and blood pressure, have been identified and extensively used for CVD risk assessment. Recently, machine learning methods leveraging image-based risk factors have garnered attention. These image-based factors, such as coronary artery calcium and heart volume, are predominantly computed using deep-learning segmentation models. However, precise segmentation annotations, essential for training these models, are labor-intensive and challenging to obtain. Our research explores whether high-precision segmentation annotation data is imperative for CVD risk assessment using these deep learning models. We hypothesize that under circumstances where the risk assessment model performance is limited, such as with kernel-based logistic regression, refined segmentation annotation might not substantially enhance the performance in predicting CVD mortality. The assessed image risk factors encompass measurements of specific anatomical heart and aorta structures, with a focus on factors like calcium deposits in the aorta and overall heart volume. This study aims to provide valuable insights into cardiovascular risk assessment, potentially leading to advanced early detection, personalized treatment strategies, and a decreased CVD burden globally.

Materials and Methods:: Data from the National Lung Screening Study Dataset [3], a publicly available source, was utilized. We analyzed 10,395 low-dose computed tomography (LDCT) scans, each corresponding to a distinct subject. Out of these, 600 subjects succumbed to CVDs over an average follow-up period of 6.5 years. For heart and aorta segmentation model training, 76 out of the 10,395 scans were randomly chosen. Two annotation techniques were employed on these 76 LDCTs: 1) Auto annotation using a commercial software, serving as the coarse annotation control group. 2) Refined annotations by J.V. and A.S. under the guidance of a radiologist from Massachusetts General Hospital. Two deep-learning models were trained using these respective annotation sets. These models were then applied to all 10,395 scans, producing two sets of risk factors: coarse and detailed, with each set containing the same 24 factors. To validate our hypothesis, a classification was performed on both sets to examine whether the more labor-intensive manual segmentation yields a higher classification accuracy using MATLAB. The classification method relied on a direct kernel transformation based on a randomly selected set of prototypes that are then passed through a logistic activation function. The performance of using the two different segmentation schemes was assessed on a validation set containing 15% of randomly selected data points, whilst the training of the classifiers involved the remaining 85% of the data.

Results, Conclusions, and Discussions::
The output of the classification model generates a value ranging from 0 to 1. By selecting a threshold to maintain a specificity of ≥ 0.8, the model utilizing refined manual annotations achieved a sensitivity of 0.5208, accuracy of 0.8012, F1 score of 0.2439, and a specificity of 0.8195. In comparison, the model based on coarse annotations registered a sensitivity of 0.5208, an accuracy of 0. 8250, an F1 score of 0. 2488, and a specificity of 0.8250.

The findings corroborate our initial hypothesis: manual segmentation does not manifestly enhance the performance in predicting patient mortality. This suggests that under situations where the risk assessment model's capacity is restricted, researchers' considerable efforts in refining algorithms to extract image-based factors may not necessarily yield significant returns. To truly enhance performance, the priority might need to shift. Instead of solely focusing on refining image-based factor extraction methods, greater attention should be dedicated to elevating the capability and performance of the risk assessment model itself. At the very least, there should be a simultaneous emphasis on both improving the quality of image-based factors and augmenting the model's overall efficiency. For instance, existing work has already highlighted the potential of deep learning in effectively measuring CVD risks directly from LDCTs [4]. As we move forward, we plan on integrating advanced models like deep learning for risk assessment. Our future investigations will delve into whether high-quality image-based factors can aid in enhancing the performance of advanced risk assessment models.



Acknowledgements (Optional): :

References (Optional): :
[1] W. H. Organization, “Cardiovascular diseases,” https://www.who.int/health-topics/cardiovascular-diseases#tab=tab 1, [Online; accessed March 10, 2023].

[2] C. W. Tsao, A. W. Aday, Z. I. Almarzooq, A. Alonso, A. Z. Beaton, M. S. Bittencourt, A. K. Boehme, A. E. Buxton, A. P. Carson, Y. Commodore-Mensah et al., “Heart disease and stroke statistics–2022 update: A report from the american heart

association,” Circulation, vol. 145, no. 8, 2022.

[3] N. L. S. T. R. Team, “Reduced lung-cancer mortality with low-dose computed tomographic screening,” New England Journal of Medicine, vol. 365, no. 5, pp. 395–409, 2011.

[4] H. Chao, H. Shan, F. Homayounieh, R. Singh, R. Doda Khera, H. Guo, et al., “Deep learning predicts cardiovascular disease risks from lung cancer screening low dose computed tomography,” Nature Communications, vol. 12, Art. no. 2963, 2021.