(M-503) An On-Site Machine Learning Assessment Tool Using Heart Rate Variability (HRV) As a Novel Biomarker for Concussion Diagnosis

Introduction:
Every year 3.8 million athletes suffer from a concussion, with 19% of athletes participating in contact sports at risk of sustaining one per year of play (1). Concussions are traumatic brian injuries that occur from a blow to the head (2). Unfortunately, up to half of concussions will go undetected leaving athletes at risk for repeated concussions, resulting in Chronic Traumatic Encephalopathy (3,4). Studies have found that 92% of ex-NFL players display signs of CTE, resulting in progressive memory decline and cognition, depression, aggressiveness, and dementia (5).
The current standard of care for on-site diagnosis, SCAT5 (a test administered on the side-line for a player with a suspected concussion), has limitations in sensitivity, objectivity, and specificity as it relies on subjective tests and self-reporting of symptoms (3-9), which increases risk of Chronic Traumatic Encephalopathy (3,4). Studies identified varied test-retest reliability of SCAT5, ranging from 20% to 52% sensitivity and 21% to 91% specificity (7-8). Additionally, athletes may deliberately not disclose their symptoms to avoid removal from the sport (9).
Therefore, we developed a machine learning assessment tool that uses heart rate variability (HRV) as a biomarker for concussion diagnosis. This concept is based on the physiological link between concussions and autonomic nervous system (ANS) dysregulation, which can be detected by changes in HRV (12-16). Our tool uses commercially available heart rate bands to monitor an athlete's HRV during play and compares this measurement to their baseline to identify concussions on-site after a possible traumatic incident (Fig 1).

Materials and Methods:
Training & Developing Classifier: We trained our algorithm with publicly available Physionet datasets Healthy RR, CEBSDB, and CHARIS (N=601). Our classifier uses a pre-processing pipeline to adjust for differences in ECG data between various publicly available datasets which involve streamlining file types, locating/detecting RR intervals, and removing noise artifactions. We then chose several features to calculate from the R-R interval data: Range, IQR, Variance, Standard Deviation, and Coefficient of Variance (most salient features related HRV). Our classifier uses a logistic regression model with a 0.75-0.25 training and testing split of the publicly available data (N= 451 training, N= 150 testing).
Data Collection Method: In order to ensure our classifier is able to differentiate concussion versus similar ANS disturbances, we conducted testing of athletes under a variety of conditions. We performed both TBI and non-TBI perturbation tests to contribute to scientific validation, enhancing the classifier’s diagnostic capabilities and confidence in its effectiveness. We collected ECG data using Frontier X2 from a group of 15 collegiate athletes ages (20 ± 2) years. We performed baseline testing of subjects at rest without previous activity. ECG was recorded throughout. We then performed baseline testing of subjects while active. After collecting data from baseline testing, we conducted both non-TBI and TBI perturbation tests. Non-TBI perturbation tests included protocols to elicit ANS responses.The data we collected from these experiments (N=30 samples, as each test is 1 sample) was used to test our ML classifier algorithm to establish the feasibility.


Results, Conclusions, and Discussions:
Currently, Neurobeat’s logistic regression classifier utilizes 295 ECG recordings from TBI patients and 306 ECG recordings from healthy patients. We chose to use a Logistic Regression model because we only screened for 2 outputs. We used a max_iteration value of 500, an ‘lbfgs’ solver, and a classification:test ratio of (0.75: 0.25). We randomized our test-seed to include cross-validation in our logistic regression model.
The results demonstrated that our classification algorithm had an accuracy rate of 89.33%, and cross-validation results demonstrated an average accuracy rate of ~86% (Table 4, Fig 2). NeuroBeat’s model uses standard deviation to differentiate between concussion and healthy readings (Fig 3.1). This supports our hypothesis that HRV is different in healthy and concussed individuals, which can be measured through Standard Deviation, which was used to differentiate between the two groups (Fig 3.1).These results are promising and suggest that the relationship between HRV and concussions should be looked into further.
Our final objective was to evaluate the efficacy of our algorithm by utilizing our own collected data and compare our results to the standard of care (SCAT5). We trained our classifier on all 601 samples of the pre-existing data, and tested using the 30 test samples of data we collected from 15 individual athletes. Running the algorithm on our collected data set had a 83.33% accuracy rate and a false positive rate of 17.2%.The confidence threshold was set at 50%. The beta coefficient results (N=30) indicates that Standard Deviation is most implicated in determining whether the sample of HRV is concussed or healthy (Fig 4), as we expected.
These results suggest the potential sensitivity of the underlying physiological mechanism of HRV, which further IRB-approved studies can confirm. We hope to get an IRB study approval in the future to collect more data, especially from concussed individuals. We can then improve our algorithm and discover the best thresholds to set to maximize true positive and minimize the false negative rates.

Acknowledgements (Optional): Our team worked under the guidance of Dr. Ross Venook, Senior Lecturer in Bioengineering at Stanford University. We were also funded by the Biodesign NEXT grant, awarded through the Stanford Byers Center for Biodesign.

References (Optional):
Centers for Disease Control and Prevention. (n.d.). Press release. Centers for Disease Control and Prevention. Retrieved February 10, 2023, from https://www.cdc.gov/media/pressrel/2007/r070607.htm
Centers for Disease Control and Prevention. (2019, February 12). What is a concussion? Centers for Disease Control and Prevention. Retrieved February 10, 2023, from https://www.cdc.gov/headsup/basics/concussion_whatis.html
Galgano, M. A., Cantu, R., & Chin, L. S. (2016). Chronic traumatic encephalopathy: The impact on athletes. Cureus. https://doi.org/10.7759/cureus.532
Mez, J., Daneshvar, D. H., Kiernan, P. T., Abdolmohammadi, B., Alvarez, V. E., Huber, B. R., Alosco, M. L., Solomon, T. M., Nowinski, C. J., McHale, L., Cormier, K. A., Kubilus, C. A., Martin, B. M., Murphy, L., Baugh, C. M., Montenigro, P. H., Chaisson, C. E., Tripodis, Y., Kowall, N. W., … McKee, A. C. (2017). Clinicopathological evaluation of chronic traumatic encephalopathy in players of American football. JAMA, 318(4), 360. https://doi.org/10.1001/jama.2017.8334
Researchers find CTE in 345 of 376 former NFL players studied. Chobanian Avedisian School of Medicine Researchers Find CTE in 345 of 376 Former NFL Players Studied Comments. (1969, December 28). Retrieved February 10, 2023, from https://www.bumc.bu.edu/busm/2023/02/06/researchers-find-cte-in-345-of-376-former-nfl-players-studied/
Petit, K. M., Savage, J. L., Bretzin, A. C., Anderson, M., & Covassin, T. (2020, August 1). The Sport Concussion Assessment Tool-5 (SCAT5): Baseline assessments in NCAA Division I collegiate student-athletes. International journal of exercise science. Retrieved February 10, 2023, from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7449330/
Hänninen, T., Parkkari, J., Howell, D. R., Palola, V., Seppänen, A., Tuominen, M., Iverson, G. L., & Luoto, T. M. (2021). Reliability of the sport concussion assessment tool 5 baseline testing: A 2-week test–retest study. Journal of Science and Medicine in Sport, 24(2), 129–134. https://doi.org/10.1016/j.jsams.2020.07.014
Harmon, K. G., Whelan, B. M., Aukerman, D. F., Bohr, A. D., Nerrie, J. M., Elkinton, H. A., Holliday, M., Poddar, S. K., Chrisman, S. P. D., & McQueen, M. B. (2022, February 1). Diagnostic accuracy and reliability of sideline concussion evaluation: A prospective, case-controlled study in college athletes comparing newer tools and established tests. British Journal of Sports Medicine. Retrieved February 10, 2023, from https://bjsm.bmj.com/content/56/3/144.long