Margaret Butler Fellow Argonne National laboratory Westmont, Illinois, United States
Introduction::
Large language models (LLMs) have captured attention of researchers across different scientific fields. LLM-based automated conversational assistants offer an opportunity to create interactive dialogue, with widespread applications in e-commerce, home automation and finance. However, sensitive data access issues, model retraining, long compute time and lack of real-time results have limited the direct use of LLMs in healthcare applications. Healthcare domain is ripe to take advantage of the near-human efficiency and accuracy of LLMs due to ever increasing gap between healthcare personnel and population that requires healthcare assistance. In this work, we introduce a lightweight approach, priming LLMs, to develop an automated health assistant that relies upon fundamental theories of behavior science and unstructured clinical notes. We test our model under two different health scenarios, first for providing fitness coaching to users and second to provide clinical assistance to discharged patients.
Materials and Methods::
We rely on the Fogg’s behavioral model, that triangulates the user state of a patient with respect to motivation, ability, and propensity of a patient, to prime the large language model to function as a personalized and automated fitness coach. For our second application of providing clinical assistance during patient discharge, we use the de-identified MIMIC IV unstructured clinical note dataset that consists of 331,794 patient discharge summaries and classify patients based on the reason for admission to the hospital. We evaluate our proposed approach for these two applications by simulating user-assistant conversations under various scenarios of fitness coaching and clinical discharge conditions. To this end, we use GPT-3 as the LLM and compare our approach of primed LLM to the unprimed LLM. Subsequently, we conduct a quantitative reviewer evaluation with domain experts to evaluate our model efficacy.
Results, Conclusions, and Discussions:: Results: Our proposed approach allows the LLM to be personalized to a particular user state and disease condition of a patient and is therefore capable of providing a first line of support to patients. In the case of health coaching, we found the ratings for the primed conversations to be significantly higher in terms of domain knowledge and patient empathy with a significant uplift in actionability. We also found that sentence length and conversation length were higher in primed LLMs compared to naive context aware LLMs. Subsequently, we conducted a quantitative reviewer evaluation and report that the primed architectures were overall more appropriate and demonstrated higher empathy. In a similar set of experiments, we will explore the feasibility of using unstructured clinical notes to prime LLMs and enable them to function as a personalized and automated health assistant to help interact with patients during discharge.
Discussion and Conclusions: Automated and personalized health coach assistants have the potential to reduce the cost of fitness and need for trained coaches who are required to cater the ever-increasing population suffering from non-communicable diseases and the rampant sedentary lifestyles. Given the rise of interest in personal health monitoring systems and increasing disparity with respect to the number of trained healthcare personnel versus the number of people who require medical assistance, responsible, personalized, and automated deep learning-based methods such as LLMs offer an attractive solution. Without any retraining, this work provides the first necessary step as a proof-of-concept study of how lightweight approaches such as priming can be used to develop personalized health assistants. In particular, we demonstrate how behavior science principles and unstructured medical data can be used to encode user-specific information in LLMs and enable them to function as an automated healthcare assistants for a more principled user experience without the need for any additional model retraining.
