(K-432) Affective Biometrics of Deep Brain Stimulation Target Engagement

Introduction:: Deep brain stimulation (DBS) is emerging as a promising therapy for a variety of neuropsychiatric illnesses.
Changes to emotional expression in clinical contexts, known as affect, are poorly characterized alongside DBS, but may convey important information for accurate DBS implantation and therapeutic management - together called target engagement.
In this study, we demonstrate the utility of emotion expression estimates (EEE) in assessing target engagement during DBS.
We first demonstrate the use of validated emotion expression models in publically available DBS patient videos.
Classification of active stimulation, stimulation ON or stimulation OFF, is done through the emotion expression estimates (EEE) generated from the models.
We then identify the most informative emotions for the classification.
To better understand how DBS modulates emotional expressions, we perform dimensionality reduction and identify the major patterns of emotions.
We use both linear (PCA) and nonlinear (tSNE and UMAP) dimensionality reduction, comparing it to a trajectory-aware dimensionality reduction (tPHATE).
This work presents a proof-of-principle for using EEEs to characterize DBS target engagement, and enables future work to characterize and leverage emotion expression changes for adaptive DBS.

Materials and Methods:: Datasets: We collate patient DBS videos publicly available on YouTube to analyse. Distinct time periods of stimulation OFF and stimulation ON are determined through the clinician.

Emotion Expression Models - We use a previously validated facial emotion expression deep learning model trained on multicultural facial mimicry. These models transform from the pixel-space of videos into emotion expression spaces within ~40 dimensions of emotion expression. Models are applied to segments of video under an assumption of stationarity.

Timeseries Analysis:
Timeseries were analysed for each emotion expression dimension separately. Timeseries are analysed with simple statistics, including mean, median, max, and min. These statistics are shown to be insufficient in capturing known differences in emotion expression trajectories. We then perform a frequency-domain analysis as a surrogate for autocorrelation analyses, demonstrating inferential equivalence with computational speed ups.

Dimensionality Reduction:

Linear - Principal Components Analysis (PCA), implemented by scikit-learn Python library, was used to identify the largest feature correlations within each stimulation condition. A secondary analysis was performed with PCA across both stimulation conditions, within a single video.

Nonlinear -
Nonlinear dimensionality reduction was implemented via scikit-learn Python library. Two approaches were compared: tSNE and UMAP. In both approaches, local and global distances are relatively preserved as data is moved from the ~40 dimensional space into a flat 2D, 3D, and 4D map. In each case, the separability of DBS stimulation condition was assessed. A time-aware  approach, T-PHATE, is also demonstrated as a more reliable way to separate distinct stimulation conditions from full-dimensional EEEs.

Results, Conclusions, and Discussions:: Results and Discussion:
1) We demonstrate that DBS videos can be processed with EEE models. We show the distribution of emotions for each stimulation condition, across all ~40 emotion expression dimensions.
2) We demonstrate the use of PCA to identify the linear dimensions of largest variance, both within and across DBS conditions. We see that (a) the components differ in the two analyses, and (b) the effects of DBS initiation are evident largely in positively-valenced emotions.
3) We showed that nonlinear dimensionality reductions are capable of clustering (a) similar emotion expressions, and (b) similar DBS conditions, solely from the model EEE output.
4) We introduce the use of T-PHATE to provide more robust classification of the DBS target engagement along the emotion expression trajectory. We see that T-PHATE is able to cleanly separate stimulation OFF to stimulation ON transitions using just the EEE timeseries. We then identify the emotions that are most conserved through the T-PHATE mapping to arrive at the final 2D map.

Conclusions:
EEE provide insights into the active stimulation state of DBS, while also providing an interpretable feature space linking facial expression to DBS status via emotion expression.

Acknowledgements (Optional): :

References (Optional): :