Multiscale, Adversarial, Anatomically Constrained Digital Image Registration Algorithm for Cardiac Motion Estimation in Adult and Fetal Echo

Introduction:: Cardiac Motion estimation from echocardiography images, typically achieved via temporal image registration, is a good method to obtain physiological features of hearts to evaluate cardiac function, detect dysfunction and assessing disease severity. Motion estimation is a good way to obtain myocardial strains, with an accuracy on par with speckle tracking [1], and it can be used to accurately estimate stoke volume and ejection fraction. The extracted motion information are also important inputs for biomechanics simulations of cardiac function to understand intracardiac flow and myocardial stresses [2]. To date, however, clinical echocardiographic measurements have problems with precision, especially for fetal echocardiography, due to a range of issues ranging from subjective manual inputs, inter-observer variability, to the difference between ultrasound machines and processing algorithm from various vendors. For example, measurements of fetal left ventricular strains can vary from group-to-group by more than one-fold [3]. Deep learning image registration is a good way to resolve imprecision, by removing manual processes and standardizing processing algorithm. Our objective is thus to develop a novel deep learning image registration (DLIR) algorithm to address precision issues, and also to achieve improved DLIR performance by adopting strategies that can improve on the current vanilla DLIR.


Materials and Methods:: We propose improvements to the vanilla deep learning image registration (DLIR) algorithm for extracting the myocardial motion from both fetal and adult echocardiography to achieve a higher accuracy, by adding a combination of several strategies (Figure 1). Our proposal is to add (1) a multi-scale training scheme, (2) an adversarial scheme to regularize the loss function, and (3) an anatomic shape encoder to preserve physiological myocardial anatomical topology. The algorithm was trained to evaluate adult echo using the publicly available CAMUS dataset of adult echo of 500 patients (with manually obtained left ventricular myocardium segmentation truths), and to evaluate fetal echo using 140 2D echo videos extracted from 4D fetal echo images of 15 patients (with similar segmentation truths). Image registration results were evaluated by warping the moving image according to the registration output for comparison with the fixed image. Performance is valuated using (1) mean square error comparing the fixed image and warped moving image, (2) the Hausdorff distance between the myocardial segmentation in the fixed image and that in the warped moving image, and (3) the Dice coefficient comparing the same segmentations.


Results, Conclusions, and Discussions:: Results are demonstrated in Table 1. For the adult echo dataset in the 2-chamber view, by adding shape-encoder to the vanilla DLIR, DICE coefficient improved from 0.8616 to 0.8868; by adding adversarial network to vanilla DLIR, DICE, DICE improved from 0.8616 to 0.8845. Adding both the shape-encoder and adversarial network to vanilla DLIR, however, improved DICE coefficient to 0.9098, suggesting that the shape-encoder and adversarial network brought non-overlapping benefits. Analysis of results showed that the shape-encoder prevented registrations that resulted in irregularly shaped myocardium and ensured greater uniformity in myocardial thickness, while the adversarial network enabled the warped moving image to have image features that were close to the fixed image. Finally, adding multiscale training to the shape-encoder and adversarial DLIR led to even further improvements with DICE coefficient of 0.9186. Interestingly, this final result was more accurate than using the gold standard non-machine learning Elastix algorithm for DLIR, suggesting that deep learning can enhance accuracy besides processing time. Similar results were obtained for the adult echo 4-chamber view dataset, and for the fetal echo dataset.

In conclusion, we proposed 3 strategies to improve the vanilla DLIR, our results suggested the addition of each of our three proposed strategies provided a different advantage to registration and quantifiable improvements, and the combination of all three strategies gave synergistic improvements, which could go beyond the non-ML gold standard approach.


Acknowledgements (Optional): : EPSRC doctoral training programme PhD scholarship (Hasan).

References (Optional): : [1] Heyde B, Jasaityte R, Barbosa D, Robesyn V, Bouchez S, Wouters P, Maes F, Claus P, D'hooge J. Elastic image registration versus speckle tracking for 2-D myocardial motion estimation: a direct comparison in vivo. IEEE transactions on medical imaging. 2012 Nov 27;32(2):449-59.

[2] Wong HS, Li B, Tulzer A, Tulzer G, Yap CH. Fluid Mechanical Effects of Fetal Aortic Valvuloplasty for Cases of Critical Aortic Stenosis with Evolving Hypoplastic Left Heart Syndrome. Annals of Biomedical Engineering. 2023 Feb 13:1-4.

[3] van Oostrum NH, de Vet CM, van der Woude DA, Kemps HM, Oei SG, van Laar JO. Fetal strain and strain rate during pregnancy measured with speckle tracking echocardiography: A systematic review. European Journal of Obstetrics & Gynecology and Reproductive Biology. 2020 Jul 1;250:178-87.