Deep learning based on co-registered ultrasound and photoacoustic imaging improves the assessment of rectal cancer treatment response

Introduction:: Preoperative chemoradiotherapy (CRT) followed by total mesorectal excision is the routine treatment for rectal cancer. For patients who have complete treatment response, the nonsurgical watch-and-wait approach potentially reduces surgical complications and improves patients’ quality of life by avoiding surgery, which often entails permanent colostomy, altogether. However, no imaging techniques have achieved satisfactory sensitivity to complete response, make it challenging to monitor treatment response noninvasively and identify complete responders ^[4,5]. This is because the tumor bed maintains an abnormal appearance on ultrasound and MRI due to fibrosis or edema formation even though no residual cancer remains. However, as vascular normalization occurs in treatment responders, we hypothesize that ultrasound combined with photoacoustic imaging can improve assessment of CRT response.

In our previous work, we developed a coregistered ultrasound and photoacoustic (US-PAM) probe and demonstrated that photoacoustic signals in cancer could be accurately distinguished from normal tissue. In this work, we have collected in vivo images from 21 patients and developed a deep learning model based on coregistered dual-modality images to differentiate cancer from normal tissue. The aim of this study is to more accurately identify complete responders with no residual cancer in their tumor bed with the addition of coregistered photoacoustic imaging.

Materials and Methods:: An in-house endorectal probe scanned 360^o coregistered USPA scans at 1 fps. A 1064 nm Nd:YAG laser with a pulse repetition rate of 1 kHz illuminated ~1.5 mm² to generate photoacoustic signals. A 20MHz focused ultrasound transducer with the focal length of 12.7 mm was used to detect both photoacoustic and ultrasound signals.

A deep neural network was developed to differentiate cancer from normal and normalized tissue, after identifying and excluding image artifacts. The model architecture is illustrated in Fig 1a. The model input was coregistered ultrasound and photoacoustic images with a set of reference normal USPA images from the same patient. The purpose of adding reference normal images was to compensate for interpatient variability and make the model more generalizable. In vivo data from 11 patients and ex vivo data from 4 patients were labeled to train the model. In vivo data from 3 different patients were used for validation to identify the stopping point during model training; another non-overlapping set of in vivo data from 3 more patients were used for testing the model performance. To facilitate surgeons’ decision making using our model, guided back-propagation was implemented to generate hotspots to indicate suspicious cancer regions based on the model predictions.


Results, Conclusions, and Discussions:: Results

Compared to the same model trained on only ultrasound images, the model based on coregistered US-PAM images and individualized reference normal images showed significantly superior performance when differentiating cancer from normal and normalized tissue. The accuracy improved from 82.9±1.3% to 92.4±0.6%, while the AUC increased from 0.917±0.01 to 0.968±0.004. The improvement was most notable when classifying images of tumor bed with no residual cancer. After adding a third category of image artifact, the final 3-class classification accuracy was 89.1±0.8%, compared to 75.1±1.6% from model trained on only ultrasound images.

Conclusions

Deep learning model based on coregistered USPA images and individualized prior information of normal tissue images achieves high accuracy at differentiating cancer from normal and normalized tissue after CRT. Our findings support the proposition that the addition of photoacoustic imaging to conventional ultrasound improves treatment response assessment.

Acknowledgements (Optional): :

References (Optional): :