Deep Image Prior for Undersampling High-speed Photoacoustic Microscopy

Introduction:: Photoacoustic microscopy (PAM) is an emerging imaging method combining light and sound. However, limited by the laser’s repetition rate, state-of-the-art high-speed PAM technology often sacrifices spatial sampling density (i.e., undersampling) for increased imaging speed over a large field-of-view. Deep learning (DL) methods have recently been used to improve sparsely sampled PAM images; however, these methods often require time-consuming pre-training and large training dataset with ground truth. Here, we propose the use of deep image prior (DIP) to improve the image quality of undersampled PAM images. Unlike other DL approaches, DIP requires neither pre-training nor fully-sampled ground truth, enabling its flexible and fast implementation on various imaging targets. Our results have demonstrated substantial improvement in PAM images with as few as 1.4% of the fully sampled pixels on high-speed PAM. Our approach outperforms interpolation, is competitive with pre-trained supervised DL method, and is readily translated to other high-speed, undersampling imaging modalities.


Materials and Methods:: Deep image prior: We employed a non-pretrained DIP model to improve undersampled PAM images and increase the imaging speed of fast PAM systems [1]. DIP model iteratively seeks an optimized, fully sampled image x_k that approximates the undersampled image x₀, given a known downsampling mask m (Fig. 1).

CNN for the deep prior employs U-Net with some modifications [2]. U-Net is a popular network architecture, especially for medical imaging applications. Its encoder-decoder framework effectively extracts features in the compression path and reconstructs the target from those features in the decompression path.

Data Acquisition and Undersampling Mask Construction: Two PAM systems were used in this study. The first was the traditional slow-scanning PAM system using motorized stages [3, 4] with a B-scan rate of 1 Hz, providing fully-sampled images as the testing dataset. The second system was a fast-scanning PAM system using a polygon scanner, with a B-scan rate of 1 kHz. Limited by laser’s repetition rate, the fast-scanning PAM provided only undersampled images.

We artificially generated the undersampled images from the fully sampled imaged. We denoted [Sx,Sy] as the undersampling ratio in the x and y axis to construct binary undersampling masks of “0” and “1”. The effective pixels are defined as the pixels in the undersampled image exacted from the fully sampled image. For example, the undersampling ratio [5,1] provides 20% effective pixels.


Results, Conclusions, and Discussions:: Results:

DIP on in vivo Vascular Data: Representative outputs from DIP and compared methods (interpolations and pre-trained FD U-Net mode) are shown in Fig. 2. The improvements by DIP are significant even for high undersampling ratios (< 20% effective pixels), as shown in Fig. 2. Particularly, at [7,3] and [10,5], blood vessels recovered by DIP remain relatively continuous and smooth - a key attribute of DIP, which does not learn the vascular continuity from any ground truth. In contrast, interpolation-based methods result in discontinuous and aliased vessels. From the vessel profiles, DIP accurately reconstruct the correct signal intensities along different directions. As expected, the pre-trained FD U-Net yields slightly better performance than DIP with sharper and more continuous vessels than the DIP results (Fig. 2(c)).

We further applied the DIP model to the mouse vasculature data acquired by fast-scanning PAM, with an extreme undersampling ratio of [6,12] (only 1.4% effective pixels). Similar to the results by slow-scanning PAM. DIP and FD U-Net show improved vessel continuity and edge smoothness. The pre-trained FD U-Net, however, has checkerboard artifacts due to either residual scanning effects or transposed convolution (Fig. 3(c)). DIP is visually free of unidentified artifacts, and has the best vessel connectivity and least edge jags.

Discussion and Conclusion: We introduce a novel DL method DIP to reconstruct undersampled PAM images for increased imaging speed. DIP does not require pre-training on a large and diverse dataset with paired ground truth. Our experimental results demonstrate the feasibility of applying DIP on improving PAM images of in vivo vasculature, with as few as 1.4% effective pixels. Our method may provide a practical solution for high-speed imaging applications with severe undersampling.


Acknowledgements (Optional): :

References (Optional): : [1] D. Ulyanov, A. Vedaldi, and V. Lempitsky, "Deep image prior," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 9446-9454.
[2] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in International Conference on Medical image computing and computer-assisted intervention, 2015, pp. 234-241: Springer.
[3] W. Liu et al., "Quad-mode functional and molecular photoacoustic microscopy," Scientific reports, vol. 8, no. 1, pp. 1-10, 2018.
[4] M. Chen et al., "Simultaneous photoacoustic imaging of intravascular and tissue oxygenation," Optics letters, vol. 44, no. 15, pp. 3773-3776, 2019.