(E-168) A novel algorithm for full-resolution registration of giga-pixel images with sub-cellular accuracy

Introduction:: A comprehensive understanding of the pathobiology of cancer development is essential for its early detection and effective treatment. While two-dimensional (2D) analysis of H&E-stained whole slide tissue sections is the gold standard in anatomic pathology, three-dimensional (3D) analysis is necessary for providing a more holistic understanding of the tumor microenvironment. By reconstructing the tumor and its surroundings in 3D, complex patterns and interactions between cells can be identified, which may be difficult to discern in 2D. In addition, 3D analysis can assess spatial relationships between different cellular and acellular components of the tumor microenvironment which can inform treatment decisions.



3D reconstruction of tumor microenvironment is typically done by registering down-sampled whole slide images of its 2D serial sections. Although downsampling provides good registration, the resulting low-resolution reconstruction results in information loss and a less accurate characterization of the 3D architecture of the tumor microenvironment. Ideally, we would like to reconstruct images at the full resolution they were acquired. However, such full-resolution registration of gigapixel images remains a challenge due to memory and computational constraints. To address this challenge, we have developed a new approach that utilizes a sparse representation of the whole slide tissue sample to align serial sections at full resolution and achieve their accurate sub-cellular registration without compromising speed and efficiency.


Materials and Methods:: We generated a sparse representation of the whole slide image (~ 6.5 billion pixels) by extracting its tissue contour¹ with only ~0.05 million pixels. This sparse representation requires only 0.0008% of the memory resources, allowing for faster computations, while still resulting in a high level of accuracy for registration. We utilized this representation to perform a global Euclidean registration followed by a local elastic registration². Successive application of this two-step process across consecutive serial sections allowed 3D registration of the complete stack of 2D serial sections.


Results, Conclusions, and Discussions:: Results:
Figure 1 illustrates an example of our full-resolution registration algorithm. The degree of registration was measured using the Normalized Cross Correlation (NCC) metric, which ranges between -1 and +1, where +1 represents a perfect correlation between the reference and moving images. Figure 1 (A) is a visualization of the degree of overlap between the reference and moving image, after using our algorithm for global registration but before local registration. The reference image is displayed in green pseudo-color, while the moving image is shown in red pseudo-color. The complete overlap between the two images is depicted as gray. The NCC score obtained was +0.293, indicating a poor correlation between the images. Figure 1 (B) is a visualization of the degree of overlap between the reference and moving image, after using our algorithm for global registration as well as local registration. An NCC score of +0.895 was observed, indicating good correlation and sub-cellular registration accuracy between the two images at full resolution without any downsampling. The zoomed-in regions of the images demonstrate the sub-cellular precision achieved by our registration method. This is illustrated by the accurate alignment of the nuclei in two distinct serial sections. This is the first time registration of high-resolution images has been achieved with such high accuracy.


Conclusions and Discussions:
We have developed a new registration algorithm that achieves full-resolution alignment between serial sections with sub-cellular accuracy. To the best of our knowledge, this is the first time that registration of gigapixel images has been achieved without any downsampling. The algorithm we have developed for image registration is not constrained by the full resolution of the images and can be applied to ultra-structural images, such as those produced by electron microscopy, in addition to histology images.


Acknowledgements (Optional): :

References (Optional): : 1. Lorensen, W. E., & Cline, H. E. (1998). Marching cubes: a high resolution 3D surface construction algorithm. Seminal graphics: pioneering efforts that shaped the field, 347-353.

2. Pérez, J. S., Meinhardt-Llopis, E., & Facciolo, G. (2013). TV-L1 optical flow estimation. Image Processing On Line, 2013, 137-150.