(E-172) Leveraging public datasets for automatic ex vivo hemisphere segmentation

Deep learning-based brain MRI segmentation has been extensively studied, including different scan sequences, different subject populations etc. However, the performance of existing methods drops significantly when presented with datasets that are structurally different. Our dataset contains ex vivo brain hemispheres from donors. In this case, the target data distribution is largely different from any training set that existing algorithm employs. To overcome this issue, we introduce a series of data augmentation methods to utilize public datasets for training a model that generalizes well on our ex vivo brain hemisphere dataset.

In this paper, we first introduce the public dataset we use and the augmentation methods we applied to better generalizability. We will then present the qualitative and quantitative results and discuss the future directions of this work.

One major challenge is that our dataset is in T2-weighted MRI while most public datasets are in T1. The CURIOUS[3] dataset is one of the largest T2-weighted MRI source. However, it was not designed for segmentation tasks, hence lacks segmentation labels. We use SynthSeg[1], a state of the art algorithm for vivo brain segmentation to provide the CURIOUS[3] dataset with the ground truth labels.

To mimic our ex vivo hemisphere data, we took several steps to augment the CURIOUS[3] dataset. First, we implemented Hemisphere Masking, transforming the whole brain data into a hemisphere, preserving 23 regions of interest, such as the Left Cerebral White Matter, Cortex, Thalamus, and others. Secondly, we applied the Random Convex Polygon method, simulating the ex vivo hemispheres in formalin by randomly selecting 100 points on the boundary to form a polygon, approximating our dataset's conditions. Thirdly, we used the bias field augmentation of the training dataset to account for the significant intensity-non-uniformity in our dataset. We generated random bias fields to augment our dataset during training, further aligning it with our data. Lastly, we utilized conventional data augmentation techniques such as rotation (0 to 20 degrees) and translation (0 to 20 millimeters) to increase the model's robustness, ensuring it can locate target regions and not just follow patterns. After training, the model is finetuned on one case from our dataset with a small learning rate. We trained a standard 3D UNet[2] with Dice loss function.

Fig. 2 illustrates the qualitative segmentation results in the axial view.  From left to right, 2.a is the original image, 2.b is the ground truth, 2.c is our method, and 2.d is SynthSeg[1]. The result in 2.e is predicted by the model before finetuning on our own dataset, and the prediction in 2.f is from a model trained without the proposed data augmentations.

In 2.f, we show that the prediction completely fails when trained on the CURIOUS dataset without the proposed augmentations. In 2.e, the results exhibited substantial improvement due to our augmentation, although it still slightly diverged from the ground truth, especially on small structures.
Consequently, our proposed augmentations have been proven to be effective. In 2.c, SynthSeg[1], on the other hand, shows visible flaws on the larger structures such as white matter and cortex. Similar conclusions have been drawn from Fig. 3.

Table 1 shows the quantitative segmentation performance of our method and SynthSeg[1]. The proposed method generally performs on par with SynthSeg[1], if not better, in larger structures. In smaller structures, such as the amygdala and caudate, we do see SynthSeg[1] benefiting from its more extensive training process. Overall, the performance of our method is comparable with SynthSeg. Although in some structures SynthSeg shows better quantitative performance, we do see from the visualizations that porposed method generally produces more reasonable predictions. This could be caused partially by the manual segmentation quality. With more labeled data for fine-tuning in the future, we can further improve the performance of our model. 

[1] Billot, Benjamin, et al. "SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining." Medical image analysis 86 (2023): 102789.

[2] Çiçek, Özgün, et al. "3D U-Net: learning dense volumetric segmentation from sparse annotation." Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19. Springer International Publishing, 2016.

[3] Xiao, Y., Fortin, M., Unsgård, G., Rivaz, H. and Reinertsen, I. (2017), REtroSpective Evaluation of Cerebral Tumors (RESECT): A clinical database of pre-operative MRI and intra-operative ultrasound in low-grade glioma surgeries. Med. Phys.. doi:10.1002/mp.12268 (primary)