(E-191) Inception-Transmorph: multi-contrast brain tumor segmentation in MRI images

Introduction:: Brain tumor localization and segmentation from magnetic resonance imaging (MRI) are essential for diagnosing and prognosis of patients with gliomas as each brain imaging modality gives unique and key details related to different parts of the tumor. In the past decade, Unet and its variant have achieved state-of-the-art performance in challenging medical image segmentation tasks. Recently, vision transformer models have shown prominence in computer vision. Specifically, the self-attention module in ViT-based models allows for modeling long-range information by pairwise interaction between token embeddings and hence leading to more effective local and global contextual representations. Leveraging the advantage of the UNet and Vision transformer, Transmorph shows good potential in medical image segmentation tasks. Also, due to the sparsity of the tumor tissue, the algorithm could be biased toward achieving higher specificity than sensitivity. The selection of the loss function could also have a significant impact on the model predictions. Although region-based or distribution-based loss functions have been widely utilized in previous studies, localized loss functions have not been thoroughly investigated. As a result, we proposed a novel deep-learning architecture based on Unet and Vision transformer, which can fuse different modalities of MRI images: T1, T1 with contrast agent (T1c), T2, and Fluid-attenuated Inversion Recovery (FLAIR). In addition, we explore the potential benefits of localized loss functions through the use of a compound loss function, which aims to improve segmentation accuracy. Finally, we compare the performance of Transmorph models to Unet-based models to determine the effectiveness of our proposed approach.

Materials and Methods:: Multi-parametric MRI scans from 1251 patients were used for BraTS2022, all these patients were provided with segmentation labels. A randomly selected subset consists 100 subjects were used in this study for training efficiency purposes. A cascaded network[1] was developed by using two blocks derived from the Inception architecture to combine the four imaging modalities, and the output of the Inception module was then fed into the TransMorph model which is responsible for the image segmentation task (Figure 1). Two novel loss functions were introduced in this study. The edge loss function, based on canny edge detection, was utilized to penalize the distance between the ground truth segmentation masks and predicted segmentation masks using a standard mean square error loss. Additionally, a mask loss was developed to regulate the relative location of the tumor core, enhancing the tumor and the whole tumor. Our network was implemented using PyTorch 1.2. The network weights were updated using the Adam optimizer, with a batch size of 1 and a decaying learning rate with an initial value of α=1e-5. The hyperparameters of the model are optimized using Optuna. In order to select the most optimal neural network architecture, We also evaluated four different model architectures for the semantic segmentation task: Vanilla-Unet, Optimzied UNet[2], Attention U-Net, and Inception U-Net. Then compared the results to the Inception-Transmorph model.


Results, Conclusions, and Discussions:: We conducted an evaluation of our proposed method and compared it to five state-of-the-art models, using two sets of loss functions. In the first testing set, we used only the mean square error (MSE) loss and cross-entropy loss, which have been widely used in previous research. In the second experiment, we also implemented a compound loss including edge loss, and mask loss, together with the loss functions from the first testing. We used the Dice score and the 95th percentile Hausdorff distance score to evaluate the model performance (Table 1), with the highest scores highlighted in bold. Notably, the Inception-Transmorph model achieved the second-best performance in the Dice score on the validation set, reaching 79.86%, and the best Hausdorff distance score which is 4.01.


The experimental results suggest that the optimized UNet model outperforms the other models in terms of overall performance. However, the Inception-Transmorph model shows promise due to its comparable performance to the optimized UNet and better localization capabilities. Moreover, the compound loss function shows better performance compared to other combinations of loss functions during training. This novel loss function combines pixel-level and lesion-level loss to enhance segmentation results at both levels. Researchers interested in employing the compound loss function should consider using edge loss and mask loss in tandem. Although edge loss helps address intensity dependence and ensures correct boundary location, it may lead to incorrect tumor core identification for pixels outside the whole tumor region where edges are detected. To mitigate this issue, mask loss can regulate the model's predictions regarding the inclusion relation between three labels at an early stage of training.
In this paper, we introduced Inception-Transmorph which is a novel transformer-based architecture for multi-modal MRI segmentation of brain tumors. We show that our segmentation architecture together with localized loss function is robust, scalable, and more accurate than existing voxel-based segmentation methods and could be the foundation of a new class for brain tumor segmentation.





Acknowledgements (Optional): : There is no funding source to declare.

References (Optional): : [1] Abderezaei, J., Pionteck, A., Chopra, A., & Kurt, M. (2022). 3D Inception-Based TransMorph: Pre-and Post-operative Multi-contrast MRI Registration in Brain Tumors. arXiv preprint arXiv:2212.04579.
[2] Futrega, Michał, et al. "Optimized U-Net for brain tumor segmentation." Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part II. Cham: Springer International Publishing, 2022.