A Fully Automated Multimodal MRI-Based Multi-Task Learning for Glioma Segmentation and IDH Genotyping

Release time:2025-03-05

DOI number:10.1109/TMI.2022.3142321

Journal:IEEE TRANSACTIONS ON MEDICAL IMAGING

Key Words:BRAIN-TUMOR SEGMENTATION，T2-FLAIR MISMATCH，GRADE GLIOMAS，NETWORK，MODEL，CLASSIFICATION，RADIOMICS，1P/19Q

Abstract:The accurate prediction of isocitrate dehydrogenase (IDH) mutation and glioma segmentation are important tasks for computer-aided diagnosis using preoperative multimodal magnetic resonance imaging (MRI). The two tasks are ongoing challenges due to the significant inter-tumor and intra-tumor heterogeneity. The existing methods to address them are mostly based on single-task approaches without considering the correlation between the two tasks. In addition, the acquisition of IDH genetic labels is expensive and costly, resulting in a limited number of IDH mutation data for modeling. To comprehensively address these problems, we propose a fully automated multimodal MRI-based multi-task learning framework for simultaneous glioma segmentation and IDH genotyping. Specifically, the task correlation and heterogeneity are tackled with a hybrid CNN-Transformer encoder that consists of a convolutional neural network and a transformer to extract the shared spatial and global information learned from a decoder for glioma segmentation and a multi-scale classifier for IDH genotyping. Then, a multi-task learning loss is designed to balance the two tasks by combining the segmentation and classification loss functions with uncertain weights. Finally, an uncertainty-aware pseudo-label selection is proposed to generate IDH pseudo-labels from larger unlabeled data for improving the accuracy of IDH genotyping by using semi-supervised learning. We evaluate our method on a multi-institutional public dataset. Experimental results show that our proposed multi-task network achieves promising performance and outperforms the single-task learning counterparts and other existing state-of-the-art methods. With the introduction of unlabeled data, the semi-supervised multi-task learning framework further improves the performance of glioma segmentation and IDH genotyping. The source codes of our framework are publicly available at https://github.com/miacsu/MTTU-Net.git.

Note:Cheng, JH (Cheng, Jianhong) [1] , [2] ; Liu, J (Liu, Jin) [1] ; Kuang, HL (Kuang, Hulin) [1] ; Wang, JX (Wang, Jianxin) [1]

Indexed by:Journal paper

Volume:41

Issue:6

Page Number:1520-1532

Translation or Not:no

Date of Publication:2022-06-01

Included Journals:SCI