## A dual-task mutual learning framework for predicting post-thrombectomy cerebral hemorrhage

Caiwen Jiang 1 2 , , 4 2 6

Tianyu Wang , Xiaodan Xing , Mianxin Liu , Guang GLYPH&lt;12&gt; GLYPH&lt;12&gt; )

Yang 2 7 8 9 , , , , Zhongxiang Ding 4( ) , and Dinggang Shen 1 3 5( , ,

- 1 School of Biomedical Engineering &amp; State Key Laboratory of Advanced Medical Materials and Devices, ShanghaiTech University, Shanghai, China {jiangcw,dgshen}@shanghaitech.edu.cn
- 2 Bioengineering Department and Imperial-X, Imperial College London, London, UK 3 Shanghai United Imaging Intelligence Co., Ltd., Shanghai, China
- 5 Shanghai Clinical Research and Trial Center, Shanghai, China
- 4 Department of Radiology, Affiliated Hangzhou First People's Hospital, Westlake University School of Medicine, Hangzhou, China
- 6 Shanghai Artificial Intelligence Laboratory, Shanghai, China
- 8 Cardiovascular Research Centre, Royal Brompton Hospital, London, UK
- 7 National Heart and Lung Institute, Imperial College London, London, UK
- 9 School of Biomedical Engineering &amp; Imaging Sciences, King's College London, London, UK

Abstract. Ischemic stroke is a severe condition caused by the blockage of brain blood vessels, and can lead to the death of brain tissue due to oxygen deprivation. Thrombectomy has become a common treatment choice for ischemic stroke due to its immediate effectiveness. But, it carries the risk of postoperative cerebral hemorrhage. Clinically, multiple CT scans within 0-72 hours post-surgery are used to monitor for hemorrhage. However, this approach exposes radiation dose to patients, and may delay the detection of cerebral hemorrhage. To address this dilemma, we propose a novel prediction framework for measuring postoperative cerebral hemorrhage using only the patient's initial CT scan. Specifically, we introduce a dual-task mutual learning framework to takes the initial CT scan as input and simultaneously estimates both the follow-up CT scan and prognostic label to predict the occurrence of postoperative cerebral hemorrhage. Our proposed framework incorporates two attention mechanisms, i.e., self-attention and interactive attention. Specifically, the self-attention mechanism allows the model to focus more on highdensity areas in the image, which are critical for diagnosis (i.e., potential hemorrhage areas). The interactive attention mechanism further models the dependencies between the interrelated generation and classification tasks, enabling both tasks to perform better than the case when conducted individually. Validated on clinical data, our method can generate follow-up CT scans better than state-of-the-art methods, and achieves an accuracy of 86 37% . in predicting follow-up prognostic labels. Thus, our work thus contributes to the timely screening of post-thrombectomy

cerebral hemorrhage, and could significantly reform the clinical process of thrombectomy and other similar operations related to stroke.

Keywords: Postoperative cerebral hemorrhage · Prediction of hemorrhage progression · Dual-task mutual learning · Interactive attention.

## 1 Introduction

Ischemic stroke is a medical emergency caused by the blockage of blood vessels in the brain. Its timely treatment is crucial for reducing brain damage and other complications associated with stroke [23, 39]. Thrombectomy, favored for its quick effectiveness, has emerged as a common option for treating ischemic stroke [13]. However, the procedure may damage blood vessels and require perfusion of contrast agents for vascular visualization, which could introduce a risk of postoperative cerebral hemorrhage. In this context, the timely screening of cerebral hemorrhage after thrombectomy is an essential clinical task.

In the clinic, cerebral hemorrhage is monitored by two to three CT scans conducted within 0-72 hours post-surgery [15]. However, two to three CT scans cannot cover the entire period of cerebral hemorrhage (i.e., 0-72h post-surgery), often resulting in delayed detection, which can postpone the initiation of necessary treatment. Additionally, multiple CT scans within a short period also pose a significant radiation risk to patients. To address this dilemma, in this paper, we make the first attempt to predict the occurrence of hemorrhage within 0-72h post-surgery based only on the patient's initial CT scan.

There are already extensive studies on disease prediction [1, 17, 22, 47]. For example, Hu et al propose a framework that combines CNN and transformer for predicting the progression trends of mild cognitive impairment [17]. Alsekait et al integrate the support vector machine into deep learning models to predict the development of chronic kidney disease [1]. However, these studies, which directly predict future prognostic labels from images, often lack intermediate evidence, rendering the prediction less convincing. Consequently, some studies attempt to achieve prediction by generating future images [14, 16]. For instance, Han et al adopt the regularized generative adversarial networks to generate images of future time points for predicting the risk of osteoarthritis [16]. Such approaches can provide more information, thereby making the outcomes more persuasive. In fact, estimating future prognostic labels and images does not conflict to each other, as there exists an inherent connection between the two tasks. Thus, we believe that performing both tasks simultaneously could potentially achieve better results than conducting them separately.

To this end, we design a dual-task interactive learning framework to simultaneously predict the follow-up CT scan and prognostic label from the patient's initial CT scan for achieving postoperative cerebral hemorrhage prediction. Through dual-task interactive learning, we can capture dependencies between the interrelated generation and classification tasks, allowing both tasks to perform better than the case when performed separately. Our proposed framework employs a combination of self-attention and interactive attention mecha-

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00940v1-figure-67b734ab-14a8-4fac-ae66-929c22a053bc.jpg)

𝟒𝟒

×

𝟒𝟒

×

𝟒𝟒

×

𝟒𝟒

Fig. 1. Overview of our proposed dual-task interactive learning framework.

nisms. The self-attention mechanism enables the model to focus more on highdensity areas that are critical for diagnosis. Meanwhile, the interactive attention mechanism models dependencies between the interrelated generation and classification tasks, significantly reducing computational complexity while enhancing the performance of each task. Extensive experiments on clinical data show that our method can generate higher-quality follow-up CT scans and achieve more accurate prognostic label prediction than state-of-the-art methods.

The main contributions of our work include i) the first attempt to achieve early prediction of postoperative cerebral hemorrhage by estimating follow-up CT scans and prognostic labels from initial scans, and ii) the development of a novel dual-task interactive learning framework for this task. Extensive experiments also demonstrate the effectiveness of our method on collected datasets.

## 2 Method

Our proposed dual-task interactive learning framework is shown in Fig. 1. Given an initial CT scan, it is first processed by the patch partitioning block into a series of tokens that can be handled by subsequent transformer blocks. Then, these tokens alternately pass through transformer and patch merging blocks to extract features. The extracted features are subsequently input into two taskspecific branches. Correspondingly, in each branch, the extracted features alternately pass through three transformer and patch expanding blocks to improve resolution. The final features are then fed into the corresponding task heads to predict the results. Throughout this whole process, in addition to the use of selfattention mechanism, we apply the interactive attention mechanism to perform attention interactions at the corresponding feature levels. In the following, we will introduce the details of our method.

## 2.1 Spatial Alignment of Image Pairs

Due to patient posture and physiological movements, there is a significant spatial misalignment between initial and follow-up scans. Therefore, we need to perform

data preprocessing to spatially align the initial and follow-up scans. As shown in the left part of Fig. 2, for the input of two CT scans, to eliminate background interference, we first use the TotalSegmentator [44], an open-access tool based on nnU-Net and trained with more than one thousand samples, to segment the brain regions from both CT scans. Subsequently, we apply an affine registration method [3] to align the segmented brain regions. In this way, we can obtain spatially-aligned brain region image pairs for the latter model training.

## 2.2 Model Architecture

We propose a dual-task interactive learning framework consisting of five types of blocks, i.e., patch partitioning, transformer, patch merging, patch expanding, and task head. Among them, the patch partitioning block splits the input into multiple non-overlapping patches, with the features of each patch being the concatenation of the raw voxel values.

For the transformer blocks, we use the same window operation as Swintransformer [28], i.e., computing attention in the partitioned windows, instead of the whole images or feature maps. Specifically, each transformer block contains a regular window-based multi-head self-attention (W-MSA) module and a shifted window-based MSA (SW-MSA) module, followed by a 2-layer multilayer perceptron (MLP). Layer normalization (LN) is applied before each MSA module and MLP layer, and residual connections are applied after each module.

Patch merging and patch expanding blocks can be regarded as two opposite operations. The patch merging block merges adjacent tokens along the height and width dimensions in a non-overlapping manner to generate new tokens. In implementation, our merging scope is 2 × 2 ; therefore, after passing the patch merging layer, the height ( H ) and width ( W ) dimensions of the features are halved, and the C dimension is quadrupled. Then, a linear mapping is applied to halve the channel dimension of the concatenated tokens. Correspondingly, the patch expanding block first doubles the channel dimension of the input features through linear mapping, and then reshapes the features to double the height and width dimensions while reducing the channel dimension.

The task-specific heads are used to predict the corresponding task results from the features. The generation head and classification head are each composed of a single linear layer and a single softmax layer, respectively. We employ a weighted sum [11] to dynamically adjust the training weights of each task-specific loss according to their gradients. The task-specific loss is calculated between the ground truth and the final predictions for each task. In particular, we use both L 1 loss and adversarial loss for generation task, and a cross-entropy loss for classification task.

## 2.3 Self-Attention and Interactive Attention

Due to the higher density of blood compared to normal brain tissue, cerebral hemorrhage regions typically appear as high-density areas in CT images. To better extract features from CT images that may contain high-density areas, our

Fig. 2. Left: Data preprocessing workflow for obtaining the spatially-aligned initial and follow-up brain images. Right: Details of the two attention mechanisms (i.e., selfattention, and interactive attention) involved in the proposed framework.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00940v1-figure-c96156ba-daf0-457f-93b2-f26be5cbfb73.jpg)

proposed framework employs two attention mechanisms, including self-attention and interactive attention, as shown in the right part of Fig. 2.

Self-Attention: The self-attention is executed during feature extraction. By computing self-attention in the encoder, we enable the model to focus more on high-density areas in the image, which are critical for diagnosis. Specifically, the input x SA first passes through three linear layers to obtain query q SA and key k SA . Then, the attention value A SA is calculated as:

$$A _ { _ { S A } } = s o f t m a x ( \frac { q _ { _ { S A } } k _ { _ { S A } } } { \sqrt { C _ { _ { S A } } } } + B ),$$

where C SA is the number of channels and B is the position bias. Finally, the output of a particular self-attention head is A SA · v SA . In this way, we can apply adaptive weights to different areas of the image or feature map, allowing the model to focus more on high-density areas.

Interactive Attention: It is known existing strong correlation between followup CT scans and prognostic labels. For example, follow-up CT scans can be used to diagnose prognostic labels, and prognostic labels can roughly describe follow-up CT scans. Therefore, the task of generating follow-up CT scans and the task of predicting prognostic labels should also be interrelated. To capture task dependencies beyond shared encoder parameters, we design an interactive attention mechanism in decoders to further capture the relationship between these two tasks, reducing computational overhead while enhancing the performance of both tasks.

In our implementation, we set the generation task as the reference task. For a specific interactive attention calculation in the reference task decoder (i.e., the generation decoder), let x G denote the previous block output, and x SA denote the output of the corresponding transformer block in the encoder. As shown in the right part of Fig. 2, the generation decoder takes both x G and x SA as input. The standard method of computing self-attention is to obtain key, query, and value vectors only from its own previous output x G . In contrast, in the interactive attention calculation, we compute the query q SA and key k SA from x SA (from the encoder). Meanwhile, the value v G is still computed using the previous block output x G since the final output should be related to the generation task.

Table 1. Quantitative results of ablation analysis, in terms of PSNR, SSIM, ACC, and

AUC.

| Method        | Generation      | Generation      | Classification   | Classification   |
|---------------|-----------------|-----------------|------------------|------------------|
| Method        | PSNR [ dB ] ↑   | SSIM [%] ↑      | ACC [%] ↑        | AUC [%] ↑        |
| S-CNN         | 22.28(2.31)     | 86.45(2.37)     | 79.28(2.93)      | 82.43(3.12)      |
| D-CNN         | 25.53(1.26)     | 89.37(2.13)     | 82.36(2.85)      | 84.92(3.22)      |
| S-Transformer | 25.47(1.23)     | 90.42(1.18)     | 83.64(2.13)      | 86.54(2.64)      |
| D-Transformer | 26.92(1.18)     | 92.17(1.13)     | 85.23(1.95)      | 89.75(2.05)      |
| Ours          | 28 . 57(1 . 02) | 92 . 48(1 . 12) | 86 . 37(1 . 84)  | 92 . 32(2 . 14)  |

For the classification decoder, we adopt the same scheme as above, but we only calculate A SA in the decoder of reference task (i.e., generation task), and then feed it to the classification decoder directly. Note that we can also take the classification task as the reference task. However, we find that, taking the generation task as the reference task can lead to better outcomes through experiments. We apply the same procedure to all transformer blocks in the generation and classification decoders.

## 3 Experiments

## 3.1 Dataset and Implementation

We collect 200 samples for our dataset, and each sample contains the initial CT scan, the final follow-up CT scan, and the follow-up prognostic label (i.e., hemorrhagic transformation and non-hemorrhagic transformation). The followup prognostic label is determined by doctors based on the final follow-up CT scan. Of these 200 scans, 160 are used for training and 40 for testing. During the evaluation, we conduct five-fold cross-validation to exclude randomness.

In our implementation, experiments are conducted on the PyTorch platform using two NVIDIA Tesla A100 GPUs and an Adam optimizer with an initial learning rate of 0 001 . . All images are resampled to a voxel spacing of 1 × × 1 1 mm 3 with the size of 256 × 256 × 128 , and their intensity is normalized within [ 0 1 , ] by min-max normalization. To augment the training samples and reduce the usage of GPU memory, the original image is randomly cropped to the size of 96 × 96 × 96 as input. To quantify our results, we use Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) [21] to evaluate the generation task, and ACCuracy (ACC) and Area Under the Curve (AUC) to evaluate the classification task.

## 3.2 Ablation Analysis

To evaluate the effectiveness of each network component in our dual-task mutual learning framework, we designed another four variants, including: 1) S-CNN, consisting of a CNN encoder and a CNN decoder; 2) D-CNN, consisting of a CNN encoder and two CNN decoders; 3) S-Transformer, consisting of a transformer encoder and a transformer decoder; 4) D-Transformer, consisting of a transformer

Table 2. Quantitative comparison of our method with several state-of-the-art generation and classification methods, in terms of PSNR, SSIM, ACC, and AUC, where ∗ denotes CNN-based method and † denotes Transformer-based method.

| Generation      | Generation                             | Generation                             | Classification                         | Classification                  | Classification                  |
|-----------------|----------------------------------------|----------------------------------------|----------------------------------------|---------------------------------|---------------------------------|
| Method          | PSNR [ dB ] ↑                          | SSIM [%] ↑                             | Method                                 | ACC [%] ↑                       | AUC [%] ↑                       |
| cGAN ∗ [18]     | 23.32(1.75)                            | 85.44(1.86)                            | VGG ∗ [32]                             | 79.46(3.75)                     | 82.12(2.97)                     |
| SAGAN ∗ [25]    | 24.84(1.43)                            | 88.34(1.54)                            | ResNet ∗ [50]                          | 81.46(4.12)                     | 85.76(3.46)                     |
| TransUNet † [8] | 26.12(1.22)                            | 89.45(1.26)                            | Trans-RNN † [4]                        | 83.02(2.17)                     | 87.96(2.56)                     |
| ResViT † [12]   | 27.34(1.23)                            | 89.73(1.14)                            | Res-Trans † [42]                       | 84 . 66(2 . 43)                 | 89 . 42(3 . 12)                 |
| Ours †          | 28 . 57(1 . 02) 92 . 48(1 . 12) Ours † | 28 . 57(1 . 02) 92 . 48(1 . 12) Ours † | 28 . 57(1 . 02) 92 . 48(1 . 12) Ours † | 86 . 37(1 . 84) 92 . 32(2 . 14) | 86 . 37(1 . 84) 92 . 32(2 . 14) |

encoder and two transformer decoders. D-Transformer has the same architecture as our method, but without adopting the interactive attention mechanism in decoders. In addition, for S-CNN and S-Transformer, we need to use two separate models to perform the generation and classification tasks, respectively.

The quantitative results are provided in Table 1, from which, we can find the following observations. (1) D-CNN/D-Transformer achieves better performance than S-CNN/S-Transformer. This proves that a dual-task learning framework is more appropriate than a single-task framework for our interrelated generation and classification tasks. (2) The transformer-based S-Transformer and DTransformer, respectively, achieve better results than the CNN-based S-CNN and D-CNN. This may be because the transformers can capture global information by focusing on high-density areas crucial for diagnosis, thereby benefiting both tasks. (3) Our method achieves better results on both generation and classification tasks than D-Transformer and other variants. This demonstrates that the interactive attention mechanism can strengthen the connection between generation and classification tasks, thus resulting in better performance. These three comparisons conjointly verify the effective design of our proposed framework, where the dual-task learning strategy , transformer-based architecture , and interactive attention mechanism all can benefit our tasks.

## 3.3 Comparison with State-of-the-art Methods

Furthermore, we compare our method with several state-of-the-art generation and classification methods. The generation methods include cGAN [18], SAGAN [25], TransUNet [8], and ResViT [12]. The classification methods include VGG [32], ResNet [50], Transformer-RNN (Trans-RNN) [4], and ResNet-Transformer (ResTrans) [42]. The quantitative results and visualizations of the generated outcomes are provided in Table 2 and Fig. 3, respectively.

Quantitative Comparison: Quantitative results are provided in Table 2. It can be observed that, overall, transformer-based methods outperform CNN-based methods on both generation and classification tasks. This may be attributed to the transformer structure's superior ability to extract and focus on high-density areas crucial for diagnosis. This validates our selection of the transformer-based architecture. Further, among all the transformer-based methods, our method

Fig. 3. Visual comparison of follow-up scans produced by five different methods. From left to right are the input (initial scan), results by five other comparison methods (2nd-5th columns) and our method (6th column), and the ground truth (GT, i.e., the follow-up scan). The corresponding difference maps between the generated results and GT are shown in the 2nd and 4th rows, where darker color indicates larger differences. Red dotted boxes show the areas for detailed comparison.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00940v1-figure-89287345-a153-44ae-bd4e-81f92b8db6ba.jpg)

achieves the best performance. This demonstrates that employing a dual-task framework to simultaneously perform interrelated generation and classification tasks yields better performance than performing any of those tasks individually.

Qualitative Comparison: We provide a visual comparison of follow-up scans generated by five different methods in Fig. 3. First, compared to other methods, our method can generate the overall optimal images, characterized by the least noise, fewest artifacts but clearest structure. Second, in terms of detail, our method can also most accurately generate the high-density areas (i.e., areas marked by red boxes) that are crucial for predicting cerebral hemorrhage. Finally, the lightest color in the difference map demonstrates our method can generate lung images with the smallest difference from the ground truth. Such key observations demonstrate that our method is superior to those state-of-theart methods in generation task.

## 4 Conclusion

In this paper, to preemptively determine the occurrence of cerebral hemorrhage post-thrombectomy, we have presented a novel prediction method based solely on the patient's initial CT scan, i.e., simultaneously predicting the follow-up CT and prognostic label from the initial scan. To achieve this goal, we design a dualtask mutual learning framework by proposing three novel strategies including 1)

dual-task learning strategy, 2) transformer-based architecture, and 3) interactive attention mechanism. Among them, the transformer-based architecture enables the model to focus more on the areas important for diagnosing cerebral hemorrhage. Dual-task learning strategy and interactive attention mechanism capture the dependencies between the interrelated generation and classification tasks to improve performance while effectively reducing computational complexity. Validated on the collected clinical dataset demonstrates that our method is designed effectively and can achieve superior performance quantitatively and qualitatively over the state-of-the-art methods.

Acknowledgments. This work was supported in part by National Natural Science Foundation of China (grant numbers U23A20295, 62131015, 62250710165), the STI 2030-Major Projects (No. 2022ZD0209000), Shanghai Municipal Central Guided Local Science and Technology Development Fund (grant number YDZX20233100001001), the China Ministry of Science and Technology (STI2030Major Projects-2022ZD0213100), The Key R&amp;D Program of Guangdong Province, China (grant numbers 2023B0303040001, 2021B0101420006), the ERC IMI (10100 5122), the H2020 (952172), the MRC (MC/PC/21013), the Royal Society (IEC\NS FC\211235), the NVIDIA Academic Hardware Grant Program, the SABER project supported by Boehringer Ingelheim Ltd, NIHR Imperial Biomedical Research Centre (RDA01), Wellcome Leap Dynamic Resilience, UKRI guarantee funding for Horizon Europe MSCA Postdoctoral Fellowships (EP/Z002206/1), and the UKRI Future Leaders Fellowship (MR/V023799/1).

Declaration of Competing Interest. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## References

- 1. Alsekait, D., Saleh, H., Gabralla, L., Alnowaiser, K., El-Sappagh, S., Sahal, R., El-Rashidy, N.: Toward comprehensive chronic kidney disease prediction based on ensemble deep learning models. Applied Sciences 13 (6), 3937 (2023)
- 3. Avants, B., Tustison, N., Song, G., et al.: Advanced normalization tools (ANTS). Insight j 2 (365), 1-35 (2009)
- 2. Armanious, K., Jiang, C., Fischer, M., Küstner, T., Hepp, T., Nikolaou, K., Gatidis, S., Yang, B.: MedGAN: Medical image translation using GANs. Computerized medical imaging and graphics 79 , 101684 (2020)
- 4. Ayoub, M., Liao, Z., Hussain, S., Li, L., Zhang, C., Wong, K.: End to end vision transformer architecture for brain stroke assessment based on multi-slice classification and localization using computed tomography. Computerized Medical Imaging and Graphics 109 , 102294 (2023)
- 5. Bhattacharjee, D., Zhang, T., Süsstrunk, S., Salzmann, M.: MulT: An end-toend multitask learning transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 12031-12041 (2022)

- 6. Bodanapally, U., Shanmuganathan, K., Issa, G., Dreizin, D., Li, G., Sudini, K., Fleiter, T.: Dual-energy CT in hemorrhagic progression of cerebral contusion: overestimation of hematoma volumes on standard 120-kv images and rectification with virtual high-energy monochromatic images after contrast-enhanced wholebody imaging. American Journal of Neuroradiology 39 (4), 658-662 (2018)
- 8. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A., Zhou, Y.: Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)
- 7. Cao, B., Zhang, H., Wang, N., Gao, X., Shen, D.: Auto-GAN: self-supervised collaborative learning for medical image synthesis. Proceedings of the AAAI conference on artificial intelligence 34 (07), 10486-10493 (2020)
- 9. Chen, J., Wei, J., Li, R.: TarGAN: Target-aware generative adversarial networks for multi-modality medical image translation. International Conference on Medical Image Computing and Computer-Assisted Intervention pp. 24-33 (2021)
- 11. Chen, Z., Badrinarayanan, V., Lee, C., Rabinovich, A.: Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. International conference on machine learning pp. 794-803 (2018)
- 10. Chen, L., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40 , 834-848 (2017)
- 12. Dalmaz, O., Yurt, M., Çukur, T.: Resvit: Residual vision transformers for multimodal medical image synthesis. IEEE Transactions on Medical Imaging 41 (10), 2598-2614 (2022)
- 14. Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321 , 321-331 (2018)
- 13. Derex, L., Cho, T.: Mechanical thrombectomy in acute ischemic stroke. Revue Neurologique 173 (3), 106-113 (2017)
- 15. Grkovski, R., Acu, L., Ahmadli, U., Terziev, R., Schubert, T., Wegener, S., Kulcsar, Z., Husain, S., Alkadhi, H., Winklhofer, S.: A novel dual-energy CT method for detection and differentiation of intracerebral hemorrhage from contrast extravasation in stroke patients after endovascular thrombectomy: Feasibility and first results. Clinical Neuroradiology pp. 1-7 (2022)
- 17. Hu, Z., Wang, Z., Jin, Y., Hou, W.: VGG-TSwinformer: Transformer-based deep learning model for early alzheimer's disease prediction. Computer Methods and Programs in Biomedicine 229 , 107291 (2023)
- 16. Han, T., Kather, J., Pedersoli, F., Zimmermann, M., Keil, S., Schulze-Hagen, M., Terwoelbeck, M., Isfort, P., Haarburger, C., Kiessling, F., et al.: Image prediction of disease progression for osteoarthritis by style-based manifold extrapolation. Nature Machine Intelligence 4 (11), 1029-1039 (2022)
- 18. Isola, P., Zhu, J., Zhou, T., Efros, A.: Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition pp. 1125-1134 (2017)
- 20. Jia, H., Yap, P., Shen, D.: Iterative multi-atlas-based multi-image segmentation with tree-based registration. NeuroImage 59 (1), 422-430 (2012)
- 19. Jia, H., Wu, G., Wang, Q., Shen, D.: ABSORB: Atlas building by self-organized registration and bundling. NeuroImage 51 (3), 1057-1070 (2010)
- 21. Jiang, C., Pan, Y., Cui, Z., Nie, D., Shen, D.: Semi-supervised standard-dose PET image generation via region-adaptive normalization and structural consistency constraint. IEEE Transactions on Medical Imaging 42 (10), 2974-2987 (2023)

- 22. Jiang, C., Pan, Y., Wang, T., Chen, Q., Yang, J., Ding, L., Liu, J., Ding, Z., Shen, D.: S2DGAN: Generating dual-energy CT from single-energy CT for real-time determination of intracerebral hemorrhage. International Conference on Information Processing in Medical Imaging pp. 375-387 (2023)
- 24. Jiang, Y., Chang, S., Wang, Z.: TransGAN: Two transformers can make one strong GAN. arXiv preprint arXiv:2102.07074 1 (3) (2021)
- 23. Jiang, C., Wang, T., Pan, Y., Ding, Z., Shen, D.: Real-time diagnosis of intracerebral hemorrhage by generating dual-energy CT from single-energy CT. Medical Image Analysis 95 , 103194 (2024)
- 25. Lan, H., D., A., Toga, A., Sepehrband, F.: Three-dimensional self-attention conditional GAN with spectral normalization for multimodal neuroimaging synthesis. Magnetic Resonance in Medicine 86 (3), 1718-1733 (2021)
- 27. Liu, X., Yu, L., Primak, A., McCollough, C.: Quantitative imaging of element composition and mass fraction using dual-energy CT: Three-material decomposition. Medical physics 36 (5), 1602-1609 (2009)
- 26. Lee, K., Chang, H., Jiang, L., Zhang, H., Tu, Z., Liu, C.: VitGAN: Training GANs with vision transformers. arXiv preprint arXiv:2107.04589 (2021)
- 28. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision pp. 10012-10022 (2021)
- 30. Luo, Y., Wang, Y., Zu, C., Zhan, B., Wu, X., Zhou, J., Shen, D., Zhou, L.: 3D transformer-GAN for high-quality PET reconstruction. International Conference on Medical Image Computing and Computer-Assisted Intervention pp. 276-285 (2021)
- 29. Lu, Z., Li, Z., Wang, J., Shen, D.: Two-stage self-supervised cycle-consistency network for reconstruction of thin-slice MR images. arXiv preprint arXiv:2106.15395 (2021)
- 31. Lyu, T., Zhao, W., Zhu, Y., Wu, Z., Zhang, Y., Chen, Y., Luo, L., Li, S., Xing, L.: Estimating dual-energy CT imaging from single-energy CT data with material decomposition convolutional neural network. Medical image analysis 70 , 102001 (2021)
- 33. Mangesius, S., Janjic, T., Steiger, R., Haider, L., Rehwald, R., Knoflach, M., Widmann, G., Gizewski, E., Grams, A.: Dual-energy computed tomography in acute ischemic stroke: state-of-the-art. European Radiology 31 (6), 4138-4147 (2021)
- 32. Mahjoubi, M., Hamida, S., Siani, L., Cherradi, B., A., E., Raihani, A.: Deep learning for cerebral hemorrhage detection and classification in head CT scans using CNN. International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET) pp. 1-8 (2023)
- 34. Pan, K., Cheng, P., Huang, Z., Lin, L., Tang, X.: Transformer-based T2-weighted MRI synthesis from T1-weighted images. 2022 44th Annual International Conference of the IEEE Engineering in Medicine &amp; Biology Society (EMBC) pp. 50625065 (2022)
- 36. Pan, Y., Liu, M., Xia, Y., Shen, D.: Disease-image-specific learning for diagnosisoriented neuroimage synthesis with incomplete multi-modality data. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (5), 1675-1686 (2021)
- 35. Pan, Y., Liu, M., Lian, C., Xia, Y., Shen, D.: Spatially-constrained fisher representation for brain disease identification with incomplete multi-modal neuroimages. IEEE Transactions on Medical Imaging 39 (9), 2965-2975 (2020)
- 37. Peiris, H., Hayat, M., Chen, Z., Egan, G., Harandi, M.: A volumetric transformer for accurate 3D tumor segmentation. arXiv preprint arXiv:2111.13300 (2021)

## 12 Authors Suppressed Due to Excessive Length

- 38. Peiris, H., Hayat, M., Chen, Z., Egan, G., Harandi, M.: A robust volumetric transformer for accurate 3D tumor segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention pp. 162-172 (2022)
- 40. Sundaram, S., Hulkund, N.: GAN-based data augmentation for chest X-ray classification. arXiv preprint arXiv:2107.02970 (2021)
- 39. Shao, Y., Xu, Y., Li, Y., Wen, X., He, X.: A new classification system for postinterventional cerebral hyperdensity: The influence on hemorrhagic transformation and clinical prognosis in acute stroke. Neural Plasticity 2021 (2021)
- 41. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L.: Attention is all you need. Advances in neural information processing systems 30 (2017)
- 43. Wang, Y., Yu, B., Wang, L., Zu, C., Lalush, D., Lin, W., Wu, X., Zhou, J., Shen, D., Zhou, L.: 3D conditional generative adversarial networks for high-quality PET image estimation at low dose. Neuroimage 174 , 550-562 (2018)
- 42. Wang, X., Liu, Z., Li, J., Xiong, G.: Vision transformer-based classification study of intracranial hemorrhage. International Conference on Computer Vision, Image and Deep Learning &amp; International Conference on Computer Engineering and Applications (CVIDL &amp; ICCEA) pp. 1-8 (2022)
- 44. Wasserthal, J., Breit, H., Meyer, M., Pradella, M., Hinck, D., Sauter, A., Heye, T., Boll, D., Cyriac, J., Yang, S., et al.: Totalsegmentator: Robust segmentation of 104 anatomic structures in CT images. Radiology: Artificial Intelligence 5 (5) (2023)
- 46. Xiang, L., Qiao, Y., Nie, D., An, L., Lin, W., Wang, Q., Shen, D.: Deep autocontext convolutional neural networks for standard-dose PET image estimation from low-dose PET/MRI. Neurocomputing 267 , 406-416 (2017)
- 45. Wu, G., Jia, H., Wang, Q., Shen, D.: SharpMean: groupwise registration guided by sharp mean image and tree-based registration. NeuroImage 56 (4), 1968-1981 (2011)
- 47. Xie, S., Yu, Z., Lv, Z.: Multi-disease prediction based on deep learning: A survey. CMES-Computer Modeling in Engineering &amp; Sciences 128 (2) (2021)
- 49. Yang, H., Sun, J., Carass, A., Zhao, C., Lee, J., Prince, J., Xu, Z.: Unsupervised MR-to-CT synthesis using structure-constrained cycleGAN. IEEE Transactions on Medical Imaging 39 (12), 4249-4261 (2020)
- 48. Xuan, K., Si, L., Zhang, L., Xue, Z., Wang, Q.: Reduce slice spacing of MR images by super-resolution learned without ground-truth. arXiv preprint arXiv:2003.12627 (2020)
- 50. Zhou, Q., Zhu, W., Li, F., Yuan, M., Zheng, L., Liu, X.: Transfer learning of the ResNet-18 and DenseNet-121 model used to diagnose intracranial hemorrhage in CT scanning. Current Pharmaceutical Design 28 (4), 287-295 (2022)
Caiwen Jiang Tianyu Wang Xiaodan Xing Mianxin Liu Guang Yang Zhongxiang Ding Dinggang Shen
A dual-task mutual learning framework for predicting post-thrombectomy cerebral hemorrhage
Ischemic stroke is a severe condition caused by the blockage of brain blood
vessels, and can lead to the death of brain tissue due to oxygen deprivation.
Thrombectomy has become a common treatment choice for ischemic stroke due to
its immediate effectiveness. But, it carries the risk of postoperative cerebral
hemorrhage. Clinically, multiple CT scans within 0-72 hours post-surgery are
used to monitor for hemorrhage. However, this approach exposes radiation dose
to patients, and may delay the detection of cerebral hemorrhage. To address
this dilemma, we propose a novel prediction framework for measuring
postoperative cerebral hemorrhage using only the patient's initial CT scan.
Specifically, we introduce a dual-task mutual learning framework to takes the
initial CT scan as input and simultaneously estimates both the follow-up CT
scan and prognostic label to predict the occurrence of postoperative cerebral
hemorrhage. Our proposed framework incorporates two attention mechanisms, i.e.,
self-attention and interactive attention. Specifically, the self-attention
mechanism allows the model to focus more on high-density areas in the image,
which are critical for diagnosis (i.e., potential hemorrhage areas). The
interactive attention mechanism further models the dependencies between the
interrelated generation and classification tasks, enabling both tasks to
perform better than the case when conducted individually. Validated on clinical
data, our method can generate follow-up CT scans better than state-of-the-art
methods, and achieves an accuracy of 86.37% in predicting follow-up prognostic
labels. Thus, our work thus contributes to the timely screening of
post-thrombectomy cerebral hemorrhage, and could significantly reform the
clinical process of thrombectomy and other similar operations related to
stroke.