Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images

Ali HatamizadehVishwesh NathYucheng TangDong YangHolger RothDaguang Xu

article2022BrainLes@MICCAI2,577 citations

Presents Swin UNETR, an architecture that couples a hierarchical shifted-window transformer encoder with a convolutional decoder to capture multi-scale contextual dependencies for multi-modal 3D brain tumor MRI segmentation.

Listen

Accurate identification and boundary delineation of brain tumors in three-dimensional magnetic resonance imaging (MRI) scans are vital for clinical diagnosis, treatment planning, and surgical intervention. While standard deep learning methods rely heavily on convolutional neural networks, their local receptive fields limit their ability to capture broader, long-range anatomical context across tumors of varying shapes and sizes. The article sets out to develop and evaluate a new artificial intelligence architecture, termed Swin UNETR, designed to improve 3D brain tumor semantic segmentation by combining a hierarchical shifted-window vision transformer encoder with a standard convolutional decoder across multiple spatial resolutions.

To evaluate the system, the authors trained and benchmarked Swin UNETR on the multi-institutional BraTS 2021 dataset, comprising 1,251 multi-modal MRI training subjects and 219 validation cases. The method processes four standard MRI sequences simultaneously and evaluates predictions using cross-validation and official blind challenge submissions. In internal cross-validation across five folds, Swin UNETR outperformed prominent prior benchmark architectures, including nnU-Net, SegResNet, and TransBTS, by 0.4% to 0.7% across tumor sub-regions and achieved an overall average overlap score of 0.913. When submitted to the official BraTS 2021 competition server, an ensemble of ten Swin UNETR models ranked among the top-performing solutions out of more than 2,000 challenge submissions, marking the first competitive top-tier placement by a transformer-based model in this benchmark.

These findings indicate that integrating multi-scale hierarchical transformers into standard medical segmentation pipelines yields superior contextual representations without sacrificing local detail. In clinical and technological workflows, this approach reduces segmentation errors, providing more reliable volumetric insights that support surgical planning and risk assessment. Organizations developing medical AI tools should consider adopting hierarchical transformer encoders as a standard foundation for complex 3D image analysis pipelines, while pursuing further clinical validation on external institutional datasets before operational deployment.

NVIDIA Vanderbilt University computer vision convolutional neural networks encoder-decoder Segmentation Swin Brain

arXiv: 2201.01266 Project-MONAI/research-contributions

Paper: UNETR: Transformers for 3D Medical Image Segmentation, Ali Hatamizadeh et al. (2021). Reading UNETR first is essential because the source paper directly builds upon its architecture for 3D medical image segmentation.
Paper: Swin Transformer: Hierarchical Vision Transformer using Shifted Windows, Ze Liu et al. (2021). This foundational work on the Swin Transformer provides the necessary background on shifted-window attention mechanisms utilized by the source model.

Paper: U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation, Chenxin Li et al. (2025). This subsequent study extends the biomedical segmentation paradigm by replacing standard linear operations with Kolmogorov-Arnold Network layers.
Paper: VMamba: Visual State Space Model, Yue Liu et al. (2024). This later research follows up on transformer-based vision architectures by introducing state-space models to achieve linear-time visual processing.

Cover for Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images

Abstract

Semantic segmentation of brain tumors is a fundamental medical image analysis task involving multiple MRI imaging modalities that can assist clinicians in diagnosing the patient and successively studying the progression of the malignant entity. In recent years, Fully Convolutional Neural Networks (FCNNs) approaches have become the de facto standard for 3D medical image segmentation. The popular "U-shaped" network architecture has achieved state-of-the-art performance benchmarks on different 2D and 3D semantic segmentation tasks and across various imaging modalities. However, due to the limited kernel size of convolution layers in FCNNs, their performance of modeling long-range information is sub-optimal, and this can lead to deficiencies in the segmentation of tumors with variable sizes. On the other hand, transformer models have demonstrated excellent capabilities in capturing such long-range information in multiple domains, including natural language processing and computer vision. Inspired by the success of vision transformers and their variants, we propose a novel segmentation model termed Swin UNEt TRansformers (Swin UNETR). Specifically, the task of 3D brain tumor semantic segmentation is reformulated as a sequence to sequence prediction problem wherein multi-modal input data is projected into a 1D sequence of embedding and used as an input to a hierarchical Swin transformer as the encoder. The swin transformer encoder extracts features at five different resolutions by utilizing shifted windows for computing self-attention and is connected to an FCNN-based decoder at each resolution via skip connections. We have participated in BraTS 2021 segmentation challenge, and our proposed model ranks among the top-performing approaches in the validation phase. Code: this https URL

1 Introduction
2 Related work
3 Swin UNETR
3.1 Encoder
3.2 Decoder
3.3 Loss Function
3.4 Implementation Details
3.5 Dataset and Model Ensembling
4 Results and Discussion
5 Conclusion
References

Knowls

Knowl 1 — Swin UNETR Architecture Overview
model/method
Swin UNETR (Swin UNEt TRansformers) is a 3D medical image segmentation architecture that couples a hierarchical 3D Swin Transformer encoder with a fully convolutional neural network (FCNN) decoder via multi-scale skip connections in a U-shaped topology.

Given a 3D multi-modal input volume $X \in \mathbb{R}^{H \times W \times D \times S}$ , where $(H, W, D)$ denotes spatial resolution and $S$ denotes input modalities (e.g., $S = 4$ for T1, T1Gd, T2, and FLAIR MRI), the network divides the input into non-overlapping 3D patches of size $2 \times 2 \times 2$ . These patches are projected into a $C$ -dimensional embedding space and processed through 4 hierarchical stages of shifted-window self-attention blocks. Multi-scale feature representations extracted at stages $0, 1, 2, 3, 4$ and the bottleneck are transmitted through skip connections to residual convolution blocks in the decoder. The decoder progressively upsamples and fuses representations to yield multi-channel voxel predictions corresponding to nested tumor sub-regions: Enhancing Tumor (ET), Whole Tumor (WT), and Tumor Core (TC).
Knowl 2 — 3D Swin Transformer Hierarchical Encoder
model/method
The encoder of Swin UNETR computes hierarchical multi-scale representations using 3D window-based self-attention. For an input volume with patch dimensions $2 \times 2 \times 2$ across $S = 4$ MRI channels, initial patch tokens have dimension $2 \times 2 \times 2 \times 4 = 32$ . A linear embedding layer projects these into an embedding space of dimension $C = 48$ , yielding an initial token resolution of $\frac{H}{2} \times \frac{W}{2} \times \frac{D}{2}$ .

The encoder contains 4 stages, each consisting of 2 transformer blocks ( $L = 8$ total layers). Self-attention is computed locally inside 3D windows of size $M \times M \times M$ ( $7 \times 7 \times 7$ ). In consecutive layers, window partitions are shifted by $(\lfloor M/2 \rfloor, \lfloor M/2 \rfloor, \lfloor M/2 \rfloor) = (3, 3, 3)$ voxels with 3D cyclic shifting to allow cross-window information exchange.

Between consecutive stages, a patch-merging layer reduces spatial resolution by grouping $2 \times 2 \times 2$ neighboring patches and concatenating their features to create a $4C$ -dimensional embedding, followed by a linear projection to $2C$ . Consequently, stages 1, 2, 3, and 4 produce feature representations of dimensions $\frac{H}{2} \times \frac{W}{2} \times \frac{D}{2} \times C$ , $\frac{H}{4} \times \frac{W}{4} \times \frac{D}{4} \times 2C$ , $\frac{H}{8} \times \frac{W}{8} \times \frac{D}{8} \times 4C$ , and $\frac{H}{16} \times \frac{W}{16} \times \frac{D}{16} \times 8C$ , respectively.
Knowl 3 — Shifted Window Self-Attention Formulation
equation
For two consecutive transformer blocks at layers $l$ and $l+1$ in the Swin Transformer encoder, token features are updated using regular window-based multi-head self-attention (W-MSA) and shifted window-based multi-head self-attention (SW-MSA):

$\hat{z}^l = \text{W-MSA}(\text{LN}(z^{l-1})) + z^{l-1}$

$z^l = \text{MLP}(\text{LN}(\hat{z}^l)) + \hat{z}^l$

$\hat{z}^{l+1} = \text{SW-MSA}(\text{LN}(z^l)) + z^l$

$z^{l+1} = \text{MLP}(\text{LN}(\hat{z}^{l+1})) + \hat{z}^{l+1}$

where $z^{l-1}$ , $\hat{z}^l$ , $z^l$ , $\hat{z}^{l+1}$ , and $z^{l+1}$ denote intermediate feature embeddings, $\text{LN}(\cdot)$ denotes Layer Normalization, and $\text{MLP}(\cdot)$ denotes a multi-layer perceptron. Within each partitioned 3D window, self-attention is computed as:

$\text{Attention}(Q, K, V) = \text{Softmax}\left(\frac{QK^\top}{\sqrt{d}}\right)V$

where $Q, K, V \in \mathbb{R}^{M^3 \times d}$ are query, key, and value representations within the window, $M^3$ is the number of tokens in the 3D window, and $d$ is the query/key channel dimension.
Knowl 4 — CNN-Based Residual Decoder with Skip Connections
model/method
The decoder of Swin UNETR reconstructs the final segmentation mask from the multi-scale representations produced by the encoder. At each encoder stage $i \in \{0, 1, 2, 3, 4\}$ and at the bottleneck ( $i = 5$ ), the extracted feature maps are reshaped into 3D tensors of shape $\frac{H}{2^i} \times \frac{W}{2^i} \times \frac{D}{2^i}$ .

Each feature representation is passed through a residual block comprising two $3 \times 3 \times 3$ convolutional layers normalized by instance normalization. Starting from the bottleneck ( $i=5$ ), spatial resolution is doubled via 3D deconvolutional layers, and the upsampled features are concatenated along channels with the skip-connection output from stage $i-1$ . The concatenated representations are subsequently processed by another residual block. At the final full resolution ( $H \times W \times D$ ), a $1 \times 1 \times 1$ convolutional layer followed by a sigmoid activation generates the final multi-channel segmentation probability maps.
Knowl 5 — Multi-Class Soft Dice Loss
equation
The Swin UNETR model optimizes network weights during training using the voxel-wise soft Dice loss function:

$\mathcal{L}(G, Y) = 1 - \frac{2}{J} \sum_{j=1}^{J} \frac{\sum_{i=1}^{I} G_{i,j} Y_{i,j}}{\sum_{i=1}^{I} G_{i,j}^2 + \sum_{i=1}^{I} Y_{i,j}^2}$

where $I$ denotes the total number of voxels in the image volume, $J$ denotes the number of semantic classes, $Y_{i,j} \in [0, 1]$ represents the predicted probability of voxel $i$ belonging to class $j$ , and $G_{i,j} \in \{0, 1\}$ is the binary one-hot ground-truth label for class $j$ at voxel $i$ .

Knowl 6 — Swin UNETR Architecture Configuration and Complexity

data/table

The architectural specifications, hyperparameter settings, parameter counts, and floating point operations (FLOPs) of Swin UNETR are defined as follows:

The initial feature dimension is $C = 48$ , scaling to a bottleneck embedding size of 768. The encoder uses 4 stages with 2 transformer blocks per stage ( $[2, 2, 2, 2]$ ) and attention head configurations of $[3, 6, 12, 24]$ across stages. The window size for 3D self-attention is $[7, 7, 7]$ . The full architecture has 61.98 million parameters and requires 394.84 GFLOPs.

Knowl 7 — Experimental and Training Setup on BraTS 2021
experimental setup
Swin UNETR was trained and evaluated on the BraTS 2021 dataset consisting of 1251 multi-modal 3D MRI scans for training and 219 validation scans (four modalities: native T1, post-contrast T1Gd, T2, and T2-FLAIR) with rigid alignment, skull-stripping, isotropic resolution of $1 \times 1 \times 1\text{ mm}^3$ , and dimensions $240 \times 240 \times 155$ .
- Data Preprocessing & Augmentation: Non-zero voxels were normalized to zero mean and unit standard deviation. Training used randomly cropped $128 \times 128 \times 128$ voxel patches. Data augmentations included random axis mirror flipping ( $p=0.5$ per axis), random per-channel intensity shifting in range $[-0.1, 0.1]$ , and random per-channel intensity scaling in range $[0.9, 1.1]$ .
- Training Details: The model was implemented in PyTorch and MONAI, trained across 8 NVIDIA V100 GPUs on a DGX-1 cluster with batch size 1 per GPU for 800 epochs. Optimization used a learning rate of $0.0008$ , linear warmup, and a cosine annealing learning rate scheduler.
- Inference & Ensembling: Sliding window inference was executed with a 0.7 overlap factor for adjacent voxels. The final submission ensemble aggregated 10 Swin UNETR models obtained from two separate 5-fold cross-validation runs (80:20 train-validation split).

Knowl 8 — Five-Fold Cross-Validation Performance Comparison on BraTS 2021

data/table

A five-fold cross-validation experiment on the 1251 BraTS 2021 training cases compared Swin UNETR against nnU-Net, SegResNet, and TransBTS across Enhancing Tumor (ET), Whole Tumor (WT), and Tumor Core (TC) sub-regions using mean Dice score:

	Swin UNETR				nnU-Net				SegResNet				TransBTS
Dice Score	ET	WT	TC	Avg.	ET	WT	TC	Avg.	ET	WT	TC	Avg.	ET	WT	TC	Avg.
Fold 1	0.876	0.929	0.914	0.906	0.866	0.921	0.902	0.896	0.867	0.924	0.907	0.899	0.856	0.910	0.897	0.883
Fold 2	0.908	0.938	0.919	0.921	0.899	0.933	0.919	0.917	0.900	0.933	0.915	0.916	0.885	0.919	0.903	0.902
Fold 3	0.891	0.931	0.919	0.913	0.886	0.929	0.914	0.910	0.884	0.927	0.917	0.909	0.866	0.903	0.898	0.889
Fold 4	0.890	0.937	0.920	0.915	0.886	0.927	0.914	0.909	0.888	0.921	0.916	0.908	0.868	0.910	0.901	0.893
Fold 5	0.891	0.934	0.917	0.914	0.880	0.929	0.917	0.909	0.878	0.930	0.912	0.906	0.867	0.915	0.893	0.892
Avg.	0.891	0.933	0.917	0.913	0.883	0.927	0.913	0.908	0.883	0.927	0.913	0.907	0.868	0.911	0.898	0.891

Swin UNETR outperforms all competing methods across every individual fold and on average across all classes, exceeding the nearest models by 0.8% on ET, 0.6% on WT, 0.4% on TC, and 0.5% on average. TransBTS, which limits transformer attention solely to the network bottleneck, demonstrates the lowest performance among the evaluated models.

Knowl 9 — BraTS 2021 Validation and Testing Benchmark Performance

data/table

The 10-model ensemble of Swin UNETR was evaluated on the official BraTS 2021 validation set (219 subjects) and testing set, measuring mean Dice score and 95% Hausdorff distance (in millimeters) on Enhancing Tumor (ET), Whole Tumor (WT), and Tumor Core (TC):

	Dice			Hausdorff (mm)
Dataset	ET	WT	TC	ET	WT	TC
Validation	0.858	0.926	0.885	6.016	5.831	3.770
Testing	0.853	0.927	0.876	16.326	4.739	15.309

On the validation set, Swin UNETR achieved mean Dice scores of 0.858 (ET), 0.926 (WT), and 0.885 (TC). On the hidden testing dataset, ET and WT Dice scores remained consistent at 0.853 and 0.927, while the TC Dice score decreased by 0.9% to 0.876.

Coverage note — No substantial contributed material was omitted.

References

1.Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Landman, B.A., Litjens, G., Menze, B., Ronneberger, O., Summers, R.M., van Ginneken, B., et al.: The medical segmentation decathlon. arXiv preprint arXiv:2106.05735 (2021)
2.Baid, U., Ghodasara, S., Mohan, S., Bilello, M., Calabrese, E., Colak, E., Farahani, K., Kalpathy-Cramer, J., Kitamura, F.C., Pati, S., et al.: The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02314 (2021)
3.Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., John Freymann, K.F., Davatzikos, C.: Segmentation labels and radiomic features for the pre-operative scans of the tcga-gbm collection. The Cancer Imaging Archive (2017), https://doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q
4.Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., John Freymann, K.F., Davatzikos, C.: Segmentation labels and radiomic features for the pre-operative scans of the tcga-lgg collection. The Cancer Imaging Archive (2017), https://doi.org/10.7937/K9/TCIA.2017.GJQ7R0EF
5.Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., Freymann, J., Farahani, K., Davatzikos, C.: Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientific data 4 (9 2017)
6.Bakas, S., Reyes, M., et Int, Menze, B.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. In: arXiv:1811.02629 (2018)
7.Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Shinohara, R.T., Berger, C., Ha, S.M., Rozycki, M., et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv preprint arXiv:1811.02629 (2018)
8.Bao, H., Dong, L., Wei, F.: Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254 (2021)
9.Caron, M., Touvron, H., Misra, I., J'egou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision (2021)
10.Chen, C., Liu, X., Ding, M., Zheng, J., Li, J.: 3d dilated multi-fiber network for real-time brain tumor segmentation in mri. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 184–192. Springer (2019)
11.Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., Zhou, Y.: Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)
12.Ç i̧cek, O., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3d u-net: ¨ learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention. pp. 424–432. Springer (2016)
13.Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
14.Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. In: International Conference on Learning Representations (2020)
15.Grover, V.P., Tognarelli, J.M., Crossey, M.M., Cox, I.J., Taylor-Robinson, S.D., McPhail, M.J.: Magnetic resonance imaging: principles and techniques: lessons for clinicians. Journal of clinical and experimental hepatology 5(3), 246–255 (2015)
16.Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H., Xu, D.: Unetr: Transformers for 3d medical image segmentation. arXiv preprint arXiv:2103.10504 (2021)
17.Hoover, J.M., Morris, J.M., Meyer, F.B.: Use of preoperative magnetic resonance imaging t1 and t2 sequences to determine intraoperative meningioma consistency. Surgical neurology international 2 (2011)
18.Huo, Y., Xu, Z., Xiong, Y., Aboud, K., Parvathaneni, P., Bao, S., Bermudez, C., Resnick, S.M., Cutting, L.E., Landman, B.A.: 3d whole brain segmentation using spatially localized atlas network tiles. NeuroImage 194, 105–119 (2019)
19.Isensee, F., Jaeger, P.F., Full, P.M., Vollmuth, P., Maier-Hein, K.: nnu-net for brain tumor segmentation. In: BrainLes@MICCAI (2020)
20.Isensee, F., J¨ager, P.F., Full, P.M., Vollmuth, P., Maier-Hein, K.H.: nnu-net for brain tumor segmentation. In: International MICCAI Brainlesion Workshop. pp. 118–132. Springer (2020)
21.Jiang, Z., Ding, C., Liu, M., Tao, D.: Two-stage cascaded u-net: 1st place solution to brats challenge 2019 segmentation task. In: International MICCAI Brainlesion Workshop. pp. 231–241. Springer (2019)
22.Kamnitsas, K., W. Bai, E.F., McDonagh, S., Sinclair, M., Pawlowski, N., Rajchl, M., Lee, M., Kainz, B., Rueckert, D., Glocker, B.: Ensembles of multiple models and architectures for robust brain tumour segmentation. In: International Conf. on Medical Image Computing and Computer Assisted Intervention. Multimodal Brain Tumor Segmentation Challenge (MICCAI, 2017). LNCS (2017)
23.Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Medical image analysis 36, 61–78 (2017)
24.Liu, D., Zhang, H., Zhao, M., Yu, X., Yao, S., Zhou, W.: Brain tumor segmention based on dilated convolution refine networks. In: 2018 IEEE 16th International Conference on Software Engineering Research, Management and Applications (SERA). pp. 113–120. IEEE (2018)
25.Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision (2021)
26.Liu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., Hu, H.: Video swin transformer. arXiv preprint arXiv:2106.13230 (2021)
27.Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3431–3440 (2015)
28.Louis, D.N., Ohgaki, H., Wiestler, O.D., Cavenee, W.K., Burger, P.C., Jouvet, A., Scheithauer, B.W., Kleihues, P.: The 2007 who classification of tumours of the central nervous system. Acta neuropathologica 114(2), 97–109 (2007)
29.Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., Lanczi, L., Gerstner, E.R., Weber, M.A., Arbel, T., Avants, B.B., Ayache, N., Buendia, P., Collins, D.L., Cordier, N., Corso, J.J., Criminisi, A., Das, T., Delingette, H., Demiralp, C., Durst, C.R., Dojat, M., Doyle, S., Festa, J., Forbes, F., Geremia, E., Glocker, B., Golland, P., Guo, X., Hamamci, A., Iftekharuddin, K.M., Jena, R., John, N.M., Konukoglu, E., Lashkari, D., Mariz, J.A., Meier, R., Pereira, S., Precup, D., Price, S.J., Raviv, T.R., Reza, S.M.S., Ryan, M.T., Sarikaya, D., Schwartz, L.H., Shin, H.C., Shotton, J., Silva, C.A., Sousa, N., Subbanna, N.K., Szekely, G., Taylor, T.J., Thomas, O.M., Tustison, N.J., Unal, G.B., Vasseur, F., Wintermark, M., Ye, D.H., Zhao, L., Zhao, B., Zikic, D., Prastawa, M., Reyes, M., Leemput, K.V.: The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)
30.Milletari, F., Navab, N., Ahmadi, S.A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 fourth international conference on 3D vision (3DV). pp. 565–571. IEEE (2016)
31.Myronenko, A.: 3D MRI brain tumor segmentation using autoencoder regularization. In: BrainLes, Medical Image Computing and Computer Assisted Intervention (MICCAI). pp. 311–320. LNCS, Springer (2018), https://arxiv.org/abs/1810.11654
32.Myronenko, A., Hatamizadeh, A.: Robust semantic segmentation of brain tumor regions from 3d mris. In: International MICCAI Brainlesion Workshop. pp. 82–89. Springer (2019)
33.Nie, D., Zhang, H., Adeli, E., Liu, L., Shen, D.: 3d deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. In: International conference on medical image computing and computer-assisted intervention. pp. 212–220. Springer (2016)
34.Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C., Dosovitskiy, A.: Do vision transformers see like convolutional neural networks? Advances in Neural Information Processing Systems 34 (2021)
35.Simpson, A.L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., Van Ginneken, B., Kopp-Schneider, A., Landman, B.A., Litjens, G., Menze, B., et al.: A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063 (2019)
36.Tang, Y., Yang, D., Li, W., Roth, H., Landman, B., Xu, D., Nath, V., Hatamizadeh, A.: Self-supervised pre-training of swin transformers for 3d medical image analysis. arXiv preprint arXiv:2111.14791 (2021)
37.Ulyanov, D., Vedaldi, A., Lempitsky, V.: Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016)
38.Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Advances in neural information processing systems. pp. 5998–6008 (2017)
39.Wang, W., Chen, C., Ding, M., Yu, H., Zha, S., Li, J.: Transbts: Multimodal brain tumor segmentation using transformer. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 109–119. Springer (2021)
40.Xie, Y., Zhang, J., Shen, C., Xia, Y.: Cotr: Efficiently bridging cnn and transformer for 3d medical image segmentation. arXiv preprint arXiv:2103.03024 (2021)
41.Zacharaki, E.I., Wang, S., Chawla, S., Soo Yoo, D., Wolf, R., Melhem, E.R., Davatzikos, C.: Classification of brain tumor type and grade using mri texture and shape in a machine learning scheme. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 62(6), 1609–1618 (2009)
42.Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P.H., et al.: Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6881–6890 (2021)
43.Zhou, C., Chen, S., Ding, C., Tao, D.: Learning contextual and attentive information for brain tumor segmentation. In: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2018). Multimodal Brain Tumor Segmentation Challenge (BraTS 2018). BrainLes 2018 workshop. LNCS, Springer (2018)

Citation

MLA

Hatamizadeh, A., et al. “Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images”. arXiv, 2022, http://arxiv.org/abs/2201.01266v1.

APA

Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H., & Xu, D. (2022). Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images. arXiv. http://arxiv.org/abs/2201.01266v1

Chicago

Hatamizadeh, A., V. Nath, Y. Tang, D. Yang, H. Roth, and D. Xu. 2022. “Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images”. arXiv. http://arxiv.org/abs/2201.01266v1.

Harvard

Hatamizadeh, A. et al. (2022) “Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2201.01266v1.

Vancouver

1. Hatamizadeh A, Nath V, Tang Y, Yang D, Roth H, Xu D (2022) Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images. arXiv

BibTeX

@article{hatamizadeh2022swin,
  title = {Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images},
  author = {Hatamizadeh, Ali and Nath, Vishwesh and Tang, Yucheng and Yang, Dong and Roth, Holger and Xu, Daguang},
  year = {2022},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2201.01266v1},
  eprint = {2201.01266}
}

Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Embed Dimension	Feature Size	Number of Blocks	Window Size	Number of Heads	Parameters	FLOPs
768	48	[2, 2, 2, 2]	[7, 7, 7]	[3, 6, 12, 24]	61.98M	394.84G

Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Swin UNETR Architecture Overview

Knowl 2 — 3D Swin Transformer Hierarchical Encoder

Knowl 3 — Shifted Window Self-Attention Formulation

Knowl 4 — CNN-Based Residual Decoder with Skip Connections

Knowl 5 — Multi-Class Soft Dice Loss

Knowl 6 — Swin UNETR Architecture Configuration and Complexity

Knowl 7 — Experimental and Training Setup on BraTS 2021

Knowl 8 — Five-Fold Cross-Validation Performance Comparison on BraTS 2021

Knowl 9 — BraTS 2021 Validation and Testing Benchmark Performance

References

Citation

BibTeX

Source Code

Access the Paper