Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations

Carole H. SudreWenqi LiTom Kamiel Magda VercauterenSébastien OurselinM. Jorge Cardoso

article2017DLMIA/ML-CDS@MICCAI2,807 citations

Proposes adapting the Generalized Dice overlap into a deep learning loss function to solve severe class imbalance problems in 2D and 3D medical image segmentation.

Listen

Medical image segmentation often targets rare pathologies such as brain tumors or white matter lesions that occupy only a tiny fraction of each scan. This severe class imbalance destabilizes standard deep-learning training and produces unreliable results. The issue has grown more pressing as deep-learning tools move into routine clinical use where missing small lesions can affect diagnosis or treatment planning.

The article set out to compare the training stability of three established loss functions designed for imbalance and to test whether the Generalized Dice overlap, already used as an evaluation metric, could serve as a more robust loss function for both balanced and highly unbalanced segmentation tasks.

The authors reimplemented four representative networks—two 2D and two 3D—and trained them on the public BRATS tumor dataset and an in-house set of 524 subjects with age-related white-matter hyperintensities. They systematically varied learning rates and patch sizes to produce different levels of foreground-background imbalance, then measured final Dice scores over the last 200 training iterations.

The Generalized Dice loss proved consistently more stable than weighted cross-entropy, the two-class Dice loss, or the sensitivity-specificity loss, especially when imbalance exceeded roughly 1 percent and when higher learning rates were used. Weighted cross-entropy frequently failed to train under the most unbalanced conditions, while the Dice loss and sensitivity-specificity loss showed large performance drops or network-dependent behavior. On the held-out 3D test set the Generalized Dice loss maintained higher and less variable Dice scores across sampling strategies.

These results indicate that the choice of loss function can determine whether a segmentation model succeeds or fails in realistic clinical settings where lesions are small and variable. Adopting the Generalized Dice loss reduces the need for exhaustive hyper-parameter tuning and improves the chance of detecting punctate lesions that other losses overlook.

The authors recommend using the Generalized Dice loss as the default for any new unbalanced segmentation task and note that the implementation is already available in the open-source NiftyNet package. They also call for further experiments on more extreme imbalance ratios, such as those encountered when detecting lacunes or perivascular spaces.

The study examined only two imaging tasks and four networks without data augmentation, so results may not generalize to every anatomy or architecture. Nevertheless, the consistent advantage of the Generalized Dice loss across the tested range of imbalance and hyper-parameters gives reasonable confidence that it offers a practical improvement for many current applications.

University College London deep learning medical images network architecture Generalized Dice Loss Weighted Cross-Entropy Sensitivity Specificity

arXiv: 1707.03237 NifTK/NiftyNet

Paper: V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation, Fausto Milletari et al. (2016). Introduces the standard Dice loss for 3D biomedical segmentation, which the source explicitly generalizes to handle multi-class and extreme class imbalance.
Paper: U-Net: Convolutional Networks for Biomedical Image Segmentation, Olaf Ronneberger et al. (2015). Presents the foundational U-Net architecture and weighted cross-entropy formulations evaluated and extended for imbalanced medical image segmentation.
Paper: 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation, Özgün Çiçek et al. (2016). Establishes the 3D U-Net architecture used directly as one of the baseline deep networks analyzed under varying degrees of class imbalance.
Paper: Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation, Konstantinos Kamnitsas et al. (2016). Provides the multi-scale 3D CNN baseline framework and brain lesion segmentation setting benchmarked in the source's comparative loss study.
Paper: Brain tumor segmentation with Deep Neural Networks, Mohammad Havaei et al. (2015). Details the BRATS brain tumor segmentation task and the fundamental challenges of severe foreground-background class imbalance in neural networks.
Paper: Fully convolutional networks for semantic segmentation, Jonathan Long et al. (2015). Introduces end-to-end fully convolutional networks for dense pixel-wise segmentation underlying the 2D network architectures evaluated in the study.

Paper: UNet++: A Nested U-Net Architecture for Medical Image Segmentation, Zongwei Zhou et al. (2018). Extends medical image segmentation by redesigning skip connections into nested dense pathways and leveraging hybrid overlap losses for fine-scale lesion boundary capture.
Paper: UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation, Huimin Huang et al. (2020). Builds on overlap-based training objectives by combining multi-scale skip connections with a compound loss targeting severe organ-scale imbalance.
Paper: Attention U-Net: Learning Where to Look for the Pancreas, Ozan Oktay et al. (2018). Incorporates attention gates into 3D U-Net architectures to suppress background noise and improve segmentation of small, highly unbalanced target organs like the pancreas.
Paper: UNETR: Transformers for 3D Medical Image Segmentation, Ali Hatamizadeh et al. (2021). Applies transformer encoders to volumetric medical segmentation, advancing beyond CNNs while continuing to optimize overlap metrics on unbalanced MRI and CT benchmarks.
Paper: Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images, Ali Hatamizadeh et al. (2022). Continues the brain tumor segmentation line on BraTS data by pairing shifted-window transformer backbones with overlap-optimized loss objectives.
Paper: Class-Balanced Loss Based on Effective Number of Samples, Yin Cui et al. (2019). Develops a generalized theoretical approach to class-imbalanced loss design by scaling training penalties via the effective number of samples across unbalanced datasets.

Cover for Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations

Abstract

Deep-learning has proved in recent years to be a powerful tool for image analysis and is now widely used to segment both 2D and 3D medical images. Deep-learning segmentation frameworks rely not only on the choice of network architecture but also on the choice of loss function. When the segmentation process targets rare observations, a severe class imbalance is likely to occur between candidate labels, thus resulting in sub-optimal performance. In order to mitigate this issue, strategies such as the weighted cross-entropy function, the sensitivity function or the Dice loss function, have been proposed. In this work, we investigate the behavior of these loss functions and their sensitivity to learning rate tuning in the presence of different rates of label imbalance across 2D and 3D segmentation tasks. We also propose to use the class re-balancing properties of the Generalized Dice overlap, a known metric for segmentation assessment, as a robust and accurate deep-learning loss function for unbalanced tasks.

Topics

deep learning medical images network architecture segmentation methods automatic segmentations convolutional neural networks tuning method

Keywords

Generalized Dice Loss Weighted Cross-Entropy Sensitivity Specificity Deep Learning Class Imbalance UNet DeepMedic HighResNet Patch Size Learning Rate

1 Introduction
2 Methods
2.1 Loss functions for unbalanced data
2.2 Deep learning framework
3 Experiments and Results
3.1 Experiments
3.2 2D Results
3.3 3D Results
4 Discussion
References

Knowls

Knowl 1 — Generalized Dice Loss with Inverse-Volume Weighting
model/method
The Generalized Dice Loss ( $GDL$ ) adapts the Generalized Dice Score into a differentiable loss function for training deep convolutional neural networks on segmentation tasks subject to severe class imbalance. For a segmentation problem with candidate class labels $l \in \{1, \dots, L\}$ over an image of $N$ voxels, let $r_{ln} \in \{0, 1\}$ denote the reference ground-truth binary indicator for class $l$ at voxel $n$ , and let $p_{ln} \in [0, 1]$ denote the corresponding predicted class probability. The Generalized Dice Loss is defined as:

$GDL = 1 - 2 \frac{\sum_{l=1}^L w_l \sum_{n=1}^N r_{ln} p_{ln}}{\sum_{l=1}^L w_l \sum_{n=1}^N (r_{ln} + p_{ln})}$

where $w_l$ is a class-specific weight introduced to provide invariance to varying class volumes. In the volume-weighted variant, denoted $GDL_v$ , the weights are defined as the inverse square of the reference class volumes:

$w_l = \frac{1}{\left(\sum_{n=1}^N r_{ln}\right)^2}$

This weighting corrects each class's contribution by the inverse of its volume, counteracting the intrinsic bias of the standard Dice score toward large structures and ensuring balanced gradients across small and large regions.
Knowl 2 — Analytical Gradient of Two-Class Generalized Dice Loss
equation
For a two-class (foreground vs. background) segmentation problem with $N$ voxels, let $r_n \in \{0, 1\}$ and $p_n \in [0, 1]$ denote the reference ground truth and predicted probability for the foreground label at voxel $n$ , respectively. Let $w_1$ denote the foreground class weight and $w_2$ denote the background class weight. The gradient of the Generalized Dice Loss ( $GDL$ ) with respect to the predicted foreground probability $p_i$ at voxel $i \in \{1, \dots, N\}$ is:

$\frac{\partial GDL}{\partial p_i} = -2 \frac{(w_1^2 - w_2^2) \left[\sum_{n=1}^N p_n r_n - r_i \sum_{n=1}^N (p_n + r_n)\right] + N w_2 (w_1 + w_2)(1 - 2 r_i)}{\left[ (w_1 - w_2) \sum_{n=1}^N (p_n + r_n) + 2 N w_2 \right]^2}$

This analytical gradient can be evaluated during stochastic gradient descent and generalizes directly to multi-class segmentation settings.
Knowl 3 — Formulations of Benchmark Loss Functions for Imbalanced Binary Segmentation
model/method
In binary segmentation over $N$ voxels where $r_n \in \{0, 1\}$ is the reference label and $p_n \in [0, 1]$ is the predicted foreground probability at voxel $n$ (with background reference $1 - r_n$ and background probability $1 - p_n$ ), three alternative loss functions designed to address class imbalance are formulated as follows:
1. Weighted Cross-Entropy (WCE):
$WCE = -\frac{1}{N} \sum_{n=1}^N \left[ w r_n \log(p_n) + (1 - r_n) \log(1 - p_n) \right]$

where the foreground weight $w$ is defined dynamically from predictions as $w = \frac{N - \sum_{n=1}^N p_n}{\sum_{n=1}^N p_n}$ .
1. Two-Class Dice Loss ( $DL_2$ ):
$DL_2 = 1 - \frac{\sum_{n=1}^N p_n r_n + \epsilon}{\sum_{n=1}^N (p_n + r_n) + \epsilon} - \frac{\sum_{n=1}^N (1 - p_n)(1 - r_n) + \epsilon}{\sum_{n=1}^N (2 - p_n - r_n) + \epsilon}$

where $\epsilon > 0$ is a smoothing parameter to avoid division by zero when reference and prediction sets are empty.
1. Sensitivity-Specificity Loss ( $SS$ ):
$SS = \lambda \frac{\sum_{n=1}^N (r_n - p_n)^2 r_n}{\sum_{n=1}^N r_n + \epsilon} + (1 - \lambda) \frac{\sum_{n=1}^N (r_n - p_n)^2 (1 - r_n)}{\sum_{n=1}^N (1 - r_n) + \epsilon}$

where $\lambda$ balances sensitivity and specificity (set to $\lambda = 0.05$ ), and $\epsilon > 0$ prevents division by zero.

Knowl 4 — Quantitative 3D White Matter Hyperintensity Segmentation Results

data/table

The performance of DeepMedic and HighResNet on 3D age-related white matter hyperintensity segmentation was evaluated across three patch sizes (Small: S, Moderate: M, Large: L), three learning rates ( $\log_{10}(\text{LR}) \in \{-5, -4, -3\}$ ), and four loss functions ( $WCE$ , $DL_2$ , $SS$ , $GDL_v$ ). The table reports the median Dice Similarity Coefficient (DSC) and interquartile range over the final 200 training iterations:

		DeepMedic				HighResNet
Patch	$\log_{10}(\text{LR})$	WCE	$DL_2$	SS	$GDL_v$	WCE	$DL_2$	SS	$GDL_v$
S	-5	0.49 (0.17)	0.44 (0.19)	0.42 (0.14)	0.46 (0.17)	0 (0)	0 (0)	0.06 (0.15)	0.47 (0.32)
S	-4	0.58 (0.20)	0.60 (0.15)	0.61 (0.22)	0.61 (0.18)	0 (0)	0.71 (0.18)	0.34 (0.20)	0.74 (0.15)
S	-3	0.61 (0.12)	0.59 (0.14)	0.63 (0.15)	0.60 (0.15)	0 (0)	0 (0)	0 (0)	0 (0)
M	-5	0.05 (0.07)	0.05 (0.07)	0.05 (0.06)	0.04 (0.06)	0 (0)	0.60 (0.27)	0.15 (0.13)	0.64 (0.19)
M	-4	0.09 (0.11)	0.07 (0.09)	0.08 (0.09)	0.08 (0.10)	0 (0)	0.71 (0.20)	0.20 (0.20)	0.69 (0.20)
M	-3	0.45 (0.31)	0.42 (0.31)	0.17 (0.24)	0.48 (0.32)	0 (0)	0 (0)	0 (0)	0.65 (0.23)
L	-5	0.01 (0.03)	0.01 (0.03)	0.01 (0.03)	0.01 (0.03)	0 (0)	0.54 (0.27)	0.03 (0.06)	0.50 (0.32)
L	-4	0.01 (0.04)	0.02 (0.04)	0.02 (0.04)	0.01 (0.04)	0 (0)	0.57 (0.32)	0.08 (0.19)	0.60 (0.30)
L	-3	0.21 (0.33)	0.18 (0.30)	0.05 (0.12)	0.20 (0.33)	0 (0)	0.62 (0.18)	0.22 (0.15)	0.49 (0.34)

These results demonstrate that under severe 3D class imbalance (foreground ratio down to 0.002), Weighted Cross-Entropy completely fails to converge on HighResNet (DSC = 0), and Sensitivity-Specificity exhibits severely reduced DSC (0.03--0.34). While $DL_2$ achieves competitive scores at lower learning rates, it fails at $\text{LR} = 10^{-3}$ on small and moderate patches. In contrast, $GDL_v$ maintains high performance across patch sizes and is the only loss capable of training HighResNet at $\text{LR} = 10^{-3}$ on moderate patches (DSC = 0.65).

Knowl 5 — Quantitative 2D Brain Tumor Segmentation Results on BRATS

data/table

The performance of UNet and TwoPathCNN on the 2D BRATS neuro-oncological tumor segmentation dataset was assessed across patch sizes (Small: S, Moderate: M, Large: L), learning rates ( $\log_{10}(\text{LR}) \in \{-5, -4, -3\}$ ), and four loss functions ( $WCE$ , $DL_2$ , $SS$ , $GDL_v$ ). The table reports the median DSC and interquartile range over the final 200 training iterations:

		UNet				TwoPathCNN
Patch	$\log_{10}(\text{LR})$	WCE	$DL_2$	SS	$GDL_v$	WCE	$DL_2$	SS	$GDL_v$
S	-5	0.71 (0.17)	0.73 (0.13)	0.37 (0.17)	0.75 (0.14)	0.56 (0.48)	0 (0)	0.53 (0.41)	0.49 (0.44)
S	-4	0.77 (0.18)	0.76 (0.13)	0.74 (0.16)	0.80 (0.12)	0.80 (0.12)	0.79 (0.11)	0.81 (0.12)	0.80 (0.12)
S	-3	0.70 (0.17)	0.72 (0.15)	0.39 (0.16)	0.72 (0.15)	0 (0)	0 (0)	0.77 (0.11)	0.72 (0.15)
M	-5	0.71 (0.23)	0.70 (0.22)	0.65 (0.25)	0.74 (0.19)	0 (0)	0.73 (0.18)	0.69 (0.21)	0.73 (0.19)
M	-4	0.73 (0.18)	0.70 (0.22)	0.61 (0.25)	0.72 (0.19)	0.77 (0.16)	0.76 (0.17)	0.71 (0.18)	0.76 (0.17)
M	-3	0.68 (0.23)	0.67 (0.21)	0.70 (0.26)	0.69 (0.22)	0 (0)	0.71 (0.22)	0.67 (0.21)	0.72 (0.19)
L	-5	0.63 (0.46)	0.62 (0.40)	0.49 (0.42)	0.56 (0.44)	0.62 (0.50)	0.50 (0.41)	0.50 (0.38)	0.56 (0.35)
L	-4	0.68 (0.34)	0.64 (0.44)	0.18 (0.24)	0.66 (0.39)	0.64 (0.42)	0.59 (0.43)	0.52 (0.38)	0.64 (0.35)
L	-3	0.59 (0.39)	0.57 (0.53)	0.16 (0.22)	0.59 (0.45)	0.77 (0.12)	0.77 (0.14)	0.79 (0.12)	0.79 (0.11)

The data shows that an intermediate learning rate of $10^{-4}$ produces optimal or near-optimal segmentation across networks and loss functions. However, when trained with TwoPathCNN at $\text{LR} = 10^{-3}$ , $WCE$ and $DL_2$ collapse entirely (DSC = 0 (0)) on small patches and $WCE$ collapses on moderate patches, whereas $GDL_v$ remains consistently stable (DSC = 0.72).

Knowl 6 — Experimental Setup for 2D and 3D Class-Imbalanced Segmentation

experimental setup

The evaluation of segmentation loss functions encompasses four deep neural network architectures across 2D and 3D medical image segmentation benchmarks:

2D Architecture & Dataset: UNet and TwoPathCNN were evaluated on the BRATS benchmark for binary brain tumor segmentation. Training was conducted for 1,000 iterations without data augmentation.
3D Architecture & Dataset: DeepMedic and HighResNet were evaluated on an in-house dataset of 524 subjects with age-related white matter hyperintensities (WMH) using T1-weighted, T2-weighted, and FLAIR MRI. Intensities were normalized via z-scoring relative to white matter intensity. Training was run for 3,000 iterations without data augmentation, and 10% of data was held out for test evaluation.
Hyperparameters: Three log-spaced learning rates were tested: $10^{-3}$ , $10^{-4}$ , and $10^{-5}$ . Training patches were selected under the constraint of containing at least one foreground voxel.
Patch and Batch Size Configurations:

	UNet			TwoPathCNN			DeepMedic			HighResNet
Batch Size	5	3	1	5	3	1	5	3	1	5	3	1
Initial Patch Size	56	64	88	51	63	85	51	63	87	51	63	85
Effective Patch Size	16	24	48	19	31	53	3	15	39	15	27	49
Imbalance Ratio	0.52	0.33	0.15	0.29	0.25	0.16	0.20	0.01	0.002	0.02	0.01	0.003

Knowl 7 — Robustness of Generalized Dice Loss under Severe Imbalance and Punctate Lesion Detection
empirical result
Empirical evaluation on 3D white matter hyperintensity test data shows that the Generalized Dice Loss with volume weighting ( $GDL_v$ ) exhibits distinct advantages over Weighted Cross-Entropy ( $WCE$ ), two-class Dice Loss ( $DL_2$ ), and Sensitivity-Specificity ( $SS$ ):
1. Robustness to Extreme Imbalance: When foreground-to-background volume ratios drop to severe levels (0.02% to 0.2% of 3D patch volume), $WCE$ fails to learn, and $SS$ exhibits major degradation in test DSC. Overlap-based loss functions are substantially more robust to high imbalance.
2. Hyperparameter Stability: $GDL_v$ maintains stable convergence across a broad hyperparameter space, whereas $DL_2$ fails at elevated learning rates ( $10^{-3}$ ) and $SS$ performance varies strongly with network architecture.
3. Small and Punctate Structure Sensitivity: Qualitative analysis on 3D MRI test volumes indicates that $GDL_v$ provides superior capability to detect and segment small, punctate lesions that are systematically missed or under-segmented when training with $DL_2$ or $SS$ .

Coverage note — No substantial contributed material was omitted from the paper.

References

1.Brosch, T., Yoo, Y., Tang, L.Y., Li, D.K., Traboulsee, A., Tam, R.: Deep convolutional encoder networks for multiple sclerosis lesion segmentation. In: MICCAI 2015. pp. 3–11. Springer (2015)
2.Crum, W., Camara, O., Hill, D.: Generalized Overlap Measures for Evaluation and Validation in Medical Image Analysis. IEEE TMI 25(11), 1451–1461 (nov 2006)
3.Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.M., Larochelle, H.: Brain tumor segmentation with deep neural networks. MIA 35, 18–31 (2017)
4.Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. MIA 36, 61–78 (feb 2017)
5.Lai, M.: Deep learning for medical image segmentation. arXiv:1505.02000 (2015)
6.Li, W., Wang, G., Fidon, L., Ourselin, S., Cardoso, M.J., Vercauteren, T.: On the compactness, efficiency, and representation of 3d convolutional networks: Brain parcellation as a pretext task. In: IPMI 2017
7.Menze, B.H.e.a.: The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE TMI 34(10), 1993–2024 (oct 2015)
8.Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV). pp. 565–571. IEEE (oct 2016)
9.Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: MICCAI. pp. 234–241. Springer (2015)
10.Zheng, Y., Liu, D., Georgescu, B., Nguyen, H., Comaniciu, D.: 3d deep learning for efficient and robust landmark detection in volumetric data. In: MICCAI 2015. pp. 565–572. Springer (2015)

Citation

MLA

Sudre, C. H., et al. “Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations”. Lecture Notes in Computer Science, Springer International Publishing, 2017, pp. 240–48, https://doi.org/10.1007/978-3-319-67558-9_28.

APA

Sudre, C. H., Li, W., Vercauteren, T., Ourselin, S., & Jorge Cardoso, M. (2017). Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations. In Lecture Notes in Computer Science (pp. 240–248). Springer International Publishing. https://doi.org/10.1007/978-3-319-67558-9_28

Chicago

Sudre, C. H., W. Li, T. Vercauteren, S. Ourselin, and M. Jorge Cardoso. 2017. “Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations”. In Lecture Notes in Computer Science. Springer International Publishing. https://doi.org/10.1007/978-3-319-67558-9_28.

Harvard

Sudre, C.H. et al. (2017) “Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations”, Lecture Notes in Computer Science. Springer International Publishing, pp. 240–248. Available at: https://doi.org/10.1007/978-3-319-67558-9_28.

Vancouver

1. Sudre CH, Li W, Vercauteren T, Ourselin S, Jorge Cardoso M (2017) Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations. In: Lecture Notes in Computer Science. Springer International Publishing, pp 240–248

BibTeX

@inbook{Sudre_2017, title={Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations}, ISBN={9783319675589}, ISSN={1611-3349}, url={http://dx.doi.org/10.1007/978-3-319-67558-9_28}, DOI={10.1007/978-3-319-67558-9_28}, booktitle={Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support}, publisher={Springer International Publishing}, author={Sudre, Carole H. and Li, Wenqi and Vercauteren, Tom and Ourselin, Sebastien and Jorge Cardoso, M.}, year={2017}, pages={240–248} }

Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF