Focal and Efficient IOU Loss for Accurate Bounding Box Regression

Yi-Fan ZhangWeiqiang RenZhang ZhangZhen JiaLiang WangTieniu Tan

article2021Neurocomputing1,943 citations

Proposes the Focal-EIOU loss function to accelerate convergence and improve localization accuracy in object detection by explicitly optimizing geometric discrepancies and concentrating training on high-quality bounding boxes.

Listen

Modern computer vision systems rely heavily on object detection to identify and locate items within images. A core technical challenge in these systems is bounding box regression, which refines predicted spatial boundaries to match actual target objects. Existing loss functions used to guide this optimization process suffer from two primary shortcomings: geometric formulation inefficiencies that slow down model training and harm localization precision, and an extreme sample imbalance where abundant low-quality anchor boxes generate harmful gradients that drown out the smaller volume of highly informative, high-quality candidates.

The article sets out to design and evaluate improved loss functions that directly resolve geometric optimization errors and sample imbalance in bounding box regression. Specifically, the authors introduce the Efficient Intersection over Union (EIOU) loss alongside a specialized sample-reweighting mechanism termed Focal-EIOU loss to demonstrate faster training convergence and superior object localization accuracy across standard visual benchmarks.

To evaluate this framework, the authors conducted controlled synthetic simulations analyzing optimization dynamics across diverse bounding box scales and aspect ratios. They subsequently integrated the proposed loss functions into multiple leading object detection models—including Faster R-CNN, Mask R-CNN, RetinaNet, ATSS, PAA, and DETR—and benchmarked localization performance against existing methods on the standard COCO 2017 dataset containing over 115,000 training images.

The investigation produced several key findings. First, EIOU loss explicitly and independently minimizes differences in overlap area, center point distance, and side dimensions, directly resolving the conflicting gradient and aspect ratio distortions found in earlier metrics. Second, synthetic experiments confirmed that Focal-EIOU achieves the fastest convergence speed while maximizing the proportion of high-quality regressed boxes. Third, on the COCO dataset, replacing standard baseline loss functions with Focal-EIOU yielded consistent accuracy gains across all evaluated detection architectures, delivering a 1.6% Average Precision improvement on RetinaNet (increasing from 35.9% to 37.5%) and boosting Faster R-CNN performance by up to 1.6% compared to the 0.1% to 0.72% gains achieved by prior methods.

These findings demonstrate that separating geometric regression factors while reweighting model focus toward high-quality proposals substantially enhances model precision and training efficiency without increasing architectural inference costs. By accelerating convergence and mitigating gradient noise, organizations developing vision systems can train more dependable detectors using fewer computational iterations.

Teams deploying vision systems should adopt Focal-EIOU loss as a straightforward upgrade over traditional loss functions in existing object detection pipelines. The authors recommend configuring the sample focusing parameter to moderate suppression levels (setting gamma to 0.5) and balancing loss weights around 2.5 to avoid over-suppressing informative gradients or destabilizing optimization schedules.

Practitioners should exercise caution in applications that prioritize tiny visual objects. The empirical analysis indicates that while Focal-EIOU excels at identifying medium and large elements, it exhibits slightly reduced sensitivity and confidence on very small items compared to standard overlap metrics. Overall confidence in the performance improvements across general object detection tasks remains high due to thorough validation across multiple architectures and standard benchmarks.

Horizon Robotics Institute of Automation, Chinese Academy of Sciences University of Chinese Academy of Sciences localization performance object localization optimization Focal-EIOU loss bounding box regression Efficient IOU Loss

arXiv: 2101.08158

Paper: Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression, Zhaohui Zheng et al. (2019). It introduces Distance-IoU (DIoU) and Complete-IoU (CIoU) losses, which EIOU explicitly decomposes and refines into separate side-length and center-point discrepancy terms.
Paper: Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression, Hamid Rezatofighi et al. (2019). It establishes Generalized IoU (GIoU) loss to handle non-overlapping bounding boxes, laying the foundational framework of IoU-based regression losses optimized in this work.
Paper: Focal Loss for Dense Object Detection, Tsung-Yi Lin et al. (2017). It introduces the Focal Loss mechanism to address sample imbalance during dense detector training, which is adapted in the source paper into a regression-specific Focal-EIOU loss.
Paper: Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection, Xiang Li et al. (2020). It explores quality estimation and continuous focal loss formulations for bounding box regression, providing essential context on addressing sample quality imbalances in localization.

No sufficiently relevant recommendations were found.

Cover for Focal and Efficient IOU Loss for Accurate Bounding Box Regression

Abstract

In object detection, bounding box regression (BBR) is a crucial step that determines the object localization performance. However, we find that most previous loss functions for BBR have two main drawbacks: (i) Both $\ell_n$ -norm and IOU-based loss functions are inefficient to depict the objective of BBR, which leads to slow convergence and inaccurate regression results. (ii) Most of the loss functions ignore the imbalance problem in BBR that the large number of anchor boxes which have small overlaps with the target boxes contribute most to the optimization of BBR. To mitigate the adverse effects caused thereby, we perform thorough studies to exploit the potential of BBR losses in this paper. Firstly, an Efficient Intersection over Union (EIOU) loss is proposed, which explicitly measures the discrepancies of three geometric factors in BBR, i.e., the overlap area, the central point and the side length. After that, we state the Effective Example Mining (EEM) problem and propose a regression version of focal loss to make the regression process focus on high-quality anchor boxes. Finally, the above two parts are combined to obtain a new loss function, namely Focal-EIOU loss. Extensive experiments on both synthetic and real datasets are performed. Notable superiorities on both the convergence speed and the localization accuracy can be achieved over other BBR losses.

Topics

localization performance object localization optimization real data sets

Keywords

Focal-EIOU loss bounding box regression Efficient IOU Loss localization accuracy Smooth L1 Loss object detection RetinaNet baseline

1 Introduction
2 Related Work
2.1 Loss Functions for BBR
2.2 Effective Example Mining
3 Efficient Intersection over Union Loss
3.1 Limitations of IOU-Based Losses
3.1.1 Limitations of IOU Loss
3.1.2 Limitations of Generalized IOU Loss
3.1.3 Limitations of Complete IOU Loss
3.2 The Proposed Method
4 Focal Loss For BBR
4.1 FocalL1 Loss
4.2 Focal-EIOU Loss
5 Experiments
5.1 Datasets and Evaluation Metrics
5.2 Implementation Details
5.3 Simulation Experiments
5.4 Ablation Experiments
5.4.1 Overall Ablation Studies
5.4.2 Effect of the Tradeoff Weight
5.4.3 Ablation Studies on FocalL1 Loss
5.4.4 Ablation Studies on Focal-EIOU Loss.
5.5 Incorporations with State-of-the-Arts
5.6 Discussion on Focal-EIOU loss and error set analysis
6 Conclusion
References

Knowls

Knowl 1 — Efficient Intersection over Union Loss
equation
The Efficient Intersection over Union (EIOU) loss measures the discrepancy between a predicted bounding box $B$ and a target bounding box $B^{gt}$ by decomposing the penalty into three geometric components: overlap area loss $\mathcal{L}_{\text{IOU}}$ , central point distance loss $\mathcal{L}_{\text{dis}}$ , and side length loss $\mathcal{L}_{\text{asp}}$ :

$\mathcal{L}_{\text{EIOU}} = \mathcal{L}_{\text{IOU}} + \mathcal{L}_{\text{dis}} + \mathcal{L}_{\text{asp}} = 1 - \text{IOU} + \frac{\rho^2(b, b^{gt})}{(w^c)^2 + (h^c)^2} + \frac{\rho^2(w, w^{gt})}{(w^c)^2} + \frac{\rho^2(h, h^{gt})}{(h^c)^2}$

where:
- $\text{IOU} = \frac{|B \cap B^{gt}|}{|B \cup B^{gt}|}$ is the intersection over union between $B$ and $B^{gt}$ .
- $b = (x, y)$ and $b^{gt} = (x^{gt}, y^{gt})$ denote the central coordinate points of $B$ and $B^{gt}$ .
- $\rho(b, b^{gt}) = \|b - b^{gt}\|_2$ denotes the Euclidean distance between center points.
- $\rho(w, w^{gt}) = |w - w^{gt}|$ and $\rho(h, h^{gt}) = |h - h^{gt}|$ denote the Euclidean distances between predicted and target box widths ( $w, w^{gt}$ ) and heights ( $h, h^{gt}$ ).
- $w^c$ and $h^c$ are the width and height of the smallest enclosing bounding box $C$ covering both $B$ and $B^{gt}$ .
By directly penalizing width and height discrepancies normalized by the enclosing dimensions, EIOU allows independent gradient updates for width and height.
Knowl 2 — Focal-EIOU Loss Formulation and Batch-Weighted Normalization
equation
To mitigate bounding box regression (BBR) sample imbalance, where numerous low-overlap anchor boxes dominate model gradients over informative high-quality anchors, the Focal-EIOU loss weights the EIOU loss by an IOU power factor:

$\mathcal{L}_{\text{Focal-EIOU}} = \text{IOU}^\gamma \mathcal{L}_{\text{EIOU}}$

where $\text{IOU} = \frac{|B \cap B^{gt}|}{|B \cup B^{gt}|}$ , $\mathcal{L}_{\text{EIOU}}$ is the Efficient IOU loss, and $\gamma \ge 0$ is a hyperparameter governing the inhibition degree of low-quality examples (default $\gamma = 0.5$ ).

To prevent slow convergence during early training phases caused by small weighting coefficients, the loss is normalized over all $n$ anchor-target pairs in a mini-batch by the sum of their assigned weights:

$\mathcal{L}_{\text{Focal-EIOU}} = \frac{\sum_{i=1}^n W_i \cdot \mathcal{L}_{\text{EIOU}, i}}{\sum_{i=1}^n W_i}$

where for the $i$ -th anchor-target pair, $\mathcal{L}_{\text{EIOU}, i}$ is its EIOU loss and $W_i = \text{IOU}_i^\gamma$ is its assigned weight.
Knowl 3 — FocalL1 Loss for Bounding Box Regression
equation
The FocalL1 loss is a bounding box regression loss designed to suppress gradients from low-quality anchor outliers while increasing the gradient contribution of well-regressed inliers. For a scalar bounding box coordinate error $x = |B_i - B_i^{gt}| \ge 0$ , the loss function $\mathcal{L}_f(x)$ is defined as:

$\mathcal{L}_f(x) = \begin{cases} -\frac{\alpha x^2 (2 \ln(\beta x) - 1)}{4}, & 0 < x \le 1 \\ -\alpha \ln(\beta) x + C, & x > 1 \end{cases}$

subject to $1/e \le \beta \le 1$ , with normalization parameter $\alpha = e\beta$ , and continuity constant $C = \frac{2\alpha \ln(\beta) + \alpha}{4}$ .

The corresponding gradient magnitude $g(x) = \frac{\partial \mathcal{L}_f}{\partial x}$ is:

$g(x) = \begin{cases} -\alpha x \ln(\beta x), & 0 < x \le 1 \\ -\alpha \ln(\beta), & x > 1 \end{cases}$

The gradient satisfies four criteria:
1. $g(x) \to 0$ as $x \to 0$ .
2. The gradient magnitude increases rapidly near small errors, reaching its maximum value of $1.0$ at $x^* = \frac{1}{e\beta}$ , and is suppressed at large errors.
3. The parameter $\beta$ flexibly controls the inhibition degree of large regression errors.
4. The gradient family is normalized to the range $(0, 1]$ .
For 4-coordinate regression with predicted box $B = (x, y, w, h)$ and target $B^{gt} = (x^{gt}, y^{gt}, w^{gt}, h^{gt})$ , total localization loss is:

$\mathcal{L}_{\text{Loc}} = \sum_{i \in \{x, y, w, h\}} \mathcal{L}_f(|B_i - B_i^{gt}|)$
Knowl 4 — Theoretical Limitations of the Aspect Ratio Formulation in CIOU Loss
theoretical result
The Complete IOU (CIOU) loss incorporates an aspect ratio penalty term $\alpha v$ , where:

$v = \frac{4}{\pi^2} \left(\arctan\frac{w^{gt}}{h^{gt}} - \arctan\frac{w}{h}\right)^2, \quad \alpha = \frac{v}{(1 - \text{IOU}) + v}$

Three analytical flaws impair this aspect ratio formulation:
1. Scale ambiguity: The variable $v$ measures only the ratio discrepancy rather than true dimensional errors. Any predicted box satisfying $w = k w^{gt}$ and $h = k h^{gt}$ for any scale factor $k \in \mathbb{R}^+$ yields $v = 0$ , even when the absolute dimensions differ substantially from the target.
2. Coupled opposing gradients: The partial derivatives of $v$ with respect to predicted width $w$ and height $h$ satisfy: $\frac{\partial v}{\partial w} = -\frac{h}{w} \frac{\partial v}{\partial h}$ Because $\frac{\partial v}{\partial w}$ and $\frac{\partial v}{\partial h}$ always have opposite signs, optimizing $v$ forces one dimension to decrease whenever the other increases. This creates conflicting optimization directions when both dimensions simultaneously need expansion ( $w < w^{gt}$ and $h < h^{gt}$ ) or simultaneous contraction ( $w > w^{gt}$ and $h > h^{gt}$ ).
3. Misguided trajectory: Minimizing $v$ can increase aspect ratio similarity by driving $(w, h)$ further away from $(w^{gt}, h^{gt})$ , slowing down convergence toward the ground-truth box size.
Knowl 5 — Gradient Vanishing Mechanism in Direct Focal Reweighting of IOU Losses
theoretical result
Directly substituting the EIOU loss value $\mathcal{L}_{\text{EIOU}}$ into the scalar error $x$ of the FocalL1 formulation $\mathcal{L}_f(x)$ degrades regression performance due to gradient vanishing on high-quality anchors.

For scalar coordinate regression with error $\ell_1(B_i) = |B_i - B_i^{gt}|$ , the gradient of the loss $\mathcal{L}_f(\ell_1(B_i))$ with respect to predicted box parameter $B_i$ is:

$\frac{\partial \mathcal{L}_f(\ell_1(B_i))}{\partial B_i} = \frac{\partial \mathcal{L}_f}{\partial \ell_1} \cdot \frac{\partial \ell_1}{\partial B_i}$

Here, $\frac{\partial \ell_1}{\partial B_i} \in \{-1, +1\}$ is constant, so $\frac{\partial \mathcal{L}_f}{\partial \ell_1}$ maintains adequate gradient magnitude even when errors are small.

In contrast, if $\ell_1(B_i)$ is replaced by $\mathcal{L}_{\text{EIOU}}(B, B^{gt})$ , the gradient becomes:

$\frac{\partial \mathcal{L}_f(\mathcal{L}_{\text{EIOU}})}{\partial B_i} = \frac{\partial \mathcal{L}_f}{\partial \mathcal{L}_{\text{EIOU}}} \cdot \frac{\partial \mathcal{L}_{\text{EIOU}}}{\partial B_i}$

As the predicted box nears the target box, $\mathcal{L}_{\text{EIOU}} \to 0$ , causing $\frac{\partial \mathcal{L}_{\text{EIOU}}}{\partial B_i} \to 0$ . Simultaneously, $\frac{\partial \mathcal{L}_f}{\partial \mathcal{L}_{\text{EIOU}}} \to 0$ for small regression errors. The product of these two near-zero quantities diminishes the total gradient excessively on well-aligned anchors, neutralizing the benefit of focal reweighting. Reweighting via $\text{IOU}^\gamma \mathcal{L}_{\text{EIOU}}$ resolves this because the weight $\text{IOU}^\gamma$ approaches $1$ (maximum weight) as overlap approaches unity.
Knowl 6 — Simulation Algorithm for Evaluating Bounding Box Regression Loss Convergence
algorithm
A synthetic bounding box regression experiment evaluates convergence speed and localization accuracy across loss functions by optimizing anchor boxes directly via gradient descent over $T = 200$ iterations with learning rate annealing.
Input: Number of target aspect ratios and scales $A \in \{3, 7\}$ , anchor set $\{\{B_{n,s}\}_{s=1}^S\}_{n=1}^N$ where $S = A \times A$ combinations of scales and aspect ratios at $N$ coordinate points, target boxes $\{B_i^{gt}\}_{i=1}^A$ centered at $(10, 10)$ with area $100$ .
Output: Regression error history $E \in \mathbb{R}^T$ and IOU history $I \in \mathbb{R}^{T \times N \times S \times A}$ .
$(T, E, I) \leftarrow (200, 0, 0)$
for $t = 1$ to $T$ do
    for $n = 1$ to $N$ do
        for $s = 1$ to $S$ do
            for $i = 1$ to $A$ do
                if $t \le 0.8T$ then
                     $\mu = 0.1$
                else if $t \le 0.9T$ then
                     $\mu = 0.01$
                else
                     $\mu = 0.001$
                end if
                 $\nabla B_{n,s}^{t-1} = \frac{\partial \mathcal{L}(B_{n,s}^{t-1}, B_i^{gt})}{\partial B_{n,s}^{t-1}}$
                 $B_{n,s}^t = B_{n,s}^{t-1} + \mu \nabla B_{n,s}^{t-1}$
                 $E(t) = E(t) + |B_{n,s}^t - B_i^{gt}|$
                 $I(t, n, s, i) = \text{IOU}(B_{n,s}^t, B_i^{gt})$
            end for
        end for
    end for
end for
return $E, I$
In Setup 1 ( $A=7, N=1000$ ), anchors are generated across a $20 \times 20$ region with 7 aspect ratios ( $1:4, 1:3, 1:2, 1:1, 2:1, 3:1, 4:1$ ) and 7 scales ( $50, 67, 75, 100, 133, 150, 200$ ). In Setup 2 ( $A=3, N=100$ ), anchors are localized in a $2.5 \times 2.5$ square around target $(10, 10)$ with 3 aspect ratios ( $1:3, 1:1, 3:1$ ) and 3 scales ( $50, 100, 150$ ).

Knowl 7 — Ablation Comparison of Loss Functions on COCO 2017 Object Detection

data/table

An ablation study evaluates the localization accuracy of various bounding box regression loss functions integrated into a RetinaNet detector with a ResNet-50-FPN backbone evaluated on the COCO val-2017 benchmark.

Method	$\text{AP}$	$\text{AP}_{50}$	$\text{AP}_{75}$	$\text{AP}_S$	$\text{AP}_M$	$\text{AP}_L$
Baseline (SmoothL1)	35.9	55.2	38.4	21.2	39.5	48.4
IOU	36.5	55.6	38.9	20.9	40.1	48.0
GIOU	36.5	55.6	39.0	20.7	40.2	48.2
CIOU	36.7	55.7	39.2	20.6	40.4	49.0
FocalL1	36.5	55.8	38.9	21.2	39.8	48.8
EIOU	37.0	55.7	39.5	20.7	40.5	49.5
Focal-EIOU (v1)	36.8	55.4	39.5	20.9	40.0	49.1
Focal-EIOU	37.5	56.1	40.0	21.1	40.9	49.8

The results demonstrate:

EIOU outperforms CIOU (+0.3% AP) and baseline SmoothL1 (+1.1% AP) by replacing the coupled aspect-ratio penalty with independent width and height penalties.
FocalL1 improves baseline SmoothL1 by +0.6% AP.
Focal-EIOU (v1), which plugs EIOU directly into the FocalL1 formulation, underperforms standard EIOU (36.8% vs. 37.0% AP) due to gradient attenuation.
Focal-EIOU with $\text{IOU}^\gamma$ reweighting achieves 37.5% AP (+1.6% AP over baseline), with substantial gains on medium ( $\text{AP}_M = 40.9\%$ ) and large objects ( $\text{AP}_L = 49.8\%$ ).

Knowl 8 — Object Detection Performance Improvements Across Multiple Detectors on COCO 2017

data/table

Replacing the default bounding box regression loss functions in multiple detector architectures with Focal-EIOU yields consistent Average Precision gains on the COCO val-2017 benchmark across two-stage, one-stage, anchor-free/adaptive, and transformer-based models.

Method	Backbone	$\text{AP}$	$\text{AP}_{50}$	$\text{AP}_{75}$	$\text{AP}_S$	$\text{AP}_M$	$\text{AP}_L$
Faster R-CNN	ResNet-50-FPN	37.3	58.2	40.3	21.3	40.9	48.0
Faster R-CNN*	ResNet-50-FPN	38.9	59.1	42.4	21.2	41.1	50.2
Faster R-CNN	ResNeXt-101-32x4d-FPN	41.2	62.1	45.1	24.0	45.5	53.5
Faster R-CNN*	ResNeXt-101-32x4d-FPN	42.4	63.1	46.8	24.2	46.3	54.1
Mask R-CNN	ResNet-50-FPN	38.2	58.8	41.4	21.9	40.9	49.5
Mask R-CNN*	ResNet-50-FPN	39.6	59.3	41.7	22.4	41.5	51.1
Mask R-CNN	ResNeXt-101-32x4d-FPN	41.9	62.5	45.9	24.4	46.3	54.0
Mask R-CNN*	ResNeXt-101-32x4d-FPN	43.0	63.1	46.1	24.4	47.3	56.1
RetinaNet	ResNet-50-FPN	35.9	55.2	38.4	21.2	39.5	48.4
RetinaNet*	ResNet-50-FPN	37.5	56.1	40.0	21.3	40.9	49.8
RetinaNet	ResNeXt-101-32x4d-FPN	40.8	60.9	43.7	22.9	44.5	54.6
RetinaNet*	ResNeXt-101-32x4d-FPN	41.8	61.4	44.7	21.7	45.0	55.2
ATSS	ResNet-50-FPN	39.1	57.6	42.1	22.9	42.8	51.1
ATSS*	ResNet-50-FPN	39.7	57.9	45.7	22.6	43.2	51.8
ATSS	ResNeXt-64x4d-101-DCN	50.7	68.9	56.3	33.2	52.9	62.2
ATSS*	ResNeXt-64x4d-101-DCN	51.4	69.3	57.3	32.8	53.4	64.2
PAA	ResNet-50-FPN	40.3	57.6	43.9	23.0	44.9	54.0
PAA*	ResNet-50-FPN	40.8	57.9	44.7	22.9	45.3	54.9
PAA	ResNeXt-64x4d-101-DCN	51.4	69.7	57.0	34.0	53.8	64.0
PAA*	ResNeXt-64x4d-101-DCN	52.3	70.2	58.1	33.2	54.3	66.2
DETR	ResNet-50	42.0	62.4	44.2	20.5	45.8	61.1
DETR*	ResNet-50	43.2	62.5	45.3	21.1	46.1	62.7

Models marked with an asterisk (*) use Focal-EIOU loss instead of their original bounding box regression loss functions. Focal-EIOU improves Faster R-CNN (ResNet-50) by +1.6% AP, Mask R-CNN (ResNet-50) by +1.4% AP, RetinaNet (ResNet-50) by +1.6% AP, and DETR (ResNet-50) by +1.2% AP. Improvements on ATSS and PAA (+0.5% to +0.9% AP) are smaller because both architectures already employ intrinsic loss reweighting (centerness and predicted IOU scores).

Knowl 9 — Hyperparameter Optimization and Reweighting Variations in Focal Regression Losses
empirical result
Empirical ablation experiments on the COCO val-2017 dataset identify optimal hyperparameter configurations and compare reweighting formulations for bounding box regression:
1. Trade-off Loss Weight: Varying the overall BBR loss weight from $1.0$ to $3.0$ shows that a weight of $2.5$ yields peak performance (37.0% AP for EIOU), whereas weights exceeding $2.5$ cause performance degradation due to outlier gradient overamplification.
2. FocalL1 Parameter $\beta$ : In FocalL1 regression, tuning $\beta \in [0.5, 0.9]$ achieves optimal balance at $\beta = 0.8$ , obtaining 36.5% AP compared to 35.9% for baseline SmoothL1 and 36.3% for BalancedL1.
3. Focal-EIOU Reweighting Parameter $\gamma$ : For Focal-EIOU ( $\mathcal{L}_{\text{Focal-EIOU}} = \text{IOU}^\gamma \mathcal{L}_{\text{EIOU}}$ ), $\gamma = 0.5$ achieves the optimal trade-off with 37.5% AP. Setting $\gamma > 1.0$ (e.g., $\gamma = 2.0$ ) over-suppresses harder examples and degrades AP below baseline.
4. Focal Classification Reweighting Variant ( $\mathcal{L}_{\text{Focal-EIOU}^*}$ ): Using the standard classification focal loss form $\mathcal{L}_{\text{Focal-EIOU}^*} = -(1 - \text{IOU})^\gamma \log(\text{IOU}) \mathcal{L}_{\text{EIOU}}$ causes AP to decrease sharply as $\gamma$ increases, confirming that extreme suppression of hard examples is detrimental to bounding box regression.
Knowl 10 — Scale Sensitivity and Small Object Limitation of Focal-EIOU Loss
limitation
While Focal-EIOU loss achieves substantial performance improvements on medium and large objects, error analysis reveals scale-dependent limitations:
- Small Object Degradation: On small objects, Focal-EIOU loss can be slightly inferior to standard IOU loss, occasionally assigning lower confidence scores or lower quality bounding boxes to small objects (e.g., on COCO val-2017 RetinaNet ResNet-50, small object precision $\text{AP}_S$ is $21.1\%$ for Focal-EIOU compared to $21.2\%$ for baseline SmoothL1 and FocalL1; on ATSS ResNet-50, $\text{AP}_S$ drops from $22.9\%$ to $22.6\%$ ).
- Sensitivity to Large Objects: Focal-EIOU is sensitive to large spatial elements, which occasionally leads the detector to predict spurious or misaligned bounding boxes near large background or object areas.
- Concentration of Gains on Medium and Large Scales: The method's performance gains are predominantly concentrated in medium and large objects (e.g., $\text{AP}_M$ increases from $39.5\%$ to $40.9\%$ and $\text{AP}_L$ increases from $48.4\%$ to $49.8\%$ on RetinaNet ResNet-50).

Coverage note — None was omitted; all key contributions including the EIOU loss, FocalL1 loss, Focal-EIOU loss, theoretical analyses, simulation algorithm, benchmark tables, and limitations are covered.

References

1.Sara Beery, Guanhang Wu, V. Rathod, Ronny Votel, and Jonathan Huang. Context r-cnn: Long term temporal context for per-camera object detection. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13072–13082, 2020. 2
2.Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving into high quality object detection. In CVPR, 2018. 1
3.Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213–229. Springer, 2020. 1, 2, 8
4.Kean Chen, Weiyao Lin, Jianguo Li, John See, Ji Wang, and Junni Zou. Ap-loss for accurate one-stage object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2
5.Zhiming Chen, Kean Chen, Weiyao Lin, John See, Hui Yu, Yan Ke, and Cong Yang. Piou loss: Towards accurate oriented object detection in complex environments, 2020. 2
6.P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9):1627–1645, 2010. 2
7.Spyros Gidaris and Nikos Komodakis. Locnet: Improving localization accuracy for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. 2
8.Ross Girshick. Fast r-cnn. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2015. 1, 2
9.Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014. 2
10.Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017. 1, 2, 8
11.K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9):1904–1916, 2015. 2
12.Kang Kim and Hee Seok Lee. Probabilistic anchor assignment with iou prediction for object detection. In ECCV, 2020. 2, 6, 8
13.Sven Kosub. A note on the triangle inequality for the jaccard distance. Pattern Recognition Letters, 120:36 – 38, 2019. 3
14.Xiang Li, Wenhai Wang, Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li, Jinhui Tang, and Jian Yang. Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection. arXiv preprint arXiv:2006.04388, 2020. 8
15.Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017. 1, 2, 7, 8
16.Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision – ECCV 2014, pages 740–755, Cham, 2014. Springer International Publishing. 2, 5
17.Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg. Ssd: Single shot multibox detector. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016, pages 21–37, Cham, 2016. Springer International Publishing. 2
18.Jiangmiao Pang, Kai Chen, Jianping Shi, Huajun Feng, Wanli Ouyang, and Dahua Lin. Libra r-cnn: Towards balanced learning for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019. 2
19.Jiangmiao Pang, Kai Chen, Jianping Shi, Huajun Feng, Wanli Ouyang, and Dahua Lin. Libra r-cnn: Towards balanced learning for object detection. In IEEE Conference on Computer Vision and Pattern Recognition, 2019. 2, 4, 7
20.Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017. 5
21.Q. Qian, L. Chen, H. Li, and R. Jin. Dr loss: Improving object detection by distributional ranking. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12161–12169, Los Alamitos, CA, USA, jun 2020. IEEE Computer Society. 2
22.Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. 2
23.Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 91–99. Curran Associates, Inc., 2015. 2, 8
24.Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized intersection over union: A metric and a loss for bounding box regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 2, 3, 8
25.Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. 2
26.Yue Wu, Yinpeng Chen, Lu Yuan, Zicheng Liu, Lijuan Wang, Hongzhi Li, and Yun Fu. Rethinking classification and localization for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020. 2
27.Jiahui Yu, Yuning Jiang, Zhangyang Wang, Zhimin Cao, and Thomas Huang. Unitbox: An advanced object detection network. In Proceedings of the 24th ACM International Conference on Multimedia, MM ’16, page 516–520, New York, NY, USA, 2016. Association for Computing Machinery. 1, 2, 3
28.Hongkai Zhang, Hong Chang, Bingpeng Ma, Naiyan Wang, and Xilin Chen. Dynamic R-CNN: Towards high quality object detection via dynamic training. arXiv preprint arXiv:2004.06002, 2020. 1, 2, 4
29.Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z. Li. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In CVPR, 2020. 2, 8
30.Ping Wang Zhaohui Zheng, Jinze Li Wei Liu, and Dongwei Ren Rongguang Ye. Distance-iou loss: Faster and better learning for bounding box regression. In The AAAI Conference on Artificial Intelligence (AAAI), 2020. 2, 3, 8

Citation

MLA

Zhang, Y.-F., et al. “Focal and Efficient IOU Loss for Accurate Bounding Box Regression”. arXiv, 2021, http://arxiv.org/abs/2101.08158v2.

APA

Zhang, Y.-F., Ren, W., Zhang, Z., Jia, Z., Wang, L., & Tan, T. (2021). Focal and Efficient IOU Loss for Accurate Bounding Box Regression. arXiv. http://arxiv.org/abs/2101.08158v2

Chicago

Zhang, Y.-F., W. Ren, Z. Zhang, Z. Jia, L. Wang, and T. Tan. 2021. “Focal and Efficient IOU Loss for Accurate Bounding Box Regression”. arXiv. http://arxiv.org/abs/2101.08158v2.

Harvard

Zhang, Y.-F. et al. (2021) “Focal and Efficient IOU Loss for Accurate Bounding Box Regression”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2101.08158v2.

Vancouver

1. Zhang Y-F, Ren W, Zhang Z, Jia Z, Wang L, Tan T (2021) Focal and Efficient IOU Loss for Accurate Bounding Box Regression. arXiv

BibTeX

@article{zhang2021focal,
  title = {Focal and Efficient IOU Loss for Accurate Bounding Box Regression},
  author = {Zhang, Yi-Fan and Ren, Weiqiang and Zhang, Zhang and Jia, Zhen and Wang, Liang and Tan, Tieniu},
  year = {2021},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2101.08158v2},
  eprint = {2101.08158}
}

Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Focal and Efficient IOU Loss for Accurate Bounding Box Regression

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Efficient Intersection over Union Loss

Knowl 2 — Focal-EIOU Loss Formulation and Batch-Weighted Normalization

Knowl 3 — FocalL1 Loss for Bounding Box Regression

Knowl 4 — Theoretical Limitations of the Aspect Ratio Formulation in CIOU Loss

Knowl 5 — Gradient Vanishing Mechanism in Direct Focal Reweighting of IOU Losses

Knowl 6 — Simulation Algorithm for Evaluating Bounding Box Regression Loss Convergence