Soft-NMS — Improving Object Detection with One Line of Code
Navaneeth BodlaBharat SinghRama ChellappaLarry S. Davis
Proposes a drop-in replacement for greedy non-maximum suppression that continuously decays overlapping detection scores, boosting object detector accuracy across standard benchmarks without retraining or added computational overhead.
Modern computer vision applications—such as autonomous driving, surveillance, and automated image indexing—depend heavily on accurate object detection. For decades, detection pipelines have relied on traditional non-maximum suppression to eliminate duplicate predictions around detected items. However, traditional suppression uses a strict overlap threshold that sets neighboring detection scores entirely to zero, which frequently causes real objects in crowded scenes to be missed and lowers detection accuracy.
This article proposes and evaluates "Soft-NMS," an alternative algorithm that continuously decays confidence scores as a function of bounding-box overlap rather than completely eliminating nearby detections. The goal is to retain true detections with recalibrated confidence scores while avoiding additional false positives.
To test this approach, the authors integrated linear and Gaussian score-decay formulations into leading detection frameworks, including Faster-RCNN, R-FCN, and Deformable-RFCN. They evaluated performance across standard computer vision benchmarks—specifically PASCAL VOC 2007 and MS-COCO—without retraining existing models or adding computational overhead, maintaining an identical quadratic computational complexity to traditional methods.
Empirical findings demonstrate consistent improvements across standard benchmarks. Soft-NMS improved average precision by 1.7 percentage points for Faster-RCNN and R-FCN on PASCAL VOC 2007. On the MS-COCO dataset, the method improved precision by 1.1 percentage points for Faster-RCNN, 1.3 percentage points for R-FCN, and pushed single-model Deformable-RFCN performance from 39.8% to 40.9%. The largest gains—between 3% and 6%—occurred in scenes with dense object clusters, such as herds of animals, while gains were smaller (around 0.5%) on single-stage detectors like SSD and YOLOv2 that generate fewer overlapping proposals. Across varying evaluation thresholds, Gaussian decay showed broad parameter stability around its standard setting.
These results establish that detection accuracy can be enhanced immediately without altering neural network architectures, retraining models, or increasing latency. Soft-NMS reduces missed detections in dense environments while preserving low false-positive rates, making it an efficient, low-risk upgrade for production systems that rely on multi-stage proposal detectors.
Organizations deploying object detection pipelines should replace standard non-maximum suppression with Soft-NMS, specifically using the Gaussian decay setting with its default parameter value. For single-stage detectors or domains with isolated objects, engineering teams should conduct localized pilot tests to determine if the modest gains justify deployment.
While confidence in these findings is high due to consistent performance across standard benchmarks and multiple architectures, the technique remains a greedy heuristic rather than a mathematically guaranteed global optimization. Decision-makers should note that benefits are heavily concentrated in crowded scenes with high object overlap and provide negligible impact on images with isolated, non-overlapping items.
- Paper: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, Shaoqing Ren et al. (2015). Introduces the standard Faster R-CNN architecture whose post-processing relies heavily on greedy Non-Maximum Suppression, establishing the core baseline that Soft-NMS seeks to improve.
- Paper: R-FCN: Object Detection via Region-based Fully Convolutional Networks, Jifeng Dai et al. (2016). Presents the Region-based Fully Convolutional Network (R-FCN) detector evaluated extensively as a primary testbed in the Soft-NMS paper.
- Paper: Deformable Convolutional Networks, Jifeng Dai et al. (2017). Provides the foundational deformable convolution mechanisms behind Deformable R-FCN, which Soft-NMS uses to push state-of-the-art detection performance.
- Paper: Fast R-CNN, Ross B. Girshick (2015). Formalizes the standard multi-stage detection pipeline and region-based suppression conventions that traditional NMS operates upon.
- Paper: SSD: Single Shot MultiBox Detector, W. Liu et al. (2015). Establishes a foundational single-stage detection framework that utilizes hard NMS filtering across dense multi-scale anchors.
- Paper: Rich feature hierarchies for accurate object detection and semantic segmentation, Ross Girshick et al. (2014). Introduces modern CNN-based object detection along with the classic greedy Non-Maximum Suppression post-processing step.
- Paper: Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression, Zhaohui Zheng et al. (2019). Extends bounding-box regression and suppression techniques by introducing DIoU-NMS, which uses center-point distance alongside overlap metrics to filter detections.
- Paper: YOLACT: Real-Time Instance Segmentation, Daniel Bolya et al. (2019). Introduces Fast NMS to parallelize duplicate box filtering on GPUs, offering an alternative efficiency-driven post-processing design for real-time instance segmentation.
- Paper: Cascade R-CNN: Delving Into High Quality Object Detection, Zhaowei Cai et al. (2017). Addresses box localization and thresholding issues through multi-stage sequential refinement rather than purely post-processing score decay.
- Paper: Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression, Hamid Rezatofighi et al. (2019). Tackles bounding-box evaluation and suppression limitations at the training loss level by formulating Generalized IoU as a direct geometric metric.
- Paper: FCOS: Fully Convolutional One-Stage Object Detection, Zhi Tian et al. (2019). Incorporates a continuous 'centerness' score to downweight low-quality predictions directly before suppression, complementing score-decay approaches.
- Paper: Objects as Points, Xingyi Zhou et al. (2019). Presents an anchor-free keypoint formulation that extracts local peak detections without requiring IoU-based NMS algorithms.
- Paper: End-to-End Object Detection with Transformers, Nicolas Carion et al. (2020). Replaces hand-crafted post-processing heuristics entirely by formulating object detection as a direct set prediction task with bipartite matching.
- Paper: MMDetection: Open MMLab Detection Toolbox and Benchmark, Kai Chen et al. (2019). Provides a modular detection benchmark and toolbox implementing modern post-processing methods including Soft-NMS.
- Paper: Object Detection With Deep Learning: A Review, Zhong-Qiu Zhao et al. (2018). Surveys the evolution of modern deep learning object detectors, contextualizing the role of post-processing components like NMS and Soft-NMS.
