Back

YOLOv3: An Incremental Improvement

Joseph RedmonAli Farhadi

article2018arXiv26,035 citations

Presents an updated real-time object detection model that matches the accuracy of leading detectors while operating more than three times faster across multiple spatial scales.

Listen

YOLOv3 introduces a series of targeted updates to the earlier YOLO object detectors, primarily by adopting proven components from other systems and training a new feature-extraction network. The work addresses the ongoing need for object detectors that deliver both high accuracy and real-time speed on standard hardware, a requirement that has grown with expanding use of vision systems in surveillance, autonomous systems, and content analysis.

The authors set out to document incremental refinements that raise detection quality while preserving the original emphasis on fast inference. They evaluated the resulting model on the COCO benchmark using both the conventional AP50 metric and the stricter COCO-style average precision across IoU thresholds, then compared runtimes against contemporary one-stage and two-stage detectors on the same Titan X or M40 GPUs.

The new Darknet-53 backbone, combined with multi-scale feature prediction and logistic objectness scoring, produces competitive results at substantially lower latency. At 320 × 320 resolution the detector reaches 28.2 mAP in 22 ms, matching SSD accuracy while running three times faster; on the AP50 metric it attains 57.9 in 51 ms, nearly identical to RetinaNet yet 3.8 times quicker. Small-object performance improved markedly relative to prior YOLO versions, although precision on medium and large objects declined and overall COCO AP remained below the best two-stage systems.

These outcomes indicate that YOLOv3 supplies a practical operating point for applications where throughput matters more than marginal gains on strict localization metrics. The speed advantage reduces hardware costs and power draw for edge or high-volume deployments, yet the model’s weaker performance at higher IoU thresholds suggests it may require post-processing or complementary methods when precise bounding-box alignment is essential. The authors also note that widespread adoption of such detectors carries societal risks if the technology is applied without regard for privacy or misuse.

Further work should focus on closing the remaining gap on strict localization, testing the detector on additional datasets that contain heavy label overlap, and exploring whether alternative evaluation protocols better reflect real-world utility. Results rest on a single benchmark and a limited set of ablation trials; broader validation across domains and hardware would increase confidence before large-scale deployment.

University of Washingtondetection algorithmssdsolid state drivesYOLOv3 methodobject detectionupsampled features
arXiv: 1804.02767pjreddie/darknet
  • Paper: You Only Look Once: Unified, Real-Time Object Detection, Joseph Redmon et al. (2016). Reading the original YOLO paper provides the foundational single-stage regression architecture and grid prediction concepts directly built upon and updated in YOLOv3.
  • Paper: YOLO9000: Better, Faster, Stronger, J. Redmon et al. (2016). YOLOv2 introduces crucial improvements like anchor boxes, batch normalization, and multi-scale training that serve as direct technical prerequisites for the architectural refinements in YOLOv3.
  • Paper: Feature Pyramid Networks for Object Detection, Tsung-Yi Lin et al. (2017). Feature Pyramid Networks establish the multi-scale feature fusion and skip-connection methodology that YOLOv3 adapts across its three output scales to dramatically improve small-object detection.
Cover for YOLOv3: An Incremental Improvement

Abstract

We present some updates to YOLO! We made a bunch of little design changes to make it better. We also trained this new network that's pretty swell. It's a little bigger than last time but more accurate. It's still fast though, don't worry. At 320x320 YOLOv3 runs in 22 ms at 28.2 mAP, as accurate as SSD but three times faster. When we look at the old .5 IOU mAP detection metric YOLOv3 is quite good. It achieves 57.9 mAP@50 in 51 ms on a Titan X, compared to 57.5 mAP@50 in 198 ms by RetinaNet, similar performance but 3.8x faster. As always, all the code is online at this https URL

Topics

detection algorithmssdsolid state drives

Keywords

YOLOv3 methodobject detectionupsampled featurespyramid networksaverage precisioninference timeanchor boxbounding box

Table of Contents

  • 1. Introduction
  • 2. The Deal
  • 2.1. Bounding Box Prediction
  • 2.2. Class Prediction
  • 2.3. Predictions Across Scales
  • 2.4. Feature Extractor
  • 2.5. Training
  • 3. How We Do
  • 4. Things We Tried That Didn't Work
  • 5. What This All Means
  • Rebuttal

Knowls

  1. Knowl 1 — Bounding Box Coordinate and Objectness Parameterization in YOLOv3

    model/method

    YOLOv3 predicts bounding box spatial coordinates and an objectness score for each candidate box using anchor box priors (dimension clusters).

    For a grid cell offset from the top-left corner of the image by coordinates (cx,cy)(c_x, c_y) and an anchor box prior with width pwp_w and height php_h, the network outputs four coordinate offsets (tx,ty,tw,th)(t_x, t_y, t_w, t_h). The predicted bounding box coordinates (bx,by,bw,bh)(b_x, b_y, b_w, b_h) are computed as:

    bx=σ(tx)+cxb_x = \sigma(t_x) + c_x by=σ(ty)+cyb_y = \sigma(t_y) + c_y bw=pwetwb_w = p_w e^{t_w} bh=phethb_h = p_h e^{t_h}

    where σ(⋅)\sigma(\cdot) is the logistic sigmoid function σ(z)=11+e−z\sigma(z) = \frac{1}{1 + e^{-z}}. Training coordinates optimizes a sum of squared error (SSE) loss against inverted ground-truth targets (t^x,t^y,t^w,t^h)(\hat{t}_x, \hat{t}_y, \hat{t}_w, \hat{t}_h).

    Objectness is predicted via logistic regression:

    • A ground-truth object is assigned to exactly one bounding box prior: the prior that exhibits the highest Intersection over Union (IoU) with that ground-truth box. The target objectness score for this prior is 11.
    • If an anchor prior is not the best match for a ground-truth box but overlaps it with an IoU>0.5\text{IoU} > 0.5, its prediction is ignored and incurs no loss.
    • Priors not assigned to any ground-truth object incur only objectness loss (with target 00) and no coordinate or classification loss.

  2. Knowl 2 — Multi-Scale Prediction and Anchor Allocation in YOLOv3

    model/method

    YOLOv3 performs bounding box predictions at three distinct scales using a top-down feature aggregation structure inspired by Feature Pyramid Networks.

    From the base convolutional feature extractor, several convolutional layers are appended to predict an output tensor at the coarsest scale. To predict at the next finer scale, the feature map from two layers prior to the output is upsampled by a factor of 2×2\times and channel-wise concatenated with an earlier, higher-resolution feature map from the backbone. Additional convolutional layers process this concatenated representation to produce the second-scale prediction tensor. This upsampling and concatenation sequence is repeated once more to generate predictions at the third, finest scale.

    At each scale, the network predicts 3 bounding boxes per spatial grid location. For an N×NN \times N grid, the output tensor shape is N×N×[3×(4+1+C)]N \times N \times [3 \times (4 + 1 + C)], where 44 corresponds to coordinate offsets (tx,ty,tw,th)(t_x, t_y, t_w, t_h), 11 is the objectness confidence, and CC is the number of class probabilities (yielding 3×(4+1+80)=2553 \times (4 + 1 + 80) = 255 output channels for the 80-class COCO dataset).

    Nine bounding box anchor priors are determined via kk-means clustering on ground-truth bounding box dimensions and partitioned across the three scales in descending order of receptive field size:

    • Coarsest scale (large objects): (116×90)(116 \times 90), (156×198)(156 \times 198), (373×326)(373 \times 326)
    • Intermediate scale (medium objects): (30×61)(30 \times 61), (62×45)(62 \times 45), (59×119)(59 \times 119)
    • Finest scale (small objects): (10×13)(10 \times 13), (16×30)(16 \times 30), (33×23)(33 \times 23)

  3. Knowl 3 — Multilabel Class Prediction with Independent Logistic Classifiers

    model/method

    YOLOv3 replaces the softmax activation function conventionally used for multi-class classification with independent logistic classifiers for each class label.

    During training, class predictions are supervised using binary cross-entropy (BCE) loss independently for each class. This formulation eliminates the mutual exclusivity assumption inherent to softmax, allowing candidate bounding boxes to be assigned multiple class labels simultaneously. This capability accommodates complex datasets with overlapping or hierarchical class taxonomies (e.g., categories where an entity is labeled as both 'Woman' and 'Person').

  4. Knowl 4 — Darknet-53 Backbone Architecture

    model/method

    Darknet-53 is a 53-layer convolutional neural network backbone designed for feature extraction, combining alternating 3×33 \times 3 and 1×11 \times 1 convolutional layers with residual shortcut connections.

    Spatial downsampling is performed using 3×33 \times 3 convolutions with stride 2 rather than pooling layers. Residual blocks consist of a 1×11 \times 1 convolution (halving channel dimensions) followed by a 3×33 \times 3 convolution (doubling channel dimensions) and an identity addition shortcut.

    Type Filters Size Stride Output
    Convolutional 32 3×33 \times 3 1 256×256256 \times 256
    Convolutional 64 3×33 \times 3 2 128×128128 \times 128
    1×1\times Residual 64 - - 128×128128 \times 128
    Convolutional 128 3×33 \times 3 2 64×6464 \times 64
    2×2\times Residual 128 - - 64×6464 \times 64
    Convolutional 256 3×33 \times 3 2 32×3232 \times 32
    8×8\times Residual 256 - - 32×3232 \times 32
    Convolutional 512 3×33 \times 3 2 16×1616 \times 16
    8×8\times Residual 512 - - 16×1616 \times 16
    Convolutional 1024 3×33 \times 3 2 8×88 \times 8
    4×4\times Residual 1024 - - 8×88 \times 8
    Avgpool - Global - 1×11 \times 1
    Connected 1000 - - 1000
    Softmax - - - 1000

  5. Knowl 5 — ImageNet Classification Performance of Darknet-53

    data/table

    Darknet-53 achieves top-1 and top-5 classification accuracy comparable to ResNet-152 on ImageNet while requiring fewer floating-point operations and delivering substantially higher inference throughput (frames per second) and GPU utilization (measured in billion floating point operations per second, BFLOP/s).

    Evaluations were conducted with single-crop 256×256256 \times 256 input resolution on an NVIDIA Titan X GPU under identical training settings:

    Backbone Top-1 (%) Top-5 (%) Bn Ops BFLOP/s FPS
    Darknet-19 74.1 91.8 7.29 1246 171
    ResNet-101 77.1 93.7 19.7 1039 53
    ResNet-152 77.6 93.8 29.4 1090 37
    Darknet-53 77.2 93.8 18.7 1457 78

    Darknet-53 is 1.5×1.5\times faster than ResNet-101 with higher accuracy, and achieves equivalent accuracy to ResNet-152 while running 2×2\times faster.

  6. Knowl 6 — COCO Object Detection Performance and Inference Latency

    data/table

    On the MS COCO benchmark, YOLOv3 matches the detection accuracy of single-shot detectors (such as SSD and DSSD) at higher inference speeds, and attains comparable performance on the AP50\text{AP}_{50} metric to state-of-the-art detectors while operating at significantly lower latency.

    Method Backbone AP\text{AP} AP50\text{AP}_{50} AP75\text{AP}_{75} APS\text{AP}_S APM\text{AP}_M APL\text{AP}_L
    Two-stage
    Faster R-CNN+++ ResNet-101-C4 34.9 55.7 37.4 15.6 38.7 50.9
    Faster R-CNN w FPN ResNet-101-FPN 36.2 59.1 39.0 18.2 39.0 48.2
    Faster R-CNN by G-RMI Inception-ResNet-v2 34.7 55.5 36.7 13.5 38.1 52.0
    Faster R-CNN w TDM Inception-ResNet-v2-TDM 36.8 57.7 39.2 16.2 39.8 52.1
    One-stage
    YOLOv2 Darknet-19 21.6 44.0 19.2 5.0 22.4 35.5
    SSD513 ResNet-101-SSD 31.2 50.4 33.3 10.2 34.5 49.8
    DSSD513 ResNet-101-DSSD 33.2 53.3 35.2 13.0 35.4 51.1
    RetinaNet-50-500 ResNet-50-FPN 32.5 50.9 - - - -
    RetinaNet-101-500 ResNet-101-FPN 34.4 53.1 - - - -
    RetinaNet-101-800 ResNet-101-FPN 39.1 59.1 42.3 21.8 42.7 50.2
    RetinaNet ResNeXt-101-FPN 40.8 61.1 44.1 24.1 44.2 51.2
    YOLOv3-320 Darknet-53 28.2 51.5 - - - -
    YOLOv3-416 Darknet-53 31.0 55.3 - - - -
    YOLOv3-608 Darknet-53 33.0 57.9 34.4 18.3 35.4 41.9

    Inference latencies on a Titan X / M40 GPU:

    • YOLOv3-320: 28.2 mAP28.2\text{ mAP} / 51.5 AP5051.5\text{ AP}_{50} in 22 ms22\text{ ms}
    • YOLOv3-416: 31.0 mAP31.0\text{ mAP} / 55.3 AP5055.3\text{ AP}_{50} in 29 ms29\text{ ms}
    • YOLOv3-608: 33.0 mAP33.0\text{ mAP} / 57.9 AP5057.9\text{ AP}_{50} in 51 ms51\text{ ms}

    For comparison, RetinaNet-101-800 achieves 57.5 AP5057.5\text{ AP}_{50} in 198 ms198\text{ ms} (3.8×3.8\times slower than YOLOv3-608), and SSD513 achieves 50.4 AP5050.4\text{ AP}_{50} in 125 ms125\text{ ms}.

  7. Knowl 7 — Localization and Object Scale Performance Profile of YOLOv3

    empirical result

    YOLOv3 exhibits distinct performance characteristics across IoU evaluation thresholds and object scales:

    1. IoU Sensitivity: While YOLOv3 scores competitively at moderate intersection thresholds (e.g., 57.9 AP5057.9\text{ AP}_{50} for YOLOv3-608), its accuracy degrades at stricter localization thresholds (34.4 AP7534.4\text{ AP}_{75}), indicating that the model successfully identifies rough object boundaries but struggles with precise bounding box edge alignment.
    2. Small Object Detection: Due to multi-scale feature concatenation, YOLOv3 achieves strong detection performance on small objects (APS=18.3\text{AP}_S = 18.3, compared to 5.05.0 in YOLOv2 and 10.210.2 in SSD513).
    3. Large Object Detection: YOLOv3 demonstrates comparatively lower performance on larger objects (APL=41.9\text{AP}_L = 41.9, compared to 51.151.1 for DSSD513 and 50.250.2 for RetinaNet).

  8. Knowl 8 — Unsuccessful Design Variations in YOLOv3

    empirical result

    Several standard detection techniques and architectural variants were experimentally evaluated during YOLOv3 development but failed to improve performance:

    • Linear anchor box offset predictions: Parameterizing (x,y)(x, y) coordinate offsets as linear multiples of the anchor box width and height using a linear activation (standard anchor box mechanism) reduced training stability and detector accuracy.
    • Direct linear (x,y)(x, y) prediction: Replacing the sigmoid/logistic activation σ(tx),σ(ty)\sigma(t_x), \sigma(t_y) with direct linear predictions degraded mAP by several points.
    • Focal Loss: Incorporating focal loss caused an approximate 2-point drop in mAP. The authors hypothesize that YOLOv3 is naturally robust to the foreground-background class imbalance addressed by focal loss because it predicts objectness confidence separately from conditional class probabilities.
    • Dual IoU training thresholds: Adopting Faster R-CNN style dual thresholds (positive if IoU≥0.7\text{IoU} \ge 0.7, ignored if IoU∈[0.3,0.7]\text{IoU} \in [0.3, 0.7], negative if IoU<0.3\text{IoU} < 0.3) did not yield viable performance improvements over YOLOv3's single-match and 0.50.5 ignore-threshold strategy.

Coverage note — None was omitted; informal commentary regarding metric philosophies and ethics in computer vision was excluded as non-technical discourse.

References

  1. 1.Analogy. Wikipedia, Mar 2018. 1
  2. 2.M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303–338, 2010. 6
  3. 3.C.-Y. Fu, W. Liu, A. Ranga, A. Tyagi, and A. C. Berg. Dssd: Deconvolutional single shot detector. arXiv preprint arXiv:1701.06659, 2017. 3
  4. 4.D. Gordon, A. Kembhavi, M. Rastegari, J. Redmon, D. Fox, and A. Farhadi. Iqa: Visual question answering in interactive environments. arXiv preprint arXiv:1712.03316, 2017. 1
  5. 5.K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 3
  6. 6.J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, et al. Speed/accuracy trade-offs for modern convolutional object detectors. 3
  7. 7.I. Krasin, T. Duerig, N. Alldrin, V. Ferrari, S. Abu-El-Haija, A. Kuznetsova, H. Rom, J. Uijlings, S. Popov, A. Veit, S. Belongie, V. Gomes, A. Gupta, C. Sun, G. Chechik, D. Cai, Z. Feng, D. Narayanan, and K. Murphy. Openimages: A public dataset for large-scale multi-label and multi-class image classification. Dataset available from https://github.com/openimages, 2017. 2
  8. 8.T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2117–2125, 2017. 2, 3
  9. 9.T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Doll'ar. Focal loss for dense object detection. arXiv preprint arXiv:1708.02002, 2017. 1, 3, 4
  10. 10.T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll'ar, and C. L. Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 2
  11. 11.W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg. Ssd: Single shot multibox detector. In European conference on computer vision, pages 21–37. Springer, 2016. 3
  12. 12.I. Newton. Philosophiae naturalis principia mathematica. William Dawson & Sons Ltd., London, 1687. 1
  13. 13.J. Parham, J. Crall, C. Stewart, T. Berger-Wolf, and D. Rubenstein. Animal population censusing at scale with citizen science and photographic identification. 2017. 4
  14. 14.J. Redmon. Darknet: Open source neural networks in c. http://pjreddie.com/darknet/, 2013–2016. 3
  15. 15.J. Redmon and A. Farhadi. Yolo9000: Better, faster, stronger. In Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pages 6517–6525. IEEE, 2017. 1, 2, 3
  16. 16.J. Redmon and A. Farhadi. Yolov3: An incremental improvement. arXiv, 2018. 4
  17. 17.S. Ren, K. He, R. Girshick, and J. Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497, 2015. 2
  18. 18.O. Russakovsky, L.-J. Li, and L. Fei-Fei. Best of both worlds: human-machine collaboration for object annotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2121–2131, 2015. 4
  19. 19.M. Scott. Smart camera gimbal bot scanlime:027, Dec 2017. 4
  20. 20.A. Shrivastava, R. Sukthankar, J. Malik, and A. Gupta. Beyond skip connections: Top-down modulation for object detection. arXiv preprint arXiv:1612.06851, 2016. 3
  21. 21.C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. 2017. 3

Citation

MLA
Redmon, J., and A. Farhadi. “YOLOv3: An Incremental Improvement”. arXiv, 2018, http://arxiv.org/abs/1804.02767v1.
APA
Redmon, J., & Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv. http://arxiv.org/abs/1804.02767v1
Chicago
Redmon, J., and A. Farhadi. 2018. “YOLOv3: An Incremental Improvement”. arXiv. http://arxiv.org/abs/1804.02767v1.
Harvard
Redmon, J. and Farhadi, A. (2018) “YOLOv3: An Incremental Improvement”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1804.02767v1.
Vancouver
1. Redmon J, Farhadi A (2018) YOLOv3: An Incremental Improvement. arXiv

BibTeX

@article{redmon2018yolov3,
  title = {YOLOv3: An Incremental Improvement},
  author = {Redmon, Joseph and Farhadi, Ali},
  year = {2018},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1804.02767v1},
  eprint = {1804.02767}
}
Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors