Back

Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement

Chunle GuoChongyi LiJichang GuoChen Change LoyJunhui HouSam KwongRunmin Cong

article2020CVPR2,346 citations

Proposes Zero-DCE, a lightweight framework that brightens low-light images by estimating image-specific tone curves using zero-reference loss functions, eliminating the dependency on paired or unpaired training datasets.

estimation methodface detectionimage enhancementImageCurveLow-Light
arXiv: 2001.06826
  • Paper: Vision Transformers for Dense Prediction, René Ranftl et al. (2021). This paper extends low-level image processing and estimation tasks by demonstrating how vision transformer backbones improve dense predictions.
Cover for Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement

Abstract

The paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network. Our method trains a lightweight deep network, DCE-Net, to estimate pixel-wise and high-order curves for dynamic range adjustment of a given image. The curve estimation is specially designed, considering pixel value range, monotonicity, and differentiability. Zero-DCE is appealing in its relaxed assumption on reference images, i.e., it does not require any paired or unpaired data during training. This is achieved through a set of carefully formulated non-reference loss functions, which implicitly measure the enhancement quality and drive the learning of the network. Our method is efficient as image enhancement can be achieved by an intuitive and simple nonlinear curve mapping. Despite its simplicity, we show that it generalizes well to diverse lighting conditions. Extensive experiments on various benchmarks demonstrate the advantages of our method over state-of-the-art methods qualitatively and quantitatively. Furthermore, the potential benefits of our Zero-DCE to face detection in the dark are discussed. Code and model will be available at this https URL.

Topics

estimation methodface detectionimage enhancementlighting conditionsreference imagecolor imagesdigital imageb spline curve

Keywords

ImageCurveLow-LightEnhancementDeepImagesLow-Light ImageLossCurve EstimationDeep Curve Estimation

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 Methodology
  • 3.1 Light-Enhancement Curve (LE-curve)
  • 3.2 DCE-Net
  • 3.3 Non-Reference Loss Functions
  • 4 Experiments
  • 4.1 Ablation Study
  • 4.2 Benchmark Evaluations
  • 4.2.1 Visual and Perceptual Comparisons
  • 4.2.2 Quantitative Comparisons
  • 4.2.3 Face Detection in the Dark
  • 5 Conclusion
  • References

Knowls

  1. Knowl 1 — Zero-Reference Deep Curve Estimation Framework

    model/method

    Zero-Reference Deep Curve Estimation (Zero-DCE) formulates low-light image enhancement not as an end-to-end image-to-image translation task, but as an image-specific curve parameter estimation task. A lightweight deep convolutional neural network (DCE-Net) takes an input low-light image and estimates a set of pixel-wise curve parameter maps. These parameter maps define high-order, nonlinear light-enhancement curves (LE-curves) that iteratively adjust the dynamic range of each pixel across the RGB color channels.

    The framework operates under a zero-reference learning paradigm: it requires neither paired low/normal-light images nor unpaired training data with adversarial discriminators. Instead, training is guided exclusively by a set of differentiable non-reference loss functions evaluating spatial consistency, exposure distribution, color constancy, and illumination smoothness.

  2. Knowl 2 — Light-Enhancement Curve (LE-Curve) Formulation

    equation

    To dynamically adjust pixel intensities within the normalized range [0,1][0, 1] while preserving neighboring contrast and differentiability, Zero-DCE defines a base quadratic curve mapping:

    LE(I(x);α)=I(x)+αI(x)(1−I(x))LE(I(x); \alpha) = I(x) + \alpha I(x)(1 - I(x))

    where xx denotes the pixel coordinates, I(x)∈[0,1]I(x) \in [0, 1] represents the normalized input pixel intensity, and α∈[−1,1]\alpha \in [-1, 1] is a trainable adjustment parameter controlling the exposure level and curve curvature.

    To achieve wider dynamic range adjustment without global over- or under-enhancement, the curve is extended into a pixel-wise, multi-iteration higher-order mapping applied separately across the three RGB channels:

    LEn(x)=LEn−1(x)+An(x)LEn−1(x)(1−LEn−1(x))LE_n(x) = LE_{n-1}(x) + A_n(x) LE_{n-1}(x)(1 - LE_{n-1}(x))

    where n∈{1,2,…,N}n \in \{1, 2, \dots, N\} represents the iteration index (with N=8N = 8 and LE0(x)=I(x)LE_0(x) = I(x)), and An(x)∈[−1,1]A_n(x) \in [-1, 1] is the estimated pixel-wise parameter map for the nn-th iteration with the same spatial dimensions as the input image II.

  3. Knowl 3 — Zero-DCE Non-Reference Loss Functions

    equation

    Zero-DCE trains the curve estimation network without reference images using a total loss function composed of four task-specific non-reference loss terms:

    Ltotal=Lspa+Lexp+WcolLcol+WtvALtvAL_{\text{total}} = L_{\text{spa}} + L_{\text{exp}} + W_{\text{col}} L_{\text{col}} + W_{\text{tvA}} L_{\text{tvA}}

    where the loss weights are fixed to Wcol=0.5W_{\text{col}} = 0.5 and WtvA=20W_{\text{tvA}} = 20.

    1. Spatial Consistency Loss (LspaL_{\text{spa}}) preserves spatial coherence and local contrast between the enhanced image YY and input image II across KK local regions of size 4×44 \times 4:

    Lspa=1K∑i=1K∑j∈Ω(i)(∣Yi−Yj∣−∣Ii−Ij∣)2L_{\text{spa}} = \frac{1}{K} \sum_{i=1}^{K} \sum_{j \in \Omega(i)} \left( |Y_i - Y_j| - |I_i - I_j| \right)^2

    where Ω(i)\Omega(i) denotes the four direct neighboring regions (top, down, left, right) of region ii, and Yi,IiY_i, I_i are average intensity values within local region ii.

    1. Exposure Control Loss (LexpL_{\text{exp}}) drives local region intensity toward a well-exposed gray level E=0.6E = 0.6 across MM non-overlapping patches of size 16×1616 \times 16:

    Lexp=1M∑k=1M∣Yk−E∣L_{\text{exp}} = \frac{1}{M} \sum_{k=1}^{M} |Y_k - E|

    1. Color Constancy Loss (LcolL_{\text{col}}) enforces the Gray-World hypothesis to prevent color casts across the enhanced RGB channels:

    Lcol=∑∀(p,q)∈{(R,G),(R,B),(G,B)}(Jp−Jq)2L_{\text{col}} = \sum_{\forall (p,q) \in \{(R,G), (R,B), (G,B)\}} (J^p - J^q)^2

    where JpJ^p is the global mean intensity of color channel pp in the enhanced image.

    1. Illumination Smoothness Loss (LtvAL_{\text{tvA}}) enforces spatial gradient smoothness across the curve parameter maps AncA_n^c to prevent neighboring artifact discontinuities over N=8N=8 iterations and channels c∈{R,G,B}c \in \{R, G, B\}:

    LtvA=1N∑n=1N∑c∈{R,G,B}(∣∇xAnc∣+∣∇yAnc∣)2L_{\text{tvA}} = \frac{1}{N} \sum_{n=1}^{N} \sum_{c \in \{R,G,B\}} \left( |\nabla_x A_n^c| + |\nabla_y A_n^c| \right)^2

    where ∇x\nabla_x and ∇y\nabla_y represent horizontal and vertical spatial gradient operators.

  4. Knowl 4 — DCE-Net Architecture

    model/method

    The Deep Curve Estimation Network (DCE-Net) is a fully convolutional neural network designed to estimate pixel-wise curve parameter maps. It consists of seven convolutional layers with symmetric skip connections (concatenations between symmetric layer pairs).

    • Layers 1 through 6 each employ 32 convolutional filters of size 3×33 \times 3 with stride 1, each followed by a Rectified Linear Unit (ReLU) activation function.
    • Downsampling and batch normalization layers are omitted to prevent the disruption of local neighboring pixel spatial relations.
    • Layer 7 uses a 3×33 \times 3 convolution followed by a Tanh activation function, producing 2424 output feature maps corresponding to N=8N = 8 iterations with 3 curve parameter maps (for R, G, and B channels) per iteration.
    • DCE-Net contains 79,416 trainable parameters and requires 5.21 GFLOPs for an input image of size 256×256×3256 \times 256 \times 3.

  5. Knowl 5 — Training Setup and Multi-Exposure Data Strategy

    experimental setup

    DCE-Net is trained on 360 multi-exposure sequences from the Part1 subset of the SICE dataset, containing both under-exposed and over-exposed photographs to allow bidirectional dynamic range adjustment. A total of 2,422 images are used for training and 600 for validation, all resized to 512×512×3512 \times 512 \times 3.

    Training is performed with the PyTorch framework on a single NVIDIA GeForce RTX 2080 Ti GPU with a batch size of 8. Network filter weights are initialized from a zero-mean Gaussian distribution with a standard deviation of 0.02, and biases are initialized as constants. Optimization is performed using the Adam optimizer with default parameters and a fixed learning rate of 10−410^{-4}. Total training takes approximately 30 minutes.

  6. Knowl 6 — Full-Reference Image Quality Assessment on SICE Part2 Benchmark

    data/table

    Zero-DCE was quantitatively evaluated against state-of-the-art low-light enhancement methods on the Part2 subset of the SICE dataset, comprising 767 paired low/normal light images resized to 1200×900×31200 \times 900 \times 3. Image quality was evaluated using Peak Signal-to-Noise Ratio (PSNR in dB, higher is better), Structural Similarity Index (SSIM, higher is better), and Mean Absolute Error (MAE, lower is better).

    Method PSNR SSIM MAE
    SRIE 14.41 0.54 127.08
    LIME 16.17 0.57 108.12
    Li et al. 15.19 0.54 114.21
    RetinexNet 15.99 0.53 104.81
    Wang et al. 13.52 0.49 142.01
    EnlightenGAN 16.21 0.59 102.78
    Zero-DCE 16.57 0.59 98.78

    Despite using zero reference images during training, Zero-DCE achieves the best PSNR (16.57 dB), tied-best SSIM (0.59), and lowest MAE (98.78) compared to conventional, supervised CNN-based, and unsupervised GAN-based models.

  7. Knowl 7 — Perceptual Quality and User Study Across Standard Benchmarks

    data/table

    Zero-DCE was evaluated on 202 testing images across five standard low-light benchmarks: NPE (84 images), LIME (10 images), MEF (17 images), DICM (64 images), and VV (24 images). Evaluation includes a User Study (US, score range 1 to 5, higher is better, evaluated by 15 independent human subjects assessing exposure, color balance, and noise/artifacts) and the no-reference Perceptual Index (PI, lower is better).

    Method NPE LIME MEF DICM VV Average
    SRIE 3.65 / 2.79 3.50 / 2.76 3.22 / 2.61 3.42 / 3.17 2.80 / 3.37 3.32 / 2.94
    LIME 3.78 / 3.05 3.95 / 3.00 3.71 / 2.78 3.31 / 3.35 3.21 / 3.03 3.59 / 3.04
    Li et al. 3.80 / 3.09 3.78 / 3.02 2.93 / 3.61 3.47 / 3.43 2.87 / 3.37 3.37 / 3.72
    RetinexNet 3.30 / 3.18 2.32 / 3.08 2.80 / 2.86 2.88 / 3.24 1.96 / 2.95 2.58 / 3.06
    Wang et al. 3.83 / 2.83 3.82 / 2.90 3.13 / 2.72 3.44 / 3.20 2.95 / 3.42 3.43 / 3.01
    EnlightenGAN 3.90 / 2.96 3.84 / 2.83 3.75 / 2.45 3.50 / 3.13 3.17 / 4.71 3.63 / 3.22
    Zero-DCE 3.81 / 2.84 3.80 / 2.76 4.13 / 2.43 3.52 / 3.04 3.24 / 3.33 3.70 / 2.88

    Table values are formatted as US / PI. Zero-DCE achieves the highest overall average subjective rating (3.70) and the best average perceptual index (2.88).

  8. Knowl 8 — Runtime and Computational Efficiency Comparison

    data/table

    Inference runtimes of different low-light image enhancement methods were benchmarked using the average execution time across 32 images of resolution 1200×900×31200 \times 900 \times 3.

    Method Runtime (seconds) Platform
    SRIE 12.1865 MATLAB (CPU)
    LIME 0.4914 MATLAB (CPU)
    Li et al. 90.7859 MATLAB (CPU)
    RetinexNet 0.1200 TensorFlow (GPU)
    Wang et al. 0.0210 TensorFlow (GPU)
    EnlightenGAN 0.0078 PyTorch (GPU)
    Zero-DCE 0.0025 PyTorch (GPU)

    Zero-DCE achieves an inference time of 0.0025 seconds per 1200×900×31200 \times 900 \times 3 image on an NVIDIA GeForce GTX 2080Ti GPU, processing images at approximately 500 frames per second for 640×480×3640 \times 480 \times 3 inputs, which is over three times faster than EnlightenGAN and orders of magnitude faster than conventional and supervised CNN baselines.

  9. Knowl 9 — Impact of Zero-DCE on Dark Face Detection

    empirical result

    Zero-DCE was evaluated as a pre-processing enhancement step for dark face detection on 6,000 images from the training and validation sets of the DARK FACE dataset. The baseline face detector was a Dual Shot Face Detector (DSFD) trained on WIDER FACE.

    Applying Zero-DCE low-light enhancement prior to face detection substantially improves DSFD precision-recall curves and average precision (AP) over raw unenhanced dark images (raw image AP is 0.231). Zero-DCE achieves competitive detection AP against RetinexNet (both achieving top AP performance among tested enhancers) and performs especially well in the high-recall regime by brightening dark facial features without washing out well-lit background regions.

  10. Knowl 10 — Ablation Analysis on Losses, Architecture, and Data Distribution

    empirical result

    Ablation experiments on Zero-DCE validate its core design choices:

    1. Loss Components: Removing spatial consistency loss LspaL_{\text{spa}} decreases output contrast; omitting exposure control loss LexpL_{\text{exp}} fails to recover dark regions; removing color constancy loss LcolL_{\text{col}} causes severe color casts by ignoring channel correlations; and removing illumination smoothness loss LtvAL_{\text{tvA}} creates visible spatial artifacts.
    2. Network Structure and Iteration Count: Testing combinations of convolutional layers l∈{3,7}l \in \{3, 7\}, feature channels f∈{16,32}f \in \{16, 32\}, and iterations n∈{1,8,16}n \in \{1, 8, 16\} indicates that single iteration (n=1n=1) produces insufficient dynamic range expansion. The 7-layer, 32-channel, 8-iteration model (7-32-8) balances parameter size, inference speed, and restoration performance.
    3. Training Data Distribution: Training solely on low-light images (without multi-exposure sequences) leads the network to over-enhance already well-lit image regions. Including multi-exposure images (both under-exposed and over-exposed) during zero-reference training is required for balanced bidirectional dynamic range mapping.

Coverage note — None was omitted; all primary methodological components, mathematical formulations, network design parameters, training procedures, ablation studies, and empirical benchmark evaluations are included.

References

  1. 1.Yochai Blau and Tomer Michaeli. The perception-distortion tradeoff. In CVPR, 2018. 7
  2. 2.Gershon Buchsbaum. A spatial processor model for object colour perception. J. Franklin Institute, 310(1):1–26, 1980. 5
  3. 3.Vladimir Bychkovsky, Sylvain Paris, Eric Chan, and Fredo Durand. Learning photographic global tonal adjustment with a database of input/output image pairs. In CVPR, 2011. 2, 6
  4. 4.Jianrui Cai, Shuhang Gu, and Lei Zhang. Learning a deep single image contrast enhancer from multi-exposure image. IEEE Transactions on Image Processing, 27(4):2049–2026, 2018. 5, 6, 7
  5. 5.Chen Chen, Qifeng Chen, Jia Xu, and Koltun Vladlen. Learning to see in the dark. In CVPR, 2018. 5
  6. 6.Yusheng Chen, Yuching Wang, Manhsin Kao, and Yungyu Chuang. Deep photo enhancer: Unpaired learning for image enhancement from photographs with gans. In CVPR, 2018. 5
  7. 7.Dinu Coltuc, Philippe Bolon, and Jean-Marc Chassery. Exact histogram specification. IEEE Transactions on Image Processing, 15(5):1143–1152, 2006. 2
  8. 8.Xueyang Fu, Delu Zeng, Yue Huang, Xiao-Ping Zhang, and Xinghao Ding. A weighted variational model for simultaneous reflectance and illumination estimation. In CVPR, 2016. 2, 6, 7, 8
  9. 9.Xiaojie Guo, Yu Li, and Haibin Ling. Lime: Low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing, 26(2):982–993, 2017. 2, 6, 7, 8
  10. 10.Haidi Ibrahim and Nicholas Sia Pik Kong. Brightness preserving dynamic histogram equalization for image contrast enhancement. IEEE Transactions on Consumer Electronics, 53(4):1752–1758, 2007. 2
  11. 11.Andrey Ignatov, Nikolay Kobyshev, Radu Timofte, Kenneth Vanhoey, and Luc Van Gool. Wespe: Weakly supervised photo enhancer for digital cameras. In CVPRW, 2018. 5
  12. 12.Yifan Jiang, Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang, Xiaohui Shen, Jianchao Yang, Pan Zhou, and Zhangyang Wang. EnlightenGAN: Deep light enhancement without paired supervision. In CVPR, 2019. 1, 2, 3, 5, 6, 7, 8
  13. 13.Edwin H Land. The retinex theory of color vision. Scientific American, 237(6):108–128, 1977. 2
  14. 14.Chulwoo Lee, Chul Lee, and Chang-Su Kim. Contrast enhancement based on layered difference representation. In ICIP, 2012. 6
  15. 15.Chulwoo Lee, Chul Lee, and Chang-Su Kim. Contrast enhancement based on layered difference representation of 2d histograms. IEEE Transactions on Image Processing, 22(12):5372–5384, 2013. 2
  16. 16.Chongyi Li, Chunle Guo, and Jichang Guo. Underwater image color correction based on weakly supervised color transfer. IEEE Signal Processing Letters, 25(3):323–327, 2018. 5
  17. 17.Chongyi Li, Jichang Guo, Fatih Porikli, and Yanwei Pang. Lightennet: a convolutional neural network for weakly illuminated image enhancement. Pattern Recognition Letters, 104:15–22, 2018. 5
  18. 18.Jian Li, Yabiao Wang, Changan Wang, Ying Tai, Jianjun Qian, Jian Yang, Chengjie Wang, Jilin Li, and Feiyuen Huang. Dsfd: Dual shot face detector. In CVPR, 2019. 8
  19. 19.Mading Li, Jiaying Liu, Wenhan Yang, Xiaoyan Sun, and Zongming Guo. Structure-revealing low-light image enhancement via robust retinex model. IEEE Transactions on Image Processing, 27(6):2828–2841, 2018. 2, 6, 7, 8
  20. 20.Kin Gwn Lore, Adedotun Akintayo, and Soumik Sarkar. Llnet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognition, 61:650–662, 2017. 2
  21. 21.Chao Ma, Chih-Yuan Yang, Xiaokang Yang, and Ming-Hsuan Yang. Learning a no-reference quality metric for single-image super-resolution. Computer Vision and Image Understanding, 158:1–16, 2017. 7
  22. 22.Kede Ma, Kai Zeng, and Zhou Wang. Perceptual quality assessment for multi-exposure image fusion. IEEE Transactions on Image Processing, 24(11):3345–3356, 2015. 6
  23. 23.Tom Mertens, Jan Kautz, and Frank Van Reeth. Exposure fusion. In PCCGA, 2007. 5
  24. 24.Tom Mertens, Jan Kautz, and Frank Van Reeth. Exposure fusion: A simple and practrical alterrnative to high dynamic range photography. Computer Graphics Forum, 28(1):161–171, 2009. 5
  25. 25.Anish Mittal, Rajiv Soundararajan, and Alan C. Bovik. Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters, 20(3):209–212, 2013. 7
  26. 26.Yanyun Qu, Yizi Chen, Jingying Huang, and Yuan Xie. Enhanced pix2pix dehazing network. In CVPR, 2019. 7
  27. 27.J Alex Stark. Adaptive image contrast enhancement using generalizations of histogram equalization. IEEE Transactions on Image Processing, 9(5):889–896, 2000. 2
  28. 28.Ruixing Wang, Qing Zhang, Chi-Wing Fu, Xiaoyong Shen, Wei-Shi Zheng, and Jiaya Jia. Underexposed photo enhancement using deep illumination estimation. In CVPR, 2019. 1, 2, 5, 6, 7, 8
  29. 29.Shuhang Wang, Jin Zheng, Hai-Miao Hu, and Bo Li. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Transactions on Image Processing, 22(9):3538–3548, 2013. 2, 6
  30. 30.Wenguan Wang, Qiuxia Lai, Huazhu Fu, Jianbing Shen, and Haibin Ling. Salient object detection in the deep learning era: An in-depth survey, 2019. 5
  31. 31.Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. 7
  32. 32.Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. Deep retinex decomposition for low-light enhancement. In BMVC, 2018. 2, 5, 6, 7, 8
  33. 33.Peng Xu. Deep learning for free-hand sketch: A survey, 2020. 5
  34. 34.Shuo Yang, Ping Luo, Chen-Change Loy, and Xiaoou Tang. Wider face: A face detection benchmark. In CVPR, 2016. 8
  35. 35.Runsheng Yu, Wenyu Liu, Yasen Zhang, Zhi Qu, Deli Zhao, and Bo Zhang. Deepexposure: Learning to expose photos with asynchronously reinforced adversarial learning. In NeurIPS, 2018. 5
  36. 36.Lu Yuan and Jian Sun. Automatic exposure correction of consumer photographs. In ECCV, 2012. 2
  37. 37.Ye Yuan, Wenhan Yang, Wenqi Ren, Jiaying Liu, Walter J Scheirer, and Wang Zhangyang. Ug+ track 2: A collective benchmark effort for evaluating and advancing image understanding in poor visibility environments, 2019. arXiv arXiv:1904.04474. 6, 8
  38. 38.Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV, 2017. 2

Citation

MLA
Guo, C., et al. “Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement”. CVPR 2020, 2020, http://arxiv.org/abs/2001.06826v2.
APA
Guo, C., Li, C., Guo, J., Loy, C. C., Hou, J., Kwong, S., & Cong, R. (2020). Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement. CVPR 2020. http://arxiv.org/abs/2001.06826v2
Chicago
Guo, C., C. Li, J. Guo, et al. 2020. “Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement”. CVPR 2020. http://arxiv.org/abs/2001.06826v2.
Harvard
Guo, C. et al. (2020) “Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement”, CVPR 2020 [Preprint]. Available at: http://arxiv.org/abs/2001.06826v2.
Vancouver
1. Guo C, Li C, Guo J, Loy CC, Hou J, Kwong S, Cong R (2020) Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement. CVPR 2020

BibTeX

@article{guo2020zero,
  title = {Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement},
  author = {Guo, Chunle and Li, Chongyi and Guo, Jichang and Loy, Chen Change and Hou, Junhui and Kwong, Sam and Cong, Runmin},
  year = {2020},
  journal = {CVPR 2020},
  url = {http://arxiv.org/abs/2001.06826v2},
  eprint = {2001.06826}
}
Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE