Recovering high dynamic range radiance maps from photographs

Paul E. DebevecJitendra Malik

article1997SIGGRAPH2,140 citations

Proposes a seminal algorithm that recovers camera response curves and fuses differently exposed photographs into true high dynamic range radiance maps using standard imaging equipment.

Listen

Real-world scenes often exhibit extreme variations in illumination, such as bright sunlight alongside deep shadows, that exceed the recording capabilities of standard film and digital cameras. Conventional imaging devices apply unknown, nonlinear mappings during exposure, development, and digitization, which distorts the relationship between true scene brightness and the resulting digital pixel values. In addition, single exposures routinely suffer from clamped highlights and lost shadow detail, limiting their utility for accurate image processing, computer graphics rendering, and visual compositing.

The article sets out to demonstrate a practical method for recovering the aggregate nonlinear response function of any imaging system using standard equipment, and to reconstruct a single high dynamic range radiance map from a sequence of differently exposed photographs.

The authors implemented a self-calibrating computational technique that leverages reciprocity—the physical property that exposure is the product of light intensity and shutter duration. Using multiple photographs taken from a fixed viewpoint across varying, known exposure times, the system establishes a linear least-squares formulation solved via singular value decomposition. Credibility is supported by empirical validation across both digital sensor systems and traditional 35mm film scanned to digital media, sampling only several dozen pixel locations across eleven to sixteen exposure brackets.

The evaluation yielded several key findings in order of significance. First, the algorithm successfully extracts the complete, continuous response function of the imaging pipeline in seconds using as few as 28 to 50 pixel samples across the exposure sequence, eliminating the need for photometric measurement tools or calibration charts. Second, combining multiple exposures with an inverted response curve yields composite radiance maps that capture extreme illumination ranges, recovering over four to five orders of magnitude of useful dynamic range where the brightest points are up to 250,000 times brighter than the darkest. Third, using recovered high dynamic range radiance maps instead of raw digital images dramatically improves the physical realism of standard image-processing tasks, such as generating synthetic motion blur that correctly retains bright highlight saturation matching physical reference photographs.

These findings mean that practitioners can capture physically accurate optical measurements of real-world scenes without specialized, expensive radiometers. This significantly enhances workflows in image-based modeling, global illumination evaluation, and visual effects compositing by allowing imagery from different cameras and exposures to be merged consistently into a unified radiance space. Furthermore, the recovered response functions can be inverted to re-render composite scenes through virtual cameras, expanding creative flexibility while reducing production costs and data inconsistencies.

Organizations handling visual data should consider adopting multi-exposure acquisition protocols and the response recovery pipeline when capturing complex lighting environments. For standard shoots, practitioners can pre-compute an imaging system's response curve using a few calibration exposures and then capture only the minimum number of brackets needed to cover the scene's dynamic range. While manual pixel selection was originally used to solve the linear system, stakeholders should automate sample selection from uniform image regions to streamline production workflows.

The method relies on several boundary conditions, specifically requiring a static scene, fixed illumination, consistent aperture settings to prevent optical variations, and known, accurate shutter durations. The algorithm reconstructs relative radiance up to a scaling factor rather than absolute values, requiring an external calibration light source if absolute photometric units are required. Confidence in the underlying mathematical approach is high given the demonstrated alignment between synthetic simulations and ground-truth physical photographs under standard operating conditions.

  • Paper: Light field rendering, Marc Levoy et al. (1996). Provides foundational concepts in image-based rendering and sampling light fields directly from real images without explicit 3D geometry.
  • Paper: The lumigraph, S. Gortler et al. (1996). Introduces optical flow and plenoptic sampling techniques for capturing and rendering real-world appearance from multi-view photographs.
  • Paper: The rendering equation, J. Kajiya (1986). Establishes the fundamental physical and mathematical formulation of radiance transport across scenes required for accurate photometric modeling.
  • Paper: A reflectance model for computer graphics, Robert L. Cook et al. (1981). Supplies the physical optics foundation for material reflectance and highlight behavior necessary for analyzing camera response to dynamic lighting.
  • Paper: Shape and motion from image streams under orthography: a factorization method, Carlo Tomasi et al. (1992). Demonstrates singular value decomposition matrix factorization methods for robustly decomposing multi-image measurement matrices.
Cover for Recovering high dynamic range radiance maps from photographs

Abstract

We present a method of recovering high dynamic range radiance maps from photographs taken with conventional imaging equipment. In our method, multiple photographs of the scene are taken with different amounts of exposure. Our algorithm uses these differently exposed photographs to recover the response function of the imaging process, up to factor of scale, using the assumption of reciprocity. With the known response function, the algorithm can fuse the multiple photographs into a single, high dynamic range radiance map whose pixel values are proportional to the true radiance values in the scene. We demonstrate our method on images acquired with both photochemical and digital imaging processes. We discuss how this work is applicable in many areas of computer graphics involving digitized photographs, including image-based modeling, image compositing, and image processing. Lastly, we demonstrate a few applications of having high dynamic range radiance maps, such as synthesizing realistic motion blur and simulating the response of the human visual system.

Table of Contents

  • 1 Introduction
  • 1.1 Applications
  • 1.2 Background
  • 2 The Algorithm
  • 2.1 Film Response Recovery
  • 2.2 Constructing the High Dynamic Range Radiance Map
  • 2.2.1 Storage
  • 2.3 How many images are necessary?
  • 2.4 Recovering extended dynamic range from single exposures
  • 2.5 Obtaining Absolute Radiance
  • 2.6 Color
  • 2.7 Taking virtual photographs
  • 3 Results
  • 4 Conclusion
  • Acknowledgments
  • References
  • A Matlab Code

Knowls

  1. Knowl 1 — Optimization Objective for Imaging Response Curve Recovery

    equation

    The inverse response function g(z)=ln⁡f−1(z)g(z) = \ln f^{-1}(z), which maps discrete pixel brightness values z∈[Zmin⁡,Zmax⁡]z \in [Z_{\min}, Z_{\max}] to log exposures, is recovered simultaneously with unknown scene log irradiances ln⁡Ei\ln E_i by minimizing the weighted quadratic objective function:

    O=∑i=1N∑j=1P[w(Zij)(g(Zij)−ln⁡Ei−ln⁡Δtj)]2+λ∑z=Zmin⁡+1Zmax⁡−1[w(z)g′′(z)]2\mathcal{O} = \sum_{i=1}^{N} \sum_{j=1}^{P} \left[ w(Z_{ij}) \left( g(Z_{ij}) - \ln E_i - \ln \Delta t_j \right) \right]^2 + \lambda \sum_{z=Z_{\min}+1}^{Z_{\max}-1} \left[ w(z) g''(z) \right]^2

    subject to the scale normalization constraint:

    g(Zmid)=0,where Zmid=12(Zmin⁡+Zmax⁡)g(Z_{\text{mid}}) = 0, \quad \text{where } Z_{\text{mid}} = \frac{1}{2}(Z_{\min} + Z_{\max})

    where:

    • NN is the number of sampled spatial pixel locations.
    • PP is the number of photographs taken with different exposures.
    • Zij∈[Zmin⁡,Zmax⁡]Z_{ij} \in [Z_{\min}, Z_{\max}] is the discrete pixel value at location ii in photograph jj.
    • Δtj\Delta t_j is the exposure duration for photograph jj.
    • EiE_i is the sensor irradiance at pixel location ii.
    • w(z)w(z) is a hat weighting function that gives zero or low weight to extreme, noisy, or saturated pixel values.
    • g′′(z)=g(z−1)−2g(z)+g(z+1)g''(z) = g(z - 1) - 2g(z) + g(z + 1) is the discrete second derivative of gg, providing smoothness and coupling across discrete pixel levels.
    • λ\lambda is a scalar regularization hyperparameter balancing data fidelity against curve smoothness.
    • The constraint g(Zmid)=0g(Z_{\text{mid}}) = 0 removes the arbitrary additive constant ambiguity α\alpha in (g+α)(g + \alpha) and (ln⁡Ei+α)(\ln E_i + \alpha), defining unit exposure at the midpoint pixel value.
  2. Knowl 2 — High Dynamic Range Radiance Map Reconstruction

    equation

    Given a recovered inverse response function g(z)=ln⁡f−1(z)g(z) = \ln f^{-1}(z), the relative log irradiance ln⁡Ei\ln E_i at pixel location ii is reconstructed by combining measurements across all PP differently exposed photographs using a weighted average:

    ln⁡Ei=∑j=1Pw(Zij)(g(Zij)−ln⁡Δtj)∑j=1Pw(Zij)\ln E_i = \frac{\sum_{j=1}^{P} w(Z_{ij}) \left( g(Z_{ij}) - \ln \Delta t_j \right)}{\sum_{j=1}^{P} w(Z_{ij})}

    where:

    • ZijZ_{ij} is the pixel value at location ii in photograph jj.
    • Δtj\Delta t_j is the exposure duration of photograph jj.
    • w(z)w(z) is the hat weighting function assigning highest confidence to pixel values located near the center of the imaging system's working range.

    This weighted fusion reduces radiometric noise and film grain artifacts while preventing saturated, under-exposed, and blooming pixels from corrupting the reconstructed radiance map.

  3. Knowl 3 — Reciprocity Principle for Response Function Calibration

    assumption

    The response recovery method assumes imaging sensor reciprocity: the sensor exposure XX equals the product of film/sensor irradiance EE and exposure duration Δt\Delta t:

    X=EΔtX = E \Delta t

    Under reciprocity, halving EE and doubling Δt\Delta t produces an identical sensor response. The final digitized pixel value Z∈[Zmin⁡,Zmax⁡]Z \in [Z_{\min}, Z_{\max}] is modeled as a monotonic nonlinear aggregate mapping ff of sensor exposure:

    Zij=f(EiΔtj)Z_{ij} = f(E_i \Delta t_j)

    where ii indexes pixel locations, jj indexes exposure durations, EiE_i is the film irradiance at pixel ii (assumed constant across images of a static scene), and Δtj\Delta t_j is the exposure time for photograph jj. Reciprocity holds within 1/31/3 stop for typical print films for exposure durations between 10 s10\,\text{s} and 1/10,000 s1/10{,}000\,\text{s}, and holds in CCD arrays assuming charge accumulation is proportional to the total absorbed photon count over the integration period.

  4. Knowl 4 — Hat Weighting Function for Response Calibration and Radiance Fusion

    definition

    To account for reduced smoothness, poor data fitting, and saturation at the extreme ends of the camera response curve, a symmetric hat weighting function w(z)w(z) is defined over discrete pixel values z∈[Zmin⁡,Zmax⁡]z \in [Z_{\min}, Z_{\max}]:

    w(z)={z−Zmin⁡,for z≤12(Zmin⁡+Zmax⁡)Zmax⁡−z,for z>12(Zmin⁡+Zmax⁡)w(z) = \begin{cases} z - Z_{\min}, & \text{for } z \le \frac{1}{2}(Z_{\min} + Z_{\max}) \\ Z_{\max} - z, & \text{for } z > \frac{1}{2}(Z_{\min} + Z_{\max}) \end{cases}

    For 8-bit digitized imagery where Zmin⁡=0Z_{\min} = 0 and Zmax⁡=255Z_{\max} = 255, w(z)w(z) increases linearly from 00 at z=0z=0 to a maximum of 128128 at z=128z=128, and decreases linearly back to 00 at z=255z=255. In response curve optimization and radiance map fusion, w(z)w(z) suppresses noisy near-zero values and saturated pixel values subject to blooming.

  5. Knowl 5 — Linear Least-Squares Camera Response Solver

    algorithm

    The inverse response function gg and the log irradiance values ln⁡E\ln E are computed by constructing and solving an overdetermined linear system Ax=bAx = b via Singular Value Decomposition (SVD).

    Input: Pixel matrix ZZ of size N×PN \times P containing values in [0,255][0, 255], log exposure durations vector BB of size PP, smoothness weight λ\lambda, weighting function ww defined over integers 0…2550 \dots 255
    Output: Log exposure vector gg of size 256, log irradiance vector ln⁡E\ln E of size NN
    n←256n \leftarrow 256
    M←N⋅P+n+1M \leftarrow N \cdot P + n + 1
    Initialize matrix AA of size M×(n+N)M \times (n + N) with all zeros
    Initialize vector bb of size MM with all zeros
    k←1k \leftarrow 1
    for i←1i \leftarrow 1 to NN do
        for j←1j \leftarrow 1 to PP do
            z←Z[i,j]z \leftarrow Z[i, j]
            wij←w(z)w_{ij} \leftarrow w(z)
            A[k,z+1]←wijA[k, z + 1] \leftarrow w_{ij}
            A[k,n+i]←−wijA[k, n + i] \leftarrow -w_{ij}
            b[k]←wij⋅B[j]b[k] \leftarrow w_{ij} \cdot B[j]
            k←k+1k \leftarrow k + 1
        end for
    end for
    A[k,129]←1A[k, 129] \leftarrow 1
    b[k]←0b[k] \leftarrow 0
    k←k+1k \leftarrow k + 1
    for z←1z \leftarrow 1 to n−2n - 2 do
        A[k,z]←λ⋅w(z)A[k, z] \leftarrow \lambda \cdot w(z)
        A[k,z+1]←−2⋅λ⋅w(z)A[k, z + 1] \leftarrow -2 \cdot \lambda \cdot w(z)
        A[k,z+2]←λ⋅w(z)A[k, z + 2] \leftarrow \lambda \cdot w(z)
        k←k+1k \leftarrow k + 1
    end for
    x←solve_linear_least_squares(A,b)x \leftarrow \text{solve\_linear\_least\_squares}(A, b)
    g←x[1…n]g \leftarrow x[1 \dots n]
    ln⁡E←x[n+1…n+N]\ln E \leftarrow x[n + 1 \dots n + N]
    return g,ln⁡Eg, \ln E
  6. Knowl 6 — Pixel Sampling and Exposure Overlap Requirements

    model/method

    The requirements for reconstructing the response function gg and the complete radiance map are:

    1. Overdetermination Condition: Solving for NN unknown log irradiances and (Zmax⁡−Zmin⁡)(Z_{\max} - Z_{\min}) discrete response samples requires N(P−1)>(Zmax⁡−Zmin⁡)N(P - 1) > (Z_{\max} - Z_{\min}). For 8-bit images (Zmax⁡−Zmin⁡=255Z_{\max} - Z_{\min} = 255) with P=11P = 11 exposures, N≈50N \approx 50 sampled pixels is sufficient.
    2. Pixel Selection Strategy: Sampled pixels must span an even distribution of brightnesses across [Zmin⁡,Zmax⁡][Z_{\min}, Z_{\max}], be spatially distributed across the image, and reside in regions of low intensity variance to avoid optical blur and sub-pixel irradiance variations.
    3. Exposure Overlap for Calibration: A minimum of two differently exposed photographs is required to recover g(z)g(z), provided their exposure amounts overlap such that some pixels lie in the active working range (between the toe and shoulder) in both photographs.
    4. Dynamic Range Coverage: If a scene has a dynamic range of RR and the sensor/film working dynamic range is FF, the minimum number of exposure brackets needed to ensure full scene coverage is ⌈R/F⌉\lceil R / F \rceil.
  7. Knowl 7 — Color Channel Calibration and Absolute Radiance Conversion

    model/method

    For multi-channel RGB imagery, response curve recovery and radiance map fusion are executed for the red, green, and blue channels independently. Because each channel is recovered up to an independent scale factor, setting g(Zmid)=0g(Z_{\text{mid}}) = 0 assumes the imaging system responds achromatically to neutral gray at ZmidZ_{\text{mid}}.

    When imaging systems are calibrated for specific illuminants (such as daylight or tungsten), or when absolute radiance is required:

    • Inter-channel color balance and absolute radiometric scale can be calibrated by imaging a reference luminaire of known spectral radiance and scaling each channel's reconstructed values accordingly.
    • Approximate absolute radiance scaling can be estimated from film speed (ASA rating), aperture size (f-number), and exposure time.
  8. Knowl 8 — Virtual Photography and Linear Processing on High Dynamic Range Radiance Maps

    model/method

    Standard linear image processing operations (such as synthetic motion blur, optical filtering, and blurring) yield erroneous results when applied to conventional low dynamic range (LDR) images because bright highlight pixels are nonlinearly compressed and clamped to maximum display values prior to filtering, causing blurred highlights to lose intensity and look muddy.

    Applying linear spatial filters directly to high dynamic range radiance maps, and subsequently passing the filtered radiance map through the recovered imaging response function Z=f(EΔt)=g−1(ln⁡E+ln⁡Δt)Z = f(E \Delta t) = g^{-1}(\ln E + \ln \Delta t) for a chosen exposure duration Δt\Delta t, produces physically accurate motion blur and highlight saturation matching real camera motion.

  9. Knowl 9 — Extended Dynamic Range Recovery from Single Film Negatives

    model/method

    Print film captures a wider optical dynamic range than standard 8-bit desktop film scanners deliver in a single pass. Two scanning strategies extract this extended dynamic range:

    1. Slide-Mode Scanning: Scanning the print negative under the scanner's slide film mode forces the scanner to capture the entire optical density range within an 8-bit image. The response curve of the slide-mode scanning pipeline is calibrated and used to linearize the single scan into a radiance map.
    2. Dual-Density Scanning: Scanning the same physical negative twice under differing scanner density or brightness adjustment settings, recovering the respective response functions for both settings, and combining the two scans using multi-exposure radiance fusion.
  10. Knowl 10 — Empirical Response Recovery and Dynamic Range Results on CCD and Print Film

    empirical result

    The calibration and radiance reconstruction algorithms were demonstrated on digital and photochemical imaging pipelines:

    • Digital CCD Camera: Evaluated on a Kodak DCS460 digital camera capturing 11 grayscale exposures ranging from 1/30 s1/30\,\text{s} to 30 s30\,\text{s} in 1-stop increments at f/8. The recovered response curve revealed a deliberate nonlinear S-shaped tone mapping applied by the camera hardware/firmware to mimic film response, and the reconstructed radiance map spanned over 4 orders of magnitude of dynamic range.
    • Color Print Film and PhotoCD: Evaluated on Fuji 100 ASA 35mm film scanned via Kodak PhotoCD with 16 exposures from 1/1000 s1/1000\,\text{s} to 30 s30\,\text{s} in 1-stop increments at f/8. The recovered red, green, and blue curves showed close agreement in high exposures but an elevated blue response at low exposures (characterizing a slight blue cast in deep shadows). The resulting HDR radiance map of the scene spanned a dynamic range exceeding five orders of magnitude (250,000:1250{,}000:1).

Coverage note — None was omitted; all core algorithms, mathematical derivations, calibration methods, operational constraints, and experimental findings from the paper were converted into standalone knowls. External tone reproduction operators cited for visualization purposes were omitted as they are not contributions of this paper.

References

  1. 1.ADAMS, A. Basic Photo, 1st ed. Morgan & Morgan, Hastings-on-Hudson, New York, 1970.
  2. 2.CHEN, E. QuickTime VR - an image-based approach to virtual environment navigation. In SIGGRAPH ’95 (1995).
  3. 3.DEBEVEC, P. E., TAYLOR, C. J., AND MALIK, J. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In SIGGRAPH ’96 (August 1996), pp. 11–20.
  4. 4.FAUGERAS, O. Three-Dimensional Computer Vision. MIT Press, 1993.
  5. 5.FERWERDA, J. A., PATTANAIK, S. N., SHIRLEY, P., AND GREENBERG, D. P. A model of visual adaptation for realistic image synthesis. In SIGGRAPH ’96 (1996), pp. 249–258.
  6. 6.GORTLER, S. J., GRZESZCZUK, R., SZELISKI, R., AND COHEN, M. F. The Lumigraph. In SIGGRAPH ’96 (1996), pp. 43–54.
  7. 7.HORN, B. K. P. Robot Vision. MIT Press, Cambridge, Mass., 1986, ch. 10, pp. 206–208.
  8. 8.JAMES, T., Ed. The Theory of the Photographic Process. Macmillan, New York, 1977.
  9. 9.KAUFMAN, J. E., Ed. IES Lighting Handbook; the standard lighting guide, 7th ed. Illuminating Engineering Society, New York, 1987, p. 24.
  10. 10.KOLB, C., MITCHELL, D., AND HANRAHAN, P. A realistic camera model for computer graphics. In SIGGRAPH ’95 (1995).
  11. 11.LAVEAU, S., AND FAUGERAS, O. 3-D scene representation as a collection of images. In Proceedings of 12th International Conference on Pattern Recognition (1994), vol. 1, pp. 689–691.
  12. 12.LEVOY, M., AND HANRAHAN, P. Light field rendering. In SIGGRAPH ’96 (1996), pp. 31–42.
  13. 13.MADDEN, B. C. Extended intensity range imaging. Tech. rep., GRASP Laboratory, University of Pennsylvania, 1993.
  14. 14.MANN, S., AND PICARD, R. W. Being ’undigital’ with digital cameras: Extending dynamic range by combining differently exposed pictures. In Proceedings of IS&T 46th annual conference (May 1995), pp. 422–428.
  15. 15.MCMILLAN, L., AND BISHOP, G. Plenoptic Modeling: An image-based rendering system. In SIGGRAPH ’95 (1995).
  16. 16.SCHLICK, C. Quantization techniques for visualization of high dynamic range pictures. In Fifth Eurographics Workshop on Rendering (Darmstadt, Germany) (June 1994), pp. 7–18.
  17. 17.SZELISKI, R. Image mosaicing for tele-reality applications. In IEEE Computer Graphics and Applications (1996).
  18. 18.TANI, T. Photographic sensitivity : theory and mechanisms. Oxford University Press, New York, 1995.
  19. 19.THEUWISSEN, A. J. P. Solid-state imaging with charge-coupled devices. Kluwer Academic Publishers, Dordrecht; Boston, 1995.
  20. 20.TUMBLIN, J., AND RUSHMEIER, H. Tone reproduction for realistic images. IEEE Computer Graphics and Applications 13, 6 (1993), 42–48.
  21. 21.WARD, G. J. Measuring and modeling anisotropic reflection. In SIGGRAPH ’92 (July 1992), pp. 265–272.
  22. 22.WARD, G. J. The radiance lighting simulation and rendering system. In SIGGRAPH ’94 (July 1994), pp. 459–472.
  23. 23.WARD, G. J., RUSHMEIER, H., AND PIATKO, C. A visibility matching tone reproduction operator for high dynamic range scenes. Tech. Rep. LBNL-39882, Lawrence Berkeley National Laboratory, March 1997.

Citation

MLA
Debevec, P. E., and J. Malik. “Recovering High Dynamic Range Radiance Maps from Photographs”. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH '97, 1997, pp. 369–78, https://doi.org/10.1145/258734.258884.
APA
Debevec, P. E., & Malik, J. (1997). Recovering high dynamic range radiance maps from photographs. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH '97, 369–378. https://doi.org/10.1145/258734.258884
Chicago
Debevec, P. E., and J. Malik. 1997. “Recovering High Dynamic Range Radiance Maps from Photographs”. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH '97, 369–78. https://doi.org/10.1145/258734.258884.
Harvard
Debevec, P.E. and Malik, J. (1997) “Recovering high dynamic range radiance maps from photographs”, Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97. ACM Press, pp. 369–378. Available at: https://doi.org/10.1145/258734.258884.
Vancouver
1. Debevec PE, Malik J (1997) Recovering high dynamic range radiance maps from photographs. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97. ACM Press, pp 369–378

BibTeX

@inproceedings{Debevec_1997, series={SIGGRAPH ’97}, title={Recovering high dynamic range radiance maps from photographs}, url={http://dx.doi.org/10.1145/258734.258884}, DOI={10.1145/258734.258884}, booktitle={Proceedings of the 24th annual conference on Computer graphics and interactive techniques  - SIGGRAPH ’97}, publisher={ACM Press}, author={Debevec, Paul E. and Malik, Jitendra}, year={1997}, pages={369–378}, collection={SIGGRAPH ’97} }
Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF