Super-resolution from a single image

Daniel GlasnerShai BagonMichal Irani

article2009ICCV2,028 citations

Unifies classical multi-image reconstruction and example-based learning into a single framework that generates high-resolution details from a lone input image by exploiting cross-scale and within-scale patch redundancies.

Listen

Recovering high-resolution detail from low-resolution imagery is a persistent challenge across visual computing, defense, intelligence, and consumer media. Existing techniques historically faced major trade-offs: classical multi-image super-resolution relies on multiple precisely aligned low-resolution inputs and fails beyond modest magnification limits (typically less than a factor of 2), while example-based methods require massive external image training databases that can introduce artificial, erroneous features ("hallucinations"). The article aims to evaluate and demonstrate a unified super-resolution framework capable of generating high-quality image enhancements from as little as a single low-resolution input, without relying on external training databases or prior examples.

The authors develop an approach centered on the natural property of patch redundancy, showing that small image fragments (e.g., 5×5 pixels) naturally repeat both within the same image scale and across different down-scaled representations of the image. The article validates this premise through statistical analysis on 300 natural images from the Berkeley Segmentation Database. By treating recurring within-scale patches as independent observations, the framework establishes classical reconstruction constraints; concurrently, cross-scale patch matches provide natural high-to-low resolution exemplar pairs. The system solves the combined system of equations across a coarse-to-fine scale cascade, using iterative back-projection to ensure strict consistency with the original low-resolution image.

The findings confirm strong empirical and practical performance across several dimensions. First, the statistical analysis reveals high internal redundancy: over 90% of patches in natural images recur at least nine times within their native scale, and over 80% of edge and texture patches recur across significant scale reductions (down to 41% of original size). Second, combining within-scale and cross-scale constraints successfully reconstructs genuine high-frequency details (such as fine railings and distinct text) that standard interpolation and classical single-image methods blur or miss. Third, while the cross-scale example-based component provides the primary resolution boost, the classical constraint component effectively anchors the solution to the input data, preventing erroneous feature hallucinations. Finally, qualitative and quantitative benchmarks demonstrate that this self-contained single-image method achieves visual quality comparable to or exceeding leading external database-driven and edge-modeled approaches at magnification factors up to 4x.

These findings indicate that organizations can achieve state-of-the-art super-resolution without collecting, curating, or maintaining massive external reference datasets, thereby lowering data-storage costs and eliminating privacy or licensing risks associated with third-party training data. Furthermore, because the framework adaptively adjusts based on localized patch repetition, it reduces the risk of generating false artifacts in safety- or compliance-critical imaging domains.

Stakeholders seeking to deploy super-resolution workflows should consider adopting internal self-similarity pipelines as a lightweight alternative to database-heavy models, particularly when operating on standalone images. The source also supports extending this unified formulation into existing multi-frame video pipelines or hybrid database architectures to maximize reconstruction fidelity. Practitioners should note that the primary limitation occurs in image regions lacking cross-scale self-similarity—such as unique, non-repeating margin text—where the method gracefully falls back to classical enhancement limits without inventing artificial details. Overall confidence in the underlying methodology is high for natural imagery featuring standard textural and perspective redundancies.

Weizmann Institute of Science database systems image patches image super-resolution single-image super-resolution example-based super-resolution multi-image super-resolution

Glasner et al (2009).pdf

Paper: Image quilting for texture synthesis and transfer, Alexei A. Efros et al. (2001). Establishes the fundamental patch-based image synthesis paradigm that underpins example-based super-resolution without requiring complex parametric generative models.
Paper: An Iterative Image Registration Technique with an Application to Stereo Vision, B. D. Lucas et al. (1981). Introduces the standard differential alignment framework needed to compute subpixel misalignments and fuse recurring patch observations in classical multi-image super-resolution.

Paper: Single image super-resolution from transformed self-exemplars, Jia-Bin Huang et al. (2015). Directly generalizes internal self-similarity super-resolution by expanding the internal patch dictionary through perspective and affine transformations.
Paper: Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding, M. Bevilacqua et al. (2012). Builds on example-based single-image super-resolution by introducing a fast neighbor-embedding strategy with nonnegative weights.
Paper: Image Super-Resolution Using Deep Convolutional Networks, Chao Dong et al. (2014). Transitions single-image super-resolution from handcrafted patch recurrence and dictionary search to end-to-end deep convolutional neural networks.
Paper: Deep Image Prior, Dmitry Ulyanov et al. (2017). Continues the paradigm of learning super-resolution priors solely from a single image without external training data by utilizing untrained deep network architectures.
Paper: Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network, Wenzhe Shi et al. (2016). Advances single-image super-resolution efficiency by extracting features at the low-resolution scale and upsampling via sub-pixel convolution.
Paper: Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, Christian Ledig et al. (2017). Pioneers perceptual and adversarial loss formulations to recover fine, photo-realistic high-frequency textures in single-image super-resolution.
Paper: Accurate Image Super-Resolution Using Very Deep Convolutional Networks, Jiwon Kim et al. (2016). Extends deep learning-based single-image super-resolution by exploiting deeper residual networks to capture broader contextual receptive fields across multiple scales.
Paper: SwinIR: Image Restoration Using Swin Transformer, Jingyun Liang et al. (2021). Modernizes single-image restoration and super-resolution by replacing purely local convolutional operators with shifted-window self-attention mechanisms.

Cover for Super-resolution from a single image

Abstract

Methods for super-resolution can be broadly classified into two families of methods: (i) The classical multi-image super-resolution (combining images obtained at subpixel misalignments), and (ii) Example-Based super-resolution (learning correspondence between low and high resolution image patches from a database). In this paper we propose a unified framework for combining these two families of methods. We further show how this combined approach can be applied to obtain super resolution from as little as a single image (with no database or prior examples). Our approach is based on the observation that patches in a natural image tend to redundantly recur many times inside the image, both within the same scale, as well as across different scales. Recurrence of patches within the same image scale (at subpixel misalignments) gives rise to the classical super-resolution, whereas recurrence of patches across different scales of the same image gives rise to example-based super-resolution. Our approach attempts to recover at each pixel its best possible resolution increase based on its patch redundancy within and across scales.

Topics

database systems image patches image super-resolution single images classifiers

Keywords

single-image super-resolution example-based super-resolution multi-image super-resolution cross-scale patch correspondences blur kernels subpixel shifts

1. Introduction
2. Patch Redundancy in a Single Image
3. Single Image SR - A Unified Framework
3.1. Employing in-scale patch redundancy
3.2. Employing cross-scale patch redundancy
3.3. Combining Classical and Example-Based SR
4. Experimental Results
References

Knowls

Knowl 1 — Unified Single-Image Super-Resolution Formulation
model/method
The unified super-resolution (SR) framework simultaneously incorporates classical multi-image reconstruction constraints and database-free example-based constraints into a single linear system.

Let $L = I_0$ be the input low-resolution image and $H = I_n$ be the unknown target high-resolution image, related by the point spread function (PSF) blur kernel $B$ and total scale factor $s$ : $L = (H * B) \downarrow_s$ , where $\downarrow$ denotes subsampling. A continuous scale progression is discretized into a cascade of images $I_0, I_1, \dots, I_n = H$ with scale factors $s_l = \alpha^l$ relative to $L$ for a fixed step $\alpha > 1$ , and associated blur kernels $B_l$ .

Each patch $Q_l$ extracted at resolution level $l \in \{0, 1, \dots, n-1\}$ (either within-scale for $l=0$ or transferred from cross-scale parent patches for $l > 0$ ) induces a set of linear equations on the target high-resolution pixels $H(q)$ :

$Q_l(p) = (H * B_{n-l}) \downarrow_{s_{n-l}}(p) = \sum_{q_i \in \text{Support}(B_{n-l})} H(q_i) B_{n-l}(q_i - p)$

where $B_{n-l}$ is a compactly supported blur kernel accounting for the residual scale gap $(n-l)$ between intermediate level $l$ and the target level $n$ . Each equation is weighted by a reliability score based on the patch similarity. As $l \to n$ , $B_{n-l}$ approaches the Dirac delta function $\delta$ , improving the conditioning of the linear system. If only within-scale matches are found ( $l=0$ ), the system reduces to classical single-image SR; if no matches are found, it reduces to localized deblurring.
Knowl 2 — Cross-Scale Patch Recurrence for Internal Example-Based Super-Resolution
model/method
Example-based super-resolution (SR) learns low-to-high resolution patch mappings without an external training database by exploiting patch recurrence across different scales within the same image.

A downsampled pyramid of known images is constructed from the low-resolution input image $L = I_0$ : $I_{-l} = (L * B_l) \downarrow_{s_l}$ for scale levels $l = 1, \dots, m$ , where $B_l$ is the downscaling blur kernel and $s_l = \alpha^l$ .

For each pixel $p$ in $L$ and its surrounding patch $P_0(p)$ (typically $5 \times 5$ pixels):
1. An Approximate Nearest Neighbor search locates the most similar patch $P_{-l}(\tilde{p})$ in a coarser image $I_{-l}$ .
2. The higher-resolution 'parent' patch $Q_0(s_l \cdot \tilde{p})$ corresponding to $P_{-l}(\tilde{p})$ is extracted directly from the input image $L$ .
3. The pair $[P_{-l}(\tilde{p}), Q_0(s_l \cdot \tilde{p})]$ forms an internal low-res/high-res correspondence, providing an estimate $Q_l(s_l \cdot p)$ for the unknown high-resolution parent patch of $P_0(p)$ at intermediate scale level $l$ in the target cascade.
Knowl 3 — Coarse-to-Fine Single-Image Super-Resolution with Iterative Back-Projection
algorithm
The unified single-image super-resolution algorithm reconstructs the high-resolution target image $H$ incrementally across scale levels $l = 0, \dots, n-1$ using internal patch correspondences and back-projection consistency.
Input: Low-resolution image $L$ , scaling factor step $\alpha$ (e.g., $\alpha = 1.25$ or $\alpha = 2^{1/3}$ ), target level $n$ , patch size (e.g., $5 \times 5$ ), number of neighbors $k = 9$ .
Output: High-resolution image $H = I_n$ .
Construct down-scaled image pyramid $I_{-m}, \dots, I_{-1}$ from $L$ via $I_{-l} = (L * B_l) \downarrow_{s_l}$ .
Initialize $I_0 = L$ .
for $l = 0$ to $n-1$ do
    For each patch in $I_l$ , search for $k$ nearest neighbors across known resolution levels $\{I_{-m}, \dots, I_l\}$ using Approximate Nearest Neighbors.
    Extract parent patches from the appropriate higher-resolution levels to form candidate high-resolution patches $Q$ .
    Formulate the linear system of equations for $I_{l+1}$ where each patch $Q$ induces constraints $Q = (I_{l+1} * B_{\text{res}}) \downarrow_{s_{\text{res}}}$ , weighted by patch similarity scores.
    Solve the weighted linear system to recover an initial estimate of $I_{l+1}$ .
    Simulate low-resolution image: $L_{\text{sim}} = (I_{l+1} * B_{l+1}) \downarrow_{s_{l+1}}$ .
    Compute error residual: $E = L - L_{\text{sim}}$ .
    Back-project residual $E$ onto $I_{l+1}$ to enforce consistency with the input $L$ .
end for
return $H = I_n$
Knowl 4 — Empirical Patch Recurrence Statistics Across Image Scales
empirical result
An empirical evaluation on 300 natural grayscale images from the Berkeley Segmentation Database measures the frequency of small ( $5 \times 5$ ) patch repetitions within the same scale and across scaled-down resolutions ( $I_s$ scaled by $1.25^s$ for $s = 0, -1, \dots, -6$ , down to $0.26 = 1.25^{-6}$ of the original dimensions).

When evaluating all patches across the database:
- Over $90\%$ of patches have $\ge 9$ similar patches within the original scale ( $s = 0$ ).
- Over $80\%$ of patches have $\ge 9$ similar patches at scale factor $0.41 = 1.25^{-4}$ .
- $70\%$ of patches have $\ge 9$ similar patches at scale factor $0.26 = 1.25^{-6}$ .
When evaluating only the top $25\%$ highest-variance patches (retaining edges, corners, and texture while removing uniform regions):
- Over $80\%$ of high-variance patches have $\ge 9$ similar patches within the original scale.
- Over $70\%$ have $\ge 9$ similar patches at scale factor $0.41$ .
- $60\%$ have $\ge 9$ similar patches at scale factor $0.26$ .
The lowest scale in which a patch recurs establishes the upper bound for its potential resolution increase using internal self-similarity alone.
Knowl 5 — Patch Similarity Metric via Sub-Pixel Shift Self-Distance
definition
Patch similarity between two $5 \times 5$ patches $P_1$ and $P_2$ is computed by measuring their Gaussian-weighted Sum of Squared Differences (SSD) after subtracting their respective mean intensity (DC component).

Because textured and high-variance patches produce larger baseline SSD errors than smooth patches under minor misalignments, a patch-adaptive similarity threshold is defined for each source patch $P$ . The threshold distance $d_{\text{thresh}}(P)$ is the Gaussian-weighted SSD between $P$ and a copy of $P$ shifted by $0.5$ pixels. A candidate patch $P'$ is classified as similar to $P$ if and only if:

$d(P, P') \le d_{\text{thresh}}(P) = d\left(P, P_{\text{shift}(0.5)}\right)$
Knowl 6 — In-Scale Patch Redundancy as Classical Super-Resolution Constraints
model/method
Classical multi-image super-resolution relies on multiple low-resolution frames of the same scene captured with subpixel shifts to construct an overdetermined linear system for the unknown high-resolution pixels.

In a single image $L = (H * B) \downarrow_s$ , recurring patches $P_1, \dots, P_k$ similar to a patch $P$ at pixel $p$ occur at subpixel misalignments (determined at $1/s$ pixel accuracy, where $s$ is the magnification scale factor). These patches are treated as independent observations of the same underlying high-resolution scene structure. Each matching patch $P_j$ induces a linear constraint on the high-resolution pixels $H(q)$ within the support of blur kernel $B$ :

$P_j(p) = (H * B)(q) = \sum_{q_i \in \text{Support}(B)} H(q_i) B(q_i - q)$

Each constraint is weighted by the similarity score of $P_j$ to $P$ . Combining these constraints for $k$ neighbors (typically $k=9$ ) converts the underdetermined single-image problem into a determined linear system, achieving magnification up to factors of $\approx 1.6 - 2.0$ using only within-scale redundancy.
Knowl 7 — Complementary Roles of Classical and Example-Based Constraints in Unified Super-Resolution
theoretical result
The unified super-resolution framework represents an optimization problem combining a data fidelity term and two distinct regularization priors:
1. Data Term: The physical image degradation model $(H * B) \downarrow_s = L$ , requiring the reconstructed high-resolution image $H$ to remain consistent with the input low-resolution image $L$ .
2. Example-Based Prior: Generated by cross-scale patch correspondences, this prior provides high-frequency details and edge sharpens beyond the Nyquist frequency of $L$ , overcoming the theoretical resolution limits ( $<2\times$ ) of classical reconstruction methods.
3. Classical Multi-Patch Prior: Generated by within-scale subpixel patch recurrences, this prior enforces subpixel geometric consistency.
The classical reconstruction constraints constrain the solution space and prevent the example-based component from hallucinating erroneous high-resolution textures or artifacts that are inconsistent with the observed low-resolution data.
Knowl 8 — Color Channel Processing in Single-Image Super-Resolution
model/method
For super-resolution of color images, the input image is transformed from RGB color space to the YIQ color space.

The unified single-image super-resolution algorithm is applied solely to the luminance ( $Y$ ) channel, which contains the high-frequency structural content (edges, corners, and fine textures). The chromatic channels ( $I$ and $Q$ ), which are dominated by low-frequency color variations, are magnified using standard bicubic interpolation. The super-resolved $Y$ channel and the bicubic-interpolated $I$ and $Q$ channels are then combined and converted back to RGB.

Coverage note — None was omitted; all key theoretical formulations, empirical redundancy measurements, patch matching thresholds, unified equations, algorithm steps, and color pipeline have been extracted.

References

1.S. Arya and D. M. Mount. Approximate nearest neighbor queries in fixed dimensions. In SODA, 1993.
2.S. Baker and T. Kanade. Hallucinating faces. In Automatic Face and Gesture Recognition, 2000.
3.S. Baker and T. Kanade. Limits on super-resolution and how to break them. PAMI, (9), 2002.
4.A. Buades, B. Coll, and J. M. Morel. A review of image denoising algorithms, with a new one. SIAM MMS, (2), 2005.
5.D. Capel. Image Mosaicing and Super-Resolution. Springer–Verlag, 2004.
6.M. Ebrahimi and E. Vrscay. Solving the inverse problem of image zooming using "self-examples". In Image Analysis and Recognition, 2007.
7.A. A. Efros and T. K. Leung. Texture synthesis by non-parametric sampling. In ICCV, 1999.
8.S. Farsiu, M. Robinson, M. Elad, and P. Milanfar. Fast and robust multiframe super resolution. T-IP, (10), 2004.
9.R. Fattal. Image upsampling via imposed edge statistics. In SIGGRAPH, 2007.
10.W. Freeman, E. Pasztor, and O. Carmichael. Learning low-level vision. IJCV, (1), 2000.
11.W. T. Freeman, T. R. Jones, and E. C. Pasztor. Example-based super-resolution. Comp. Graph. Appl., (2), 2002.
12.M. Irani and S. Peleg. Improving resolution by image registration. CVGIP, (3), 1991.
13.K. Kim and Y. Kwon. Example-based learning for single-image SR and JPEG artifact removal. MPI-TR, (173), 08.
14.Z. Lin and H. Shum. Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation. PAMI, (1), 04.
15.G. Peyre, S. Bougleux, and L. D. Cohen. Non-local regularization of inverse problems. In ECCV, 2008.
16.M. Protter, M. Elad, H. Takeda, and P. Milanfar. Generalizing the nonlocal-means to super-resolution reconstruction. T-IP, 09.
17.D. Simakov, Y. Caspi, E. Shechtman, and M. Irani. Summarizing visual data using bidirectional similarity. CVPR, 2008.
18.N. Suetake, M. Sakano, and E. Uchino. Image super-resolution based on local self-similarity. Opt.Rev., (1), 2008.
19.J. Sun, Z. Xu, and H. Shum. Image super-resolution using gradient profile prior. In CVPR, 2008.

Citation

MLA

Glasner, D., et al. “Super-resolution from a Single Image”. 2009 IEEE 12th International Conference on Computer Vision, 2009, pp. 349–56, https://doi.org/10.1109/ICCV.2009.5459271.

APA

Glasner, D., Bagon, S., & Irani, M. (2009). Super-resolution from a single image. 2009 IEEE 12th International Conference on Computer Vision, 349–356. https://doi.org/10.1109/ICCV.2009.5459271

Chicago

Glasner, D., S. Bagon, and M. Irani. 2009. “Super-resolution from a Single Image”. 2009 IEEE 12th International Conference on Computer Vision, 349–56. https://doi.org/10.1109/ICCV.2009.5459271.

Harvard

Glasner, D., Bagon, S. and Irani, M. (2009) “Super-resolution from a single image”, 2009 IEEE 12th International Conference on Computer Vision. IEEE, pp. 349–356. Available at: https://doi.org/10.1109/ICCV.2009.5459271.

Vancouver

1. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In: 2009 IEEE 12th International Conference on Computer Vision. IEEE, pp 349–356

BibTeX

@inproceedings{Glasner_2009, title={Super-resolution from a single image}, url={http://dx.doi.org/10.1109/ICCV.2009.5459271}, DOI={10.1109/iccv.2009.5459271}, booktitle={2009 IEEE 12th International Conference on Computer Vision}, publisher={IEEE}, author={Glasner, Daniel and Bagon, Shai and Irani, Michal}, year={2009}, month=Sept, pages={349–356} }

Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE

Super-resolution from a single image

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Unified Single-Image Super-Resolution Formulation

Knowl 2 — Cross-Scale Patch Recurrence for Internal Example-Based Super-Resolution

Knowl 3 — Coarse-to-Fine Single-Image Super-Resolution with Iterative Back-Projection

Knowl 4 — Empirical Patch Recurrence Statistics Across Image Scales

Knowl 5 — Patch Similarity Metric via Sub-Pixel Shift Self-Distance

Knowl 6 — In-Scale Patch Redundancy as Classical Super-Resolution Constraints

Knowl 7 — Complementary Roles of Classical and Example-Based Constraints in Unified Super-Resolution

Knowl 8 — Color Channel Processing in Single-Image Super-Resolution

References

Citation

BibTeX

Access the Paper