Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization

Weisheng DongLei ZhangGuangming ShiXiaolin Wu

article2010IEEE Transactions on Image Processing1,391 citationsBest Paper Award

Presents an adaptive sparse representation framework that dynamically selects optimal dictionary bases, autoregressive models, and non-local self-similarity constraints for each image patch, significantly improving image deblurring and super-resolution reconstruction quality.

Listen

Digital images frequently suffer from blur and low resolution due to optical limits, motion, and sensor constraints. Restoring high-quality images from degraded inputs is an ill-posed mathematical problem with multiple possible solutions. Conventional approaches often use universal representation dictionaries or rigid smoothing constraints, which tend to generate severe visual artifacts, blur fine textures, or introduce artificial edges.

The article develops and evaluates a comprehensive restoration framework that combines adaptive sparse domain selection with dual adaptive regularizations. The main objective is to significantly improve image deblurring and single-image super-resolution by tailoring local dictionary bases and statistical constraints to the specific structural properties of each image patch.

The approach pre-trains compact sub-dictionaries and local predictive models by clustering over 700,000 natural image patches into distinct pattern categories. During restoration, the algorithm dynamically selects the best-matched sub-dictionary and local autoregressive model for each degraded patch, while also integrating non-local self-similarity constraints derived from repetitive patterns across the image. The optimization is solved using an iterative shrinkage algorithm that adaptively reweights sparsity parameters based on local statistics. Performance was rigorously tested on standard benchmark images and an extensive 1,000-image dataset under varying blur and noise conditions.

The experimental findings show that the proposed framework consistently outperforms existing state-of-the-art restoration methods across objective quality metrics and perceptual evaluations. For image deblurring, the method achieves average peak signal-to-noise ratio improvements of up to 0.85 dB over the leading alternative (BM3D) while markedly reducing edge ringing. In single-image super-resolution, it surpasses top competing methods by an average of 1.13 dB in noiseless scenarios and 0.77 dB under noisy conditions, delivering substantially sharper edges. Furthermore, the model demonstrates strong stability across different training datasets and varying numbers of cluster categories.

These results indicate that replacing rigid, universal dictionaries with locally tailored sparse representations offers substantial gains in reconstruction fidelity and noise suppression. For decision-makers and technical leaders in fields such as medical imaging, remote sensing, digital surveillance, and consumer electronics, this method provides a robust pathway to enhance visual asset quality without creating misleading artificial details.

To adopt this framework, organizations should incorporate parallel computing architectures to mitigate the processing time, which currently ranges from 2 to 5 minutes per standard image on central processing units. Implementing accelerated first-order optimization techniques or graphics processor pipelines is recommended before deploying the solution into real-time operational environments.

Confidence in these findings is high given the validation across a diverse 1,000-image benchmark. However, practitioners should note that small patch configurations (such as 3x3 pixels) can occasionally introduce faint artifacts in flat regions, making 7x7 patches the recommended default. Computational latency remains the primary boundary condition for time-sensitive applications.

Hong Kong Polytechnic University McMaster University Xidian University adaptive regularization genetic selection image deblurring adaptive sparse domain selection image deblurring image super-resolution

arXiv: 1012.1184

Paper: Non-local sparse models for image restoration, Julien Mairal et al. (2009). This paper establishes simultaneous sparse coding by unifying non-local self-similarity with adaptive sparse representations, providing the direct foundation for incorporating non-local priors into sparse restoration frameworks.
Paper: Online dictionary learning for sparse coding, Julien Mairal et al. (2009). This work introduces efficient online dictionary learning algorithms for natural image patches, enabling the training of the multiple basis sets and dictionaries employed by sparse domain selection methods.
Paper: A non-local algorithm for image denoising, Antoni Buades et al. (2005). This foundational paper presents the non-local means image prior, which formalizes the patch redundancy and non-local self-similarity regularizer integrated into the source method.
Paper: Super-resolution from a single image, Daniel Glasner et al. (2009). This paper introduces single-image super-resolution driven by internal patch redundancy and self-similarity across scales, motivating the non-local structural modeling utilized in the source restoration framework.
Paper: Fast Image Deconvolution using Hyper-Laplacian Priors, Dilip Krishnan et al. (2009). This paper develops fast non-convex image deconvolution using alternating minimization and half-quadratic splitting, setting standard optimization schemes for sparsity-regularized deblurring.
Paper: Emergence of simple-cell receptive field properties by learning a sparse code for natural images, Bruno A. Olshausen et al. (1996). This seminal work demonstrates that natural image patches are intrinsically sparse under learned basis sets, providing the fundamental theoretical justification for sparse representation modeling in image restoration.

Paper: Learning a Deep Convolutional Network for Image Super-Resolution, Chao Dong et al. (2014). This paper recasts traditional patch-based sparse coding restoration pipelines into end-to-end learned deep convolutional neural networks for single-image super-resolution.
Paper: Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding, Marco Bevilacqua et al. (2012). This work develops a low-complexity single-pass super-resolution framework by replacing iterative sparse reconstruction with nonnegative neighbor embedding over patch dictionaries.
Paper: Learning Deep CNN Denoiser Prior for Image Restoration, Kai Zhang et al. (2017). This paper integrates learned deep denoisers into model-based variable splitting optimization, modernizing the adaptive regularization and deblurring/super-resolution paradigms of prior sparse methods.
Paper: Single image super-resolution from transformed self-exemplars, Jia-Bin Huang et al. (2015). This study extends internal self-similarity priors for single-image super-resolution by allowing geometric perspective and affine transformations on candidate exemplar patches.
Paper: Deep Image Prior, Dmitry Ulyanov et al. (2017). This work explores unsupervised image restoration by demonstrating that untrained deep convolutional architectures inherently act as structured, adaptive image priors.
Paper: Denoising Diffusion Restoration Models, Bahjat Kawar et al. (2022). This paper leverages pre-trained generative diffusion models as versatile priors to solve linear inverse restoration problems like deblurring and super-resolution without task-specific training.

Cover for Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization

Abstract

As a powerful statistical image modeling technique, sparse representation has been successfully used in various image restoration applications. The success of sparse representation owes to the development of l1-norm optimization techniques, and the fact that natural images are intrinsically sparse in some domain. The image restoration quality largely depends on whether the employed sparse domain can represent well the underlying image. Considering that the contents can vary significantly across different images or different patches in a single image, we propose to learn various sets of bases from a pre-collected dataset of example image patches, and then for a given patch to be processed, one set of bases are adaptively selected to characterize the local sparse domain. We further introduce two adaptive regularization terms into the sparse representation framework. First, a set of autoregressive (AR) models are learned from the dataset of example image patches. The best fitted AR models to a given patch are adaptively selected to regularize the image local structures. Second, the image non-local self-similarity is introduced as another regularization term. In addition, the sparsity regularization parameter is adaptively estimated for better image restoration performance. Extensive experiments on image deblurring and super-resolution validate that by using adaptive sparse domain selection and adaptive regularization, the proposed method achieves much better results than many state-of-the-art algorithms in terms of both PSNR and visual perception.

I. Introduction
II. Related Works
III. Sparse Representation with Adaptive Sparse Domain Selection
A. Learning the sub-dictionaries
B. Adaptive selection of the sub-dictionary
C. Adaptively reweighted sparsity regularization
IV. Spatially Adaptive Regularization
A. Training the AR models
B. Adaptive selection of the AR model for regularization
C. Adaptive regularization by non-local similarity
V. Summary of the Algorithm
VI. Experimental Results
A. Training datasets
B. Experimental settings
C. Experimental results on de-blurring
D. Experimental results on single image super-resolution
E. Experimental results on a 1000-image dataset
F. Discussions on the computational cost
VII. Conclusion
References

Knowls

Knowl 1 — ASDS-AReg Image Restoration Framework
model/method
The Adaptive Sparse Domain Selection with Spatially Adaptive Regularization (ASDS-AReg) framework restores a clean image $x \in \mathbb{R}^{N_I}$ from a degraded observation $y \in \mathbb{R}^M$ modeled as:

$y = DHx + v$

where $H$ is a blur operator, $D$ is a down-sampling operator (the identity matrix in pure deblurring tasks), and $v \sim \mathcal{N}(0, \sigma_n^2 I)$ is additive white Gaussian noise.

Let $R_i$ denote the patch extraction matrix extracting the $i$ -th patch $x_i = R_i x \in \mathbb{R}^n$ of size $\sqrt{n} \times \sqrt{n}$ from image $x$ , for $i = 1, \dots, N$ . The image is reconstructed from patch representations using the synthesis operator $\Phi \circ \alpha$ defined by:

$\hat{x} = \Phi \circ \alpha = \left(\sum_{i=1}^N R_i^T R_i\right)^{-1} \sum_{i=1}^N R_i^T \Phi_{k_i} \alpha_i$

where $\Phi_{k_i}$ is a compact orthonormal sub-dictionary adaptively assigned to patch $i$ , and $\alpha_i$ is its representation vector.

The overall ASDS-AReg optimization problem combines data fidelity, an autoregressive (AR) local structure regularization term, a non-local (NL) self-similarity regularization term, and an adaptively reweighted $\ell_1$ -norm sparsity penalty:

$\hat{\alpha} = \arg\min_\alpha \left\{ \|y - DH(\Phi \circ \alpha)\|_2^2 + \gamma \|(I - A)(\Phi \circ \alpha)\|_2^2 + \eta \|(I - B)(\Phi \circ \alpha)\|_2^2 + \sum_{i=1}^N \sum_{j=1}^n \lambda_{i,j} |\alpha_{i,j}| \right\}$

where $\gamma > 0$ and $\eta > 0$ are regularization weights, $A$ is the globally aggregated AR model matrix, $B$ is the globally aggregated non-local similarity weight matrix, $\alpha_{i,j}$ is the coefficient corresponding to the $j$ -th atom of sub-dictionary $\Phi_{k_i}$ , and $\lambda_{i,j}$ is its adaptive sparsity weight.
Knowl 2 — Learning Compact Sub-Dictionaries via Truncated Principal Component Analysis
model/method
Instead of training a single over-complete dictionary, the ASDS framework learns $K$ compact sub-dictionaries $\{\Phi_k\}_{k=1}^K$ offline from clustered natural image patches:
1. Patch Extraction and Filtering: A dataset of patches of size $\sqrt{n} \times \sqrt{n}$ is extracted from training images. Patches with intensity variance $\text{Var}(s_i) \le \Delta$ (with threshold $\Delta = 16$ ) are excluded to focus exclusively on meaningful structural details and edges.
2. Feature-Space Clustering: High-pass filtering is applied to the selected training patch set $S = [s_1, \dots, s_M] \in \mathbb{R}^{n \times M}$ to obtain $S_h = [s_1^h, \dots, s_M^h]$ . $K$ -means clustering partitions $S_h$ into $K$ clusters $\{C_1, \dots, C_K\}$ with centroids $\mu_k \in \mathbb{R}^n$ , dividing $S$ into $K$ subsets $S_k \in \mathbb{R}^{n \times m_k}$ .
3. Principal Component Analysis and Truncation: For each subset $S_k$ , PCA is computed on its covariance matrix $\Omega_k$ , yielding an orthogonal eigenvector transformation matrix $P_k$ . To balance the $\ell_2$ approximation error with the $\ell_1$ sparsity penalty, only the first $r_o$ principal eigenvectors $\Phi_r = [p_1, \dots, p_r]$ are retained:
$r_o = \arg\min_r \left\{ \|S_k - \Phi_r \Lambda_r\|_F^2 + \lambda \|\Lambda_r\|_1 \right\}$

where $\Lambda_r = \Phi_r^T S_k$ . The resulting compact sub-dictionary for cluster $k$ is $\Phi_k = [p_1, p_2, \dots, p_{r_o}] \in \mathbb{R}^{n \times r_o}$ .
Knowl 3 — Robust Sub-Dictionary and AR Model Selection via Subspace Projection
model/method
For a given image patch estimate $\hat{x}_i$ , its corresponding sub-dictionary $\Phi_{k_i}$ and AR model $a_{k_i}$ are adaptively selected online by comparing its high-pass filtered patch $\hat{x}_i^h$ against the cluster centroids $\{\mu_1, \dots, \mu_K\}$ .

To increase selection robustness against image noise and artifacts in intermediate iterations, matching is performed in the low-dimensional principal subspace of the cluster centroids:

Let $U = [\mu_1, \mu_2, \dots, \mu_K] \in \mathbb{R}^{n \times K}$ denote the matrix containing all cluster centroids. Singular value decomposition of the covariance matrix of $U$ provides a projection matrix $\Phi_c$ formed by the leading eigenvectors. The optimal cluster index $k_i$ for patch $\hat{x}_i$ is determined by:

$k_i = \arg\min_k \|\Phi_c^T \hat{x}_i^h - \Phi_c^T \mu_k\|_2$

The selected sub-dictionary $\Phi_{k_i}$ and AR model $a_{k_i}$ are then assigned to the patch at location $i$ .
Knowl 4 — MAP-Based Non-Local Adaptive Sparsity Parameter Estimation
equation
Under a Bayesian Maximum a Posteriori (MAP) framework where observation noise is independent Gaussian $\mathcal{N}(0, \sigma_n^2)$ and representation coefficients $\alpha_{i,j}$ follow an independent zero-mean Laplacian distribution with standard deviation $\sigma_{i,j}$ , the regularizer weight $\lambda_{i,j}$ for coefficient $\alpha_{i,j}$ is given by:

$\lambda_{i,j} = \frac{2\sqrt{2}\sigma_n^2}{\hat{\sigma}_{i,j} + \varepsilon}$

where $\varepsilon$ is a small positive constant to prevent division by zero, and $\hat{\sigma}_{i,j}$ is the estimated standard deviation of coefficient $\alpha_{i,j}$ .

The parameter $\hat{\sigma}_{i,j}$ is estimated non-locally: for patch estimate $\hat{x}_i$ , the algorithm finds its $L$ most similar patches $\{\hat{x}_i^l\}_{l=1}^L$ across the image. The sparse coding coefficients of these similar patches over the chosen sub-dictionary $\Phi_{k_i}$ are $\hat{\alpha}_i^l = \Phi_{k_i}^T \hat{x}_i^l$ . The standard deviation $\hat{\sigma}_{i,j}$ is then computed as the sample standard deviation of the $j$ -th entry across the set $\{\hat{\alpha}_i^l\}_{l=1}^L$ .
Knowl 5 — Autoregressive Model Regularization
model/method
To exploit local spatial correlation and promote piecewise stationarity:
1. Offline AR Training: For each patch cluster $S_k$ , an 8th-order 2D autoregressive (AR) model defined on a $3 \times 3$ window is trained. The AR coefficient vector $a_k \in \mathbb{R}^8$ is computed via least squares:
$a_k = \arg\min_a \sum_{s_i \in S_k} (s_i - a^T q_i)^2$

where $s_i$ is the central pixel value of patch $s_i$ and $q_i \in \mathbb{R}^8$ contains the eight surrounding neighbor pixels of $s_i$ .
1. Regularization Formulation: Given patch $x_i$ with central pixel $x_i$ , neighbor vector $\chi_i$ , and assigned AR model $a_{k_i}$ , the AR prediction error is minimized across all image pixels:
$\sum_{x_i \in x} \|x_i - a_{k_i}^T \chi_i\|_2^2 = \|(I - A)x\|_2^2$

where $A \in \mathbb{R}^{N_I \times N_I}$ is a sparse matrix whose entries are:

$A(i,j) = \begin{cases} a_{k_i}(m), & \text{if pixel } j \text{ is the } m\text{-th neighbor in } \chi_i \text{ of pixel } i \\ 0, & \text{otherwise} \end{cases}$
Knowl 6 — Non-Local Self-Similarity Regularization
model/method
To exploit non-local structural redundancies in natural images, each patch estimate $\hat{x}_i$ is matched to its $L$ closest neighbor patches $\{\hat{x}_i^l\}_{l=1}^L$ based on Euclidean distance $e_i^l = \|\hat{x}_i - \hat{x}_i^l\|_2^2$ .

The central pixel $x_i$ of patch $x_i$ is predicted by the weighted average of the center pixels $x_i^l$ of its similar patches, with non-local weights $b_i^l$ defined by:

$b_i^l = \frac{1}{c_i} \exp\left(-\frac{e_i^l}{h}\right), \quad c_i = \sum_{l=1}^L \exp\left(-\frac{e_i^l}{h}\right)$

where $h$ is a filtering bandwidth parameter and $c_i$ is the normalization scalar.

Expressed across the entire image, the non-local prediction error regularizer is:

$\sum_{x_i \in x} \left\| x_i - \sum_{l=1}^L b_i^l x_i^l \right\|_2^2 = \|(I - B)x\|_2^2$

where $B \in \mathbb{R}^{N_I \times N_I}$ is a sparse matrix whose entries are:

$B(i,j) = \begin{cases} b_i^l, & \text{if pixel } j \text{ is the central pixel of the } l\text{-th similar patch to } x_i \\ 0, & \text{otherwise} \end{cases}$
Knowl 7 — Iterative Shrinkage Algorithm for ASDS-AReg Image Restoration
algorithm
The ASDS-AReg optimization problem is formulated in augmented linear form as:

$\hat{\alpha} = \arg\min_\alpha \left\{ \|\tilde{y} - K(\Phi \circ \alpha)\|_2^2 + \sum_{i=1}^N \sum_{j=1}^n \lambda_{i,j} |\alpha_{i,j}| \right\}, \quad \tilde{y} = \begin{bmatrix} y \\ 0 \\ 0 \end{bmatrix}, \quad K = \begin{bmatrix} DH \\ \gamma (I - A) \\ \eta (I - B) \end{bmatrix}$
Input: Degraded image $y$ , operators $D, H$ , parameters $\gamma, \eta, P, e, Max\_Iter$ , step scale $r = 4.7$
Output: Restored image $\hat{x}$
Compute initial image estimate $\hat{x}^{(0)}$ using iterated wavelet shrinkage
Select initial sub-dictionaries $\Phi_{k_i}$ and AR models $a_{k_i}$ for each patch using centroid subspace projection
Calculate initial non-local weights $b_i$ and construct sparse matrices $A$ and $B$
Compute $U = (DH)^T DH$ and $V = \gamma^2 (I - A)^T (I - A) + \eta^2 (I - B)^T (I - B)$
Set $k = 0$
while $\|\hat{x}^{(k)} - \hat{x}^{(k+1)}\|_2^2 / N > e$ and $k < Max\_Iter$ do
     $\hat{x}^{(k+1/2)} = \hat{x}^{(k)} + \frac{1}{r} \left( (DH)^T y - U \hat{x}^{(k)} - V \hat{x}^{(k)} \right)$
    for each patch $i = 1$ to $N$ do
         $\alpha_i^{(k+1/2)} = \Phi_{k_i}^T R_i \hat{x}^{(k+1/2)}$
        for each atom $j = 1$ to $n$ do
             $\tau_{i,j} = \lambda_{i,j} / r$
             $\alpha_{i,j}^{(k+1)} = \text{sgn}(\alpha_{i,j}^{(k+1/2)}) \max(0, |\alpha_{i,j}^{(k+1/2)}| - \tau_{i,j})$
        end for
    end for
     $\hat{x}^{(k+1)} = \left(\sum_{i=1}^N R_i^T R_i\right)^{-1} \sum_{i=1}^N R_i^T \Phi_{k_i} \alpha_i^{(k+1)}$
    if $\text{mod}(k, P) == 0$ then
        Update sub-dictionaries $\Phi_{k_i}$ , AR models $a_{k_i}$ , weights $b_i$ , matrices $A, B, V$ , and weights $\lambda_{i,j}$ using $\hat{x}^{(k+1)}$
    end if
     $k = k + 1$
end while
return $\hat{x}^{(k)}$
Default hyperparameters: $L = 10$ , update period $P = 100$ , patch size $7 \times 7$ with 5-pixel overlap ( $N = N_I / 4$ ). For deblurring: $\gamma = 0.0775, \eta = 0.1414, \tau_{i,j} = \lambda_{i,j} / 4.7$ . For noiseless super-resolution: $\gamma = 0.0894, \eta = 0.2, \tau_{i,j} = 0.18 / \hat{\sigma}_{i,j}$ . For noisy super-resolution ( $\sigma_n = 5$ ): $\gamma = 0.2828, \eta = 0.5, \tau_{i,j} = \lambda_{i,j} / 16.6$ . Convergence typically occurs in 700–1000 iterations.
Knowl 8 — Deblurring and Super-Resolution Performance on Benchmark Images
empirical result
The ASDS-AR-NL method was evaluated against state-of-the-art baselines across standard test images (luminance channel) for both deblurring and $3\times$ single image super-resolution:
1. Deblurring Performance:
  - Under uniform $9 \times 9$ blur kernel with noise level $\sigma_n = \sqrt{2}$ , ASDS-AR-NL-TD2 achieved an average PSNR of 27.47 dB and SSIM of 0.7943, outperforming iterated wavelet shrinkage (25.56 dB / 0.7217), constrained TV (26.15 dB / 0.7443), spatially weighted TV (26.46 dB / 0.7544), $\ell_0$ -norm frame deblurring (26.36 dB / 0.7500), and BM3D deblurring (26.97 dB / 0.7748).
  - Under uniform $9 \times 9$ blur kernel with $\sigma_n = 2$ , ASDS-AR-NL-TD2 achieved 26.75 dB / 0.7646, exceeding BM3D (26.35 dB / 0.7487) by 0.40 dB.
  - Under Gaussian blur (standard deviation 3), ASDS-AR-NL-TD2 achieved 24.72 dB / 0.6834 (at $\sigma_n = \sqrt{2}$ ) and 24.56 dB / 0.6720 (at $\sigma_n = 2$ ), outperforming BM3D (24.57 dB and 24.38 dB, respectively).
2. Super-Resolution Performance ( $3\times$ scaling factor):
  - For noiseless images ( $\sigma_n = 0$ ), ASDS-AR-NL-TD2 achieved an average PSNR of 29.16 dB and SSIM of 0.8463, surpassing iterated wavelet shrinkage (28.03 dB / 0.8115), Softcuts (27.49 dB / 0.7910), Yang et al. sparse representation (27.69 dB / 0.7954), and Marquina–Osher TV (27.49 dB / 0.8190).
  - For noisy images ( $\sigma_n = 5$ ), ASDS-AR-NL-TD2 achieved 27.82 dB / 0.7867, outperforming Softcuts (27.05 dB / 0.7657), Yang et al. (26.34 dB / 0.7090), and wavelet shrinkage (26.49 dB / 0.7048).

Knowl 9 — Quantitative Restoration Results on 1000 Natural Images Dataset

data/table

To establish robustness across diverse image textures and edges, the ASDS-AR-NL-TD2 algorithm was evaluated on a dataset of 1000 natural images ( $256 \times 256$ sub-images collected from Flickr and BSDS):

Method	Uniform Blur ( $\sigma_n=\sqrt{2}$ )	Uniform Blur ( $\sigma_n=2$ )	Gaussian Blur ( $\sigma_n=\sqrt{2}$ )	Gaussian Blur ( $\sigma_n=2$ )
ASDS-AR-NL-TD2	29.36 dB (0.8397)	28.66 dB (0.8163)	26.22 dB (0.7335)	26.10 dB (0.7261)
BM3D	28.51 dB (0.8139)	27.96 dB (0.7966)	26.09 dB (0.7297)	25.91 dB (0.7209)
$\ell_0$ Frame Deblurring	28.26 dB (0.8081)	27.41 dB (0.7763)	25.63 dB (0.7072)	25.37 dB (0.6934)

For $3\times$ single-image super-resolution on the 1000-image dataset:

The results demonstrate that ASDS-AR-NL-TD2 consistently delivers superior PSNR and SSIM across large-scale natural image evaluations.

Knowl 10 — Robustness to Number of Clusters and Patch Size Selection
empirical result
Ablation experiments on the 1000-image dataset evaluated the effects of cluster count $K$ and patch dimensions:
1. Cluster Count Robustness: Sub-dictionaries and AR models were trained using $K = 100, 200,$ and $400$ clusters. For deblurring (uniform blur, $\sigma_n = \sqrt{2}$ ), the average PSNR (SSIM) values were 29.29 dB (0.8379) for $K = 100$ , 29.36 dB (0.8397) for $K = 200$ , and 29.31 dB (0.8380) for $K = 400$ . For super-resolution ( $\sigma_n = 0$ ), performance remained virtually identical: 27.51 dB (0.7971) for $K=100$ , 27.52 dB (0.7974) for $K=200$ , and 27.53 dB (0.7975) for $K=400$ , demonstrating stability across cluster counts.
2. Patch Size Selection: Testing patch sizes of $3 \times 3$ , $5 \times 5$ , and $7 \times 7$ on the 1000-image dataset produced similar average PSNRs: 29.60 dB ( $3 \times 3$ ), 29.56 dB ( $5 \times 5$ ), and 29.36 dB ( $7 \times 7$ ) for deblurring; and 27.51 dB ( $3 \times 3$ ), 27.54 dB ( $5 \times 5$ ), and 27.53 dB ( $7 \times 7$ ) for super-resolution. However, visual evaluation showed that smaller patches ( $3 \times 3$ and $5 \times 5$ ) introduce visible reconstruction artifacts in flat smooth regions, whereas $7 \times 7$ patches suppress them, leading to $7 \times 7$ being selected as the optimal patch size.

Coverage note — Individual per-image numerical result rows from Tables 1 through 6, 10, and 11 were summarized into aggregate benchmark performance knowls to ensure high conceptual density and avoid data redundancy.

References

1.M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging. Bristol, U.K.: IOP, 1998.
2.D. Donoho, “Compressed sensing,” IEEE Trans. on Information Theory, vol. 52, no. 4, pp. 1289-1306, April 2006.
3.E. Candès and T. Tao, “Near optimal signal recovery from random projections: Universal encoding strategies?” IEEE Trans. on Information Theory, vol. 52, no. 12, pp. 5406 - 5425, December 2006.
4.E. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. on Information Theory, vol. 52, no. 2, pp. 489 - 509, February 2006.
5.M. R. Banham and A. K. Katsaggelos, “Digital image restoration,” IEEE Trans. Signal Processing Mag., vol. 14, no. 2, pp. 24-41, Mar. 1997.
6.L. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Phys. D, vol. 60, pp. 259–268, 1992.
7.T. Chan, S. Esedoglu, F. Park, and A. Yip, “Recent developments in total variation image restoration,” in Mathematical Models of Computer Vision, N. Paragios, Y. Chen, and O. Faugeras, Eds. New York: Springer Verlag, 2005.
8.S. Kindermann, S. Osher, and P. W. Jones, “Deblurring and denoising of images by nonlocal functionals,” Multiscale Modeling and Simulation, vol. 4, no. 4, pp. 1091-1115, 2005.
9.R. Molina, J. Mateos, and A. K. Katsaggelos, “Blind deconvolution using a variational approach to parameter, image, and blur estimation,” IEEE Trans. On Image Process., vol. 15, no. 12, pp 3715-3727, Dec. 2006.
10.I. Daubechies, M. Defriese, and C. DeMol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Commun. Pure Appl. Math., vol.57, pp.1413-1457, 2004.
11.P. Combettes, and V. Wajs, “Signal recovery by proximal forward-backward splitting,” SIAM J.Multiscale Model.Simul., vol.4, pp.1168-1200, 2005.
12.J.M. Bioucas Dias, and M.A.T. Figueiredo. “A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. Image Proc., vol.16, no.12, pp.2992-3004, 2007.
13.M. Elad, M.A.T. Figueiredo, and Y. Ma, “On the Role of Sparse and Redundant Representations in Image Processing,” Proceedings of IEEE, Special Issue on Applications of Compressive Sensing & Sparse Representation, June 2010.
14.J. Mairal, M. Elad, and G. Sapiro, “Sparse Representation for Color Image Restoration,” IEEE Trans. on Image Processing, vol. 17, no. 1, pages 53-69, Jan. 2008.
15.J. Mairal, F. Bach, J. Ponce, G. Sapiro and A. Zisserman, “Non-Local Sparse Models for Image Restoration,” International Conference on Computer Vision, Tokyo, Japan, 2009.
16.M. Irani and S. Peleg, “Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency,” Journal of Visual Communication and Image Representation, vol. 4, no. 4, pp. 324-335, Dec. 1993.
17.S. D. Babacan, R. Molina, and A. K. Katsaggelos, “Total variation super resolution using a variational approach,” in Proc. Int. Conf. Image Proc, pp. 641-644, Oct. 2008.
18.J. Oliveira, J. M. Bioucas-Dia, M. Figueiredo, and, “Adaptive total variation image deblurring: a majorization-minimization approach,” Signal Processing, vol. 89, no. 9, pp. 1683-1693, Sep. 2009.
19.M. Lysaker and X. Tai, “Iterative image restoration combining total variation minimization and a second-order functional,” International Journal of Computer Vision, vol. 66, no. 1, pp. 5-18, 2006.
20.X. Zhang, M. Burger, X. Bresson, and S. Osher, “Bregmanized nonlocal regularization for deconvolution and sparse reconstruction,” 2009. UCLA CAM Report (09-03).
21.D. L. Donoho, “De-noising by soft thresholding,” IEEE Trans. Information Theory, vol. 41, pp. 613-627, May 1995.
22.M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, Dec. 2006.
23.M. J. Fadili and J. L. Starck, “Sparse representation-based image deconvolution by iterative thresholding,” Astronomical Data Analysis, Marseille, France, Sep. 2006.
24.J. Bobin, J. Starck, J. Fadili, Y. Moudden, and D. Donoho, “Morphological Component Analysis: An Adaptive Thresholding Strategy”, IEEE Trans. Image processing, vol. 16, no. 11, pp. 2675-2681, 2007.
25.J. Yang, J. Wright, Y. Ma, and T. Huang, “Image super-resolution as sparse representation of raw image patches,” IEEE Computer Vision and Pattern Recognition, Jun. 2008.
26.M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006.
27.J. Mairal, G. Sapiro, and M. Elad, “Learning Multiscale Sparse Representations for Image and Video Restoration,” SIAM Multiscale Modeling and Simulation, vol. 7, no. 1, pages 214-241, April 2008.
28.R. Rubinstein, M. Zibulevsky, and M. Elad, “Double sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation,” IEEE Trans. Signal Processing, vol. 58, no. 3, pp. 1553-1564, March 2010.
29.J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, “Supervised dictionary learning,” Advances in Neural Information Processing Systems (NIPS’08), pp. 1033-1040.
30.G. Monaci and P. Vanderqheynst, “Learning structured dictionaries for image representation,” in Proc. IEEE Int. conf. Image Process., pp. 2351-2354, Oct. 2004.
31.R. Rubinstein, A.M. Bruckstein, and M. Elad, “Dictionaries for sparse representation modeling,” Proceedings of IEEE, Special Issue on Applications of Compressive Sensing & Sparse Representation, vol. 98, no. 6, pp. 1045-1057, June, 2010.
32.B. K. Gunturk, A. U. Batur, Y. Altunbasak, M. H. Hayes III, and R. M. Mersereau, “Eigenface-based super-resolution for face recognition,” in Proc. Int. conf. Image Process., pp. 845-848, Oct. 2002.
33.X. Wu, K. U. Barthel, and W. Zhang, “Piecewise 2D autoregression for predictive image coding,” in Proc. Int. Conf. Image Process., vol. 3, pp. 901-904, Oct. 1998.
34.X. Li and M. T. Orchard, “New edge-directed interpolation,” IEEE Trans. Image Process., vol. 10, no. 10, pp. 1521-1527, Oct. 2001.
35.X. Zhang and X. Wu, “Image interpolation by 2-D autoregressive modeling and soft-decision estimation,” IEEE Trans. Image Process., vol. 17, no. 6, pp. 887-896, Jun. 2008.
36.A. Buades, B. Coll, and J. M. Morel, “A review of image denoising algorithms, with a new one,” Multisc. Model. Simulat., vol. 4, no. 2, pp. 490.530, 2005.
37.K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd Edition, Academic Press, 1991.
38.L. Zhang, R. Lukac, X. Wu, and D. Zhang, “PCA-based spatially adaptive denoising of CFA images for single-sensor digital cameras,” IEEE Trans. Image Process., vol. 18, no. 4, pp. 797-812, Apr. 2009.
39.L. Zhang, W. Dong, D. Zhang, and G. Shi, “Two-stage image denoising by principal component analysis with local pixel grouping,” Pattern Recognition, vol. 43, pp. 1531-1549, Apr. 2010.
40.X. Wu, X. Zhang, and J. Wang, “Model-guided adaptive recovery of compressive sensing,” in Proc. Data Compression Conference, pp. 123-132, 2009.
41.M. Protter, M. Elad, H. Takeda, and P. Milanfar, “Generalizing the nonlocal-means to super-resolution reconstruction,” IEEE Trans. On Image Process., vol. 18, no. 1, pp. 36-51, Jan. 2009.
42.A. Beck and M. Teboulle, “Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems,” IEEE Trans. On Image Process., vol. 18, no. 11, pp. 2419-2434, Nov. 2009.
43.S. Dai, M. Han, W. Xu, Y. Wu, Y. Gong, and A. K. Katsaggelos, “SoftCuts: a soft edge smoothness prior for color image super-resolution,” IEEE Trans. Image Process., vol. 18, no. 5, pp. 969-981, May 2009.
44.Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error measurement to structural similarity,” IEEE Trans. Image Process., vol. 3, no. 4, pp. 600-612, Apr. 2004.
45.G. Chantas, N. P. Galatsanos, R. Molina, A. K. Katsaggelos, “Variational Bayesian image restoration with a product of spatially weighted total variation image priors,” IEEE Trans. Image Process., vol. 19, no. 2, pp. 351-362, Feb. 2010.
46.J. Portilla, “Image restoration through l0 analysis-based sparse optimization in tight frames,” in Proc. IEEE Int. conf. Image Process., pp. 3909-3912, Nov. 2009.
47.A. Marquina, and S. J. Osher, “Image super-resolution by TV-regularization and Bregman iteration,” J. Sci. Comput., vol. 37, pp. 367-382, 2008.
48.S. Mallat and Z. Zhang, “Matching pursuits with time-frequency dictionaries,” IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
49.S. Chen, D. Donoho, and M. Saunders, “Atomic decompositions by basis pursuit,” SIAM Review, vol. 43, pp. 129-159, 2001.
50.W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Comput. Graph. Appli., pp. 56-65, 2002.
51.G. Narkiss and M. Zibulevsky, “Sequential subspace optimization method for large-scale unconstrained optimization,” Technical Report CCIT No. 559, Technion, Israel Institute of Technology, Haifa, 2005.
52.A. Beck and M. Teboulle, “A fast iterative shrinkage thresholding algorithm for linear inverse problems,” SIAM J. Imaging Sciences, vol. 2, no. 1:183202, 2009.
53.M. Elad, I. Yavneh, “A plurality of sparse representation is better than the sparsest one alone,” IEEE Trans. Information Theory, vol. 55, no. 10, pp. 4701-4714, Oct. 2009.
54.M. Protter, I. Yavneh, and M. Elad, “Closed-form MMSE estimation for signal denoising under sparse representation modeling over a unitary dictionary,” IEEE Trans. Signal Processing, vol. 58, no. 7, pp. 3471-3484, July, 2010.
55.A. M. Bruckstein, D. L. Donoho, and M. Elad, “From sparse solutions of systems of equations to sparse modeling of signals and images,” SIAM Review, vol. 51, no. 1, pp. 34-81, Feb. 2009.
56.K. Engan, S. Aase, and J. Husoy, “Multi-frame compression: theory and design,” Signal Processing, vol. 80, no. 10, pp. 2121-2140, Oct. 2000.
57.E. J. Candes, “Compressive sampling,” in Proc. of the International Congress of Mathematicians, Madrid, Spain, Aug. 2006.
58.K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image restoration by sparse 3D transform-domain collaborative filtering,” in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol. 6812, 2008.
59.E. Candes, M. B. Wakin, and S. P. Boyd, “Enhancing sparsity by reweighted l1 minimization,” Journal of Fourier Analysis and Applications, vol. 14, pp. 877-905, Dec. 2008.
60.J. A. Tropp and S. J. Wright, “Computational methods for sparse solution of linear inverse problems,” Proceedings of IEEE, Special Issue on Applications of Compressive Sensing & Sparse Representation, vol. 98, no. 6, pp. 948-958, June, 2010.
61.D. Field, “What is the goal of sensory coding?” Neural Computation, vol. 6, pp. 559-601, 1994.
62.B. Olshausen and D. Field, “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature, vol. 381, pp. 607-609, 1996.
63.B. Olshausen and D. Field, “Sparse coding with an overcomplete basis set: a strategy employed by V1?” Vision Research, vol. 37, pp. 3311-3325, 1997.

Citation

MLA

Weisheng Dong, et al. “Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization”. IEEE Transactions on Image Processing, vol. 20, no. 7, 2011, pp. 1838–57, https://doi.org/10.1109/TIP.2011.2108306.

APA

Weisheng Dong, Lei Zhang, Guangming Shi, & Xiaolin Wu. (2011). Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization. IEEE Transactions on Image Processing, 20(7), 1838–1857. https://doi.org/10.1109/TIP.2011.2108306

Chicago

Weisheng Dong, Lei Zhang, Guangming Shi, and Xiaolin Wu. 2011. “Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization”. IEEE Transactions on Image Processing 20 (7): 1838–57. https://doi.org/10.1109/TIP.2011.2108306.

Harvard

Weisheng Dong et al. (2011) “Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization”, IEEE Transactions on Image Processing, 20(7), pp. 1838–1857. Available at: https://doi.org/10.1109/TIP.2011.2108306.

Vancouver

1. Weisheng Dong, Lei Zhang, Guangming Shi, Xiaolin Wu (2011) Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization. IEEE Transactions on Image Processing 20:1838–1857

BibTeX

@article{Weisheng_Dong_2011, title={Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization}, volume={20}, ISSN={1941-0042}, url={http://dx.doi.org/10.1109/TIP.2011.2108306}, DOI={10.1109/tip.2011.2108306}, number={7}, journal={IEEE Transactions on Image Processing}, publisher={Institute of Electrical and Electronics Engineers (IEEE)}, author={Weisheng Dong and Lei Zhang and Guangming Shi and Xiaolin Wu}, year={2011}, month=July, pages={1838–1857} }

Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Method	Super-Resolution ( $\sigma_n=0$ )	Super-Resolution ( $\sigma_n=5$ )
ASDS-AR-NL-TD2	27.53 dB (0.7975)	26.56 dB (0.7444)
Yang et al.	26.26 dB (0.7444)	25.34 dB (0.6711)
Marquina–Osher TV	26.09 dB (0.7705)	25.31 dB (0.7156)

Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — ASDS-AReg Image Restoration Framework

Knowl 2 — Learning Compact Sub-Dictionaries via Truncated Principal Component Analysis

Knowl 3 — Robust Sub-Dictionary and AR Model Selection via Subspace Projection

Knowl 4 — MAP-Based Non-Local Adaptive Sparsity Parameter Estimation

Knowl 5 — Autoregressive Model Regularization

Knowl 6 — Non-Local Self-Similarity Regularization

Knowl 7 — Iterative Shrinkage Algorithm for ASDS-AReg Image Restoration

Knowl 8 — Deblurring and Super-Resolution Performance on Benchmark Images

Knowl 9 — Quantitative Restoration Results on 1000 Natural Images Dataset

Knowl 10 — Robustness to Number of Clusters and Patch Size Selection

References

Citation

BibTeX

Access the Paper