Back

MemNet: A Persistent Memory Network for Image Restoration

Ying TaiJian YangXiaoming LiuChunyan Xu

article2017ICCV1,748 citations

Introduces MemNet, a persistent memory network that integrates recursive and gating units to overcome long-term dependency loss in deep models, delivering superior performance across image denoising, super-resolution, and JPEG deblocking.

Listen

Image restoration is a foundational capability in digital imaging, tasked with reconstructing clean, high-fidelity images from inputs degraded by noise, low resolution, or compression artifacts. While deep learning models have become the standard solution for these tasks, increasing network depth often leads to the long-term dependency problem, where earlier learned information fades and fails to influence deeper processing stages. This degradation of information flow limits the ability of deep neural networks to accurately recover fine details and textures.

The article demonstrates an 80-layer neural network architecture called MemNet, which introduces persistent memory to resolve long-term dependency issues in deep image restoration models. The system evaluates the effectiveness of this memory framework across three primary restoration applications: image denoising, single-image super-resolution, and compression artifact removal.

The approach introduces a modular memory block comprising a recursive unit and an adaptive gate unit. The recursive unit generates short-term representations across different levels of detail, while dense connections supply long-term representations from preceding blocks. The gate unit dynamically regulates how much previous information to retain and how much new information to store. MemNet couples this mechanism with multi-level supervised training and residual learning. The authors validated the framework across standard benchmark datasets, including the Berkeley Segmentation Dataset, Set5, Set14, Urban100, Classic5, and LIVE1, testing performance across varying noise levels, scaling factors, and compression quality settings.

The findings confirm that MemNet achieves state-of-the-art restoration quality across all three evaluated applications. First, MemNet outperforms existing models in numerical image quality metrics while producing sharper edges, clearer patterns, and fewer artifacts. Second, structural ablation tests show that dense long-term connections are essential for preserving and recovering mid-to-high frequency details that typical feedforward networks lose. Third, MemNet demonstrates superior parameter and data efficiency; an 80-layer configuration with roughly 677,000 parameters achieved better reconstruction accuracy than competing 20-layer models requiring more than 1.7 million parameters and larger training datasets. Finally, depth scaling experiments revealed continuous performance gains as depth increased up to 212 layers.

These results demonstrate that a single, unified deep learning architecture can effectively handle multiple restoration tasks and varying corruption levels without requiring task-specific structural redesigns. By solving the long-term dependency challenge through feature-level gating rather than expanding model parameter width, organizations can achieve higher image fidelity at lower parameter footprints. This balance offers practical advantages for image-processing pipelines by controlling memory overhead while improving visual quality.

For practical adoption, engineering teams should evaluate MemNet as a unified baseline for image enhancement tasks, adjusting the number of memory blocks to balance latency and reconstruction accuracy for specific hardware targets. While the model delivers high confidence across standard synthetic benchmarks, future work should evaluate performance on real-world sensor corruptions and explore deployment optimizations for real-time edge environments.

Michigan State UniversityNanjing University of Science and Technologyadaptive learningconvolutional neural networksdeblockingMemNetpersistent memory networkimage restoration
arXiv: 1708.02209tyshiwo/MemNet
Cover for MemNet: A Persistent Memory Network for Image Restoration

Abstract

Recently, very deep convolutional neural networks (CNNs) have been attracting considerable attention in image restoration. However, as the depth grows, the long-term dependency problem is rarely realized for these very deep models, which results in the prior states/layers having little influence on the subsequent ones. Motivated by the fact that human thoughts have persistency, we propose a very deep persistent memory network (MemNet) that introduces a memory block, consisting of a recursive unit and a gate unit, to explicitly mine persistent memory through an adaptive learning process. The recursive unit learns multi-level representations of the current state under different receptive fields. The representations and the outputs from the previous memory blocks are concatenated and sent to the gate unit, which adaptively controls how much of the previous states should be reserved, and decides how much of the current state should be stored. We apply MemNet to three image restoration tasks, i.e., image denosing, super-resolution and JPEG deblocking. Comprehensive experiments demonstrate the necessity of the MemNet and its unanimous superiority on all three tasks over the state of the arts. Code is available at this https URL.

Topics

adaptive learningconvolutional neural networksdeblockingimage restorationjpegrandom access memorysuperresolutionimage denoising

Keywords

MemNetpersistent memory networkimage restorationimage superresolutionimage denoisingmemory blocklong-term memoryconvolutional auto-encoder

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 MemNet for Image Restoration
  • 3.1 Basic Network Architecture
  • 3.2 Memory Block
  • 3.3 Multi-Supervised MemNet
  • 3.4 Dense Connections for Image Restoration
  • 4 Discussions
  • 5 Experiments
  • 5.1 Implementation Details
  • 5.2 Ablation Study
  • 5.3 Gate Unit Analysis
  • 5.4 Comparision with Non-Persistent CNN Models
  • 5.5 Comparisons with State-of-the-Art Models
  • 5.6 Comparison on Different Network Depths
  • 6 Conclusions
  • References

Knowls

  1. Knowl 1 — Memory Block with Recursive and Gating Mechanisms

    model/method

    The core building block of the Persistent Memory Network (MemNet) is the memory block, designed to solve the long-term dependency problem in deep restoration networks by combining short-term and long-term memories through an adaptive gating mechanism.

    A memory block Mm\mathcal{M}_m (m∈{1,2,…,M}m \in \{1, 2, \dots, M\}) consists of a recursive unit and a gate unit:

    1. Recursive Unit (Short-Term Memory): Generates multi-level representations under different receptive fields using recursive applications of a 2-convolutional-layer residual block with pre-activation (Batch Normalization followed by ReLU, denoted τ\tau). Given the input Bm−1B_{m-1} to the mm-th memory block, the rr-th recursion out of RR total recursions is formulated as:

    Hmr=Rm(Hmr−1)=F(Hmr−1,Wm)+Hmr−1H_m^r = \mathcal{R}_m(H_m^{r-1}) = \mathcal{F}(H_m^{r-1}, W_m) + H_m^{r-1}

    F(Hmr−1,Wm)=Wm2τ(Wm1τ(Hmr−1))\mathcal{F}(H_m^{r-1}, W_m) = W_m^2 \tau(W_m^1 \tau(H_m^{r-1}))

    where Hm0=Bm−1H_m^0 = B_{m-1}, Wm={Wm1,Wm2}W_m = \{W_m^1, W_m^2\} are the shared convolutional weights across all RR recursions in block mm, and HmrH_m^r is the rr-th intermediate representation. The short-term memory tensor is the concatenation across all RR recursive states:

    Bmshort=[Hm1,Hm2,…,HmR]B_m^{\text{short}} = [H_m^1, H_m^2, \dots, H_m^R]

    1. Long-Term Memory: Formed by concatenating the initial extracted feature map B0B_0 and all preceding memory block outputs:

    Bmlong=[B0,B1,…,Bm−1]B_m^{\text{long}} = [B_0, B_1, \dots, B_{m-1}]

    1. Gate Unit (Persistent Memory Mechanism): Concatenates both memory types into Bmgate=[Bmshort,Bmlong]B_m^{\text{gate}} = [B_m^{\text{short}}, B_m^{\text{long}}] and feeds them into a 1×11 \times 1 convolutional layer with weights WmgateW_m^{\text{gate}} followed by activation τ\tau:

    Bm=fmgate(Bmgate)=Wmgateτ(Bmgate)B_m = f_m^{\text{gate}}(B_m^{\text{gate}}) = W_m^{\text{gate}} \tau(B_m^{\text{gate}})

    The learned channel-wise weights in WmgateW_m^{\text{gate}} adaptively regulate how much long-term memory to preserve from earlier states and how much new short-term information from the current state to store into BmB_m.

  2. Knowl 2 — Persistent Memory Network (MemNet) Architecture

    model/method

    The Persistent Memory Network (MemNet) is an end-to-end, deep convolutional architecture for image restoration tasks (such as image denoising, super-resolution, and JPEG deblocking) that maintains long-range feature propagation across stacked memory blocks.

    The basic architecture consists of three sequential modules:

    1. Feature Extraction Net (FENet): A single convolutional layer with 64 filters of kernel size 3×33 \times 3 that extracts initial feature representations B0B_0 from a corrupted low-quality input image xx (grayscale or luminance channel):

    B0=fext(x)B_0 = f_{\text{ext}}(x)

    1. Stacked Memory Blocks: A sequence of MM memory blocks M1,M2,…,MM\mathcal{M}_1, \mathcal{M}_2, \dots, \mathcal{M}_M. Each block receives its immediate predecessor's output along with dense long-term skip connections from all prior block outputs and B0B_0:

    Bm=Mm(Bm−1)=Mm(Mm−1(…(M1(B0))… ))B_m = \mathcal{M}_m(B_{m-1}) = \mathcal{M}_m(\mathcal{M}_{m-1}(\dots(\mathcal{M}_1(B_0))\dots))

    1. Reconstruction Net (ReconNet) with Global Residual Learning: A single 3×33 \times 3 convolutional layer frecf_{\text{rec}} with 1 filter (or 64 filters internally) that reconstructs the restoration residual rather than predicting the target image directly:

    y=D(x)=frec(BM)+xy = \mathcal{D}(x) = f_{\text{rec}}(B_M) + x

    In a standard 80-layer configuration (M=6M=6 memory blocks, each containing R=6R=6 recursions of 2-layer residual units, plus FENet and ReconNet), the network has 677K parameters when all intermediate convolutions use 64 feature channels.

  3. Knowl 3 — Multi-Supervised MemNet Training Objective

    model/method

    To enhance gradient propagation and leverage intermediate state representations, MemNet employs a multi-supervision strategy where every memory block generates an intermediate restored image through a shared reconstruction network, combined into a learned weighted ensemble.

    For a network with MM memory blocks, the intermediate prediction ymy_m at block mm is generated by:

    ym=f^rec(x,Bm)=x+frec(Bm)y_m = \hat{f}_{\text{rec}}(x, B_m) = x + f_{\text{rec}}(B_m)

    where frecf_{\text{rec}} is the shared reconstruction network, xx is the degraded input image, and BmB_m is the output of the mm-th memory block.

    The final prediction yy is computed as a weighted average over all MM intermediate predictions:

    y=∑m=1Mwm⋅ymy = \sum_{m=1}^M w_m \cdot y_m

    where {wm}m=1M\{w_m\}_{m=1}^M are scalar weights learned automatically via backpropagation during training.

    Given NN training pairs {(x(i),x~(i))}i=1N\{(x^{(i)}, \tilde{x}^{(i)})\}_{i=1}^N, where x~(i)\tilde{x}^{(i)} is the ground truth image corresponding to corrupted patch x(i)x^{(i)}, the total loss function is:

    L(Θ)=α2N∑i=1N∥x~(i)−∑m=1Mwm⋅ym(i)∥2+1−α2MN∑m=1M∑i=1N∥x~(i)−ym(i)∥2\mathcal{L}(\Theta) = \frac{\alpha}{2N} \sum_{i=1}^N \left\| \tilde{x}^{(i)} - \sum_{m=1}^M w_m \cdot y_m^{(i)} \right\|^2 + \frac{1-\alpha}{2MN} \sum_{m=1}^M \sum_{i=1}^N \|\tilde{x}^{(i)} - y_m^{(i)}\|^2

    where Θ\Theta denotes the network parameters, and α\alpha is a loss weight balancing the ensemble prediction against the individual intermediate predictions, set empirically to α=1M+1\alpha = \frac{1}{M + 1}.

  4. Knowl 4 — Frequency Compensation via Dense Long-Term Connections in Deep Restoration Networks

    empirical result

    In deep feedforward convolutional neural networks for image restoration, mid- and high-frequency spatial components attenuate as depth increases. Dense long-term connections from early to late memory blocks counteract this degradation by directly injecting preserved frequency components into deeper layers.

    To quantify this effect, 2D power spectra of intermediate outputs from an 80-layer MemNet (with 6 memory blocks, M=6M=6, R=6R=6) were compared against an identical architecture without long-term connections (MemNet_NL):

    1. 2D power spectra of intermediate image outputs at block 4 and block 6 were converted to 1D spectral densities by radially integrating power along concentric frequency rings from low to high frequency.
    2. In MemNet_NL, the 4th memory block retains more mid-frequency content than the 6th memory block, verifying that deeper feedforward layers without persistent memory lose mid-frequency signals.
    3. In full MemNet, the 6th memory block absorbs early-layer frequency components via dense skip connections and recovers significantly more high- and mid-frequency spectral density than the 4th block and outperforms the 6th block of MemNet_NL.

  5. Knowl 5 — Memory Dependency Analysis via Gate Unit Filter Weight Norms

    model/method

    To analyze how the gating mechanism in MemNet values different memory representations across network depth, a channel-wise weight norm is computed for each feature map entering the gate unit.

    For the mm-th memory block's gate unit (a 1×11 \times 1 convolution with weight tensor Wmgate∈R1×1×Lm×64W_m^{\text{gate}} \in \mathbb{R}^{1 \times 1 \times L_m \times 64}, where LmL_m is the total number of incoming feature channels), the dependency metric vmlv_m^l for the ll-th feature map (l∈{1,2,…,Lm}l \in \{1, 2, \dots, L_m\}) is:

    vml=∑i=164(Wmgate(1,1,l,i))2v_m^l = \sqrt{\sum_{i=1}^{64} \left(W_m^{\text{gate}}(1, 1, l, i)\right)^2}

    Normalized to [0,1][0, 1], larger values of vmlv_m^l indicate stronger dependency on the ll-th feature map.

    Empirical evaluation across image denoising, super-resolution, and JPEG deblocking reveals three consistent patterns:

    1. The short-term memory generated by the final recursion of the recursive unit (the last 64 channels in BmshortB_m^{\text{short}}) consistently exhibits the highest average weight norm across all blocks.
    2. Long-term memories (BmlongB_m^{\text{long}}) play an increasingly vital role in later memory blocks compared to early memory blocks, receiving higher relative weight norms than the intermediate short-term recursions (recursions 11 to R−1R-1).
    3. The average and variance of the gate weight norms decrease monotonically as the memory block index mm increases, indicating smoother feature integration in deeper blocks.

  6. Knowl 6 — Ablation Study on Long-Term and Short-Term Memory Connections

    data/table

    An ablation study evaluated the relative importance of long-term dense connections and short-term recursive connections in MemNet. Three networks with identical depth (80 layers) and channel width (64 filters) were trained on SISR and tested on the Set5 benchmark across upscaling factors ×2\times 2, ×3\times 3, and ×4\times 4:

    • MemNet_NL: Removes all long-term connections across memory blocks.
    • MemNet_NS: Removes short-term connections from recursions 11 to R−1R-1 within each recursive unit to the gate unit (retaining only the final RR-th recursion to preserve information flow).
    • MemNet: The full persistent memory network.
    Scale MemNet_NL MemNet_NS MemNet
    ×2\times 2 37.68 / 0.9591 37.71 / 0.9592 37.78 / 0.9597
    ×3\times 3 33.96 / 0.9235 34.00 / 0.9239 34.09 / 0.9248
    ×4\times 4 31.60 / 0.8878 31.65 / 0.8880 31.74 / 0.8893

    Values represent average PSNR (dB) / SSIM. MemNet significantly outperforms MemNet_NL (e.g., +0.13+0.13 dB at ×3\times 3, +0.14+0.14 dB at ×4\times 4) and MemNet_NS (e.g., +0.09+0.09 dB at ×3\times 3, +0.09+0.09 dB at ×4\times 4), demonstrating that while both connection types contribute to restoration quality, long-term connections across memory blocks have the greater impact because they span more layers and prevent loss of high-frequency information.

  7. Knowl 7 — Performance Scaling across MemNet Depth and Recursions

    data/table

    The performance of MemNet scales positively with network depth when increasing either the number of stacked memory blocks MM or the number of recursions RR per block. Evaluated on the Set5 dataset for single-image super-resolution at scale factor ×3\times 3, the configurations yield:

    Configuration M4R6 M6R6 M6R8 M10R10
    Total Depth (conv layers) 54 80 104 212
    PSNR (dB) on Set5 (×3\times 3) 34.05 34.09 34.16 34.23

    Total convolutional depth is given by 1+M×(2R+1)+11 + M \times (2R + 1) + 1 (1 feature extraction layer, MM memory blocks containing RR residual blocks of 2 layers each plus 1 gate layer, and 1 reconstruction layer). Increasing depth from 54 layers (M4R6) to 212 layers (M10R10) consistently improves PSNR, achieving a +0.18+0.18 dB overall gain without encountering optimization collapse.

  8. Knowl 8 — Efficiency and Parameter Comparison against Non-Persistent Deep Restoration Models

    data/table

    MemNet achieves superior single-image super-resolution (SISR) accuracy compared to prior deep non-persistent architectures while maintaining a compact parameter footprint through recursive parameter sharing within memory blocks.

    The table compares VDSR, DRCN, RED, and various training/supervision variants of 80-layer MemNet (with 64 filters) on the Set5 benchmark for scale factor ×3\times 3:

    Model VDSR DRCN RED MemNet (Basic) MemNet (Multi-Sup) MemNet (Full)
    Depth 20 20 30 80 80 80
    Filters 64 256 128 64 64 64
    Parameters 665K 1,774K 4,131K 677K 677K 677K
    Training Images 291 91 300 91 91 291
    Multi-Supervision No Yes No No Yes Yes
    PSNR (dB) 33.66 33.82 33.82 33.92 33.98 34.09

    Even when constrained to 91 training images and no multi-supervision, the 80-layer basic MemNet achieves 33.92 dB PSNR—outperforming the 1,774K-parameter DRCN (33.82 dB) and 4,131K-parameter RED (33.82 dB) while using only 677K parameters. Adding multi-supervision and expanding the dataset to 291 images boosts performance to 34.09 dB.

  9. Knowl 9 — Single-Image Super-Resolution Benchmark Evaluation

    data/table

    The multi-supervised 80-layer MemNet (M6R6, 64 filters) was evaluated against state-of-the-art super-resolution methods across three scaling factors (×2,×3,×4\times 2, \times 3, \times 4) on four standard benchmarks: Set5, Set14, BSD100, and Urban100. Evaluation used the luminance (Y) channel, with metrics reported as average PSNR (dB) / SSIM:

    Dataset Scale Bicubic SRCNN VDSR DRCN DnCNN DRRN MemNet
    Set5 ×2\times 2 33.66/0.9299 36.66/0.9542 37.53/0.9587 37.63/0.9588 37.58/0.9590 37.74/0.9591 37.78/0.9597
    ×3\times 3 30.39/0.8682 32.75/0.9090 33.66/0.9213 33.82/0.9226 33.75/0.9222 34.03/0.9244 34.09/0.9248
    ×4\times 4 28.42/0.8104 30.48/0.8628 31.35/0.8838 31.53/0.8854 31.40/0.8845 31.68/0.8888 31.74/0.8893
    Set14 ×2\times 2 30.24/0.8688 32.45/0.9067 33.03/0.9124 33.04/0.9118 33.03/0.9128 33.23/0.9136 33.28/0.9142
    ×3\times 3 27.55/0.7742 29.30/0.8215 29.77/0.8314 29.76/0.8311 29.81/0.8321 29.96/0.8349 30.00/0.8350
    ×4\times 4 26.00/0.7027 27.50/0.7513 28.01/0.7674 28.02/0.7670 28.04/0.7672 28.21/0.7721 28.26/0.7723
    BSD100 ×2\times 2 29.56/0.8431 31.36/0.8879 31.90/0.8960 31.85/0.8942 31.90/0.8961 32.05/0.8973 32.08/0.8978
    ×3\times 3 27.21/0.7385 28.41/0.7863 28.82/0.7976 28.80/0.7963 28.85/0.7981 28.95/0.8004 28.96/0.8001
    ×4\times 4 25.96/0.6675 26.90/0.7101 27.29/0.7251 27.23/0.7233 27.29/0.7253 27.38/0.7284 27.40/0.7281
    Urban100 ×2\times 2 26.88/0.8403 29.50/0.8946 30.76/0.9140 30.75/0.9133 30.74/0.9139 31.23/0.9188 31.31/0.9195
    ×3\times 3 24.46/0.7349 26.24/0.7989 27.14/0.8279 27.15/0.8276 27.15/0.8276 27.53/0.8378 27.56/0.8376
    ×4\times 4 23.14/0.6577 24.52/0.7221 25.18/0.7524 25.14/0.7510 25.20/0.7521 25.44/0.7638 25.50/0.7630

    A single MemNet model trained across scale factors outperforms prior deep learning methods across all test sets and upscaling factors.

  10. Knowl 10 — Image Denoising Benchmark Evaluation

    data/table

    The multi-supervised 80-layer MemNet was benchmarked for Gaussian image denoising on two standard test sets: a 14-image classic benchmark and the BSD200 test set (with images downsampled to half resolution following prior protocol). Synthetic additive white Gaussian noise at three levels (standard deviation σ∈{30,50,70}\sigma \in \{30, 50, 70\}) was added to clean images. A single unified MemNet model was trained with noise-level augmentation.

    Dataset σ\sigma BM3D EPLL PCLR PGPD WNNM RED MemNet
    14 images 30 28.49/0.8204 28.35/0.8200 28.68/0.8263 28.55/0.8199 28.74/0.8273 29.17/0.8423 29.22/0.8444
    50 26.08/0.7427 25.97/0.7354 26.29/0.7538 26.19/0.7442 26.32/0.7517 26.81/0.7733 26.91/0.7775
    70 24.65/0.6882 24.47/0.6712 24.79/0.6997 24.71/0.6913 24.80/0.6975 25.31/0.7206 25.43/0.7260
    BSD200 30 27.31/0.7755 27.38/0.7825 27.54/0.7827 27.33/0.7717 27.48/0.7807 27.95/0.8019 28.04/0.8053
    50 25.06/0.6831 25.17/0.6870 25.30/0.6947 25.18/0.6841 25.26/0.6928 25.75/0.7167 25.86/0.7202
    70 23.82/0.6240 23.81/0.6168 23.94/0.6336 23.89/0.6245 23.95/0.6346 24.37/0.6551 24.53/0.6608

    Results are reported in average PSNR (dB) / SSIM. MemNet achieves the highest score in every setting without requiring post-processing test-time augmentation (such as rotation or mirror-flip ensemble averaging used by RED).

  11. Knowl 11 — JPEG Compression Artifacts Reduction Benchmark Evaluation

    data/table

    MemNet was evaluated for JPEG deblocking on the Classic5 and LIVE1 benchmark datasets at JPEG quality factors q=10q = 10 and q=20q = 20, using MATLAB's JPEG encoder for corruption.

    Dataset Quality (qq) JPEG ARCNN TNRD DnCNN MemNet
    Classic5 10 27.82 / 0.7595 29.03 / 0.7929 29.28 / 0.7992 29.40 / 0.8026 29.69 / 0.8107
    20 30.12 / 0.8344 31.15 / 0.8517 31.47 / 0.8576 31.63 / 0.8610 31.90 / 0.8658
    LIVE1 10 27.77 / 0.7730 28.96 / 0.8076 29.15 / 0.8111 29.19 / 0.8123 29.45 / 0.8193
    20 30.07 / 0.8512 31.29 / 0.8733 31.46 / 0.8769 31.59 / 0.8802 31.83 / 0.8846

    Results show average PSNR (dB) / SSIM on the luminance channel. MemNet outperforms all previous methods, surpassing the 20-layer DnCNN by +0.29+0.29 dB / +0.27+0.27 dB on Classic5 (q=10,20q=10, 20) and by +0.26+0.26 dB / +0.24+0.24 dB on LIVE1 (q=10,20q=10, 20).

Coverage note — None was omitted; all key architectural components (MemNet, memory blocks, multi-supervision), analytical contributions (frequency spectra, gate weight norms), ablation studies, depth comparisons, and all benchmark evaluations across the three restoration tasks (denoising, super-resolution, and deblocking) are fully covered.

References

  1. 1.C. M. Bevilacqua, A. Roumy, and M. Morel. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In BMVC, 2012.
  2. 2.F. Chen, L. Zhang, and H. Yu. External patch prior guided internal clustering for image denoising. In ICCV, 2015.
  3. 3.Y. Chen and T. Pock. Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE Trans. on PAMI, 2016.
  4. 4.J. Cichon and W. Gan. Branch-specific dendritic ca2+ spikes cause persistent synaptic plasticity. Nature, 520(7546):180–185, 2015.
  5. 5.K. Dabov, A. Foi, V. Katkovnik, and K. O. Egiazarian. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. on IP, 16(8):2080–2095, 2007.
  6. 6.P. Dayan and L. F. Abbott. Theoretical neuroscience. Cambridge, MA: MIT Press, 2001.
  7. 7.C. Dong, Y. Deng, C. C. Loy, and X. Tang. Compression artifacts reduction by a deep convolutional network. In ICCV, 2015.
  8. 8.C. Dong, C. Loy, K. He, and X. Tang. Image super-resolution using deep convolutional networks. IEEE Trans. on PAMI, 38(2):295–307, 2016.
  9. 9.S. Gu, L. Zhang, W. Zuo, and X. Feng. Weighted nuclear norm minimization with application to image denoising. In CVPR, 2014.
  10. 10.J. Guo and H. Chao. Building dual-domain representations for compression artifacts reduction. In ECCV, 2016.
  11. 11.K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In ICCV, 2015.
  12. 12.K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016.
  13. 13.K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In ECCV, 2016.
  14. 14.G. Huang, Z. Liu, and K. Q. Weinberger. Densely connected convolutional networks. In CVPR, 2017.
  15. 15.J.-B. Huang, A. Singh, and N. Ahuja. Single image super-resolution from transformed self-exemplars. In CVPR, 2015.
  16. 16.S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
  17. 17.V. Jain and H. S. Seung. Natural image denoising with convolutional networks. In NIPS, 2008.
  18. 18.J. Jancsary, S. Nowozin, and C. Rother. Loss-specific training of non-parametric image restoration models: A new state of the art. In ECCV, 2012.
  19. 19.Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. In ACM MM, 2014.
  20. 20.J. Kim, J. K. Lee, and K. M. Lee. Accurate image super-resolution using very deep convolutional networks. In CVPR, 2016.
  21. 21.J. Kim, J. K. Lee, and K. M. Lee. Deeply-recursive convolutional network for image super-resolution. In CVPR, 2016.
  22. 22.A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
  23. 23.W. Lai, J. Huang, N. Ahuja, and M. Yang. Deep laplacian pyramid networks for fast and accurate super-resolution. In CVPR, 2017.
  24. 24.Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, 1998.
  25. 25.M. Liang and X. Hu. Recurrent convolutional neural network for object recognition. In CVPR, 2015.
  26. 26.X. Liu, X. Wu, J. Zhou, and D. Zhao. Data-driven sparsity-based restoration of jpeg-compressed images in dual transform-pixel domain. In CVPR, 2015.
  27. 27.X. Mao, C. Shen, and Y. Yang. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In NIPS, 2016.
  28. 28.D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV, 2001.
  29. 29.P. Milanfar. A tour of modern image filtering: new insights and methods, both practical and theoretical. IEEE Signal Processing Magazine, 30(1):106–128, 2013.
  30. 30.V. Nair and G. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010.
  31. 31.O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, and et al. Imagenet large scale visual recognition challenge. IJCV, 115(3):211–252, 2015.
  32. 32.R. K. Srivastava, K. Greff, and J. Schmidhuber. Highway networks. In NIPS, 2015.
  33. 33.C. Szegedy, W. Liu, Y. Jia, P. Sermanet, and S. Reed. Going deeper with convolutions. In CVPR, 2015.
  34. 34.Y. Tai, J. Yang, and X. Liu. Image super-resolution via deep recursive residual network. In CVPR, 2017.
  35. 35.Z. Wang, D. Liu, S. Chang, Q. Ling, Y. Yang, and T. S. Huang. D3: Deep dual-domain based fast restoration of jpeg-compressed images. In CVPR, 2016.
  36. 36.Z. Wang, D. Liu, J. Yang, W. Han, and T. S. Huang. Deep networks for image super-resolution with sparse prior. In ICCV, 2015.
  37. 37.J. Xu, L. Zhang, W. Zuo, D. Zhang, and X. Feng. Patch group based nonlocal self-similarity prior learning for image denoising. In ICCV, 2015.
  38. 38.J. Yang, J. Wright, T. S. Huang, and Y. Ma. Image super-resolution via sparse representation. IEEE Trans. on IP, 19(11):2861–2873, 2010.
  39. 39.R. Zeyde, M. Elad, and M. Protter. On single image scale-up using sparse-representations. Curves and Surfaces, pages 711–730, 2012.
  40. 40.K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang. Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Trans. on IP, 2017.
  41. 41.D. Zoran and Y. Weiss. From learning models of natural image patches to whole image restoration. In ICCV, 2011.

Citation

MLA
Tai, Y., et al. “MemNet: A Persistent Memory Network for Image Restoration”. arXiv, 2017, http://arxiv.org/abs/1708.02209v1.
APA
Tai, Y., Yang, J., Liu, X., & Xu, C. (2017). MemNet: A Persistent Memory Network for Image Restoration. arXiv. http://arxiv.org/abs/1708.02209v1
Chicago
Tai, Y., J. Yang, X. Liu, and C. Xu. 2017. “MemNet: A Persistent Memory Network for Image Restoration”. arXiv. http://arxiv.org/abs/1708.02209v1.
Harvard
Tai, Y. et al. (2017) “MemNet: A Persistent Memory Network for Image Restoration”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1708.02209v1.
Vancouver
1. Tai Y, Yang J, Liu X, Xu C (2017) MemNet: A Persistent Memory Network for Image Restoration. arXiv

BibTeX

@article{tai2017memnet,
  title = {MemNet: A Persistent Memory Network for Image Restoration},
  author = {Tai, Ying and Yang, Jian and Liu, Xiaoming and Xu, Chunyan},
  year = {2017},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1708.02209v1},
  eprint = {1708.02209}
}
Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF
License: IEEE