Back

End-to-end Optimized Image Compression

Johannes BalléValero LaparraEero P. Simoncelli

article2016ICLR2,185 citations

Establishes an end-to-end framework for learned image compression by jointly optimizing nonlinear transforms and a continuous quantization proxy for rate-distortion performance, outperforming traditional codecs like JPEG and JPEG 2000.

bit ratescompression methodsconvolutional neural networksend-to-end optimized image compressionnonlinear transform codinglossy compression
arXiv: 1611.01704
  • Paper: Adversarial Autoencoders, Alireza Makhzani et al. (2015). Adversarial autoencoders established foundational techniques for regularizing internal latent spaces that this paper adapts for variational rate-distortion image compression.
  • Paper: Density estimation using Real NVP, Laurent Dinh et al. (2016). Real NVP introduced invertible transformations and exact likelihood estimation for continuous variables, providing structural building blocks for normalizing flows used in compression models.
  • Paper: Neural Discrete Representation Learning, Aäron van den Oord et al. (2017). Vector Quantised-Variational AutoEncoders demonstrated how to learn compact latent spaces without supervision, establishing core representation techniques that inspired end-to-end compression architectures.
  • Paper: Variational image compression with a scale hyperprior, Johannes Ballé et al. (2018). This work directly extends the source framework by introducing a learned scale hyperprior to model spatial dependencies in latent representations, achieving new state-of-the-art compression performance.
  • Paper: Glow: Generative Flow with Invertible 1x1 Convolutions, Diederik P. Kingma et al. (2018). Building directly upon the generative and invertible transformation techniques established in the source, Glow introduces learned 1x1 convolutions to improve flow-based image synthesis and likelihood modeling.
  • Paper: High-Resolution Image Synthesis with Latent Diffusion Models, Robin Rombach et al. (2022). This paper leverages the foundational image compression autoencoder principles from the source to build latent diffusion models that enable efficient, high-resolution image synthesis.
Cover for End-to-end Optimized Image Compression

Abstract

We describe an image compression method, consisting of a nonlinear analysis transformation, a uniform quantizer, and a nonlinear synthesis transformation. The transforms are constructed in three successive stages of convolutional linear filters and nonlinear activation functions. Unlike most convolutional neural networks, the joint nonlinearity is chosen to implement a form of local gain control, inspired by those used to model biological neurons. Using a variant of stochastic gradient descent, we jointly optimize the entire model for rate-distortion performance over a database of training images, introducing a continuous proxy for the discontinuous loss function arising from the quantizer. Under certain conditions, the relaxed loss function may be interpreted as the log likelihood of a generative model, as implemented by a variational autoencoder. Unlike these models, however, the compression model must operate at any given point along the rate-distortion curve, as specified by a trade-off parameter. Across an independent set of test images, we find that the optimized method generally exhibits better rate-distortion performance than the standard JPEG and JPEG 2000 compression methods. More importantly, we observe a dramatic improvement in visual quality for all images at all bit rates, which is supported by objective quality estimates using MS-SSIM.

Topics

bit ratescompression methodsconvolutional neural networksgain controlgenerative modeljpegquantizersrate distortion performancessimstochastic

Keywords

end-to-end optimized image compressionnonlinear transform codinglossy compressionoptimal quantizationvariational autoencoderbinary arithmetic codingvector quantization

Table of Contents

  • 1 Introduction
  • 2 Choice of forward, inverse, and perceptual transforms
  • 3 Optimization of nonlinear transform coding model
  • 3.1 Relationship to variational generative image models
  • 4 Experimental results
  • 5 Discussion
  • References
  • 6 Appendix
  • 6.1 Network architecture and optimization
  • 6.2 Entropy code
  • 6.3 Evaluation details and additional example images

Knowls

  1. Knowl 1 — Nonlinear Transform Coding Framework for End-to-End Image Compression

    model/method

    The nonlinear transform coding framework compresses an image intensity vector x∈RNx \in \mathbb{R}^N by mapping it into a continuous latent representation y∈RMy \in \mathbb{R}^M via a parametric nonlinear analysis transform:

    y=ga(x;ϕ)y = g_a(x; \phi)

    where ϕ\phi is the parameter vector of the analysis transform. The continuous latent representation is quantized to integer coordinates using uniform scalar quantization with unit bin size:

    qi=round(yi)q_i = \text{round}(y_i)

    yielding a discrete-valued vector q∈ZMq \in \mathbb{Z}^M. The discrete representation is losslessly encoded into a bitstream with rate RR, lower-bounded by the discrete entropy H[Pq]=−E[log⁡2Pq(q)]H[P_q] = -\mathbb{E}[\log_2 P_q(q)], where the marginal probability mass function for each integer bin n∈Zn \in \mathbb{Z} is:

    Pqi(n)=∫n−12n+12pyi(t) dtP_{q_i}(n) = \int_{n - \frac{1}{2}}^{n + \frac{1}{2}} p_{y_i}(t) \, dt

    To reconstruct the image, qq is treated as a continuous vector y^=q\hat{y} = q and passed to a parametric nonlinear synthesis transform:

    x^=gs(y^;θ)\hat{x} = g_s(\hat{y}; \theta)

    where θ\theta is the synthesis parameter vector. The parameters ϕ\phi and θ\theta and the entropy distribution PqP_q are jointly optimized to minimize the rate-distortion objective functional:

    L[ga,gs,Pq]=−E[log⁡2Pq(q)]+λE[d(x,x^)]L[g_a, g_s, P_q] = -\mathbb{E}\left[\log_2 P_q(q)\right] + \lambda \mathbb{E}\left[d(x, \hat{x})\right]

    where d(x,x^)d(x, \hat{x}) is a distortion metric (such as mean squared error ∥x−x^∥22\|x - \hat{x}\|_2^2) and λ>0\lambda > 0 controls the trade-off between rate and distortion.

  2. Knowl 2 — Continuous Relaxation of Rate-Distortion Loss via Additive Uniform Noise

    model/method

    Because uniform scalar quantization q=round(y)q = \text{round}(y) has zero derivatives almost everywhere, gradient descent cannot directly optimize the discrete rate-distortion functional. To enable stochastic gradient optimization, the non-differentiable quantizer is replaced during training with an additive independent and identically distributed uniform noise source:

    y~=y+Δy,Δyi∼U(−12,12)\tilde{y} = y + \Delta y, \quad \Delta y_i \sim \mathcal{U}\left(-\frac{1}{2}, \frac{1}{2}\right)

    This continuous relaxation preserves two key properties:

    1. The probability density function of y~\tilde{y}, given by py~=py∗U(−1/2,1/2)p_{\tilde{y}} = p_y * \mathcal{U}(-1/2, 1/2), is a continuous relaxation of the discrete probability mass function PqP_q, satisfying py~(n)=Pq(n)p_{\tilde{y}}(n) = P_q(n) for all n∈ZMn \in \mathbb{Z}^M. Consequently, the differential entropy h[py~]h[p_{\tilde{y}}] serves as a smooth proxy for the discrete entropy H[Pq]H[P_q].
    2. The additive uniform noise matches the marginal moments of uniform quantization error, serving as a differentiable proxy for reconstruction distortion.

    The resulting relaxed differentiable loss function minimized during training is:

    L(θ,ϕ)=Ex,Δy[−∑ilog⁡2py~i(ga(x;ϕ)i+Δyi;ψ(i))+λd(gs(ga(x;ϕ)+Δy;θ),x)]L(\theta, \phi) = \mathbb{E}_{x, \Delta y} \left[ -\sum_i \log_2 p_{\tilde{y}_i}(g_a(x; \phi)_i + \Delta y_i; \psi^{(i)}) + \lambda d\left(g_s(g_a(x; \phi) + \Delta y; \theta), x\right) \right]

    where ψ(i)\psi^{(i)} parameterizes the marginal density model of the ii-th noisy latent coefficient.

  3. Knowl 3 — Analysis Transform with Generalized Divisive Normalization

    model/method

    The analysis transform y=ga(x;ϕ)y = g_a(x; \phi) maps an image x=u(0)x = u^{(0)} through three successive stages (k∈{0,1,2}k \in \{0, 1, 2\}), each comprising an affine convolution, spatial downsampling, and Generalized Divisive Normalization (GDN):

    1. Affine Convolution:
    ight)(m, n) + c_{k,i}$$ where $h_{k,ij}$ is a 2D convolution filter, $c_{k,i}$ is a channel bias, $i$ indexes output channels, $j$ indexes input channels, and $(m, n)$ are spatial coordinates. 2. **Spatial Downsampling**: $$w_i^{(k)}(m, n) = v_i^{(k)}(s_k m, s_k n)$$ where $s_k$ is the stage downsampling stride. 3. **Generalized Divisive Normalization (GDN)**: $$u_i^{(k+1)}(m, n) = \frac{w_i^{(k)}(m, n)}{\left(\beta_{k,i} + \sum_j \gamma_{k,ij} \left(w_j^{(k)}(m, n)\right)^2\right)^{\frac{1}{2}}}$$ where $\beta_{k,i} > 0$ is a learnable channel scale parameter and $\gamma_{k,ij} \ge 0$ is a symmetric cross-channel interaction weight matrix ($\gamma_{k,ij} = \gamma_{k,ji}$). For grayscale images, stage 0 uses 128 filters of size $9 \times 9$ with downsampling factor $s_0 = 4$. Stages 1 and 2 each use 128 filters of size $5 \times 5 \times 128$ with downsampling factors $s_1 = s_2 = 2$, reducing the spatial dimensionality by a net factor of 16 horizontally and vertically and yielding a latent output $y = u^{(3)}$ with half the total dimensionality of the input image.

  4. Knowl 4 — Synthesis Transform with Inverse Generalized Divisive Normalization

    model/method

    The synthesis transform x^=gs(y^;θ)\hat{x} = g_s(\hat{y}; \theta) reconstructs the image from the quantized latent vector y^=u^(0)\hat{y} = \hat{u}^{(0)} through three inverted stages (k∈{0,1,2}k \in \{0, 1, 2\}) comprising Inverse Generalized Divisive Normalization (IGDN), spatial upsampling, and affine convolution:

    1. Inverse GDN (IGDN): w^i(k)(m,n)=u^i(k)(m,n)⋅(β^k,i+∑jγ^k,ij(u^j(k)(m,n))2)12\hat{w}_i^{(k)}(m, n) = \hat{u}_i^{(k)}(m, n) \cdot \left(\hat{\beta}_{k,i} + \sum_j \hat{\gamma}_{k,ij} \left(\hat{u}_j^{(k)}(m, n)\right)^2\right)^{\frac{1}{2}} where β^k,i>0\hat{\beta}_{k,i} > 0 and γ^k,ij≥0\hat{\gamma}_{k,ij} \ge 0 (constrained to be symmetric: γ^k,ij=γ^k,ji\hat{\gamma}_{k,ij} = \hat{\gamma}_{k,ji}) are learnable synthesis parameters.

    2. Spatial Upsampling: v^i(k)(m,n)={w^i(k)(m/s^k,n/s^k)if m/s^k and n/s^k are integers0otherwise\hat{v}_i^{(k)}(m, n) = \begin{cases} \hat{w}_i^{(k)}(m/\hat{s}_k, n/\hat{s}_k) & \text{if } m/\hat{s}_k \text{ and } n/\hat{s}_k \text{ are integers} \\ 0 & \text{otherwise} \end{cases} where s^k\hat{s}_k is the upsampling factor for stage kk.

    3. Affine Convolution: u^i(k+1)(m,n)=∑j(h^k,ij∗v^j(k))(m,n)+c^k,i\hat{u}_i^{(k+1)}(m, n) = \sum_j \left(\hat{h}_{k,ij} * \hat{v}_j^{(k)}\right)(m, n) + \hat{c}_{k,i} where h^k,ij\hat{h}_{k,ij} is a 2D synthesis convolution kernel and c^k,i\hat{c}_{k,i} is a channel bias.

    For grayscale images, stages 0 and 1 use 128 filters (5×5×1285 \times 5 \times 128) with upsampling factors s^0=s^1=2\hat{s}_0 = \hat{s}_1 = 2, and stage 2 uses 128 filters (9×9×19 \times 9 \times 1) with upsampling factor s^2=4\hat{s}_2 = 4, outputting the reconstructed image x^=u^(3)\hat{x} = \hat{u}^{(3)}.

  5. Knowl 5 — Equivalence of Relaxed Rate-Distortion Optimization to Variational Autoencoders

    theoretical result

    When distortion is measured by mean squared error d(x,x^)=∥x−x^∥22d(x, \hat{x}) = \|x - \hat{x}\|_2^2, the continuous relaxation of the rate-distortion objective is mathematically equivalent to optimizing the variational evidence lower bound of a Variational Autoencoder (VAE) by minimizing the Kullback-Leibler divergence DKL(q(y~∣x) ∥ py∣x(y~∣x))D_{\text{KL}}(q(\tilde{y}|x) \,\|\, p_{y|x}(\tilde{y}|x)).

    Under this formulation:

    1. The generative conditional distribution is an isotropic Gaussian centered at the synthesis transform output: px∣y~(x∣y~;λ,θ)=N(x;gs(y~;θ),(2λ)−1I)p_{x|\tilde{y}}(x|\tilde{y}; \lambda, \theta) = \mathcal{N}\left(x; g_s(\tilde{y}; \theta), (2\lambda)^{-1}\mathbf{I}\right)
    2. The continuous prior distribution over latents assumes factorized independent marginals: py~(y~;ψ)=∏ipy~i(y~i;ψ(i))p_{\tilde{y}}(\tilde{y}; \psi) = \prod_i p_{\tilde{y}_i}(\tilde{y}_i; \psi^{(i)})
    3. The variational posterior is a factorized uniform distribution centered on the analysis transform output: q(y~∣x;ϕ)=∏iU(y~i;ga(x;ϕ)i,1)q(\tilde{y}|x; \phi) = \prod_i \mathcal{U}\left(\tilde{y}_i; g_a(x; \phi)_i, 1\right)

    Expanding the Kullback-Leibler divergence yields: DKL(q(y~∣x) ∥ py∣x(y~∣x))=Ey~∼q[log⁡q(y~∣x)]−Ey~∼q[log⁡px∣y~(x∣y~)]−Ey~∼q[log⁡py~(y~)]+constD_{\text{KL}}(q(\tilde{y}|x) \,\|\, p_{y|x}(\tilde{y}|x)) = \mathbb{E}_{\tilde{y} \sim q}\left[\log q(\tilde{y}|x)\right] - \mathbb{E}_{\tilde{y} \sim q}\left[\log p_{x|\tilde{y}}(x|\tilde{y})\right] - \mathbb{E}_{\tilde{y} \sim q}\left[\log p_{\tilde{y}}(\tilde{y})\right] + \text{const}

    Because q(y~∣x)q(\tilde{y}|x) is uniform on a unit-volume cube, Ey~∼q[log⁡q(y~∣x)]\mathbb{E}_{\tilde{y} \sim q}[\log q(\tilde{y}|x)] is constant. The log generative likelihood E[−log⁡px∣y~(x∣y~)]\mathbb{E}[-\log p_{x|\tilde{y}}(x|\tilde{y})] simplifies to λ∥x−gs(y~;θ)∥22+const\lambda \|x - g_s(\tilde{y}; \theta)\|_2^2 + \text{const}, and the log prior E[−log⁡py~(y~)]\mathbb{E}[-\log p_{\tilde{y}}(\tilde{y})] corresponds to the differential entropy rate term −∑ilog⁡2py~i(y~i;ψ(i))-\sum_i \log_2 p_{\tilde{y}_i}(\tilde{y}_i; \psi^{(i)}).

  6. Knowl 6 — Non-Parametric Piecewise Linear Marginal Density Estimation for Latents

    model/method

    The marginal probability density functions py~i(y~i;ψ(i))p_{\tilde{y}_i}(\tilde{y}_i; \psi^{(i)}) of the noisy latent variables are modeled non-parametrically as continuous piecewise linear splines with a fine uniform node spacing of 10 sampling points per unit interval. Because the density py~i=pyi∗U(−1/2,1/2)p_{\tilde{y}_i} = p_{y_i} * \mathcal{U}(-1/2, 1/2) is smoothed by convolution with a uniform distribution, the approximation error of the linear spline decreases arbitrarily as the sampling interval shrinks.

    The vector ψ(i)\psi^{(i)} of spline node heights is trained alongside the network parameters using stochastic gradient descent to minimize the negative expected log-likelihood:

    Lψ(ψ)=−Ey~[∑ilog⁡py~i(y~i;ψ(i))]L_\psi(\psi) = -\mathbb{E}_{\tilde{y}}\left[\sum_i \log p_{\tilde{y}_i}(\tilde{y}_i; \psi^{(i)})\right]

    After each gradient step, the spline parameters are renormalized to ensure that each density integrates to 1. Additionally, every 10610^6 iterations, the dynamic support range of the spline grid is heuristically expanded or adjusted to enclose the full range of y~i\tilde{y}_i values observed on the training dataset.

  7. Knowl 7 — Parameterization and Optimization Constraints for Stable Transform Coding Training

    algorithm

    Stable and rapid end-to-end convergence of the convolutional GDN/IGDN transform network is achieved through four structural and optimization constraints:

    1. Optimization Schedule: Transform parameters ϕ\phi and θ\theta are trained using the Adam optimizer initialized with learning rate α=10−4\alpha = 10^{-4}, dropped by a factor of 10 whenever joint rate and distortion improvements stall, down to α=10−7\alpha = 10^{-7}.
    2. DCT Parameterization of Convolutional Filters: Linear filter weights are parameterized in the Discrete Cosine Transform (DCT) domain rather than the spatial or discrete Fourier domains, accelerating optimization convergence.
    3. GDN/IGDN Parameter Constraints: To prevent numerical instability near zero, GDN parameters are reparameterized elementwise as: βk,i=(βk,i′)2−2−10,γk,ij=(γk,ij′)2−2−10\beta_{k,i} = (\beta'_{k,i})^2 - 2^{-10}, \quad \gamma_{k,ij} = (\gamma'_{k,ij})^2 - 2^{-10} After each gradient step, βk,i′\beta'_{k,i} and γk,ij′\gamma'_{k,ij} are projected onto [2−5,∞)[2^{-5}, \infty), and γk,ij′\gamma'_{k,ij} is symmetrized by averaging with its transpose: 12(γ′+γ′T)\frac{1}{2}(\gamma' + {\gamma'}^T). The IGDN parameters β^\hat{\beta} and γ^\hat{\gamma} undergo identical parameterization and projections.
    4. Filter Renormalization: To eliminate scale ambiguity between linear filters and adjacent divisive normalization layers, each linear filter is normalized after every gradient step by dividing its weights by the Euclidean norm computed across its spatial support and all input channels (for analysis filters) or all output channels (for synthesis filters).

  8. Knowl 8 — Context-Adaptive Binary Arithmetic Coding for Quantized Transform Latents

    algorithm

    The quantized integer latent vector q∈ZMq \in \mathbb{Z}^M is losslessly compressed into a bitstream using Context-Adaptive Binary Arithmetic Coding (CABAC) initialized from the learned marginal distributions:

    Input: Quantized latent vector qq, channel-specific marginal distributions PqiP_{q_i}, bounding ranges [qi,min⁡,qi,max⁡][q_{i,\min}, q_{i,\max}]
    Output: Binary arithmetic bitstream
    Encode image dimensions (two 16-bit integers), color mode (1 bit), and λ\lambda index (16-bit integer) into bitstream header
    for each latent coefficient qiq_i in qq in raster-scan order across channels and spatial locations:
        m←mode of Pqim \leftarrow \text{mode of } P_{q_i}
        Encode binary decision: is qi=mq_i = m?
        if qi≠mq_i \neq m:
            Encode binary decision: is qi>mq_i > m?
            if qi>mq_i > m:
                v←m+1v \leftarrow m + 1
                while v<qi,max⁡v < q_{i,\max} and qi≠vq_i \neq v:
                    Encode binary decision: is qi>vq_i > v?
                    v←v+1v \leftarrow v + 1
                if qi=vq_i = v:
                    Encode binary decision: is qi=vq_i = v?
                else if qi>qi,max⁡q_i > q_{i,\max}:
                    Encode magnitude difference (qi−qi,max⁡)(q_i - q_{i,\max}) using Exponential-Golomb code
            else:
                v←m−1v \leftarrow m - 1
                while v>qi,min⁡v > q_{i,\min} and qi≠vq_i \neq v:
                    Encode binary decision: is qi<vq_i < v?
                    v←v−1v \leftarrow v - 1
                if qi=vq_i = v:
                    Encode binary decision: is qi=vq_i = v?
                else if qi<qi,min⁡q_i < q_{i,\min}:
                    Encode magnitude difference (qi,min⁡−qi)(q_{i,\min} - q_i) using Exponential-Golomb code

    Context models are spatially shared across positions within each channel and initialized by subsampling the continuous marginal densities: Pqi(n)=py~i(n)P_{q_i}(n) = p_{\tilde{y}_i}(n).

  9. Knowl 9 — Rate-Distortion Performance and Perceptual Compression Quality

    empirical result

    When evaluated on the Kodak benchmark dataset and standard photographic test images against standard JPEG and JPEG 2000 codecs:

    1. Objective Rate-Distortion (PSNR): The optimized nonlinear transform coding method achieves higher Peak Signal-to-Noise Ratio (PSNR) than standard JPEG across all tested bit rates, and outperforms or matches JPEG 2000 across most images at low-to-medium bit rates (e.g., ≤0.3\le 0.3 bit/pixel). For example, on a 752×376752 \times 376 RGB photograph at 0.1060.106 bit/pixel, the proposed method achieves a luma PSNR of 32.43 dB32.43\text{ dB} compared to JPEG 2000's 29.49 dB29.49\text{ dB} at 0.1070.107 bit/pixel.
    2. Perceptual Quality (MS-SSIM): When evaluated by Multi-Scale Structural Similarity (MS-SSIM), the proposed method outperforms both JPEG and JPEG 2000 across all tested images and all bit rates. At 0.1130.113 bit/pixel, the proposed method achieves an MS-SSIM of 0.90390.9039 compared to 0.88600.8860 for JPEG 2000 and 0.80790.8079 for JPEG at 0.1210.121 bit/pixel.
    3. Artifact Characteristics: Under heavy compression, linear transform methods exhibit blocking (JPEG) or ringing and aliasing around basis functions (JPEG 2000). In contrast, the GDN-based transform progressively simplifies textures while preserving sharp, smooth edges and natural contours without ringing artifacts.

  10. Knowl 10 — Training Dataset Preprocessing Pipeline for Rate-Distortion Models

    experimental setup

    Models are trained separately for individual trade-off values λ∈[32,2048]\lambda \in [32, 2048] on a curated subset of 6507 images from the ImageNet database using the following preprocessing pipeline:

    1. Saturation Filtering: Images exhibiting severe over-saturation are identified and removed.
    2. Dithering: A small amount of uniform noise matching pixel quantization is added to continuous pixel intensity values.
    3. Randomized Cropping and Resampling: Images are randomly cropped and downsampled to 256×256256 \times 256 pixels. To eliminate high-frequency camera noise and pre-existing compression artifacts, only downsampling resampling factors <0.75< 0.75 are permitted; images smaller than the threshold required to yield 256×256256 \times 256 patches under this constraint are discarded.
    4. Channel Scaling: Grayscale models use 128 channels per convolutional stage. For RGB models and models trained at high bit rates (largest λ\lambda values), the number of feature channels per stage is increased to 192 and 256, respectively.

Coverage note — No substantial contributed material was omitted. The paper's mention of provisional experiments comparing GDN to ReLU/sigmoid activations is noted in the discussion but omitted as a standalone knowl due to its provisional nature.

References

  1. 1.Balle, Johannes, Valero Laparra, and Eero P. Simoncelli (2015). ‘‘Density Modeling of Images Using a Generalized Normalization Transformation’’. In: arXiv e-prints. Presented at the 4th Int. Conf. for Learning Representations, 2016. arXiv: 1511.06281.
  2. 2.– (2016). ‘‘End-to-end optimization of nonlinear transform codes for perceptual quality’’. In: arXiv e-prints. Presented at 2016 Picture Coding Symposium. arXiv: 1607.05006.
  3. 3.Candes, Emmanuel J. and David L. Donoho (2002). ‘‘New Tight Frames of Curvelets and Optimal Representations of Objects with C 2 Singularities’’. In: Comm. Pure Appl. Math. 57, pp. 219–266.
  4. 4.Carandini, Matteo and David J. Heeger (2012). ‘‘Normalization as a canonical neural computation’’. In: Nature Reviews Neuroscience 13. DOI: 10.1038/nrn3136.
  5. 5.Deng, J. et al. (2009). ‘‘ImageNet: A Large-Scale Hierarchical Image Database’’. In: IEEE Conf. on Computer Vision and Pattern Recognition. DOI: 10.1109/CVPR.2009.5206848.
  6. 6.Gersho, Allen and Robert M. Gray (1992). Vector Quantization and Signal Compression. Kluwer. ISBN: 978-0-7923-9181-4.
  7. 7.Gray, Robert M. and David L. Neuhoff (1998). ‘‘Quantization’’. In: IEEE Transactions on Information Theory 44.6. DOI: 10.1109/18.720541.
  8. 8.Gregor, Karol, Frederic Besse, et al. (2016). ‘‘Towards Conceptual Compression’’. In: arXiv e-prints. arXiv: 1604.08772.
  9. 9.Gregor, Karol and Yann LeCun (2010). ‘‘Learning Fast Approximations of Sparse Coding’’. In: Proceedings of the 27th International Conference on Machine Learning.
  10. 10.Heeger, David J. (1992). ‘‘Normalization of cell responses in cat striate cortex’’. In: Visual Neuroscience 9.2. DOI: 10.1017/S0952523800009640.
  11. 11.Ioffe, Sergey and Christian Szegedy (2015). ‘‘Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariance Shift’’. In: arXiv e-prints. arXiv: 1502.03167.
  12. 12.Jarrett, Kevin et al. (2009). ‘‘What is the Best Multi-Stage Architecture for Object Recognition?’’ In: 2009 IEEE 12th International Conference on Computer Vision. DOI: 10.1109/ICCV.2009. 5459469.
  13. 13.Kingma, Diederik P. and Jimmy Lei Ba (2014). ‘‘Adam: A Method for Stochastic Optimization’’. In: arXiv e-prints. Presented at the 3rd Int. Conf. for Learning Representations, 2015. arXiv: 1412.6980.
  14. 14.Kingma, Diederik P. and Max Welling (2014). ‘‘Auto-Encoding Variational Bayes’’. In: arXiv e-prints. arXiv: 1312.6114.
  15. 15.Laparra, Valero et al. (2016). ‘‘Perceptual image quality assessment using a normalized Laplacian pyramid’’. In: Proceedings of SPIE, Human Vision and Electronic Imaging XXI.
  16. 16.Leshno, Moshe et al. (1993). ‘‘Multilayer Feedforward Networks With a Nonpolynomial Activation Function Can Approximate Any Function’’. In: Neural Networks 6.6. DOI: 10.1016/S0893-6080(05)80131-5.
  17. 17.Lewicki, Michael S. and Bruno Olshausen (1998). ‘‘Inferring sparse, overcomplete image codes using an efficient coding framework’’. In: Advances in Neural Information Processing Systems 10, pp. 815–821.
  18. 18.Lyu, Siwei (2010). ‘‘Divisive Normalization: Justification and Effectiveness as Efficient Coding Transform’’. In: Advances in Neural Information Processing Systems 23, pp. 1522–1530.
  19. 19.Malo, Jesus et al. (2006). ‘‘Non-linear image representation for efficient perceptual coding’’. In: IEEE Transactions on Image Processing 15.1. DOI: 10.1109/TIP.2005.860325.
  20. 20.Mante, Valerio, Vincent Bonin, and Matteo Carandini (2008). ‘‘Functional Mechanisms Shaping Lateral Geniculate Responses to Artificial and Natural Stimuli’’. In: Neuron 58.4. DOI: 10.1016/j.neuron.2008.03.011.
  21. 21.Marpe, Detlev, Heiko Schwarz, and Thomas Wiegand (2003). ‘‘Context-Based Adaptive Binary Arithmetic Coding in the H.264/AVC Video Compression Standard’’. In: IEEE Transactions on Circuits and Systems for Video Technology 13.7. DOI: 10.1109/TCSVT.2003.815173.
  22. 22.Netravali, A. N. and J. O. Limb (1980). ‘‘Picture Coding: A Review’’. In: Proceedings of the IEEE 68.3. DOI: 10.1109/PROC.1980.11647.
  23. 23.Oord, Aaron van den, Nal Kalchbrenner, and Koray Kavukcuoglu (2016). ‘‘Pixel Recurrent Neural Networks’’. In: arXiv e-prints. arXiv: 1601.06759.
  24. 24.Rezende, Danilo Jimenez, Shakir Mohamed, and Daan Wierstra (2014). ‘‘Stochastic Backpropagation and Approximate Inference in Deep Generative Models’’. In: arXiv e-prints. arXiv: 1401.4082.
  25. 25.Rippel, Oren, Jasper Snoek, and Ryan P. Adams (2015). ‘‘Spectral Representations for Convolutional Neural Networks’’. In: Advances in Neural Information Processing Systems 28, pp. 2449–2457.
  26. 26.Rissanen, Jorma and Glen G. Langdon Jr. (1981). ‘‘Universal modeling and coding’’. In: IEEE Transactions on Information Theory 27.1. DOI: 10.1109/TIT.1981.1056282.
  27. 27.Schwartz, Odelia and Eero P. Simoncelli (2001). ‘‘Natural signal statistics and sensory gain control’’. In: Nature Neuroscience 4.8. DOI: 10.1038/90526.
  28. 28.Selesnick, Ivan W., Richard G. Baraniuk, and Nick C. Kingsbury (2005). ‘‘The Dual-Tree Complex Wavelet Transform’’. In: IEEE Signal Processing Magazine 22.6. DOI: 10.1109/MSP.2005.1550194.
  29. 29.Shannon, Claude E. (1948). ‘‘A Mathematical Theory of Communication’’. In: The Bell System Technical Journal 27.3. DOI: 10.1002/j.1538-7305.1948.tb01338.x.
  30. 30.Simoncelli, Eero P., William T. Freeman, et al. (1992). ‘‘Shiftable Multiscale Transforms’’. In: IEEE Transactions on Information Theory 38.2. DOI: 10.1109/18.119725.
  31. 31.Simoncelli, Eero P. and David J. Heeger (1998). ‘‘A model of neuronal responses in visual area MT’’. In: Vision Research 38.5. DOI: 10.1016/S0042-6989(97)00183-1.
  32. 32.Simoncelli, Eero P. and Bruno Olshausen (2001). ‘‘Natural image statistics and neural representation’’. In: Annual Review of Neuroscience 24. DOI: 10.1146/annurev.neuro.24.1.1193.
  33. 33.Sinz, Fabian and Matthias Bethge (2013). ‘‘What Is the Limit of Redundancy Reduction with Divisive Normalization?’’ In: Neural Computation 25.11. DOI: 10.1162/NECO_a_00505.
  34. 34.Szegedy, Christian et al. (2013). ‘‘Intriguing properties of neural networks’’. In: arXiv e-prints. arXiv: 1312.6199.
  35. 35.Theis, Lucas, Aaron van den Oord, and Matthias Bethge (2015). ‘‘A note on the evaluation of generative models’’. In: arXiv e-prints. Presented at the 4th Int. Conf. for Learning Representations. arXiv: 1511.01844.
  36. 36.Toderici, George et al. (2016). ‘‘Full Resolution Image Compression with Recurrent Neural Networks’’. In: arXiv e-prints. arXiv: 1608.05148.
  37. 37.Wang, Zhou, Eero P. Simoncelli, and Alan Conrad Bovik (2003). ‘‘Multi-Scale Structural Similarity for Image Quality Assessment’’. In: Conf. Rec. of the 37th Asilomar Conf. on Signals, Systems and Computers, 2004. DOI: 10.1109/ACSSC.2003.1292216.
  38. 38.Wintz, Paul A. (1972). ‘‘Transform Picture Coding’’. In: Proceedings of the IEEE 60.7. DOI: 10.1109/PROC.1972.8780.

Citation

MLA
Ballé, J., et al. “End-to-end Optimized Image Compression”. Presented At: Int'l Conf on Learning Representations, Toulon, France, April 2017, 2016, http://arxiv.org/abs/1611.01704v3.
APA
Ballé, J., Laparra, V., & Simoncelli, E. P. (2016). End-to-end Optimized Image Compression. Presented At: Int'l Conf on Learning Representations, Toulon, France, April 2017. http://arxiv.org/abs/1611.01704v3
Chicago
Ballé, J., V. Laparra, and E. P. Simoncelli. 2016. “End-to-end Optimized Image Compression”. Presented At: Int'l Conf on Learning Representations, Toulon, France, April 2017. http://arxiv.org/abs/1611.01704v3.
Harvard
Ballé, J., Laparra, V. and Simoncelli, E.P. (2016) “End-to-end Optimized Image Compression”, Presented at: Int'l Conf on Learning Representations, Toulon, France, April 2017 [Preprint]. Available at: http://arxiv.org/abs/1611.01704v3.
Vancouver
1. Ballé J, Laparra V, Simoncelli EP (2016) End-to-end Optimized Image Compression. Presented at: Int'l Conf on Learning Representations, Toulon, France, April 2017

BibTeX

@article{balle2016end,
  title = {End-to-end Optimized Image Compression},
  author = {Ballé, Johannes and Laparra, Valero and Simoncelli, Eero P.},
  year = {2016},
  journal = {Presented at: Int'l Conf on Learning Representations, Toulon, France, April 2017},
  url = {http://arxiv.org/abs/1611.01704v3},
  eprint = {1611.01704}
}
Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors