Back

Regularization of Neural Networks using DropConnect

Li WanMatthew D. ZeilerSixin ZhangYann LecunRob Fergus

article2013ICML2,658 citations

Proposes DropConnect, a generalization of Dropout that randomly sets network weights rather than layer activations to zero during training, backed by theoretical generalization bounds and state-of-the-art performance across standard image recognition benchmarks.

Listen

Large neural networks with millions of parameters readily overfit even extensive labeled datasets, limiting their practical accuracy on tasks such as image classification. Regularization methods like weight penalties and early stopping help but remain insufficient for very large models. The article addresses this by introducing DropConnect, a technique that randomly sets a subset of weights to zero during training, generalizing the earlier Dropout method that instead zeros activations.

The work sets out to evaluate whether DropConnect improves generalization over Dropout and standard training, both theoretically and across standard image benchmarks. The authors derive a generalization bound showing that model complexity scales linearly with the probability of retaining each weight. They implement the method efficiently on GPUs and test it by training convolutional networks on MNIST, CIFAR-10, SVHN, and NORB, using five independent models per condition and reporting both single-model and voting results.

DropConnect consistently reduces overfitting relative to no regularization and frequently outperforms Dropout, achieving lower test error rates as model size grows. On MNIST it reaches 0.21 percent error with voting, surpassing prior published results without elastic distortions. On CIFAR-10, twelve DropConnect models yield 9.32 percent error, improving on the previous state of the art. Similar gains appear on SVHN, while results on NORB remain competitive. Inference uses a Gaussian approximation to avoid enumerating all possible masks.

These outcomes indicate that randomly dropping weights during training allows larger fully connected layers to be used without sacrificing test performance, lowering error rates by several tenths of a percent on multiple benchmarks. The approach therefore supports deployment of higher-capacity networks in accuracy-critical applications while maintaining acceptable training times once the feature extractor dominates computation.

The experiments support immediate adoption of DropConnect in place of Dropout for fully connected layers in image-classification pipelines, especially when combined with model voting. Further gains are likely from scaling to still larger models or additional datasets, provided GPU memory layouts are optimized. The principal limitations are restriction to fully connected layers, reliance on an inference approximation whose error grows with network depth, and evaluation confined to image data; results on other domains or architectures remain untested. Confidence is high for the reported image benchmarks given consistent multi-run statistics and state-of-the-art comparisons, but caution is warranted when extrapolating beyond the evaluated conditions.

New York Universitynetwork architectureneural networksregularizationDropConnect regularizationneural networksconvolutional networks
Wan et al (2013).pdf
Cover for Regularization of Neural Networks using DropConnect

Abstract

We introduce DropConnect, a generalization of Dropout (Hinton et al., 2012), for regularizing large fully-connected layers within neural networks. When training with Dropout, a randomly selected subset of activations are set to zero within each layer. DropConnect instead sets a randomly selected subset of weights within the network to zero. Each unit thus receives input from a random subset of units in the previous layer. We derive a bound on the generalization performance of both Dropout and DropConnect. We then evaluate DropConnect on a range of datasets, comparing to Dropout, and show state-of-the-art results on several image recognition benchmarks by aggregating multiple DropConnect-trained models.

Topics

network architectureneural networksregularization

Keywords

DropConnect regularizationneural networksconvolutional networksMNIST classificationCIFAR-10 classification

Table of Contents

  • 1. Introduction
  • 2. Motivation
  • 2.1. Dropout
  • 2.2. DropConnect
  • 3. Model Description
  • 3.1. Training
  • 3.2. Inference
  • 4. Model Generalization Bound
  • 5. Implementation Details
  • 6. Experiments
  • 6.1. MNIST
  • 6.2. CIFAR-10
  • 6.3. SVHN
  • 6.4. NORB
  • 7. Discussion
  • 8. Appendix
  • 8.1. Preliminaries
  • 8.2. Bound Derivation
  • References

Knowls

  1. Knowl 1 — DropConnect Layer Formulation and Mixture Model Interpretation

    model/method

    DropConnect is a neural network regularizer for fully connected layers that introduces dynamic sparsity into the weight matrix rather than the unit activation outputs.

    For a fully connected layer with input vector v∈Rnv \in \mathbb{R}^n, weight parameters W∈Rd×nW \in \mathbb{R}^{d \times n} (which include biases corresponding to a constant input of 11), and non-linear activation function a:R→Ra: \mathbb{R} \to \mathbb{R}, the output vector r∈Rdr \in \mathbb{R}^d under DropConnect is given by:

    r=a((M⊙W)v)r = a((M \odot W)v)

    where ⊙\odot denotes the element-wise (Hadamard) matrix product, and M∈{0,1}d×nM \in \{0, 1\}^{d \times n} is a binary mask matrix with elements drawn independently from a Bernoulli distribution with retention probability pp:

    Mij∼Bernoulli(p)M_{ij} \sim \text{Bernoulli}(p)

    A distinct mask MM is sampled independently for every individual training example, instantiating a different random connectivity pattern per sample.

    The overall model f(x;θ,M)f(x; \theta, M) mapping an input xx to output predictions oo with parameters θ={Wg,W,Ws}\theta = \{W_g, W, W_s\} (where WgW_g are feature extractor parameters, WW are DropConnect layer weights, and WsW_s are classifier parameters) operates as a mixture model over 2∣M∣2^{|M|} sub-networks:

    o=EM[f(x;θ,M)]=∑Mp(M)f(x;θ,M)o = \mathbb{E}_M[f(x; \theta, M)] = \sum_{M} p(M) f(x; \theta, M)

    When p=0.5p = 0.5, all 2∣M∣2^{|M|} network structures share equal mixture weight 2−∣M∣2^{-|M|}. When the activation function satisfies a(0)=0a(0) = 0 (such as ReLU\text{ReLU} or tanh⁡\tanh), standard Dropout is a restricted special case of DropConnect where all elements in individual rows or columns of MM are tied to zero simultaneously.

  2. Knowl 2 — Stochastic Gradient Descent Training with DropConnect

    algorithm

    Training a neural network using DropConnect applies a distinct random binary connectivity mask to the weights and biases for each training sample within a mini-batch. Sharing a single mask across multiple examples in a mini-batch fails to provide sufficient regularization.

    Let xx be a training input, yy the target ground-truth vector, η\eta the learning rate, and θ={Wg,W,Ws}\theta = \{W_g, W, W_s\} the parameter set consisting of feature extractor weights WgW_g, DropConnect fully connected weights W∈Rd×nW \in \mathbb{R}^{d \times n}, and softmax classification weights WsW_s. Loss L(y,o)=−∑i=1kyiln⁡(oi)\mathcal{L}(y, o) = -\sum_{i=1}^k y_i \ln(o_i) is cross-entropy over kk classes.

    Input: Sample xx, label yy, parameters θt−1={Wg,W,Ws}\theta_{t-1} = \{W_g, W, W_s\}, learning rate η\eta, keep probability pp
    Output: Updated parameter set θt\theta_t
    Forward Pass:
        Extract features: v←g(x;Wg)v \leftarrow g(x; W_g)
        Sample mask elements independently: Mij∼Bernoulli(p)∀i,jM_{ij} \sim \text{Bernoulli}(p) \quad \forall i, j
        Compute masked pre-activations: u←(M⊙W)vu \leftarrow (M \odot W)v
        Apply activation function: r←a(u)r \leftarrow a(u)
        Compute class probabilities: o←s(r;Ws)o \leftarrow s(r; W_s)
    Backpropagate Gradients:
        Compute loss gradients ∇θL\nabla_{\theta} \mathcal{L}
        Update softmax layer: Ws←Ws−η∇WsLW_s \leftarrow W_s - \eta \nabla_{W_s} \mathcal{L}
        Update DropConnect weights: W←W−η(M⊙∇WL)W \leftarrow W - \eta (M \odot \nabla_W \mathcal{L})
        Update feature extractor: Wg←Wg−η∇WgLW_g \leftarrow W_g - \eta \nabla_{W_g} \mathcal{L} (propagating backward through M⊙WM \odot W)

    During gradient descent updates, only active connections (Mij=1M_{ij} = 1) receive weight modifications, and backpropagated gradients passed down to earlier feature extraction layers are scaled by the masked matrix M⊙WM \odot W.

  3. Knowl 3 — DropConnect Inference via Gaussian Moment Matching

    algorithm

    Evaluating test-time predictions by exhaustively marginalizing over all 2∣M∣2^{|M|} network configurations is intractable. Replacing post-activation averaging with pre-activation expectation a(E[u])a(\mathbb{E}[u]), as done in Dropout, is mathematically invalid for nonlinear functions such as ReLU\text{ReLU} because E[a(u)]≠a(E[u])\mathbb{E}[a(u)] \neq a(\mathbb{E}[u]).

    DropConnect approximates the distribution of pre-activations using moment matching. For each output neuron ii, the pre-activation ui=∑j(Wijvj)Miju_i = \sum_j (W_{ij} v_j) M_{ij} is a weighted sum of independent Bernoulli variables Mij∼Bernoulli(p)M_{ij} \sim \text{Bernoulli}(p), which is modeled as a 1D Gaussian with mean μi\mu_i and variance σi2\sigma_i^2:

    μi=EM[ui]=(pWv)i\mu_i = \mathbb{E}_M[u_i] = (p W v)_i

    σi2=VM[ui]=(p(1−p)(W⊙W)(v⊙v))i\sigma_i^2 = \mathbb{V}_M[u_i] = (p(1 - p)(W \odot W)(v \odot v))_i

    Input: Input sample xx, model parameters θ={Wg,W,Ws}\theta = \{W_g, W, W_s\}, keep probability pp, number of inference samples ZZ
    Output: Approximated expected activation vector r^\hat{r}
    Extract features: v←g(x;Wg)v \leftarrow g(x; W_g)
    Compute mean vector: μ←pWv\mu \leftarrow p W v
    Compute variance vector: σ2←p(1−p)(W⊙W)(v⊙v)\sigma^2 \leftarrow p(1 - p) (W \odot W)(v \odot v)
    for z=1z = 1 to ZZ do
        for i=1i = 1 to dd do
            Sample pre-activation: ui,z∼N(μi,σi2)u_{i,z} \sim \mathcal{N}(\mu_i, \sigma_i^2)
            Apply non-linear activation: ri,z←a(ui,z)r_{i,z} \leftarrow a(u_{i,z})
        end for
    end for
    Compute empirical mean: r^←1Z∑z=1Zrz\hat{r} \leftarrow \frac{1}{Z} \sum_{z=1}^Z r_z
    Pass r^\hat{r} to the subsequent layer

    Sampling Z=1000Z = 1000 draws in parallel per unit provides an accurate approximation of the expected activation vector at inference time.

  4. Knowl 4 — Rademacher Complexity Bound for DropConnect Networks

    theoretical result

    Let S={x1,…,xℓ}S = \{x_1, \dots, x_\ell\} be a sample of ℓ\ell training points. Consider a neural network function class F\mathcal{F} comprising a feature extractor G\mathcal{G} yielding representations in Rn\mathbb{R}^n, a DropConnect layer with weight matrix W∈Rd×nW \in \mathbb{R}^{d \times n} and connection retention probability pp, an activation function aa (such as ReLU\text{ReLU}, tanh⁡\tanh, or sigmoid), and a softmax classification layer with weights Ws∈Rk×dW_s \in \mathbb{R}^{k \times d} for kk classes.

    Assume bounded parameter weights such that max⁡∣W∣≤Bh\max |W| \le B_h and max⁡∣Ws∣≤Bs\max |W_s| \le B_s (implying the Frobenius / L2L_2 norm of WsW_s is bounded by dkBs\sqrt{dk} B_s).

    The empirical Rademacher complexity of the full DropConnect network class R^ℓ(F)\hat{\mathcal{R}}_\ell(\mathcal{F}) is bounded in terms of the empirical Rademacher complexity of the feature extractor R^ℓ(G)\hat{\mathcal{R}}_\ell(\mathcal{G}) by:

    R^ℓ(F)≤p(2kdBsndBh)R^ℓ(G)\hat{\mathcal{R}}_\ell(\mathcal{F}) \le p \left( 2 \sqrt{k d} B_s n \sqrt{d} B_h \right) \hat{\mathcal{R}}_\ell(\mathcal{G})

    This bound establishes that the Rademacher complexity of the model is a linear function of the retention probability pp. Setting p=0p = 0 reduces the model complexity to zero (the input has no influence on the output), while setting p=1p = 1 recovers the full complexity bound of an unregularized standard dense layer.

  5. Knowl 5 — Bit-Packed 2D Texture-Aligned GPU Implementation of DropConnect

    model/method

    Instantiating an independent binary mask matrix M∈{0,1}d×nM \in \{0,1\}^{d \times n} for each sample across a mini-batch of size bb generates a mask tensor of dimensions d×n×bd \times n \times b. Storing this tensor as 32-bit floating-point values requires d×n×b×4d \times n \times b \times 4 bytes (e.g., 8 GB8\text{ GB} for d=n=4096d=n=4096 with b=128b=128), exceeding typical GPU device memory and saturating memory bandwidth.

    To resolve these constraints, two GPU optimizations are employed:

    1. Bit-Level Mask Packing: Each connection indicator is encoded as a single bit rather than a float, reducing memory footprint and required memory bandwidth by 32×32\times (e.g., from 8 GB8\text{ GB} to 256 MB256\text{ MB} for 4096×4096×1284096 \times 4096 \times 128). A single 4-byte memory read fetches 32 connection masks.
    2. 2D Texture Aligned Memory: Custom CUDA kernels utilize 2D texture memory to align memory access patterns during sparse matrix operations.

    Performance benchmarks for forward pass (fprop), activation backpropagation (bprop acts), and weight backpropagation (bprop weights) on a 1024×10241024 \times 1024 layer with mini-batch size 128 (comparing an NVIDIA GTX580 GPU to a 2.67 GHz Intel Xeon CPU compiled with -O3):

    Implementation fprop (ms) bprop acts (ms) bprop weights (ms) total (ms) Speedup
    CPU float 480.2 1228.6 1692.8 3401.6 1.0×1.0\times
    CPU bit 392.3 679.1 759.7 1831.1 1.9×1.9\times
    GPU float (global memory) 21.6 6.2 7.2 35.0 97.2×97.2\times
    GPU float (tex1D memory) 15.1 6.1 6.0 27.2 126.0×126.0\times
    GPU bit (tex2D aligned memory) 2.4 2.7 3.1 8.2 414.8×414.8\times
    GPU cuBLAS + read mask weight (Lower Bound) 0.3 0.3 0.2 0.8 -

    The bit-packed 2D texture design achieves a 414.8×414.8\times speedup over the CPU float baseline and more than 4×4\times speedup over GPU float implementations.

  6. Knowl 6 — MNIST Classification Performance Across Network Architectures

    data/table

    DropConnect was evaluated on the MNIST digit classification benchmark (28×2828 \times 28 images, 10 classes, 60,000 train, 10,000 test) across fully connected and convolutional architectures.

    In a two-layer fully connected network (800--800 neurons per layer, initial learning rate 0.1, 600-400-20 epoch schedule, without data augmentation), classification error rates across 5 independently trained models and their voting ensemble were:

    Neuron Activation Regularization Model 5-Network Error (%) 5-Network Voting Error (%)
    ReLU No-Drop 1.62±0.0371.62 \pm 0.037 1.40
    ReLU Dropout 1.28±0.0401.28 \pm 0.040 1.20
    ReLU DropConnect 1.20±0.0341.20 \pm 0.034 1.12
    Sigmoid No-Drop 1.78±0.0371.78 \pm 0.037 1.74
    Sigmoid Dropout 1.38±0.0391.38 \pm 0.039 1.36
    Sigmoid DropConnect 1.55±0.0461.55 \pm 0.046 1.48
    tanh⁡\tanh No-Drop 1.65±0.0261.65 \pm 0.026 1.49
    tanh⁡\tanh Dropout 1.58±0.0531.58 \pm 0.053 1.55
    tanh⁡\tanh DropConnect 1.36±0.0541.36 \pm 0.054 1.35

    In a convolutional architecture (2-layer CNN with 32--64 feature maps + 150-neuron ReLU fully connected layer), DropConnect was evaluated under various data augmentation settings:

    Cropping Rotation / Scaling Model 5-Network Error (%) 5-Network Voting Error (%)
    No No No-Drop 0.77±0.0510.77 \pm 0.051 0.67
    No No Dropout 0.59±0.0390.59 \pm 0.039 0.52
    No No DropConnect 0.63±0.0350.63 \pm 0.035 0.57
    Yes No No-Drop 0.50±0.0980.50 \pm 0.098 0.38
    Yes No Dropout 0.39±0.0390.39 \pm 0.039 0.35
    Yes No DropConnect 0.39±0.0470.39 \pm 0.047 0.32
    Yes Yes No-Drop 0.30±0.0350.30 \pm 0.035 0.21
    Yes Yes Dropout 0.28±0.0160.28 \pm 0.016 0.27
    Yes Yes DropConnect 0.28±0.0320.28 \pm 0.032 0.21

    Combining DropConnect with cropping, scaling, rotation, and 5-network voting achieved an error rate of 0.21%0.21\%, surpassing the previous state of the art (0.23%0.23\%) without using elastic distortions.

  7. Knowl 7 — CIFAR-10 Classification Performance with DropConnect

    data/table

    DropConnect was tested on CIFAR-10 (32×3232 \times 32 RGB images, 10 classes, 50,000 train, 10,000 test) using two CNN configurations:

    1. Small Feature Extractor (3-Layer CNN + 64-Unit FC Layer, No Data Augmentation): Single model error rates after 150 training epochs:
    • No-Drop: 23.5%23.5\%
    • Dropout: 19.7%19.7\%
    • DropConnect: 18.7%18.7\%
    1. Large Feature Extractor (2 Convolutional + 2 Locally Connected Layers + 128-Neuron ReLU FC Layer): Evaluated with 24×2424 \times 24 cropping and horizontal flipping over a 700-300-50 epoch schedule:
    Model 5-Network Mean Error (%) 5-Network Voting Error (%)
    No-Drop 11.18±0.1311.18 \pm 0.13 10.22
    Dropout 11.52±0.1811.52 \pm 0.18 9.83
    DropConnect 11.10±0.1311.10 \pm 0.13 9.41

    Ensembling 12 DropConnect-trained networks with model voting reduced classification error to 9.32%9.32\%, surpassing the prior state-of-the-art result (9.5%9.5\%).

  8. Knowl 8 — SVHN and NORB Benchmark Classification Results

    data/table

    DropConnect was evaluated on Street View House Numbers (SVHN, 604,388 train/extra images, 26,032 test images) and 2-fold jittered-cluttered NORB (stereo 48×4848 \times 48 images across 6 classes, 58,320 total test images). Both benchmarks used a 4-layer feature extractor (2 convolutional and 2 locally connected layers) feeding into a 512-neuron ReLU\text{ReLU} fully connected layer.

    On SVHN, images were contrast-normalized, randomly cropped to 28×2828 \times 28, rotated, and scaled. On NORB, rotation and scaling were applied without cropping or horizontal flipping.

    Dataset Model 5-Network Error (%) 5-Network Voting Error (%)
    SVHN No-Drop 2.26±0.0722.26 \pm 0.072 1.94
    SVHN Dropout 2.25±0.0342.25 \pm 0.034 1.96
    SVHN DropConnect 2.23±0.0392.23 \pm 0.039 1.94
    NORB No-Drop 4.48±0.784.48 \pm 0.78 3.36
    NORB Dropout 3.96±0.163.96 \pm 0.16 3.03
    NORB DropConnect 4.14±0.064.14 \pm 0.06 3.23

    On SVHN, 5-network voting achieved 1.94%1.94\% test error, reducing relative error by approximately 30%30\% compared to the prior state-of-the-art benchmark (2.80%2.80\%). On NORB, 5-network voting with DropConnect achieved 3.23%3.23\% test error, improving over the previous state-of-the-art result (3.57%3.57\%).

  9. Knowl 9 — Empirical Regularization Dynamics: Capacity, Drop Rates, and Convergence Rates

    empirical result

    Empirical characterization of DropConnect on fully connected networks reveals four key properties:

    1. Capacity Scaling: When varying the number of neurons per layer (from 200 to 1600 units in a 2-layer network on MNIST), unregularized (No-Drop) networks overfit severely as capacity increases. Both Dropout and DropConnect prevent overfitting, with DropConnect maintaining a consistently lower error rate than Dropout across all wider configurations.
    2. Drop Rate Sensitivity: Sweeping the retention probability pp from 0.1 to 0.9 on a 400--400 network shows that both Dropout and DropConnect reach optimal test accuracy near p=0.5p = 0.5.
    3. Gaussian Sampling vs. Mean Inference: For DropConnect, inference via Gaussian moment-matched sampling provides a notable accuracy gain over taking the mean activation (a(E[u])a(\mathbb{E}[u])). In contrast, for Dropout, Gaussian sampling and mean approximation yield virtually identical performance.
    4. Training Convergence: DropConnect converges at a slower rate than both No-Drop and Dropout during stochastic gradient descent, requiring more training epochs to plateau, but reaches a lower asymptotic test cross-entropy error.

Coverage note — No substantial contributed material was omitted; the extracted knowls fully cover the mathematical formulation, training and inference algorithms, Rademacher complexity bound, GPU bit-packing optimizations, and empirical evaluation across all datasets.

References

  1. 1.D. Ciresan, U. Meier, and J. Schmidhuber. Multi-column deep neural networks for image classification. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), CVPR ’12, pages 3642–3649, Washington, DC, USA, 2012. IEEE Computer Society. ISBN 978-1-4673-1226-4.
  2. 2.G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580, 2012.
  3. 3.A. Krizhevsky. Learning Multiple Layers of Features from Tiny Images. Master’s thesis, University of Toront, 2009.
  4. 4.A. Krizhevsky. cuda-convnet. http://code.google.com/p/cuda-convnet/, 2012.
  5. 5.Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278 –2324, nov 1998. ISSN 0018-9219. doi: 10.1109/5.726791.
  6. 6.Y. LeCun, F. J. Huang, and L. Bottou. Learning methods for generic object recognition with invariance to pose and lighting. In Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition, CVPR’04, pages 97–104, Washington, DC, USA, 2004. IEEE Computer Society.
  7. 7.M. Ledoux and M. Talagrand. Probability in Banach Spaces. Springer, New York, 1991.
  8. 8.D. J. C. Mackay. Probable networks and plausible predictions - a review of practical bayesian methods for supervised neural networks. In Bayesian methods for backpropagation networks. Springer, 1995.
  9. 9.V. Nair and G. E. Hinton. Rectified Linear Units Improve Restricted Boltzmann Machines. In ICML, 2010.
  10. 10.Y. Netzer, T. Wang, Coates A., A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011.
  11. 11.J. Snoek, H. Larochelle, and R. A. Adams. Practical bayesian optimization of machine learning algorithms. In Neural Information Processing Systems, 2012.
  12. 12.A. S. Weigend, D. E. Rumelhart, and B. A. Huberman. Generalization by weight-elimination with application to forecasting. In NIPS, 1991.
  13. 13.M. D. Zeiler and R. Fergus. Stochastic pooling for regualization of deep convolutional neural networks. In ICLR, 2013.

Citation

MLA
Wan, L., et al. “Regularization of Neural Networks Using DropConnect”. HAL (Le Centre Pour La Communication Scientifique Directe), 2013, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.310.3768.
APA
Wan, L., Zeiler, M. D., Zhang, S., Lecun, Y., & Fergus, R. (2013). Regularization of Neural Networks using DropConnect. HAL (Le Centre Pour La Communication Scientifique Directe). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.310.3768
Chicago
Wan, L., M. D. Zeiler, S. Zhang, Y. Lecun, and R. Fergus. 2013. “Regularization of Neural Networks Using DropConnect”. HAL (Le Centre Pour La Communication Scientifique Directe). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.310.3768.
Harvard
Wan, L. et al. (2013) “Regularization of Neural Networks using DropConnect”, HAL (Le Centre pour la Communication Scientifique Directe) [Preprint]. Available at: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.310.3768.
Vancouver
1. Wan L, Zeiler MD, Zhang S, Lecun Y, Fergus R (2013) Regularization of Neural Networks using DropConnect. HAL (Le Centre pour la Communication Scientifique Directe)

BibTeX

@article{wan2013regularization,
  title = {Regularization of Neural Networks using DropConnect},
  author = {Wan, Li and Zeiler, Matthew D. and Zhang, Sixin and Lecun, Yann and Fergus, Rob},
  year = {2013},
  journal = {HAL (Le Centre pour la Communication Scientifique Directe)},
  url = {http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.310.3768}
}
Metadata:DOI registry

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors