keyword
ladder networks
A ladder network is a deep neural network architecture designed for semi-supervised learning that simultaneously learns from both labeled and unlabeled data. The model couples a feedforward encoder pathway with a complementary decoder pathway, connected at each layer through lateral skip connections that resemble the rungs of a ladder. In this setup, the encoder generates hierarchical representations while introducing noise, and the decoder progressively reconstructs the clean, uncorrupted feature representations at every level through layer-wise denoising tasks. This allows the network to be trained end-to-end using backpropagation to jointly optimize a supervised classification objective alongside unsupervised reconstruction objectives, enabling effective feature learning without requiring separate layer-by-layer pre-training.
2 items
Semi-supervised Learning with Ladder Networks
Antti Rasmus, Mathias Berglund, M. Honkala, Harri Valpola, T. Raiko
Why you should read this
Introduces a semi-supervised Ladder Network architecture that trains supervised and unsupervised reconstruction targets simultaneously via standard backpropagation, drastically reducing the labeled data needed for high-accuracy image classification without layer-wise pre-training.
We combine supervised learning with unsupervised learning in deep neural networks. The proposed model is trained to simultaneously minimize the sum of supervised and unsupervised cost functions by backpropagation, avoiding the need for layer-wise pre-training. Our work builds on the Ladder network proposed by Valpola (2015), which we extend by combining the model with supervision. We show that the resulting model reaches state-of-the-art performance in semi-supervised MNIST and CIFAR-10 classification, in addition to permutation-invariant MNIST classification with all labels.
Added
2026-09-25
Active Learning for Convolutional Neural Networks: A Core-Set Approach
Ozan Sener, Silvio Savarese
Why you should read this
Proposes a geometric core-set formulation for batch active learning in convolutional neural networks, providing theoretical performance bounds and an effective subset selection algorithm that significantly reduces labeling requirements for image classification.
Convolutional neural networks (CNNs) have been successfully applied to many recognition and learning tasks using a universal recipe; training a deep model on a very large dataset of supervised examples. However, this approach is rather restrictive in practice since collecting a large set of labeled images is very expensive. One way to ease this problem is coming up with smart ways for choosing images to be labelled from a very large collection (ie. active learning). Our empirical study suggests that many of the active learning heuristics in the literature are not effective when applied to CNNs in batch setting. Inspired by these limitations, we define the problem of active learning as core-set selection, ie. choosing set of points such that a model learned over the selected subset is competitive for the remaining data points. We further present a theoretical result characterizing the performance of any selected subset using the geometry of the datapoints. As an active learning algorithm, we choose the subset which is expected to yield best result according to our characterization. Our experiments show that the proposed method significantly outperforms existing approaches in image classification experiments by a large margin.
Added
2026-09-14