Back

Deep Bayesian Active Learning with Image Data

Yarin GalRiashat IslamZoubin Ghahramani

article2017ICML2,066 citations

Develops a Bayesian active learning framework for high-dimensional image data that uses uncertainty estimation in deep convolutional networks to substantially reduce the amount of labeled training data needed for vision tasks.

Listen

Deploying modern machine learning in complex domains such as medical imaging often requires massive volumes of labeled data. Acquiring expert annotations for high-dimensional images is time-consuming, labor-intensive, and expensive, which frequently creates severe bottlenecks for real-world adoption. Active learning addresses this issue by enabling systems to selectively query an expert to label only the most informative samples. However, conventional deep learning methods struggle within active learning frameworks because they typically require huge datasets and cannot quantify their own prediction uncertainty.

The article evaluates a practical active learning framework designed specifically for high-dimensional image data. It demonstrates how integrating Bayesian deep learning techniques allows convolutional neural networks to reliably estimate predictive uncertainty and prioritize unlabeled data for expert review.

The authors implemented this approach using Bayesian convolutional neural networks, where uncertainty estimation was approximated through Monte Carlo dropout. They evaluated several decision criteria—called acquisition functions—including mutual information maximization (known as BALD), variation ratios, and predictive entropy. The framework was tested across multiple experimental setups: digit classification on the benchmark MNIST dataset, comparisons against conventional kernel-based and semi-supervised techniques, and a practical clinical case study diagnosing melanoma from dermoscopic skin lesion images using the ISIC 2016 dataset.

The findings show that uncertainty-driven active learning drastically reduces the amount of labeled data required to achieve high accuracy. On the benchmark image task, the proposed framework achieved a 5% error rate using only 295 labeled images, compared to 835 labeled images needed under standard random selection—representing a reduction in labeling requirements of roughly 65%. With 1,000 labeled images, the framework achieved a 1.64% error rate without utilizing any extra unlabeled data, matching the performance of complex semi-supervised approaches that had access to tens of thousands of additional unlabeled images. Furthermore, Bayesian models consistently outperformed deterministic networks and traditional kernel-based approaches. In the melanoma classification task, the BALD acquisition function successfully prioritized positive cancer cases and achieved higher diagnostic discrimination than a competitive benchmark model trained on the entire dataset.

These results demonstrate that organizations can significantly cut data annotation costs, accelerate model deployment schedules, and maintain high performance in data-constrained settings. The ability of Bayesian active learning to filter out inherently noisy data while focusing on truly informative cases minimizes risk in critical domains like healthcare and quality control.

Organizations handling costly labeling processes should consider adopting Bayesian deep active learning frameworks to optimize specialist workload. When choosing acquisition functions, teams must carefully account for dataset characteristics; for example, the variation ratios approach performed well on balanced data but failed on imbalanced clinical data, where the BALD method proved superior. For immediate workflows, decision-makers should budget for the computational cost of model retraining, as retraining models from scratch after each data acquisition step took up to 20 hours per clinical experiment.

Confidence in the framework is supported by clear performance gains across multiple random splits and datasets. Nevertheless, readers should exercise caution regarding computational overhead and sample stability. The study noted substantial variation in performance across different random splits in small-scale clinical data. Future research and pilot implementations should investigate methods to update network weights incrementally without full model resets, balancing computational runtime against the risk of suboptimal convergence.

The Alan Turing InstituteUberUniversity of Cambridgeactive learningbayesian methodsconvolutional neural networksdeep Bayesian active learningBayesian convolutional neural networksacquisition functions
arXiv: 1703.02910Riashat/Active-Learning-Bayesian-Convolutional-Neural-Networks
Cover for Deep Bayesian Active Learning with Image Data

Abstract

Even though active learning forms an important pillar of machine learning, deep learning tools are not prevalent within it. Deep learning poses several difficulties when used in an active learning setting. First, active learning (AL) methods generally rely on being able to learn and update models from small amounts of data. Recent advances in deep learning, on the other hand, are notorious for their dependence on large amounts of data. Second, many AL acquisition functions rely on model uncertainty, yet deep learning methods rarely represent such model uncertainty. In this paper we combine recent advances in Bayesian deep learning into the active learning framework in a practical way. We develop an active learning framework for high dimensional data, a task which has been extremely challenging so far, with very sparse existing literature. Taking advantage of specialised models such as Bayesian convolutional neural networks, we demonstrate our active learning techniques with image data, obtaining a significant improvement on existing active learning approaches. We demonstrate this on both the MNIST dataset, as well as for skin cancer diagnosis from lesion images (ISIC2016 task).

Topics

active learningbayesian methodsconvolutional neural networksdeep learninghigh dimensional dataonline learningtransfer learning

Keywords

deep Bayesian active learningBayesian convolutional neural networksacquisition functionsvariational inferenceimage classificationstochastic regularisationapproximate inferenceMNIST dataset

Table of Contents

  • 1 Introduction
  • 2 Related Research
  • 3 Bayesian Convolutional Neural Networks
  • 4 Acquisition Functions and their Approximations
  • 5 Active Learning with Bayesian Convolutional Neural Networks
  • 5.1 Comparison of various acquisition functions
  • 5.2 Importance of model uncertainty
  • 5.3 Comparison to current active learning techniques with image data
  • 5.4 Comparison to semi-supervised learning
  • 5.5 Cancer diagnosis from lesion image data
  • 6 Future Research
  • References

Knowls

  1. Knowl 1 — Deep Bayesian Active Learning Framework via Monte Carlo Dropout

    model/method

    Deep Bayesian Active Learning (DBAL) integrates Bayesian Convolutional Neural Networks (BCNNs) into the pool-based active learning cycle for high-dimensional image data. In this framework, a model trained on an initial labelled training dataset Dtrain\mathcal{D}_{\text{train}} evaluates an acquisition function a(x,M)a(x, \mathcal{M}) across an unlabelled pool Dpool\mathcal{D}_{\text{pool}} to select one or more query points x∗=argmax⁡x∈Dpoola(x,M)x^* = \operatorname{argmax}_{x \in \mathcal{D}_{\text{pool}}} a(x, \mathcal{M}) for oracle labelling.

    To capture epistemic uncertainty in high-dimensional deep models, the network parameters ω={W1,…,WL}\omega = \{W_1, \dots, W_L\} are treated as random variables with a prior p(ω)p(\omega) (such as a standard Gaussian). Variational inference is performed using stochastic regularisation via dropout applied before every weight layer. Under this variational distribution qθ∗(ω)q_\theta^*(\omega), the predictive distribution for class c∈{1,…,C}c \in \{1, \dots, C\} given input xx is approximated by averaging TT stochastic forward passes with Monte Carlo (MC) dropout at inference time:

    p(y=c∣x,Dtrain)≈1T∑t=1Tp(y=c∣x,ω^t)p(y = c \mid x, \mathcal{D}_{\text{train}}) \approx \frac{1}{T} \sum_{t=1}^T p(y = c \mid x, \widehat{\omega}_t)

    where ω^t∼qθ∗(ω)\widehat{\omega}_t \sim q_\theta^*(\omega), and p(y=c∣x,ω^t)=softmax⁡(fω^t(x))cp(y = c \mid x, \widehat{\omega}_t) = \operatorname{softmax}(f^{\widehat{\omega}_t}(x))_c represents the softmax output of the network under the tt-th parameter sample.

  2. Knowl 2 — Monte Carlo Dropout Estimator for the BALD Acquisition Function

    equation

    The Bayesian Active Learning by Disagreement (BALD) acquisition objective selects pool points that maximise the mutual information between the predicted class label y∈{1,…,C}y \in \{1, \dots, C\} and the model parameters ω\omega:

    I[y,ω∣x,Dtrain]=H[y∣x,Dtrain]−Ep(ω∣Dtrain)[H[y∣x,ω]]I[y, \omega \mid x, \mathcal{D}_{\text{train}}] = \mathbb{H}[y \mid x, \mathcal{D}_{\text{train}}] - \mathbb{E}_{p(\omega \mid \mathcal{D}_{\text{train}})}\big[\mathbb{H}[y \mid x, \omega]\big]

    Using a variational dropout distribution qθ∗(ω)q_\theta^*(\omega) and drawing TT Monte Carlo parameter samples ω^t∼qθ∗(ω)\widehat{\omega}_t \sim q_\theta^*(\omega), let p^ct=softmax⁡(fω^t(x))c\hat{p}_c^t = \operatorname{softmax}(f^{\widehat{\omega}_t}(x))_c denote the predicted probability of class cc on sample tt. The computationally tractable MC dropout estimator I^[y,ω∣x,Dtrain]\widehat{I}[y, \omega \mid x, \mathcal{D}_{\text{train}}] is given by:

    I^[y,ω∣x,Dtrain]=−∑c=1C(1T∑t=1Tp^ct)log⁡(1T∑t=1Tp^ct)+1T∑c=1C∑t=1Tp^ctlog⁡p^ct\widehat{I}[y, \omega \mid x, \mathcal{D}_{\text{train}}] = -\sum_{c=1}^C \left(\frac{1}{T} \sum_{t=1}^T \hat{p}_c^t\right) \log \left(\frac{1}{T} \sum_{t=1}^T \hat{p}_c^t\right) + \frac{1}{T} \sum_{c=1}^C \sum_{t=1}^T \hat{p}_c^t \log \hat{p}_c^t

    As T→∞T \to \infty, I^[y,ω∣x,Dtrain]→I[y,ω∣x,Dtrain]\widehat{I}[y, \omega \mid x, \mathcal{D}_{\text{train}}] \to I[y, \omega \mid x, \mathcal{D}_{\text{train}}]. This metric identifies pool points where individual stochastic forward passes are confident in disparate classes (high variance in logits), signifying high reducible epistemic uncertainty.

  3. Knowl 3 — Uncertainty-Based Acquisition Functions for Bayesian Image Classification

    model/method

    For active classification over pool data x∈Dpoolx \in \mathcal{D}_{\text{pool}} with discrete labels y∈{1,…,C}y \in \{1, \dots, C\} and training data Dtrain\mathcal{D}_{\text{train}}, several acquisition functions can be computed from the approximate predictive posterior distribution p(y=c∣x,Dtrain)p(y = c \mid x, \mathcal{D}_{\text{train}}):

    1. Predictive Entropy (Max Entropy): Selects points that maximise the Shannon entropy of the expected predictive distribution:

    H[y∣x,Dtrain]=−∑c=1Cp(y=c∣x,Dtrain)log⁡p(y=c∣x,Dtrain)\mathbb{H}[y \mid x, \mathcal{D}_{\text{train}}] = -\sum_{c=1}^C p(y = c \mid x, \mathcal{D}_{\text{train}}) \log p(y = c \mid x, \mathcal{D}_{\text{train}})

    1. Variation Ratios: Measures dispersion by computing the deviation of the modal class probability from 1:

    variation-ratio[x]=1−max⁡cp(y=c∣x,Dtrain)\text{variation-ratio}[x] = 1 - \max_{c} p(y = c \mid x, \mathcal{D}_{\text{train}})

    1. Mean Standard Deviation (Mean STD): Computes the average standard deviation of predicted class probabilities across classes, where σc=Eq(ω)[p(y=c∣x,ω)2]−(Eq(ω)[p(y=c∣x,ω)])2\sigma_c = \sqrt{\mathbb{E}_{q(\omega)}[p(y=c \mid x, \omega)^2] - (\mathbb{E}_{q(\omega)}[p(y=c \mid x, \omega)])^2}:

    σ(x)=1C∑c=1Cσc\sigma(x) = \frac{1}{C} \sum_{c=1}^C \sigma_c

    1. Bayesian Active Learning by Disagreement (BALD): Measures mutual information between labels and parameters, isolating epistemic uncertainty from aleatoric noise.

  4. Knowl 4 — Label Efficiency of Acquisition Functions on MNIST Benchmark

    data/table

    Active learning experiments evaluated on MNIST using a Bayesian CNN (Conv-ReLU-Conv-ReLU-MaxPool-Dropout-Dense-ReLU-Dropout-Dense-Softmax) initialized with 20 balanced samples, acquiring 10 points per iteration, show substantial differences in labelling efficiency across acquisition strategies.

    Target Test Error BALD Var Ratios Max Entropy Mean STD Random
    10% 145 120 165 230 255
    5% 335 295 355 695 835

    Variation Ratios, BALD, and Max Entropy achieve the target error rates with far fewer samples than random selection. Notably, Variation Ratios achieves 5% test error using 295 labelled images compared to 835 images required by random acquisition (a reduction of greater than 64%), whereas Mean STD performs similarly to the uniform random baseline.

  5. Knowl 5 — Epistemic Uncertainty via Bayesian CNNs Outperforms Deterministic Deep Active Learning

    empirical result

    Comparing Bayesian CNNs (using test-time Monte Carlo dropout to sample parameters ω∼qθ∗\omega \sim q_\theta^*) against deterministic CNNs (setting qθ∗(ω)=δ(ω−θ)q_\theta^*(\omega) = \delta(\omega - \theta) and disabling test-time dropout) across BALD, Variation Ratios, and Max Entropy demonstrates that epistemic uncertainty is crucial for deep active learning.

    While deterministic models capture aleatoric uncertainty (data noise), they cannot quantify parameter uncertainty. In active learning benchmarks on MNIST, Bayesian CNNs attain higher classification accuracy earlier in the acquisition trajectory and converge to higher final test accuracies across all tested acquisition functions compared to deterministic counterparts.

  6. Knowl 6 — Active Learning vs. Semi-Supervised Learning Benchmarks on MNIST

    data/table

    A Bayesian CNN trained with active learning using only 1000 actively acquired labelled images achieves performance competitive with semi-supervised methods on MNIST that utilize 1000 labelled samples plus an additional 49,000 to 59,000 unlabelled samples.

    Technique Test Error
    Semi-supervised (1000 labelled + remaining unlabelled):
    Semi-supervised Embedding 5.73%
    Transductive SVM 5.38%
    Manifold Tangent Classifier (MTC) 3.64%
    Pseudo-label 3.46%
    AtlasRBF 3.68%
    Deep Generative Models (DGN) 2.40%
    Ladder Network (Γ\Gamma-model) 1.53%
    Virtual Adversarial Training 1.32%
    Active learning (1000 acquired samples only):
    Random Acquisition 4.66%
    BALD 1.80%
    Max Entropy 1.74%
    Variation Ratios 1.64%

    Active learning with Variation Ratios achieves 1.64% test error using strictly the 1000 acquired images, performing close to the Ladder Network Γ\Gamma-model (1.53%) without requiring access to unlabelled data during optimization.

  7. Knowl 7 — Deep Bayesian Active Learning Experimental Setup for Melanoma Diagnosis

    experimental setup

    Melanoma diagnosis is evaluated using dermoscopic lesion images from the ISIC 2016 dataset (900 total images: 727 benign, 173 malignant). A VGG16 architecture pre-trained on ImageNet is adapted by replacing the top classifier with two fully connected layers of size 4096 (each followed by dropout with p=0.5p = 0.5) and a 2-dimensional output layer.

    The active learning protocol operates as follows:

    1. Data Partitioning: A balanced test set of 100 negative (benign) and 100 positive (malignant) examples is set aside. The initial training set contains 80 negative and 20 positive examples, with the remainder forming the unlabelled pool.
    2. Training & Regularization: Positive training instances are augmented via horizontal and vertical flipping. The model is trained for 100 epochs with batch size 8 and weight decay set to (1−p)l2/N(1-p)l^2/N, where p=0.5p=0.5, length-scale parameter l2=0.5l^2=0.5, and NN is the current training set size.
    3. Acquisition & Model Resetting: In each acquisition cycle, the 100 most informative pool points are queried and added to the training set. To eliminate confounding effects from cumulative training time and avoid local optima, the network is reset to its ImageNet pre-trained weights at every cycle before retraining to convergence.
    4. Inference: Performance is evaluated via Area Under the ROC Curve (AUC) computed using 20 Monte Carlo dropout samples.

  8. Knowl 8 — BALD Performance and Minority-Class Acquisition in Skin Lesion Diagnosis

    empirical result

    On the imbalanced ISIC 2016 skin lesion diagnosis task, the BALD acquisition function demonstrates two key behaviors:

    1. Preferential Positive-Class Querying: BALD actively identifies malignant (positive) lesions as high-information points, acquiring substantially more positive instances per acquisition step than the uniform random baseline (which only acquires them in proportion to their natural dataset prevalence of ~20%).
    2. Accelerated AUC Convergence: BALD achieves higher AUC faster than uniform sampling across multiple test splits.
    3. Superiority over Full-Pool Training: After 4 acquisition steps, the converged model trained with BALD achieves higher test AUC than a baseline model trained on the entire pool (which achieved AUC 0.71±0.0030.71 \pm 0.003 on split 1 and 0.75±0.010.75 \pm 0.01 on split 2). This advantage is attributed to BALD targeting high epistemic uncertainty while avoiding ambiguous points characterized by high aleatoric noise (e.g., overlapping clusters with conflicting ground-truth labels).

  9. Knowl 9 — Degeneracy of Variation Ratios Under Severe Class Imbalance

    limitation

    While Variation Ratios (1−max⁡cp(y=c∣x,Dtrain)1 - \max_c p(y=c \mid x, \mathcal{D}_{\text{train}})) exhibits strong performance on balanced datasets like MNIST, it collapses on heavily imbalanced datasets such as the ISIC 2016 melanoma dataset (approx. 80% benign, 20% malignant).

    When a model trained on imbalanced data assigns a dominant probability to the majority class across virtually all pool points (with minority instances only receiving marginally higher minority probability), max⁡cp(y=c∣x,Dtrain)\max_c p(y=c \mid x, \mathcal{D}_{\text{train}}) evaluates to near-identical values across all candidate inputs. As a result, the Variation Ratios acquisition surface becomes flat and non-discriminative, rendering it unable to prioritize informative points.

  10. Knowl 10 — Computational Overhead of Complete Model Retraining in Deep Active Learning

    limitation

    To isolate the true informational benefit of acquisition functions and avoid convergence into local optima from incremental gradient steps, the deep active learning pipeline resets model parameters to their initial or pre-trained weights after each acquisition step and retrains the network to convergence on the expanded training set.

    This requirement induces severe computational costs. In fine-tuning tasks such as VGG16 on skin lesion diagnosis, retraining across multiple acquisition cycles requires approximately 20 hours per experiment repetition. Avoiding full retraining by fine-tuning continuously across cycles reduces runtime but increases the risk of being trapped in poor local optima.

Coverage note — None was omitted; all key theoretical formulations, empirical findings on MNIST and ISIC 2016, baseline comparisons (MBR, deterministic models, semi-supervised learning), and methodological limitations were captured.

References

  1. 1.Agarwal, Mohit, Damaraju, Nandita, and Chaieb, Sahbi. Dl8803. https://github.com/NanditaDamaraju/DL8803, 2016.
  2. 2.Cohn, David A, Ghahramani, Zoubin, and Jordan, Michael I. Active learning with statistical models. Journal of artificial intelligence research, 1996.
  3. 3.Cortes, Corinna and Vapnik, Vladimir. Support-vector networks. Machine learning, 20(3):273–297, 1995.
  4. 4.Deng, Jia, Dong, Wei, Socher, Richard, Li, Li-Jia, Li, Kai, and Fei-Fei, Li. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp. 248–255. IEEE, 2009.
  5. 5.fchollet. Keras. https://github.com/fchollet/keras, 2015.
  6. 6.Freeman, Linton G. Elementary applied statistics, 1965.
  7. 7.Gal, Yarin. Uncertainty in Deep Learning. PhD thesis, University of Cambridge, 2016.
  8. 8.Gal, Yarin and Ghahramani, Zoubin. Bayesian convolutional neural networks with Bernoulli approximate variational inference. ICLR workshop track, 2016a.
  9. 9.Gal, Yarin and Ghahramani, Zoubin. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. ICML, 2016b.
  10. 10.Gutman, David, Codella, Noel CF, Celebi, Emre, Helba, Brian, Marchetti, Michael, Mishra, Nabin, and Halpern, Allan. Skin lesion analysis toward melanoma detection: A challenge at the international symposium on biomedical imaging (ISBI) 2016, hosted by the international skin imaging collaboration (ISIC). arXiv preprint arXiv:1605.01397, 2016.
  11. 11.He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, and Sun, Jian. Delving deep into rectifiers: Surpassing humanlevel performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034, 2015.
  12. 12.Hernandez-Lobato, Jose Miguel and Adams, Ryan. Probabilistic backpropagation for scalable learning of Bayesian neural networks. In Proceedings of The 32nd International Conference on Machine Learning, pp. 1861–1869, 2015.
  13. 13.Hinton, Geoffrey E, Srivastava, Nitish, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
  14. 14.Holub, Alex, Perona, Pietro, and Burl, Michael C. Entropybased active learning for object recognition. In Computer Vision and Pattern Recognition Workshops, 2008. CVPRW’08. IEEE Computer Society Conference on, pp. 1–8. IEEE, 2008.
  15. 15.Houlsby, Neil, Huszár, Ferenc, Ghahramani, Zoubin, and Lengyel, Máté. Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.5745, 2011.
  16. 16.Joshi, Ajay J, Porikli, Fatih, and Papanikolopoulos, Nikolaos. Multi-class active learning for image classification. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp. 2372–2379. IEEE, 2009.
  17. 17.Jozefowicz, Rafal, Vinyals, Oriol, Schuster, Mike, Shazeer, Noam, and Wu, Yonghui. Exploring the limits of language modeling. arXiv preprint arXiv:1602.02410, 2016.
  18. 18.Kalchbrenner, Nal and Blunsom, Phil. Recurrent continuous translation models. In EMNLP, 2013.
  19. 19.Kampffmeyer, Michael, Salberg, Arnt-Borre, and Jenssen, Robert. Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2016.
  20. 20.Kendall, Alex, Badrinarayanan, Vijay, and Cipolla, Roberto. Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv preprint arXiv:1511.02680, 2015.
  21. 21.Kingma, Diederik P, Mohamed, Shakir, Rezende, Danilo Jimenez, and Welling, Max. Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems, pp. 3581–3589, 2014.
  22. 22.Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097–1105, 2012.
  23. 23.LeCun, Yann and Cortes, Corinna. The MNIST database of handwritten digits, 1998.
  24. 24.LeCun, Yann, Boser, Bernhard, Denker, John S, Henderson, Donnie, Howard, Richard E, Hubbard, Wayne, and Jackel, Lawrence D. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551, 1989.
  25. 25.Lee, Dong-Hyun. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on Challenges in Representation Learning, 2013.
  26. 26.Li, Xin and Guo, Yuhong. Adaptive active learning for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 859–866, 2013.
  27. 27.Marcus, Daniel S, Fotenos, Anthony F, Csernansky, John G, Morris, John C, and Buckner, Randy L. Open access series of imaging studies: longitudinal mri data in nondemented and demented older adults. Journal of cognitive neuroscience, 22(12):2677–2684, 2010.
  28. 28.Miyato, Takeru, Maeda, Shin-ichi, Koyama, Masanori, Nakae, Ken, and Ishii, Shin. Distributional smoothing by virtual adversarial examples. arXiv preprint arXiv:1507.00677, 2015.
  29. 29.Pitelis, Nikolaos, Russell, Chris, and Agapito, Lourdes. Semi-supervised learning using an unsupervised atlas. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 565–580. Springer, 2014.
  30. 30.Rasmus, Antti, Berglund, Mathias, Honkala, Mikko, Valpola, Harri, and Raiko, Tapani. Semi-supervised learning with ladder networks. In Advances in Neural Information Processing Systems, pp. 3546–3554, 2015.
  31. 31.Rifai, Salah, Dauphin, Yann N, Vincent, Pascal, Bengio, Yoshua, and Muller, Xavier. The manifold tangent classifier. In Advances in Neural Information Processing Systems, pp. 2294–2302, 2011.
  32. 32.Rumelhart, David E, Hinton, Geoffrey E, and Williams, Ronald J. Learning internal representations by error propagation. Technical report, DTIC Document, 1985.
  33. 33.Shannon, Claude Elwood. A mathematical theory of communication. Bell System Technical Journal, 27(3):379–423, 1948.
  34. 34.Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
  35. 35.Srivastava, Nitish, Hinton, Geoffrey, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan. Dropout: A simple way to prevent neural networks from overfitting. JMLR, 2014.
  36. 36.Sundermeyer, Martin, Schlüter, Ralf, and Ney, Hermann. LSTM neural networks for language modeling. In INTERSPEECH, 2012.
  37. 37.Sutskever, Ilya, Vinyals, Oriol, and Le, Quoc VV. Sequence to sequence learning with neural networks. In NIPS, 2014.
  38. 38.Tong, Simon. Active Learning: Theory and Applications. PhD thesis, 2001. AAI3028187.
  39. 39.Weston, Jason, Ratle, Frédéric, Mobahi, Hossein, and Collobert, Ronan. Deep learning via semi-supervised embedding. In Neural Networks: Tricks of the Trade, pp. 639–655. Springer, 2012.
  40. 40.Zhu, X, Lafferty, J, and Ghahramani, Z. Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions. In Proceedings of the ICML-2003 Workshop on The Continuum from Labeled to Unlabeled Data, pp. 58–65. ICML, 2003.

Citation

MLA
Gal, Y., et al. “Deep Bayesian Active Learning with Image Data”. arXiv, 2017, http://arxiv.org/abs/1703.02910v1.
APA
Gal, Y., Islam, R., & Ghahramani, Z. (2017). Deep Bayesian Active Learning with Image Data. arXiv. http://arxiv.org/abs/1703.02910v1
Chicago
Gal, Y., R. Islam, and Z. Ghahramani. 2017. “Deep Bayesian Active Learning with Image Data”. arXiv. http://arxiv.org/abs/1703.02910v1.
Harvard
Gal, Y., Islam, R. and Ghahramani, Z. (2017) “Deep Bayesian Active Learning with Image Data”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1703.02910v1.
Vancouver
1. Gal Y, Islam R, Ghahramani Z (2017) Deep Bayesian Active Learning with Image Data. arXiv

BibTeX

@article{gal2017deep,
  title = {Deep Bayesian Active Learning with Image Data},
  author = {Gal, Yarin and Islam, Riashat and Ghahramani, Zoubin},
  year = {2017},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1703.02910v1},
  eprint = {1703.02910}
}
Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF
License: https://creativecommons.org/licenses/by/4.0/