A Survey of Deep Active Learning

Pengzhen RenYun XiaoXiaojun ChangPo-Yao (Bernie) HuangZhihui LiXiaojiang ChenXin Wang

article2020ACM Computing Surveys1,559 citations

Presents a systematic taxonomy of deep active learning strategies to help researchers train data-hungry neural networks with minimal manual labeling effort across label-scarce application domains.

Listen

Modern deep learning models achieve high performance across many industries, but their training relies heavily on massive volumes of manually annotated data. Acquiring these labels incurs substantial financial and operational costs, particularly in specialized domains such as medical imaging, speech recognition, and robotics, where domain experts must perform the annotations. Active learning addresses this bottleneck by identifying and querying only the most informative unlabeled samples for human annotation. However, combining traditional active learning with deep learning presents structural challenges, including deep models exhibiting overconfident uncertainty estimates, deep neural networks requiring large amounts of data to prevent overfitting, and incompatibilities between the iterative query process and deep neural network training pipelines.

The article systematically reviews and evaluates the field of deep active learning, which integrates active learning sample selection with deep feature extraction. It categorizes current methodologies, examines how they address core integration challenges, reviews practical applications across diverse fields, and analyzes emerging research directions.

To synthesize the field, the article analyzes approximately 270 research publications updated through late 2020. The evaluation covers theoretical query designs, model training pipelines, and empirical implementations across visual processing, natural language processing, and other specialized domains. The examination groups query approaches into batch-mode selection, deep Bayesian active learning, density-based core-set strategies, hybrid algorithms, and automated design methods.

The review outlines several key findings. First, traditional one-by-one sample selection fails in deep learning; efficient training requires batch query strategies, though naive batching often selects redundant samples. Second, hybrid query strategies that balance model prediction uncertainty with sample diversity consistently perform more reliably than strategies relying on either uncertainty or diversity alone. Third, leveraging unlabeled data and synthetic data via generative models or semi-supervised methods provides crucial performance gains, often matching or exceeding the benefits gained from refining query strategies alone. Fourth, treating active learning and deep model training as two separated steps causes optimization divergence, whereas unified, multi-layer architectures provide superior and task-agnostic performance across varied applications.

These findings indicate that organizations can achieve significant cost savings and efficiency gains by adopting deep active learning in data-scarce and expert-dependent workflows. By reducing the volume of manual labeling required by orders of magnitude while preserving target accuracy, development cycles can be shortened. However, real-world implementations must account for the computational overhead of batch selection and iterative model updates.

Organizations implementing these systems should adopt hybrid selection strategies that account for both sample informativeness and dataset diversity rather than relying strictly on standard model confidence outputs. Practitioners should also integrate semi-supervised learning and data augmentation to make full use of unlabeled datasets. Furthermore, development teams must incorporate dynamic stopping criteria based on prediction stability rather than fixed iterations to avoid either overspending the annotation budget or under-training the model.

The field currently exhibits notable uncertainties due to inconsistent experimental settings across existing literature. Reported performance for standard random sampling baselines varies widely between studies (by as much as 8% to 13% on identical benchmark datasets), which can obscure true algorithmic gains. In addition, deep active learning methods are not universally superior for every dataset type, especially where sample importance differences are negligible. Readers should exercise caution when comparing reported gains and prioritize establishing unified evaluation baselines, reproducible benchmarks, and proper ablation studies before deploying systems into production.

Carnegie Mellon University National Institute of Technology Kurukshetra Northwest University Qilu University of Technology RMIT University active learning classification classification models deep active learning mini-batch active learning adversarial representation active learning

arXiv: 2009.00236

Paper: Deep Bayesian Active Learning with Image Data, Yarin Gal et al. (2017). This seminal work demonstrates the practical integration of Bayesian deep neural networks with uncertainty-based acquisition functions for high-dimensional active learning, establishing a core milestone reviewed in the survey.
Paper: Active Learning for Convolutional Neural Networks: A Core-Set Approach, Ozan Sener et al. (2018). This paper establishes the core-set geometric selection framework for convolutional neural networks, serving as the foundational representative method for distribution- and diversity-based deep active learning.
Paper: Improving Generalization with Active Learning, David Cohn et al. (1994). This foundational paper introduces selective sampling and region-of-uncertainty query strategies in neural networks, defining key classical active learning principles adapted by deep learning frameworks.
Paper: Active Learning with Statistical Models, David Cohn et al. (1996). This paper formulates variance reduction and statistically optimal query selection, providing the theoretical basis for modern model-dependent sample selection criteria.
Paper: Query by committee, H. Seung et al. (1992). This foundational work introduces the Query by Committee algorithm, which underpins ensemble disagreement metrics and modern committee-based deep active learning methods.
Paper: Support Vector Machine Active Learning with Applications to Text Classification, Simon Tong et al. (2001). This work establishes pool-based active learning and margin-based uncertainty sampling, formalizing concepts heavily referenced and generalized in deep active learning taxonomies.
Paper: Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions, Xiaojin Zhu et al. (2003). This paper establishes the classic paradigm of combining active learning with semi-supervised label propagation, a core hybrid strategy detailed in the survey.
Paper: A survey on semi-supervised learning, Jesper E. van Engelen et al. (2019). This comprehensive survey provides essential background on semi-supervised learning paradigms that are frequently unified with active learning in label-scarce deep learning regimes.

Paper: A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges, Moloud Abdar et al. (2020). This comprehensive survey expands specifically on uncertainty quantification methodologies in deep neural networks, which form the primary theoretical mechanism for uncertainty-based deep active learning query strategies.
Paper: Ensemble deep learning: A review, M. A. Ganaie et al. (2021). This review provides an extensive investigation into modern deep ensemble architectures and decision-fusion techniques, which directly enhance ensemble-driven disagreement query strategies in active learning.

Cover for A Survey of Deep Active Learning

Abstract

Active learning (AL) attempts to maximize the performance gain of the model by marking the fewest samples. Deep learning (DL) is greedy for data and requires a large amount of data supply to optimize massive parameters, so that the model learns how to extract high-quality features. In recent years, due to the rapid development of internet technology, we are in an era of information torrents and we have massive amounts of data. In this way, DL has aroused strong interest of researchers and has been rapidly developed. Compared with DL, researchers have relatively low interest in AL. This is mainly because before the rise of DL, traditional machine learning requires relatively few labeled samples. Therefore, early AL is difficult to reflect the value it deserves. Although DL has made breakthroughs in various fields, most of this success is due to the publicity of the large number of existing annotation datasets. However, the acquisition of a large number of high-quality annotated datasets consumes a lot of manpower, which is not allowed in some fields that require high expertise, especially in the fields of speech recognition, information extraction, medical images, etc. Therefore, AL has gradually received due attention. A natural idea is whether AL can be used to reduce the cost of sample annotations, while retaining the powerful learning capabilities of DL. Therefore, deep active learning (DAL) has emerged. Although the related research has been quite abundant, it lacks a comprehensive survey of DAL. This article is to fill this gap, we provide a formal classification method for the existing work, and a comprehensive and systematic overview. In addition, we also analyzed and summarized the development of DAL from the perspective of application. Finally, we discussed the confusion and problems in DAL, and gave some possible development directions for DAL.

Topics

active learning classification classification models deep learning information extraction internet medical images speech recognition online learning

Keywords

deep active learning mini-batch active learning adversarial representation active learning cost-effective active learning ensemble Bayesian active learning task-aware variational adversarial active learning semi-supervised learning domain adaptation transfer learning deep belief nets

1 Introduction
1.1 Deep Learning
1.2 Active Learning
2 The necessity and challenge of combining DL and AL
3 Deep Active Learning
3.1 Query Strategy Optimization in DeepAL
3.1.1 Batch Mode DeepAL (BMDAL)
3.1.2 Uncertainty-based and Hybrid Query Strategies
3.1.3 Deep Bayesian Active Learning (DBAL)
3.1.4 Density-based Methods
3.1.5 Automated Design of DeepAL
3.2 Data Expansion of Labeled Samples in DeepAL
3.3 DeepAL Generic Framework
3.4 DeepAL Stopping Strategy
4 Application of DeepAL in fields such as vision and NLP
4.1 Visual Data Processing
4.1.1 Image classification and recognition
4.1.2 Object detection and semantic segmentation
4.1.3 Video processing
4.2 Natural Language Processing (NLP)
4.2.1 Machine translation
4.2.2 Text classification
4.2.3 Semantic analysis
4.2.4 Information extraction
4.2.5 Question-answering
4.3 Other Applications
5 Discussion and future directions
6 Summary and conclusions
References

Knowls

Knowl 1 — Formal Optimization Formulation of Pool-Based Deep Active Learning
equation
In pool-based deep active learning (DeepAL), let $\mathcal{U}_n = \{\mathcal{X}, \mathcal{Y}\}$ denote an unlabeled dataset consisting of $n$ samples, where $\mathcal{X}$ is the input sample space, $\mathcal{Y}$ is the label space, and $P(x, y)$ represents the underlying data distribution with $x \in \mathcal{X}$ and $y \in \mathcal{Y}$ . Let $\mathcal{L}_m = \{X, Y\}$ denote the current labeled training set containing $m$ samples, where $X \subset \mathcal{X}$ and $Y \subset \mathcal{Y}$ . A deep neural network hypothesis $f \in \mathcal{F}$ maps inputs to predictions, $f: \mathcal{X} \rightarrow \mathcal{Y}$ .

Given a loss function $\ell(\cdot) \in \mathbb{R}^+$ , the objective of supervised DeepAL is to design an active query strategy $Q: \mathcal{U}_n \rightarrow \mathcal{L}_m$ such that the deep model trained on $\mathcal{L}_m$ minimizes expected risk over the data distribution while maintaining a label budget $m \ll n$ :
$\arg \min_{\mathcal{L}_m \subseteq \mathcal{U}_n, (x, y) \in \mathcal{L}_m, (x, y) \in \mathcal{U}_n} \mathbb{E}_{(x, y)} [\ell(f(x), y)]$
The goal is to minimize the required labeled sample size $m$ while guaranteeing that the deep model achieves a target generalization performance level.
Knowl 2 — Core Challenges in Integrating Active Learning with Deep Learning
definition
Combining active learning (AL) with deep learning (DL) into Deep Active Learning (DeepAL) encounters three fundamental structural challenges:
1. Model Uncertainty Unreliability: Standard deep networks using softmax output layers produce overconfident class probabilities. The Softmax Response (SR) of the output layer is often poorly calibrated as an uncertainty estimate, leading naive uncertainty sampling to perform on par with or worse than random selection.
2. Data-Hunger vs. Small Label Updates: Deep neural networks require large volumes of labeled data to optimize millions of parameters without severe overfitting. In contrast, traditional AL typically queries samples in a one-by-one sequential mode, which is computationally prohibitive and ineffective for training deep networks.
3. Processing Pipeline Inconsistency: Classical active learning algorithms decouple feature representation from classifier training and assume fixed feature vectors. In contrast, deep learning jointly optimizes representation learning and classification layers. Treating AL query selection and deep model training as separate pipelines can cause representation divergence and optimization instability.
Knowl 3 — Batch Mode Deep Active Learning and Joint Mutual Information
model/method
Batch Mode Deep Active Learning (BMDAL) replaces single-sample queries with batch-level candidate selection. At each acquisition step, an acquisition function $a_{\text{batch}}(\mathcal{B}, f_\theta(\mathcal{L}))$ selects a batch of candidate unlabeled data $\mathcal{B}^* = \{x_1^*, \dots, x_b^*\} \subseteq \mathcal{U}$ of size $b$ :
$\mathcal{B}^* = \arg\max_{\mathcal{B} \subseteq \mathcal{U}} a_{\text{batch}}(\mathcal{B}, f_\theta(\mathcal{L}))$
where $f_\theta(\mathcal{L})$ is the deep model parameterized by $\theta$ trained on the labeled dataset $\mathcal{L}$ .

A naive batch acquisition based on Bayesian Active Learning by Disagreement (BALD) sums individual mutual information scores independently:
$a_{\text{BALD}}(\{x_1, \dots, x_b\}, P(\omega \mid \mathcal{D}_{\text{train}})) = \sum_{i=1}^b \mathbb{I}(y_i; \omega \mid x_i, \mathcal{D}_{\text{train}})$
where $\mathbb{I}(y; \omega \mid x, \mathcal{D}_{\text{train}}) = \mathbb{H}(y \mid x, \mathcal{D}_{\text{train}}) - \mathbb{E}_{P(\omega \mid \mathcal{D}_{\text{train}})} [\mathbb{H}(y \mid x, \omega, \mathcal{D}_{\text{train}})]$ , $\omega \sim P(\omega \mid \mathcal{D}_{\text{train}})$ represents model parameters conditioned on labeled data $\mathcal{D}_{\text{train}}$ , and $\mathbb{H}(\cdot)$ denotes entropy. Because this formulation scores instances independently, it frequently selects batches containing mutually redundant samples.

To account for intra-batch correlation, BatchBALD optimizes the joint mutual information between the joint distribution of predictions $y_{1:b} = (y_1, \dots, y_b)$ for the candidate batch $x_{1:b} = (x_1, \dots, x_b)$ and model parameters $\omega$ :
$a_{\text{BatchBALD}}(\{x_1, \dots, x_b\}, P(\omega \mid \mathcal{D}_{\text{train}})) = \mathbb{I}(y_{1:b}; \omega \mid x_{1:b}, \mathcal{D}_{\text{train}}) = \mathbb{H}(y_{1:b} \mid x_{1:b}, \mathcal{D}_{\text{train}}) - \mathbb{E}_{P(\omega \mid \mathcal{D}_{\text{train}})} [\mathbb{H}(y_{1:b} \mid x_{1:b}, \omega, \mathcal{D}_{\text{train}})]$
This joint formulation maximizes collective information while explicitly penalizing intra-batch sample similarity.
Knowl 4 — Hybrid Query Strategies Balancing Uncertainty and Diversity in DeepAL
model/method
Pure uncertainty-based sampling in DeepAL causes sampling bias because queried samples cluster near existing decision boundaries and fail to represent the global data distribution. Conversely, purely diversity-based sampling risks selecting uninformative instances from low-density regions. Hybrid query strategies optimize both criteria simultaneously.

In the Exploration-P method, uncertainty is measured via predictive entropy $E(S) = \sum_{x_i \in S} \mathbb{E}(x_i)$ , where for $k$ classes, $\mathbb{E}(x) = -\sum_{j=1}^k P(y_j \mid x) \log P(y_j \mid x)$ . Subset redundancy $R(S)$ is defined via a similarity matrix $\mathbf{M}$ and feature extractor $f(x)$ :
$R(S) = \sum_{x_i \in S} \sum_{x_j \in S} \text{Sim}(x_i, x_j), \quad \text{where } \text{Sim}(x_i, x_j) = f(x_i) \mathbf{M} f(x_j)$
The acquisition objective maximizes entropy while penalizing average redundancy:
$I(S) = E(S) - \frac{\alpha}{|S|} R(S)$
where $\alpha \ge 0$ balances uncertainty and redundancy.

Alternative hybrid DeepAL approaches include:
- BADGE (Batch Active Learning by Diverse Gradient Embeddings): Computes hallucinated gradient embeddings where vector magnitudes reflect model uncertainty and vector orientations reflect diversity, selecting batches via $k$ -means++ seeding without manual hyperparameter tuning.
- WAAL (Wasserstein Adversarial Active Learning): Formulates query selection as distribution matching using the Wasserstein distance to explicitly balance uncertainty and feature diversity.
- TA-VAAL (Task-Aware Variational Adversarial Active Learning): Combines a task-loss prediction module with variational adversarial ranking to capture both task-specific uncertainty and latent distribution coverage.
Knowl 5 — Deep Bayesian Active Learning (DBAL) Framework
model/method
Deep Bayesian Active Learning (DBAL) models parameter uncertainty in deep neural networks to produce well-calibrated confidence estimates for sample selection.

Let $\mathcal{X}$ and $\mathcal{Y}$ be the input and output spaces. A neural network $f(x; \theta)$ has a prior $p(\theta)$ (typically Gaussian) over its parameters $\theta$ . The likelihood for class $c$ is $p(y = c \mid x, \theta) = \text{softmax}(f(x; \theta))$ . The parameter posterior conditioned on training data $(X, Y)$ is:
$p(\theta \mid X, Y) = \frac{p(Y \mid X, \theta) p(\theta)}{p(Y \mid X)}$
For an unlabeled data point $x^*$ , the predictive posterior distribution is obtained by marginalizing over $\theta$ :
$p(\hat{y} \mid x^*, X, Y) = \int p(\hat{y} \mid x^*, \theta) p(\theta \mid X, Y) d\theta = \mathbb{E}_{\theta \sim p(\theta \mid X, Y)} [f(x^*; \theta)]$
In practice, DBAL computes posterior samples using Monte Carlo Dropout (MC-dropout) as approximate variational inference during inference, or via Deep Probabilistic Ensembles (DPEs), enabling uncertainty metrics such as predictive entropy or mutual information (BALD) to be evaluated on high-dimensional data.
Knowl 6 — Density-Based and Core-Set Sample Selection in DeepAL
model/method
Density-based DeepAL strategies select query subsets by analyzing the geometric coverage and density distribution of the feature representation space:
1. Core-Set Active Learning: Formulates query selection as finding a core subset $\mathcal{L} \subseteq \mathcal{U}$ that minimizes the maximum distance between any unlabeled sample and its nearest selected sample (a $k$ -Center clustering problem over neural network feature embeddings).
2. Discriminative Active Learning (DAL): Casts active sample selection as a binary classification problem. A discriminator network is trained to differentiate between labeled and unlabeled instances. Unlabeled samples that the discriminator identifies as most distinct from the labeled pool are queried to align the empirical labeled distribution with the full unlabeled distribution.
3. Farthest-First Active Learning (FF-Active): Utilizes farthest-first traversal across the activation space of intermediate representation layers to ensure selected instances maximize spatial dispersion across the data manifold.
Knowl 7 — Data Expansion and Semi-Supervised Augmentation in DeepAL
model/method
To mitigate the data shortage inherent in deep networks trained on small active query sets, DeepAL incorporates unlabeled and synthetic data into the training cycle via three main strategies:
1. Cost-Effective Active Learning (CEAL) Pseudo-Labeling: High-uncertainty samples are sent to the oracle for human labeling, while high-confidence unlabeled samples are automatically assigned pseudo-labels. Both sets are iteratively merged into the labeled pool to update the model.
2. Generative Adversarial Active Learning (GAAL & BGADL):
  - GAAL: Uses Generative Adversarial Networks (GANs) to generate synthetic samples in high-information regions.
  - BGADL (Bayesian Generative Active Deep Learning): Combines Bayesian active learning, Auxiliary Classifier GANs (ACGAN), and Variational Autoencoders (VAE) to generate samples within disagreement regions across decision boundaries.
3. Variational Adversarial Active Learning (VAAL & ARAL):
  - VAAL: Jointly trains a VAE on both labeled and unlabeled data alongside an adversarial discriminator that learns to distinguish between labeled and unlabeled latent embeddings; samples with high discriminator uncertainty are queried.
  - ARAL (Adversarial Representation Active Learning): Integrates adversarial feature representation learning with sample generation to train models on labeled, unlabeled, and synthetic data concurrently.
4. Semi-Supervised Active Learning (SSAL): Performs unsupervised pre-training on all data prior to the active learning cycles, followed by supervised training on labeled data and semi-supervised training on the entire pool at each active iteration.
Knowl 8 — Task-Agnostic Multi-Layer Loss Prediction in DeepAL
model/method
Standard uncertainty sampling evaluates only the final softmax output layer of a deep neural network, neglecting the uncertainty present in intermediate hierarchical feature representations.

Learning Loss for Active Learning (LLAL) provides a task-agnostic active learning architecture by attaching a compact loss prediction module to intermediate hidden layers of the target deep neural network:
- The loss prediction module takes multi-scale feature maps from several hidden layers as input.
- It is trained jointly with the primary network to predict the target loss that an unlabeled input would produce if evaluated against its true label.
- During the query step, unlabeled samples with the highest predicted loss are queried via a top- $K$ selection strategy.
Because the loss prediction module learns to predict scalar loss values rather than task-specific probabilities, LLAL applies directly across multiple visual tasks including image classification, object detection, and human pose estimation.
Knowl 9 — Automated Design of DeepAL via Reinforcement Learning and Neural Architecture Search
model/method
Automated DeepAL automates the design of acquisition functions and deep architectures instead of relying on manual heuristics:
1. Reinforced Active Learning (RAL): Replaces fixed acquisition rules with a Bayesian Neural Network (BNN) policy network. The policy network receives prediction uncertainty distributions from a predictor BNN and adaptively updates sample selection strategies through reinforcement learning using reward feedback from the oracle.
2. Deep Reinforcement Active Learning (DRAL): Applies an active reinforcement agent to sequential instance selection (e.g., in person re-identification), where the agent queries instances and refines its selection policy based on binary oracle feedback rewards.
3. Incremental Neural Architecture Search (Active-iNAS): Integrates Neural Architecture Search (NAS) into the active learning loop, dynamically searching for optimal neural network architectures as the labeled dataset grows incrementally across active learning cycles.
Knowl 10 — Deep Active Learning Stopping Criteria and Stabilizing Predictions
model/method
Determining when to terminate active annotation is vital in DeepAL to prevent premature termination (causing model performance degradation) or over-annotation (wasting human labeling budget):
1. Predefined Stopping Criteria: Conventional DeepAL utilizes static stopping heuristics, including reaching a maximum number of iterations, exhausting a pre-allocated labeling budget, falling below a fixed threshold of classification accuracy change, or achieving a target accuracy value.
2. Stabilizing Predictions (SP): Evaluates prediction stability across active cycles on an unannotated validation set (the "stop set") drawn from the unlabeled pool. Because the stop set does not require ground-truth annotations, it incurs zero extra labeling cost. The active query loop terminates when the model's predictions on the stop set stabilize across successive learning iterations.
Knowl 11 — Evaluation Discrepancies and Random Sampling Baselines in DeepAL
limitation
Empirical evaluations in deep active learning literature face significant reproducibility challenges and inconsistent benchmarking practices:
1. Random Sampling Baseline (RSB) Variance: Reported performance of the random sampling baseline varies substantially across studies under identical nominal settings. On CIFAR-10 with a 20% labeling budget, reported RSB accuracies diverge by up to 13% between papers.
2. Inconsistent Method Replications: For the same benchmark setup (e.g., CIFAR-100 with VGG-16 backbone at 40% labeling budget), reported performance for identical DeepAL methods varies by up to 8% across independent publications.
3. Contradictory Algorithmic Conclusions: Certain studies report that diversity-based active learning universally outperforms uncertainty-based sampling and that uncertainty sampling degrades below random sampling, whereas subsequent unified frameworks (such as multi-layer loss prediction) demonstrate superior performance for uncertainty-based approaches over both diversity methods and RSB.
4. Dataset Sensitivity: On large datasets with minimal sample importance variance, training on small actively selected subsets provides negligible gains over random sampling, indicating that active subset selection is not universally advantageous across all dataset distributions.

Knowl 12 — Cross-Domain Application Taxonomy of Deep Active Learning

data/table

DeepAL has been implemented across computer vision, natural language processing, biomedical analysis, and cyber-physical systems to reduce expert labeling costs in high-dimensional tasks. The table below outlines the core application taxonomy, representative tasks, publications, and benchmark datasets surveyed.

Field	Task	Selected Methods	Representative Datasets
Vision	Image Classification	CEAL, WI-DL, DAL, DBAL	CIFAR-10/100, MNIST, Caltech-256, PaviaU, BreaKHis
Vision	Object Detection	Query-by-Committee, LiDAR-AL	PASCAL VOC, KITTI, NACTI, NEU-DET
Vision	Semantic Segmentation	FCN Suggestive Annotation, DASL	SPIM, Confocal, LIDC-IDRI, MICCAI
Vision	Video Processing	DeActive, DRAL Re-ID	OPPORTUNITY, PRID, MARS, DukeMTMC
NLP	Machine Translation	Curriculum AL, Parallel Corpus AL	OPUS, UNPC, IWSLT, WMT
NLP	Text Classification	Active CNN/RNN, Deep Bias AL	AG News, DBPedia, Amazon Reviews, Yelp Review
NLP	Semantic Analysis	Active Deep Networks (ADN), AL Fake News	Movie Reviews (MR), KDnuggets Fake News, Liar
NLP	Information Extraction	Tweet AL, Deep NER, ER Active	CoNLL-2003, OntoNotes, NCBI Disease, DBLP
NLP	Question Answering	Hidden-Space VQA, Dialogue AL	Visual Genome, VQA, CMDC
Other	Robotics Wearables	ActiveHARNet, Trajectory AL	HHAR, Crazyflie, MIT-BIH ECG, MSP-Podcast

This taxonomy demonstrates that DeepAL extends from low-dimensional biomedical and sensor signals (such as ECG and accelerometer streams) to complex multimodal and sequential structures (such as autonomous driving LiDAR point clouds, 3D medical CT scans, satellite remote sensing, and dialogue generation).

Coverage note — Specific individual empirical case studies from individual application papers cited in Section 4 were synthesized into the cross-domain application taxonomy rather than extracted as isolated knowls.

References

1.Mohammed Abdel-Wahab and Carlos Busso. 2019. Active Learning for Speech Emotion Recognition Using Deep Neural Network. In 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019, Cambridge, United Kingdom, September 3-6, 2019. IEEE, 1–7.
2.Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering 17, 6 (2005), 734–749.
3.Charu C. Aggarwal, Xiangnan Kong, Quanquan Gu, Jiawei Han, and Philip S. Yu. 2014. Active Learning: A Survey. In Data Classification: Algorithms and Applications. CRC Press, 571–606.
4.Hamed Habibi Aghdam, Abel Gonzalez-Garcia, Antonio M. López, and Joost van de Weijer. 2019. Active Learning for Deep Detection Neural Networks. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, 3671–3679.
5.Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. Massively Multilingual Neural Machine Translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers). Association for Computational Linguistics, 3874–3884.
6.Saeed S. Alahmari, Dmitry B. Goldgof, Lawrence O. Hall, and Peter R. Mouton. 2019. Automatic Cell Counting using Active Deep Learning and Unbiased Stereology. In 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019, Bari, Italy, October 6-9, 2019. IEEE, 1708–1713.
7.Bang An, Wenjun Wu, and Huimin Han. 2018. Deep Active Learning for Text Classification. In Proceedings of the 2nd International Conference on Vision, Image and Signal Processing, ICVISP 2018, Las Vegas, NV, USA, August 27-29, 2018. ACM, 22:1–22:6.
8.Olov Andersson, Mariusz Wzorek, and Patrick Doherty. 2017. Deep Learning Quadcopter Control via Risk-Aware Active Learning. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA. AAAI Press, 3812–3818.
9.Ralph G Andrzejak, Klaus Lehnertz, Florian Mormann, Christoph Rieke, Peter David, and Christian E Elger. 2001. Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state. Physical Review E 64, 6 (2001), 061907.
10.Dana Angluin. 1988. Queries and Concept Learning. Machine Learning 2, 4 (1988), 319–342.
11.Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. VQA: Visual Question Answering. In 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015. IEEE Computer Society, 2425–2433.
12.Samuel G Armato III, Geoffrey McLennan, Luc Bidaut, Michael F McNitt-Gray, Charles R Meyer, Anthony P Reeves, Binsheng Zhao, Denise R Aberle, Claudia I Henschke, Eric A Hoffman, et al. 2011. The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Medical physics 38, 2 (2011), 915–931.
13.Nabiha Asghar, Pascal Poupart, Xin Jiang, and Hang Li. 2017. Deep Active Learning for Dialogue Generation. In Proceedings of the 6th Joint Conference on Lexical and Computational Semantics, *SEM @ACM 2017, Vancouver, Canada, August 3-4, 2017. Association for Computational Linguistics, 78–83.
14.Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. 2020. Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
15.Marc Bachlin, Meir Plotnik, Daniel Roggen, Inbal Maidan, Jeffrey M Hausdorff, Nir Giladi, and Gerhard Troster. 2009. Wearable assistant for Parkinson’s disease patients with the freezing of gait symptom. IEEE Transactions on Information Technology in Biomedicine 14, 2 (2009), 436–446.
16.Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. 2017. Designing Neural Network Architectures using Reinforcement Learning. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
17.Mariaflorina Balcan, Alina Beygelzimer, and John Langford. 2009. Agnostic active learning. J. Comput. System Sci. 75, 1 (2009), 78–89.
18.Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James R. Glass. 2019. Identifying and Controlling Important Neurons in Neural Machine Translation. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
19.William H. Beluch, Tim Genewein, Andreas Nürnberger, and Jan M. Köhler. 2018. The Power of Ensembles for Active Learning in Image Classification. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. IEEE Computer Society, 9368–9377.
20.Shai Bendavid, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Machine Learning 79, 1 (2010), 151–175.
21.Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle. 2006. Greedy Layer-Wise Training of Deep Networks. (2006), 153–160.
22.Sreyasee Das Bhattacharjee, Ashit Talukder, and Bala Venkatram Balantrapu. 2017. Active learning based news veracity detection with feature weighting and deep-shallow fusion. (2017), 556–565.
23.Mustafa Bilgic and Lise Getoor. 2009. Link-based active learning. In NIPS Workshop on Analyzing Networks and Learning with Graphs.
24.John Blitzer, Mark Dredze, and Fernando Pereira. 2007. Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification. In ACL 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23-30, 2007, Prague, Czech Republic. The Association for Computational Linguistics.
25.Michael Bloodgood and Chris Callison-Burch. 2014. Bucking the Trend: Large-Scale Cost-Focused Active Learning for Statistical Machine Translation. CoRR abs/1410.5877 (2014).
26.Michael Bloodgood and K. Vijay-Shanker. 2009. Taking into Account the Differences between Actively and Passively Acquired Data: The Case of Active Learning with Support Vector Machines for Imbalanced Datasets. In Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, May 31 - June 5, 2009, Boulder, Colorado, USA, Short Papers. The Association for Computational Linguistics, 137–140.
27.Michael Bloodgood and K. Vijay-Shanker. 2014. A Method for Stopping Active Learning Based on Stabilizing Predictions and the Need for User-Adjustable Stopping. CoRR abs/1409.5165 (2014).
28.Erik Bochinski, Ghassen Bacha, Volker Eiselein, Tim J. W. Walles, Jens C. Nejstgaard, and Thomas Sikora. 2018. Deep Active Learning for In Situ Plankton Classification. In Pattern Recognition and Information Forensics - ICPR 2018 International Workshops, CVAUI, IWCF, and MIPPSNA, Beijing, China, August 20-24, 2018, Revised Selected Papers (Lecture Notes in Computer Science, Vol. 11188). Springer, 5–15.
29.Klaus Brinker. 2003. Incorporating Diversity in Active Learning with Support Vector Machines. In Machine Learning, Proceedings of the Twentieth International Conference (ICML 2003), August 21-24, 2003, Washington, DC, USA. AAAI Press, 59–66.
30.Samuel Budd, Emma C. Robinson, and Bernhard Kainz. 2019. A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis. CoRR abs/1910.02923 (2019).
31.Alex Burka and Katherine J. Kuchenbecker. 2017. How Much Haptic Surface Data Is Enough?. In 2017 AAAI Spring Symposia, Stanford University, Palo Alto, California, USA, March 27-29, 2017. AAAI Press.
32.Sylvain Calinon, Florent Guenter, and Aude Billard. 2007. On Learning, Representing, and Generalizing a Task in a Humanoid Robot. IEEE Trans. Syst. Man Cybern. Part B 37, 2 (2007), 286–298.
33.Trevor Campbell and Tamara Broderick. 2019. Automated Scalable Bayesian Inference via Hilbert Coresets. Journal of Machine Learning Research 20, 15 (2019), 1–38.
34.Haw-Shiuan Chang, Shankar Vembu, Sunil Mohan, Rheeya Uppaal, and Andrew McCallum. 2020. Using error decay prediction to overcome practical issues of deep active learning for named entity recognition. Mach. Learn. 109, 9-10 (2020), 1749–1778.
35.Bor-Chun Chen, Chu-Song Chen, and Winston H. Hsu. 2014. Cross-Age Reference Coding for Age-Invariant Face Recognition and Retrieval. In Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI (Lecture Notes in Computer Science, Vol. 8694). Springer, 768–783.
36.Xuhui Chen, Jinlong Ji, Tianxi Ji, and Pan Li. 2018. Cost-Sensitive Deep Active Learning for Epileptic Seizure Detection. In Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, BCB 2018, Washington, DC, USA, August 29 - September 01, 2018. ACM, 226–235.
37.Anfeng Cheng, Chuan Zhou, Hong Yang, Jia Wu, Lei Li, Jianlong Tan, and Li Guo. 2019. Deep Active Learning for Anchor User Prediction. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019. ijcai.org, 2151–2157.
38.Kashyap Chitta, Jose M Alvarez, Elmar Haussmann, and Clement Farabet. 2019. Training Data Distribution Search with Ensemble Active Learning. arXiv preprint arXiv:1905.12737 (2019).
39.Kashyap Chitta, Jose M. Alvarez, and Adam Lesnikowski. 2018. Large-Scale Visual Active Learning with Deep Probabilistic Ensembles. CoRR abs/1811.03575 (2018).
40.Diane J Cook and Maureen Schmitter-Edgecombe. 2009. Assessing the quality of activities in a smart environment. Methods of information in medicine 48, 5 (2009), 480.
41.Ido Dagan and Sean P. Engelson. 1995. Committee-Based Sampling For Training Probabilistic Classifiers. In Machine Learning, Proceedings of the Twelfth International Conference on Machine Learning, Tahoe City, California, USA, July 9-12, 1995. Morgan Kaufmann, 150–157.
42.Navneet Dalal and Bill Triggs. 2005. Histograms of Oriented Gradients for Human Detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), 20-26 June 2005, San Diego, CA, USA. IEEE Computer Society, 886–893.
43.Cristian Danescu-Niculescu-Mizil and Lillian Lee. 2011. Chameleons in Imagined Conversations: A New Approach to Understanding Coordination of Linguistic Style in Dialogs. In Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics, CMCL@ACL 2011, Portland, Oregon, USA, June 23, 2011. Association for Computational Linguistics, 76–87.
44.Sanjoy Dasgupta. 2011. Two faces of active learning. Theoretical Computer Science 412, 19 (2011), 1767–1781.
45.Sajib Dasgupta and Vincent Ng. 2009. Mine the Easy, Classify the Hard: A Semi-Supervised Approach to Automatic Sentiment Classification. In ACL 2009, Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2-7 August 2009, Singapore, Keh-Yih Su, Jian Su, and Janyce Wiebe (Eds.). The Association for Computer Linguistics, 701–709. https://www.aclweb.org/anthology/P09-1079/
46.Diana L. Delibaltov, Utkarsh Gaur, Jennifer Kim, Matthew Kourakis, Erin Newman-Smith, William Smith, Samuel A. Belteton, Daniel Szymanski, and B. S. Manjunath. 2016. CellECT: cell evolution capturing tool. BMC Bioinform. 17 (2016), 88.
47.Diana L. Delibaltov, Pratim Ghosh, Volkan Rodoplu, Michael Veeman, William Smith, and B. S. Manjunath. 2013. A Linear Program Formulation for the Segmentation of Ciona Membrane Volumes. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2013 - 16th International Conference, Nagoya, Japan, September 22-26, 2013, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 8149). Springer, 444–451.
48.Cheng Deng, Yumeng Xue, Xianglong Liu, Chao Li, and Dacheng Tao. 2019. Active Transfer Learning Network: A Unified Deep Joint Spectral–Spatial Feature Learning Model for Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing 57, 3 (2019), 1741–1754.
49.Jia Deng, Alex Berg, Sanjeev Satheesh, Hao Su, Aditya Khosla, and Fei-Fei Li. 2012. Large scale visual recognition challenge. www. image-net. org/challenges/LSVRC/2012 1 (2012).
50.Li Deng. 2014. A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Transactions on Signal and Information Processing 3 (2014).
51.Rezarta Islamaj Dogan, Robert Leaman, and Zhiyong Lu. 2014. NCBI disease corpus: A resource for disease name recognition and concept normalization. J. Biomed. Informatics 47 (2014), 1–10.
52.Piotr Dollár, Christian Wojek, Bernt Schiele, and Pietro Perona. 2012. Pedestrian Detection: An Evaluation of the State of the Art. IEEE Trans. Pattern Anal. Mach. Intell. 34, 4 (2012), 743–761.
53.Jeff Donahue and Karen Simonyan. 2019. Large Scale Adversarial Representation Learning. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. 10541–10551.
54.David L Donoho et al. 2000. High-dimensional data analysis: The curses and blessings of dimensionality. AMS math challenges lecture 1, 2000 (2000), 32.
55.Baolin Du, Qi Qi, Han Zheng, Yue Huang, and Xinghao Ding. 2018. Breast Cancer Histopathological Image Classification via Deep Active Learning and Confidence Boosting. In Artificial Neural Networks and Machine Learning - ICANN 2018 - 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part II (Lecture Notes in Computer Science, Vol. 11140). Springer, 109–116.
56.Xuefeng Du, Dexing Zhong, and Huikai Shao. 2019. Building an Active Palmprint Recognition System. In 2019 IEEE International Conference on Image Processing, ICIP 2019, Taipei, Taiwan, September 22-25, 2019. IEEE, 1685–1689.
57.Melanie Ducoffe and Frédéric Precioso. 2018. Adversarial Active Learning for Deep Networks: a Margin Based Approach. CoRR abs/1802.09841 (2018).
58.Rana El Kaliouby and Peter Robinson. 2004. Mind reading machines: Automated inference of cognitive mental states from video. In 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No. 04CH37583), Vol. 1. IEEE, 682–688.
59.Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. 2010. The pascal visual object classes (voc) challenge. International journal of computer vision 88, 2 (2010), 303–338.
60.Caspar J. Fall, A. Törcsvári, K. Benzineb, and G. Karetka. 2003. Automated categorization in the international patent classification. SIGIR Forum 37, 1 (2003), 10–25.
61.Meng Fang, Yuan Li, and Trevor Cohn. 2017. Learning how to Active Learn: A Deep Reinforcement Learning Approach. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017. Association for Computational Linguistics, 595–605.
62.Reza Zanjirani Farahani and Masoud Hekmatfar. 2009. Facility location: concepts, models, algorithms and case studies. Springer.
63.Di Feng, Xiao Wei, Lars Rosenbaum, Atsuto Maki, and Klaus Dietmayer. 2019. Deep Active Learning for Efficient Training of a LiDAR 3D Object Detector. In 2019 IEEE Intelligent Vehicles Symposium, IV 2019, Paris, France, June 9-12, 2019. IEEE, 667–674.
64.Rosa L. Figueroa, Qing Zeng-Treitler, Long H. Ngo, Sergey Goryachev, and Eduardo P. Wiechmann. 2012. Active learning for clinical text classification: is it better than random sampling? J. Am. Medical Informatics Assoc. 19, 5 (2012), 809–816.
65.Frederic Brenton Fitch. 1944. McCulloch Warren S. and Pitts Walter. A logical calculus of the ideas immanent in nervous activity. Bulletin of mathematical biophysics , vol. 5 (1943), pp. 115–133. Journal of Symbolic Logic 9 (1944), 49–50.
66.Jonathan Folmsbee, Xulei Liu, Margaret Brandwein-Weber, and Scott Doyle. 2018. Active deep learning: Improved training efficiency of convolutional neural networks for tissue classification in oral cavity cancer. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE, 770–773.
67.Tavis Forrester, William J McShea, RW Keys, Robert Costello, Megan Baker, and Arielle Parsons. 2013. eMammal–citizen science camera trapping as a solution for broad-scale, long-term monitoring of wildlife populations. Sustainable Pathways: Learning from the Past and Shaping the Future (2013).
68.Marguerite Frank, Philip Wolfe, et al. 1956. An algorithm for quadratic programming. Naval research logistics quarterly 3, 1-2 (1956), 95–110.
69.Alexander Freytag, Erik Rodner, and Joachim Denzler. 2014. Selecting Influential Examples: Active Learning with Expected Model Output Changes. In Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part IV (Lecture Notes in Computer Science, Vol. 8692). Springer, 562–577.
70.Yarin Gal and Zoubin Ghahramani. 2015. Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference. CoRR abs/1506.02158 (2015).
71.Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016 (JMLR Workshop and Conference Proceedings, Vol. 48). JMLR.org, 1050–1059.
72.Yarin Gal, Riashat Islam, and Zoubin Ghahramani. 2017. Deep Bayesian Active Learning with Image Data. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 (Proceedings of Machine Learning Research, Vol. 70). PMLR, 1183–1192.
73.Vijay Garla, Caroline Taylor, and Cynthia Brandt. 2013. Semi-supervised clinical text classification with Laplacian SVMs: An application to cancer case management. J. Biomed. Informatics 46, 5 (2013), 869–875.
74.Utkarsh Gaur, Matthew Kourakis, Erin Newman-Smith, William Smith, and B. S. Manjunath. 2016. Membrane segmentation via active learning with deep networks. In 2016 IEEE International Conference on Image Processing, ICIP 2016, Phoenix, AZ, USA, September 25-28, 2016. IEEE, 1943–1947.
75.Yonatan Geifman and Ran El-Yaniv. 2017. Deep Active Learning over the Long Tail. CoRR abs/1711.00941 (2017).
76.Yonatan Geifman and Ran El-Yaniv. 2019. Deep Active Learning with a Neural Architecture Search. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. 5974–5984.
77.Andreas Geiger, Philip Lenz, and Raquel Urtasun. [n.d.]. Are we ready for autonomous driving. In Proc. CVPR. 3354–3361.
78.Andreas Geiger, Philip Lenz, and Raquel Urtasun. 2012. Are we ready for autonomous driving? The KITTI vision benchmark suite. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, June 16-21, 2012. IEEE Computer Society, 3354–3361.
79.Daniel Gissin and Shai Shalev-Shwartz. 2019. Discriminative Active Learning. CoRR abs/1907.06347 (2019).
80.Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada. 2672–2680.
81.Gregory Griffin, Alex Holub, and Pietro Perona. 2007. Caltech-256 object category dataset. (2007).
82.Denis A. Gudovskiy, Alec Hodgkinson, Takuya Yamaguchi, and Sotaro Tsukizawa. 2020. Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. IEEE, 9038–9046.
83.Gautham Krishna Gudur, Prahalathan Sundaramoorthy, and Venkatesh Umaashankar. 2019. ActiveHARNet: Towards On-Device Deep Bayesian Active Learning for Human Activity Recognition. CoRR abs/1906.00108.
84.Yuhong Guo. 2010. Active Instance Sampling via Matrix Partition. In Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010. Proceedings of a meeting held 6-9 December 2010, Vancouver, British Columbia, Canada. Curran Associates, Inc., 802–810.
85.Kazim Hanbay. 2019. Deep Neural Network Based Approach for ECG Classification Using Hybrid Differential Features and Active Learning. Iet Signal Processing 13, 2 (2019), 165–175.
86.Manuel Haußmann, Fred A. Hamprecht, and Melih Kandemir. 2019. Deep Active Learning with Adaptive Acquisition. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019. ijcai.org, 2470–2476.
87.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, 770–778.
88.Tao He, Xiaoming Jin, Guiguang Ding, Lan Yi, and Chenggang Yan. 2019. Towards Better Uncertainty Sampling: Active Learning with Multiple Views for Deep Convolutional Neural Network. In IEEE International Conference on Multimedia and Expo, ICME 2019, Shanghai, China, July 8-12, 2019. IEEE, 1360–1365.
89.Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li. 2019. Bag of Tricks for Image Classification with Convolutional Neural Networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, 558–567.
90.José Miguel Hernández-Lobato and Ryan P. Adams. 2015. Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015 (JMLR Workshop and Conference Proceedings, Vol. 37). JMLR.org, 1861–1869.
91.Geoffrey E. Hinton, Simon Osindero, and Yee Whye Teh. 2006. A Fast Learning Algorithm for Deep Belief Nets. Neural Comput. 18, 7 (2006), 1527–1554.
92.Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. CoRR abs/1207.0580 (2012).
93.Martin Hirzer, Csaba Beleznai, Peter M Roth, and Horst Bischof. 2011. Person re-identification by descriptive and discriminative classification. In Scandinavian conference on Image analysis. Springer, 91–102.
94.Steven C. H. Hoi, Rong Jin, Jianke Zhu, and Michael R. Lyu. 2006. Batch mode active learning and its application to medical image classification. In Machine Learning, Proceedings of the Twenty-Third International Conference (ICML 2006), Pittsburgh, Pennsylvania, USA, June 25-29, 2006 (ACM International Conference Proceeding Series, Vol. 148). ACM, 417–424.
95.H. M. Sajjad Hossain, M. D. Abdullah Al Haiz Khan, and Nirmalya Roy. 2018. DeActive: Scaling Activity Recognition with Active Deep Learning. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 2 (2018), 66:1–66:23.
96.H. M. Sajjad Hossain and Nirmalya Roy. 2019. Active Deep Learning for Activity Recognition with Context Aware Annotator Selection. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019. ACM, 1862–1870.
97.Neil Houlsby, Ferenc Huszar, Zoubin Ghahramani, and Máté Lengyel. 2011. Bayesian Active Learning for Classification and Preference Learning. CoRR abs/1112.5745 (2011).
98.Yue Huang, Zhenwei Liu, Minghui Jiang, Xian Yu, and Xinghao Ding. 2020. Cost-Effective Vehicle Type Recognition in Surveillance Images With Deep Active Learning and Web Data. IEEE Transactions on Intelligent Transportation Systems 21, 1 (2020), 79–86.
99.Jonathan H Huggins, Trevor Campbell, and Tamara Broderick. 2016. Coresets for Scalable Bayesian Logistic Regression. (2016), 4080–4088.
100.Ahmed Hussein, Mohamed Medhat Gaber, and Eyad Elyan. 2016. Deep Active Learning for Autonomous Navigation. In Engineering Applications of Neural Networks - 17th International Conference, EANN 2016, Aberdeen, UK, September 2-5, 2016, Proceedings (Communications in Computer and Information Science, Vol. 629). Springer, 3–17.
101.Rania Ibrahim, Noha A Yousri, Mohamed A Ismail, and Nagwa M El-Makky. 2014. Multi-level gene/MiRNA feature selection using deep belief nets and active learning. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 3957–3960.
102.Prateek Jain and Ashish Kapoor. 2009. Active learning for large multi-class problems. (2009), 762–769.
103.David Janz, Jos van der Westhuizen, and José Miguel Hernández-Lobato. 2017. Actively Learning what makes a Discrete Sequence Valid. CoRR abs/1708.04465 (2017).
104.Khaled Jedoui, Ranjay Krishna, Michael Bernstein, and Fei-Fei Li. 2019. Deep Bayesian Active Learning for Multiple Correct Outputs. CoRR abs/1912.01119 (2019).
105.Michael I. Jordan. 1986. Serial Order: A Parallel Distributed Processing Approach. Advances in psychology 121 (1986), 471–495.
106.Ajay Joshi, Fatih Porikli, and Nikolaos Papanikolopoulos. 2009. Multi-class active learning for image classification. (2009), 2372–2379.
107.J. Ajay Joshi, Fatih Porikli, and Nikolaos Papanikolopoulos. 2010. Multi-class batch-mode active learning for image classification. Robotics and Automation (2010), 1873–1878.
108.Takeo Kanade, Jeffrey F Cohn, and Yingli Tian. 2000. Comprehensive database for facial expression analysis. In Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580). IEEE, 46–53.
109.Jungo Kasai, Kun Qian, Sairam Gurajada, Yunyao Li, and Lucian Popa. 2019. Low-resource Deep Entity Resolution with Transfer and Active Learning. (2019), 5851–5861.
110.Benjamin Kellenberger, Diego Marcos, Sylvain Lobry, and Devis Tuia. 2019. Half a Percent of Labels is Enough: Efficient Animal Detection in UAV Imagery Using Deep CNNs and Active Learning. IEEE Transactions on Geoscience and Remote Sensing 57, 12 (2019), 9524–9533.
111.Alex Kendall, Vijay Badrinarayanan, and Roberto Cipolla. 2017. Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding. In British Machine Vision Conference 2017, BMVC 2017, London, UK, September 4-7, 2017. BMVA Press.
112.Kwanyoung Kim, Dongwon Park, Kwang In Kim, and Se Young Chun. 2020. Task-Aware Variational Adversarial Active Learning. arXiv: Learning (2020).
113.Ross D King, Kenneth E Whelan, Ffion M Jones, Philip G K Reiser, Christopher H Bryant, Stephen Muggleton, Douglas B Kell, and Stephen G Oliver. 2004. Functional genomic hypothesis generation and experimentation by a robot scientist. Nature 427, 6971 (2004), 247–252.
114.Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings.
115.Andreas Kirsch, Joost van Amersfoort, and Yarin Gal. 2019. BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. 7024–7035.
116.Xiangnan Kong, Jiawei Zhang, and Philip S. Yu. 2013. Inferring anchor links across multiple heterogeneous social networks. In 22nd ACM International Conference on Information and Knowledge Management, CIKM’13, San Francisco, CA, USA, October 27 - November 1, 2013. ACM, 179–188.
117.Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. 2017. Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations. Int. J. Comput. Vis. 123, 1 (2017), 32–73.
118.Vikram Krishnamurthy. 2002. Algorithms for optimal scheduling and management of hidden Markov model sensors. IEEE Transactions on Signal Processing 50, 6 (2002), 1382–1397.
119.Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. Citeseer.
120.Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. (2012), 1097–1105.
121.M P Kumar, Benjamin Packer, and Daphne Koller. 2010. Self-Paced Learning for Latent Variable Models. (2010), 1189–1197.
122.Jennifer R. Kwapisz, Gary M. Weiss, and Samuel Moore. 2010. Activity recognition using cell phone accelerometers. SIGKDD Explor. 12, 2 (2010), 74–82.
123.Bogdan Kwolek, Michal Koziarski, Andrzej Bukala, Zbigniew Antosz, Boguslaw Olborski, Pawel Wąsowicz, Jakub Swadźba, and Boguslaw Cyganek. 2019. Breast Cancer Classification on Histopathological Images Affected by Data Imbalance Using Active Learning and Deep Convolutional Neural Network. (2019), 299–312.
124.Sejeong Kwon, Meeyoung Cha, and Kyomin Jung. 2017. Rumor detection over varying time windows. PloS one 12, 1 (2017), e0168344.
125.Leah S. Larkey. 1999. A Patent Search and Classification System. In Proceedings of the Fourth ACM conference on Digital Libraries, August 11-14, 1999, Berkeley, CA, USA. ACM, 179–187.
126.Yann Lecun, Yoshua Bengio, and Geoffrey E Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436–444.
127.Y. LeCun, B. Boser, S. J. Denker, D. Henderson, E. R. Howard, W. Hubbard, and D. L. Jackel. 1989. Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation (1989), 541–551.
128.Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278–2324.
129.Byungjae Lee and Kyunghyun Paeng. 2018. A Robust and Effective Approach Towards Accurate Metastasis Detection and pN-stage Classification in Breast Cancer. (2018), 841–850.
130.Christian Leibig, Vaneeda Allken, Murat Seckin Ayhan, Philipp Berens, and Siegfried Wahl. 2017. Leveraging uncertainty information from deep neural networks for disease detection. Scientific Reports 7, 1 (2017), 17816–17816.
131.David D Lewis and William A Gale. 1994. A sequential algorithm for training text classifiers. (1994), 3–12.
132.Ya Li, Keze Wang, Lin Nie, and Qing Wang. 2017. Face Recognition via Heuristic Deep Active Learning. (2017), 97–107.
133.Jianzhe Lin, Liang Zhao, Shuying Li, Rabab K Ward, and Z Jane Wang. 2018. Active-Learning-Incorporated Deep Transfer Learning for Hyperspectral Image Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 11, 11 (2018), 4048–4062.
134.Xiao Lin and Devi Parikh. 2017. Active Learning for Visual Question Answering: An Empirical Study. CoRR abs/1711.01732 (2017).
135.Peng Liu, Hui Zhang, and Kie B Eom. 2017. Active Deep Learning for Classification of Hyperspectral Images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 10, 2 (2017), 712–724.
136.Zimo Liu, Jingya Wang, Shaogang Gong, Huchuan Lu, and Dacheng Tao. 2019. Deep Reinforcement Active Learning for Human-in-the-Loop Person Re-Identification. (2019), 6122–6131.
137.Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. (2015), 3431–3440.
138.Reza Lotfian and Carlos Busso. 2017. Building naturalistic emotionally balanced speech corpus by retrieving emotional speech from existing podcast recordings. IEEE Transactions on Affective Computing (2017).
139.David G. Lowe. 1999. Object Recognition from Local Scale-Invariant Features. In Proceedings of the International Conference on Computer Vision, Kerkyra, Corfu, Greece, September 20-25, 1999. 1150–1157.
140.Xiaoming Lv, Fajie Duan, Jiajia Jiang, Xiao Fu, and Lin Gan. 2020. Deep Active Learning for Surface Defect Detection. Sensors 20, 6 (2020), 1650.
141.Ramon Maldonado and Sanda M Harabagiu. 2019. Active deep learning for the identification of concepts and relations in electroencephalography reports. Journal of Biomedical Informatics 98 (2019), 103265.
142.RG Mark, PS Schluter, G Moody, P Devlin, and D Chernoff. 1982. An annotated ECG database for evaluating arrhythmia detectors. In IEEE Transactions on Biomedical Engineering, Vol. 29. IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC 345 E 47TH ST, NEW YORK, NY . . . , 600–600.
143.Giovanna Martínez-Arellano and Svetan M. Ratchev. 2019. Towards An Active Learning Approach To Tool Condition Monitoring With Bayesian Deep Learning. In Proceedings of the 33rd International ECMS Conference on Modelling and Simulation, ECMS 2019 Caserta, Italy, June 11-14, 2019. European Council for Modeling and Simulation, 223–229.
144.Muhammad Mateen, Junhao Wen, Nasrullah, Sun Song, and Zhouping Huang. 2019. Fundus Image Classification Using VGG-19 Architecture with PCA and SVD. Symmetry 11 (2019), 1.
145.Taylor R. Mauldin, Marc E. Canby, Vangelis Metsis, Anne H. H. Ngu, and Coralys Cubero Rivera. 2018. SmartFall: A Smartwatch-Based Fall Detection System Using Deep Learning. Sensors 18, 10 (2018), 3363.
146.Ali Mottaghi and Serena Yeung. 2019. Adversarial Representation Active Learning. CoRR abs/1912.09720 (2019).
147.Sidharth Mudgal, Han Li, Theodoros Rekatsinas, AnHai Doan, Youngchoon Park, Ganesh Krishnan, Rohit Deep, Esteban Arcaute, and Vijay Raghavendra. 2018. Deep Learning for Entity Matching: A Design Space Exploration. In Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018, Houston, TX, USA, June 10-15, 2018. ACM, 19–34.
148.Prateek Munjal, Nasir Hayat, Munawar Hayat, Jamshid Sourati, and Shadab Khan. 2020. Towards Robust and Reproducible Active Learning Using Neural Networks. arXiv (2020), arXiv–2002.
149.Shikhar Murty, Patrick Verga, Luke Vilnis, Irena Radovanovic, and Andrew McCallum. 2018. Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers. Association for Computational Linguistics, 97–109.
150.Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML.
151.Ju Gang Nam, Sunggyun Park, Eui Jin Hwang, Jong Hyuk Lee, Kwang Nam Jin, Kun Young Lim, Thienkai Huy Vu, Jae Ho Sohn, Sangheum Hwang, Jin Mo Goo, et al. 2019. Development and Validation of Deep Learning–based Automatic Detection Algorithm for Malignant Pulmonary Nodules on Chest Radiographs. Radiology 290, 1 (2019), 218–228.
152.Ali Bou Nassif, Ismail Shahin, Imtinan B. Attili, Mohammad Azzeh, and Khaled Shaalan. 2019. Speech Recognition Using Deep Neural Networks: A Systematic Review. IEEE Access 7 (2019), 19143–19165.
153.T. Hieu Nguyen and Arnold Smeulders. 2004. Active learning using pre-clustering. ICML (2004), 79–79.
154.Mohammad Sadegh Norouzzadeh, Dan Morris, Sara Beery, Neel Joshi, Nebojsa Jojic, and Jeff Clune. 2019. A deep active learning system for species identification and counting in camera trap images. CoRR abs/1910.09716 (2019).
155.Augustus Odena, Christopher Olah, and Jonathon Shlens. 2017. Conditional Image Synthesis With Auxiliary Classifier GANs. (2017), 2642–2651.
156.Natalia Ostapuk, Jie Yang, and Philippe Cudre-Mauroux. 2019. ActiveLink: Deep Active Learning for Link Prediction in Knowledge Graphs. WWW ’19: The Web Conference on The World Wide Web Conference WWW 2019 (2019), 1398–1408.
157.Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment Classification using Machine Learning Techniques. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, EMNLP 2002, Philadelphia, PA, USA, July 6-7, 2002. 79–86.
158.Mercedes Eugenia Paoletti, Juan Mario Haut, Rubén Fernández-Beltran, Javier Plaza, Antonio J. Plaza, Jun Yu Li, and Filiberto Pla. 2019. Capsule Networks for Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing 57 (2019), 2145–2160.
159.Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, and Quoc V. Le. 2019. SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition. In Interspeech 2019, 20th Annual Conference of the International Speech Communication Association, Graz, Austria, 15-19 September 2019. ISCA, 2613–2617.
160.John P. Pestian, Chris Brew, Pawel Matykiewicz, D. J. Hovermale, Neil Johnson, K. Bretonnel Cohen, and Wlodzislaw Duch. 2007. A shared task involving multi-label classification of clinical free text. In Biological, translational, and clinical language processing, BioNLP@ACL 2007, Prague, Czech Republic, June 29, 2007. Association for Computational Linguistics, 97–104.
161.Jeff M Phillips. 2016. Coresets and sketches. arXiv preprint arXiv:1601.00617 (2016).
162.Robert Pinsler, Jonathan Gordon, Eric Nalisnick, and Jose Miguel Hernandezlobato. 2019. Bayesian Batch Active Learning as Sparse Subset Approximation. (2019), 6356–6367.
163.Emmanouil Antonios Platanios, Mrinmaya Sachan, Graham Neubig, and Tom M. Mitchell. 2018. Contextual Parameter Generation for Universal Neural Machine Translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. Association for Computational Linguistics, 425–435.
164.Emmanouil Antonios Platanios, Otilia Stretcu, Graham Neubig, Barnabas Poczos, and Tom M Mitchell. 2019. Competence-based curriculum learning for neural machine translation. arXiv preprint arXiv:1903.09848 (2019).
165.Remus Pop and Patric Fulop. 2018. Deep Ensemble Bayesian Active Learning : Addressing the Mode Collapse issue in Monte Carlo dropout via Ensembles. CoRR abs/1811.03897 (2018).
166.Ameya Prabhu, Charles Dognin, and Maneesh Singh. 2019. Sampling Bias in Deep Active Classification: An Empirical Study. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. Association for Computational Linguistics, 4056–4066.
167.Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards Robust Linguistic Analysis using OntoNotes. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, CoNLL 2013, Sofia, Bulgaria, August 8-9, 2013. ACL, 143–152.
168.Shalini Priya, Saharsh Singh, Sourav Kumar Dandapat, Kripabandhu Ghosh, and Joydeep Chandra. 2019. Identifying infrastructure damage during earthquake using deep active learning. (2019), 551–552.
169.Yao Qin, Nicholas Carlini, Ian J. Goodfellow, Garrison W. Cottrell, and Colin Raffel. 2019. Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition. ArXiv abs/1903.10346 (2019).
170.Zhenshen Qu, Jingda Du, Yong Cao, Qiuyu Guan, and Pengbo Zhao. 2020. Deep Active Learning for Remote Sensing Object Detection. CoRR abs/2003.08793 (2020).
171.M M Al Rahhal, Yakoub Bazi, Haikel Alhichri, Naif Alajlan, Farid Melgani, and Ronald R Yager. 2016. Deep learning approach for active classification of electrocardiogram signals. Information Sciences 345, 345 (2016), 340–354.
172.Hiranmayi Ranganathan, Shayok Chakraborty, and Sethuraman Panchanathan. 2016. Multimodal emotion recognition using deep learning architectures. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1–9.
173.Hiranmayi Ranganathan, Hemanth Venkateswara, Shayok Chakraborty, and Sethuraman Panchanathan. 2017. Deep active learning for image classification. (2017), 3934–3938.
174.Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Xiaojiang Chen, and Xin Wang. 2020. A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions. CoRR abs/2006.02903 (2020).
175.Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. (2015), 234–241.
176.Matthias Rottmann, Karsten Kahl, and Hanno Gottschalk. 2018. Deep Bayesian Active Semi-Supervised Learning. In 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018, Orlando, FL, USA, December 17-20, 2018. IEEE, 158–164.
177.Nicholas Roy and Andrew McCallum. 2001. Toward optimal active learning through monte carlo estimation of error reduction. ICML, Williamstown (2001), 441–448.
178.Soumya Roy, Asim Unmesh, and Vinay P. Namboodiri. 2018. Deep active learning for object detection. In British Machine Vision Conference 2018, BMVC 2018, Newcastle, UK, September 3-6, 2018. BMVA Press, 91.
179.David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1986. Learning internal representations by error propagation. Parallel distributed processing: explorations in the microstructure of cognition, vol. 1 (1986), 318–362.
180.David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. 1986. Learning representations by back-propagating errors. Nature 323 (1986), 533–536.
181.Ario Sadafi, Niklas Koehler, Asya Makhro, Anna Bogdanova, Nassir Navab, Carsten Marr, and Tingying Peng. 2019. Multiclass Deep Active Learning for Detecting Red Blood Cell Subtypes in Brightfield Microscopy. (2019), 685–693.
182.Mathew Salvaris, Danielle Dean, and Wee Hyong Tok. 2018. Generative Adversarial Networks. arXiv: Machine Learning (2018), 187–208.
183.Conrad Sanderson. 2008. Biometric person recognition: Face, speech and fusion. Vol. 4. VDM Publishing.
184.Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. In Proceedings of the Seventh Conference on Natural Language Learning, CoNLL 2003, Held in cooperation with HLT-NAACL 2003, Edmonton, Canada, May 31 - June 1, 2003. ACL, 142–147.
185.Yassir Saquil, Kwang In Kim, and Peter Hall. 2018. Ranking CGANs: Subjective Control over Semantic Image Attributes. (2018), 131.
186.G Sayantan, P T Kien, and K V Kadambari. 2018. Classification of ECG beats using deep belief network and active learning. Medical & Biological Engineering & Computing 56, 10 (2018), 1887–1898.
187.Melanie Lubrano Di Scandalea, Christian S Perone, Mathieu Boudreau, and Julien Cohenadad. 2019. Deep Active Learning for Axon-Myelin Segmentation on Histology Data. arXiv: Computer Vision and Pattern Recognition (2019).
188.Tobias Scheffer, Christian Decomain, and Stefan Wrobel. 2001. Active Hidden Markov Models for Information Extraction. In Advances in Intelligent Data Analysis, 4th International Conference, IDA 2001, Cascais, Portugal, September 13-15, 2001, Proceedings (Lecture Notes in Computer Science, Vol. 2189). Springer, 309–318.
189.Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019. wav2vec: Unsupervised Pre-training for Speech Recognition. ArXiv abs/1904.05862 (2019).
190.Christopher Schröder and Andreas Niekler. 2020. A Survey of Active Learning for Text Classification using Deep Neural Networks. arXiv preprint arXiv:2008.07267 (2020).
191.Ozan Sener and Silvio Savarese. 2017. A geometric approach to active learning for convolutional neural networks. arXiv preprint arXiv:1708.00489 7 (2017).
192.Ozan Sener and Silvio Savarese. 2018. Active Learning for Convolutional Neural Networks: A Core-Set Approach. international conference on learning representations (2018).
193.Burr Settles. 2009. Active learning literature survey. Technical Report. University of Wisconsin-Madison Department of Computer Sciences.
194.Burr Settles. 2012. Active Learning, volume 6 of Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool (2012).
195.Burr Settles, Mark Craven, and Soumya Ray. 2007. Multiple-Instance Active Learning. (2007), 1289–1296.
196.H S Seung, M Opper, and H Sompolinsky. 1992. Query by committee. (1992), 287–294.
197.Matthew Shardlow, Meizhi Ju, Maolin Li, Christian O’Reilly, Elisabetta Iavarone, John McNaught, and Sophia Ananiadou. 2019. A text mining pipeline using active and deep learning aimed at curating information in computational neuroscience. Neuroinformatics 17, 3 (2019), 391–406.
198.Artem Shelmanov, Vadim Liventsev, Danil Kireev, Nikita Khromov, Alexander Panchenko, Irina Fedulova, and Dmitry V. Dylov. 2019. Active Learning with Deep Pre-trained Models for Sequence Tagging of Clinical and Biomedical Texts. In 2019 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2019, San Diego, CA, USA, November 18-21, 2019. IEEE, 482–489.
199.Yanyao Shen, Hyokun Yun, Zachary C Lipton, Yakov Kronrod, and Animashree Anandkumar. 2017. Deep active learning for named entity recognition. arXiv preprint arXiv:1707.05928 (2017).
200.Changjian Shui, Fan Zhou, Christian Gagné, and Boyu Wang. 2020. Deep active learning: Unified and principled method for query and training. In International Conference on Artificial Intelligence and Statistics. PMLR, 1308–1318.
201.Aditya Siddhant and Zachary C. Lipton. 2018. Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. Association for Computational Linguistics, 2904–2909.
202.Oriane Simeoni, Mateusz Budnik, Yannis Avrithis, and Guillaume Gravier. 2019. Rethinking deep active learning: Using unlabeled data at model training. CoRR abs/1911.08177 (2019).
203.Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
204.Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. 2019. Variational Adversarial Active Learning. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, 5971–5980.
205.Asim Smailagic, Pedro Costa, Alex Gaudio, Kartik Khandelwal, Mostafa Mirshekari, Jonathon Fagert, Devesh Walawalkar, Susu Xu, Adrian Galdran, Pei Zhang, et al. 2020. O-MedAL: Online active deep learning for medical image analysis. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 10, 4 (2020), e1353.
206.Asim Smailagic, Pedro Costa, Hae Young Noh, Devesh Walawalkar, Kartik Khandelwal, Adrian Galdran, Mostafa Mirshekari, Jonathon Fagert, Susu Xu, Pei Zhang, et al. 2018. MedAL: Accurate and Robust Deep Active Learning for Medical Image Analysis. (2018), 481–488.
207.Justin S Smith, Benjamin Nebgen, Nicholas Lubbers, Olexandr Isayev, and Adrian E Roitberg. 2018. Less is more: Sampling chemical space with active learning. Journal of Chemical Physics 148, 24 (2018), 241733.
208.Kihyuk Sohn, Xinchen Yan, and Honglak Lee. 2015. Learning structured output representation using deep conditional generative models. (2015), 3483–3491.
209.Kechen Song and Yunhui Yan. 2013. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science 285 (2013), 858–864.
210.Fabio A. Spanhol, Luiz S. Oliveira, Caroline Petitjean, and Laurent Heutte. 2016. A Dataset for Breast Cancer Histopathological Image Classification. IEEE Trans. Biomed. Eng. 63, 7 (2016), 1455–1462.
211.Akash Srivastava, Lazar Valkoz, Chris Russell, U. Michael Gutmann, and A. Charles Sutton. 2017. VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning. neural information processing systems (2017), 3310–3320.
212.Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15, 1 (2014), 1929–1958.
213.Fabian Stark, Caner Hazırbas, Rudolph Triebel, and Daniel Cremers. 2015. Captcha recognition with active deep learning. In Workshop new challenges in neural computation, Vol. 2015. Citeseer, 94.
214.Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kjærgaard, Anind Dey, Tobias Sonne, and Mads Møller Jensen. 2015. Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM conference on embedded networked sensor systems. 127–140.
215.Alexandra Swanson, Margaret Kosmala, Chris Lintott, Robert Simpson, Arfon Smith, and Craig Packer. 2015. Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna. Scientific data 2, 1 (2015), 1–14.
216.Kuniyuki Takahashi, Tetsuya Ogata, Jun Nakanishi, Gordon Cheng, and Shigeki Sugano. 2017. Dynamic motion learning for multi-DOF flexible-joint robots using active–passive motor babbling through deep learning. Advanced Robotics 31, 18 (2017), 1002–1015.
217.Xu Tan, Yi Ren, Di He, Tao Qin, Zhou Zhao, and Tie-Yan Liu. 2019. Multilingual Neural Machine Translation with Knowledge Distillation. ArXiv abs/1902.10461 (2019).
218.Yao Tan, Liu Yang, Qinghua Hu, and Zhibin Du. 2019. Batch Mode Active Learning for Semantic Segmentation Based on Multi-Clue Sample Selection. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019. ACM, 831–840.
219.Joseph Taylor, H. M. Sajjad Hossain, Mohammad Arif Ul Alam, Md Abdullah Al Hafiz Khan, Nirmalya Roy, Elizabeth Galik, and Aryya Gangopadhyay. 2017. SenseBox: A low-cost smart home system. In 2017 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2017, Kona, Big Island, HI, USA, March 13-17, 2017. IEEE, 60–62.
220.Jörg Tiedemann. 2012. Parallel Data, Tools and Interfaces in OPUS. In Proceedings of the Eighth International Conference on Language Resources and Evaluation, LREC 2012, Istanbul, Turkey, May 23-25, 2012. European Language Resources Association (ELRA), 2214–2218.
221.Simon Tong. 2001. Active learning: theory and applications. Vol. 1. Stanford University USA.
222.Simon Tong and Daphne Koller. 2002. Support vector machine active learning with applications to text classification. Journal of Machine Learning Research 2, 1 (2002), 45–66.
223.Toan Tran, Thanh-Toan Do, Ian D. Reid, and Gustavo Carneiro. 2019. Bayesian Generative Active Deep Learning. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA (Proceedings of Machine Learning Research, Vol. 97). PMLR, 6295–6304.
224.Toan Tran, Trung Pham, Gustavo Carneiro, Lyle Palmer, and Ian Reid. 2017. A bayesian data augmentation approach for learning deep models. In Advances in neural information processing systems. 2797–2806.
225.Kailas Vodrahalli, Ke Li, and Jitendra Malik. 2018. Are All Training Examples Created Equal? An Empirical Study. CoRR abs/1811.12569 (2018).
226.Byron C. Wallace, Michael J. Paul, Urmimala Sarkar, Thomas A. Trikalinos, and Mark Dredze. 2014. A large-scale quantitative analysis of latent factors and sentiment in online doctor reviews. J. Am. Medical Informatics Assoc. 21, 6 (2014), 1098–1103.
227.Dan Wang and Yi Shang. 2014. A new active labeling method for deep learning. (2014), 112–119.
228.Jiannan Wang, Guoliang Li, Jeffrey Xu Yu, and Jianhua Feng. 2011. Entity Matching: How Similar Is Similar. Proc. VLDB Endow. 4, 10 (2011), 622–633.
229.K. Wang, D. Zhang, Y. Li, R. Zhang, and L. Lin. 2017. Cost-Effective Active Learning for Deep Image Classification. IEEE Transactions on Circuits and Systems for Video Technology 27, 12 (2017), 2591–2600.
230.Menglin Wang, Baisheng Lai, Zhongming Jin, Xiaojin Gong, Jianqiang Huang, and Xiansheng Hua. 2018. Deep active learning for video-based person re-identification. arXiv preprint arXiv:1812.05785 (2018).
231.Qiang Wang, Bei Li, Tong Xiao, Jingbo Zhu, Changliang Li, Derek F. Wong, and Lidia S. Chao. 2019. Learning Deep Transformer Models for Machine Translation. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 1810–1822.
232.Wenzhe Wang, Ruiwei Feng, Jintai Chen, Yifei Lu, Tingting Chen, Hongyun Yu, Danny Z Chen, and Jian Wu. 2019. Nodule-Plus R-CNN and Deep Self-Paced Active Learning for 3D Instance Segmentation of Pulmonary Nodules. IEEE Access 7 (2019), 128796–128805.
233.Wenzhe Wang, Yifei Lu, Bian Wu, Tingting Chen, Danny Z Chen, and Jian Wu. 2018. Deep Active Self-paced Learning for Accurate Pulmonary Nodule Segmentation. (2018), 723–731.
234.William Yang Wang. 2017. "Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 2: Short Papers, Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 422–426.
235.Zengmao Wang, Bo Du, Lefei Zhang, and Liangpei Zhang. 2016. A batch-mode active learning framework by querying discriminative and representative samples for hyperspectral image classification. Neurocomputing 179 (2016), 88–100.
236.Max Welling and Whye Yee Teh. 2011. Bayesian Learning via Stochastic Gradient Langevin Dynamics. ICML (2011), 681–688.
237.Yu Wu, Yutian Lin, Xuanyi Dong, Yan Yan, Wanli Ouyang, and Yi Yang. 2018. Exploit the Unknown Gradually: One-Shot Video-Based Person Re-Identification by Stepwise Learning. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. IEEE Computer Society, 5177–5186.
238.Xide Xia, Pavlos Protopapas, and Finale Doshivelez. 2016. Cost-Sensitive Batch Mode Active Learning: Designing Astronomical Observation by Optimizing Telescope Time and Telescope Choice. (2016), 477–485.
239.Ismet Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, and Dhruv Kumar Mahajan. 2019. Billion-scale semi-supervised learning for image classification. ArXiv abs/1905.00546 (2019).
240.Yilin Yan, Min Chen, Saad Sadiq, and Mei-Ling Shyu. 2017. Efficient imbalanced multimedia concept retrieval by deep learning on spark clusters. International Journal of Multimedia Data Engineering and Management (IJMDEM) 8, 1 (2017), 1–20.
241.Yilin Yan, Min Chen, Mei-Ling Shyu, and Shu-Ching Chen. 2015. Deep learning for imbalanced multimedia data classification. In 2015 IEEE international symposium on multimedia (ISM). IEEE, 483–488.
242.Jie Yang, Thomas Drake, Andreas Damianou, and Yoelle Maarek. 2018. Leveraging Crowdsourcing Data For Deep Active Learning – An Application: Learning Intents in Alexa. (2018), 23–32.
243.Lin Yang, Yizhe Zhang, Jianxu Chen, Siyuan Zhang, and Danny Z Chen. 2017. Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation. (2017), 399–407.
244.C. Yin, B. Qian, S. Cao, X. Li, J. Wei, Q. Zheng, and I. Davidson. 2017. Deep Similarity-Based Batch Mode Active Learning with Exploration-Exploitation. In 2017 IEEE International Conference on Data Mining (ICDM). 575–584.
245.Donggeun Yoo and In So Kweon. 2019. Learning Loss for Active Learning. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, 93–102.
246.Fisher Yu, Wenqi Xian, Yingying Chen, Fangchen Liu, Mike Liao, Vashisht Madhavan, and Trevor Darrell. 2018. BDD100K: A Diverse Driving Video Database with Scalable Annotation Tooling. CoRR abs/1805.04687 (2018).
247.Jiaming Zeng, Adam Lesnikowski, and Jose M. Alvarez. 2018. The Relevance of Bayesian Layer Positioning to Model Uncertainty in Deep Bayesian Active Learning. CoRR abs/1811.12535 (2018).
248.Pei Zhang, Xueying Xu, and Deyi Xiong. 2018. Active Learning for Neural Machine Translation. In 2018 International Conference on Asian Language Processing, IALP 2018, Bandung, Indonesia, November 15-17, 2018. IEEE, 153–158.
249.Shanshan Zhang, Rodrigo Benenson, and Bernt Schiele. 2017. CityPersons: A Diverse Dataset for Pedestrian Detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society, 4457–4465.
250.Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015. Character-level Convolutional Networks for Text Classification. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada. 649–657.
251.Ye Zhang, Matthew Lease, and Byron C. Wallace. 2017. Active Discriminative Text Representation Learning. (2017), 3386–3392.
252.Yizhe Zhang, Michael T. C. Ying, Lin Yang, Anil T. Ahuja, and Danny Z. Chen. 2016. Coarse-to-Fine Stacked Fully Convolutional Nets for lymph node segmentation in ultrasound images. In IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2016, Shenzhen, China, December 15-18, 2016. IEEE Computer Society, 443–448. https://doi.org/10.1109/BIBM.2016.7822557
253.Wencang Zhao, Yu Kong, Zhengming Ding, and Yun Fu. 2017. Deep Active Learning Through Cognitive Information Parcels. (2017), 952–960.
254.Ziyuan Zhao, Xiaoyan Yang, Bharadwaj Veeravalli, and Zeng Zeng. 2020. Deeply Supervised Active Learning for Finger Bones Segmentation. arXiv preprint arXiv:2005.03225 (2020).
255.Fedor Zhdanov. 2019. Diverse mini-batch Active Learning. CoRR abs/1901.05954 (2019).
256.Liang Zheng, Zhi Bie, Yifan Sun, Jingdong Wang, Chi Su, Shengjin Wang, and Qi Tian. 2016. MARS: A Video Benchmark for Large-Scale Person Re-Identification. In Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI (Lecture Notes in Computer Science, Vol. 9910). Springer, 868–884.
257.Shusen Zhou, Qingcai Chen, and Xiaolong Wang. 2010. Active Deep Networks for Semi-Supervised Sentiment Classification. (2010), 1515–1523.
258.Siqi Zhou and Angela P Schoellig. 2019. Active Training Trajectory Generation for Inverse Dynamics Model Learning with Deep Neural Networks. In 2019 IEEE 58th Conference on Decision and Control (CDC). IEEE, 1784–1790.
259.Jia-Jie Zhu and José Bento. 2017. Generative adversarial active learning. arXiv preprint arXiv:1702.07956 (2017).
260.Xiaojin Zhu, John Lafferty, and Ronald Rosenfeld. 2005. Semi-supervised learning with graphs. Ph.D. Dissertation. Carnegie Mellon University, language technologies institute, school of . . . .
261.Michal Ziemski, Marcin Junczys-Dowmunt, and Bruno Pouliquen. 2016. The United Nations Parallel Corpus v1.0. In Proceedings of the Tenth International Conference on Language Resources and Evaluation LREC 2016, Portorož, Slovenia, May 23-28, 2016. European Language Resources Association (ELRA).
262.Barret Zoph and Quoc V. Le. 2017. Neural Architecture Search with Reinforcement Learning. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings.

Citation

MLA

Ren, P., et al. “A Survey of Deep Active Learning”. arXiv, 2020, http://arxiv.org/abs/2009.00236v2.

APA

Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Gupta, B. B., Chen, X., & Wang, X. (2020). A Survey of Deep Active Learning. arXiv. http://arxiv.org/abs/2009.00236v2

Chicago

Ren, P., Y. Xiao, X. Chang, et al. 2020. “A Survey of Deep Active Learning”. arXiv. http://arxiv.org/abs/2009.00236v2.

Harvard

Ren, P. et al. (2020) “A Survey of Deep Active Learning”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2009.00236v2.

Vancouver

1. Ren P, Xiao Y, Chang X, Huang P-Y, Li Z, Gupta BB, Chen X, Wang X (2020) A Survey of Deep Active Learning. arXiv

BibTeX

@article{ren2020survey,
  title = {A Survey of Deep Active Learning},
  author = {Ren, Pengzhen and Xiao, Yun and Chang, Xiaojun and Huang, Po-Yao and Li, Zhihui and Gupta, Brij B. and Chen, Xiaojiang and Wang, Xin},
  year = {2020},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2009.00236v2},
  eprint = {2009.00236}
}

Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/

A Survey of Deep Active Learning

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Formal Optimization Formulation of Pool-Based Deep Active Learning

Knowl 2 — Core Challenges in Integrating Active Learning with Deep Learning

Knowl 3 — Batch Mode Deep Active Learning and Joint Mutual Information

Knowl 4 — Hybrid Query Strategies Balancing Uncertainty and Diversity in DeepAL

Knowl 5 — Deep Bayesian Active Learning (DBAL) Framework

Knowl 6 — Density-Based and Core-Set Sample Selection in DeepAL

Knowl 7 — Data Expansion and Semi-Supervised Augmentation in DeepAL

Knowl 8 — Task-Agnostic Multi-Layer Loss Prediction in DeepAL

Knowl 9 — Automated Design of DeepAL via Reinforcement Learning and Neural Architecture Search

Knowl 10 — Deep Active Learning Stopping Criteria and Stabilizing Predictions

Knowl 11 — Evaluation Discrepancies and Random Sampling Baselines in DeepAL

Knowl 12 — Cross-Domain Application Taxonomy of Deep Active Learning

References

Citation

BibTeX

Access the Paper