Back

Heterogeneous Uncertainty Sampling for Supervised Learning

David D. LewisJason Catlett

article1994ICML1,421 citations

Demonstrates that using a fast probabilistic classifier to actively select training instances for a more complex C4.5 rule induction model achieves lower error rates on text categorization tasks than random sampling sets ten times larger.

Listen

Building automated text categorization systems often requires human experts to label large volumes of training data, creating a costly and time-consuming bottleneck. This challenge is especially acute when target categories are rare, as conventional random sampling forces experts to review thousands of uninformative examples to find a handful of relevant instances. Uncertainty sampling addresses this by presenting experts with only the most ambiguous cases; however, high-performing rule-based classifiers are computationally prohibitive to run repeatedly inside the sampling loop across hundreds of thousands of documents.

The article demonstrates and evaluates a heterogeneous uncertainty sampling approach, where a computationally cheap probabilistic classifier selects the most informative examples to train a more complex, interpretable decision rule classifier (C4.5). The evaluation used a real-world news dataset of over 371,000 articles across ten low-frequency subject categories to determine whether this cross-model sampling delivers high classification accuracy with substantially less manual labeling.

Key findings show that decision rules trained on uncertainty samples of roughly 1,000 instances achieved lower error rates than those trained on random samples of 10,000 instances—a tenfold reduction in required training data. Because uncertainty sampling deliberately overrepresents rare categories, unadjusted decision rules tend to generate excessive false positives; however, introducing a cost-adjustment parameter (loss ratio) between 3 and 20 successfully countered this bias and proved robust across categories. At a loss ratio of 5, the 1,000-example uncertainty sample yielded statistically significant improvements over the 10,000-example random baseline. In several cases, samples as small as 299 instances produced accuracy comparable to much larger random sets.

These results indicate that organizations can drastically reduce expert labeling labor and associated costs without sacrificing model accuracy or interpretability. Deploying an inexpensive, fast model to curate training data enables the practical construction of transparent, rule-based systems directly compatible with standard database query environments. While heterogeneous sampling incurs a slight theoretical accuracy penalty compared to using identical models throughout, the computational and labor savings heavily outweigh this trade-off.

Organizations implementing text classification on large unlabeled corpora should adopt heterogeneous sampling pipelines and implement error-cost weighting to balance class skews. Practitioners should begin with a small seed of known positive examples to jump-start the sampling loop. Future work should focus on establishing stopping rules to detect when additional sampling ceases to improve accuracy and on refining methods to handle inherently noisy or borderline instances where label uncertainty is high.

AT&T Bell Laboratoriesclass labelsgenetic selectionrule inductionheterogeneous uncertainty samplingsupervised learningprobabilistic classifiers
Lewis et al (1994).pdf
  • Paper: A sequential algorithm for training text classifiers, David D. Lewis et al. (1994). This seminal paper introduces the fundamental concept of uncertainty sampling for supervised text classification that the source paper directly builds upon and extends to heterogeneous models.
  • Paper: Query by committee, H. Seung et al. (1992). This foundational work establishes the theoretical framework for query-based active learning and selective data sampling to maximize informativeness.
  • Paper: Improving Generalization with Active Learning, David Cohn et al. (1994). It provides the core formulation of selective sampling within regions of model uncertainty, serving as essential conceptual groundwork for uncertainty-driven training algorithms.
Cover for Heterogeneous Uncertainty Sampling for Supervised Learning

Abstract

Uncertainty sampling methods iteratively request class labels for training instances whose classes are uncertain despite the previous labeled instances. These methods can greatly reduce the number of instances that an expert need label. One problem with this approach is that the classifier best suited for an application may be too expensive to train or use during the selection of instances. We test the use of one classifier (a highly efficient probabilistic one) to select examples for training another (the C4.5 rule induction program). Despite being chosen by this heterogeneous approach, the uncertainty samples yielded classifiers with lower error rates than random samples ten times larger.

Topics

class labelsgenetic selectionrule inductionsupervised machine learningsvmunsupervised learning

Keywords

heterogeneous uncertainty samplingsupervised learningprobabilistic classifiersdecision treestext categorizationfeature selectionneural networksunlabeled instances

Table of Contents

  • 1 Introduction
  • 2 Background
  • 3 Heterogeneous Uncertainty Sampling
  • 4 Task and Data Set
  • 5 Training C4.5 with Text Data
  • 5.1 Uncertainty Sampling with a Probabilistic Classifier
  • 5.2 Initial Classifier
  • 5.3 Feature Selection
  • 6 Experiment Design
  • 7 Results
  • 8 Discussion
  • 9 Future Work
  • 10 Summary
  • Acknowledgements
  • References

Knowls

  1. Knowl 1 — Heterogeneous Uncertainty Sampling Framework

    model/method

    Heterogeneous uncertainty sampling is an active learning methodology in which an efficient, computationally inexpensive classifier operates inside the sequential instance selection loop, while a different, more expressive, or more interpretable induction algorithm is trained on the resulting dataset.

    In standard uncertainty sampling from large unlabeled datasets, a classifier is iteratively rebuilt on labeled data and evaluated on large volumes of unlabeled instances to identify those whose class membership is most uncertain. When the desired final classifier (such as decision rule induction via C4.5) is too computationally demanding to evaluate across hundreds of thousands of high-dimensional instances at each active learning iteration, a lightweight model (such as a linear probabilistic classifier optimized for sparse data) performs the querying. Because the queried dataset disproportionately enriches low-frequency classes and boundary instances according to the selection model, the final target induction algorithm must incorporate a mechanism (such as a loss ratio or prior adjustment) to counterbalance sample distribution skew and prevent severe prediction bias.

  2. Knowl 2 — Cost-Sensitive Loss Ratio Adaptation for C4.5 Rule Induction

    model/method

    To train C4.5 decision rules on uncertainty samples that overrepresent low-frequency classes, C4.5 is modified to accept a loss ratio parameter LRLR, defined as the ratio of the cost of a false positive error to the cost of a false negative error:

    LR=Cost(False Positive)Cost(False Negative)LR = \frac{\text{Cost}(\text{False Positive})}{\text{Cost}(\text{False Negative})}

    The algorithm modifications are:

    1. Tree Splitting Check: The standard C4.5 check that replaces a split with a leaf if the split does not decrease raw error rate is disabled, preventing the premature halting of subtree construction for rare classes.
    2. Leaf Class Assignment: Instead of majority voting, a leaf is assigned the positive class if the estimated probability of class membership exceeds the threshold 1LR+1\frac{1}{LR + 1}.
    3. Pruning: Tree pruning and rule dropping minimize expected total loss rather than raw error counts, scaling false positive errors by LRLR.
    4. Default Class Selection: The default rule class is selected based on minimizing expected loss across uncovered instances.
    5. MDL Rule Sifting: In the Minimum Description Length (MDL) encoding used to sift and select final rule subsets, the coding cost of false positive errors is multiplied by LRLR (or false negatives by 1/LR1/LR).

  3. Knowl 3 — Probabilistic Uncertainty Sampling Algorithm for Sparse Data

    algorithm

    A fast linear probabilistic classifier is trained on sparse binary text representations and used to select the most uncertain instances in balanced batches. Let w=(w1,…,wd)\mathbf{w} = (w_1, \dots, w_d) denote a binary word occurrence vector, and C∈{0,1}C \in \{0, 1\} denote category membership. The class probability is estimated by:

    P(C∣w)=exp⁡(a+b∑i=1dlog⁡P(wi∣C)P(wi∣Cˉ))1+exp⁡(a+b∑i=1dlog⁡P(wi∣C)P(wi∣Cˉ))P(C \mid \mathbf{w}) = \frac{\exp\left(a + b \sum_{i=1}^d \log \frac{P(w_i \mid C)}{P(w_i \mid \bar{C})}\right)}{1 + \exp\left(a + b \sum_{i=1}^d \log \frac{P(w_i \mid C)}{P(w_i \mid \bar{C})}\right)}

    where Cˉ\bar{C} denotes category non-membership, and a,b∈Ra, b \in \mathbb{R} are scaling parameters fit via logistic regression.

    Input: Labeled seed set LL containing 3 positive instances, unlabeled pool UU, batch size k=4k=4, total iterations T=249T=249
    for iteration t=1t = 1 to TT:
        Estimate probabilities P(wi∣C)P(w_i \mid C), P(wi∣Cˉ)P(w_i \mid \bar{C}), and P(wi)P(w_i) on LL for all words wiw_i
        Select features wiw_i with largest values of P(wi)log⁡P(wi∣C)P(wi∣Cˉ)P(w_i) \log \frac{P(w_i \mid C)}{P(w_i \mid \bar{C})}
        Compute log-likelihood ratio sum ∑i=1dlog⁡P(wi∣C)P(wi∣Cˉ)\sum_{i=1}^d \log \frac{P(w_i \mid C)}{P(w_i \mid \bar{C})} for each instance in LL
        Fit logistic regression parameters aa and bb on LL
        Compute P(C∣w)P(C \mid \mathbf{w}) for all unlabeled instances w∈U\mathbf{w} \in U
        Select the 2 instances in UU with P(C∣w)≥0.5P(C \mid \mathbf{w}) \ge 0.5 closest to 0.50.5
        Select the 2 instances in UU with P(C∣w)<0.5P(C \mid \mathbf{w}) < 0.5 closest to 0.50.5
        Query expert for labels of the 4 selected instances
        Move the 4 instances from UU to LL
    Output: Labeled training dataset LL of size 3+4×249=9993 + 4 \times 249 = 999

  4. Knowl 4 — Text Categorization Experimental Setup on AP Newswire

    experimental setup

    The experimental testbed evaluates heterogeneous uncertainty sampling on text categorization using 371,454 Associated Press (AP) newswire article titles from 1988 to early 1993, partitioned into a training pool of 319,463 titles and a test set of 51,991 titles. Titles are represented as binary word occurrence vectors across 67,331 distinct vocabulary attributes, with an average of 8.9 non-zero features per title.

    Ten binary categories based on AP keyword slug lines with low frequencies are evaluated: tickertalk (0.07%), boxoffice (0.10%), bonds (0.15%), nielsens (0.16%), burma (0.16%), dukakis (0.20%), ireland (0.24%), quayle (0.25%), budget (0.37%), and hostages (0.49%).

    Each uncertainty sampling run is initialized with 3 randomly selected positive instances to avoid early pure-negative sampling. For C4.5 training, feature reduction selects the union of: (1) words occurring in ≥0.2%\ge 0.2\% of all instances, (2) words occurring in ≥2\ge 2 positive instances in the current training set, and (3) words occurring in ≥1\ge 1 of the 3 seed instances. Baseline models are trained on random samples of sizes 1,000 and 10,000 formed by appending random documents to the same 3 seed instances. Ten independent trials are conducted per category.

  5. Knowl 5 — Empirical Sample Efficiency of Heterogeneous Uncertainty Sampling

    empirical result

    Across nine text categorization categories with positive frequencies below 0.5%, C4.5 decision rules trained on uncertainty samples of 999 instances (selected using the fast probabilistic model) achieve error rates comparable to or better than C4.5 trained on random samples of 10,000 instances, provided an adjusted loss ratio (LR≥3LR \ge 3) is used.

    When C4.5 with LR=5LR = 5 on uncertainty samples of size 999 is compared to unmodified C4.5 (LR=1LR = 1) on random samples of size 10,000, the uncertainty samples yield a statistically significant reduction in error rate (p=0.03p = 0.03 by paired tt-test across 10 runs per category). This represents a ten-fold reduction in the volume of expert annotations required to achieve equivalent or superior classification accuracy.

  6. Knowl 6 — Sensitivity and Robustness of Rule Accuracy to the Loss Ratio Parameter

    empirical result

    When C4.5 is trained on uncertainty samples without cost adjustments (LR=1LR = 1), the resulting rule sets exhibit severely degraded accuracy due to excessive false positives, caused by the artificial enrichment of positive instances in the active learning sample.

    However, the classification error rate of the induced rules is highly robust to the exact choice of loss ratio across the interval LR∈[3,20]LR \in [3, 20]. For all tested categories, test error rates remain stable across this range and consistently outperform or match the accuracy of random samples of 1,000 and 10,000 instances.

  7. Knowl 7 — Model Expressiveness and Heterogeneity Penalty

    empirical result

    On random training samples of 10,000 text instances, C4.5 decision rules (LR=1LR=1) achieve significantly lower error rates than the linear probabilistic classifier (p=0.01p = 0.01 by paired tt-test), demonstrating that C4.5 is a more accurate inductive model class for this text categorization domain.

    However, on 999-instance uncertainty samples generated by the probabilistic classifier, C4.5 (LR=5LR=5) outperforms the probabilistic classifier (LR=1LR=1) by an insignificant margin (p=0.30p = 0.30). This reduction in relative advantage indicates a modest accuracy penalty caused by heterogeneity: the sample selected by the probabilistic model is tailored to the decision boundaries of linear models rather than tree/rule partitions.

  8. Knowl 8 — Classification Error Rates and False Positive/Negative Distributions Across Categories

    data/table

    The following table compares test set percentage error rates (mean and standard deviation) and average counts of false positives (FP) and false negatives (FN) across 51,991 test instances for five conditions:

    1. Reject All: Baseline classifying all instances as negative.
    2. Uncertainty 999 C4.5 (LR=5LR=5): Modified C4.5 on 999 uncertainty instances (10 runs).
    3. Uncertainty 999 Prob (LR=1LR=1): Probabilistic model on 999 uncertainty instances (10 runs).
    4. Random 10,000 C4.5 (LR=1LR=1): Unmodified C4.5 on 10,000 random instances (10 runs).
    5. Random 10,000 Prob (LR=1LR=1): Probabilistic model on 10,000 random instances (20 runs).
    Category Reject All Uncert 999: C4.5 (LR=5LR=5) Uncert 999: Prob (LR=1LR=1) Rand 10k: C4.5 (LR=1LR=1) Rand 10k: Prob (LR=1LR=1)
    Error % Error % (SD) FP / FN Error % (SD) FP / FN Error % (SD) FP / FN Error % (SD) FP / FN
    tickertalk 0.077 0.077 (0.000) 0.0 / 40.0 0.078 (0.001) 1.3 / 39.3 0.078 (0.003) 0.8 / 39.7 0.109 (0.044) 18.3 / 38.5
    boxoffice 0.081 0.047 (0.002) 5.5 / 19.0 0.048 (0.008) 12.6 / 12.6 0.061 (0.018) 5.0 / 26.8 0.077 (0.021) 10.8 / 29.3
    bonds 0.115 0.064 (0.002) 3.6 / 29.8 0.069 (0.006) 7.9 / 28.3 0.076 (0.020) 4.7 / 34.9 0.145 (0.069) 33.6 / 41.9
    nielsens 0.167 0.094 (0.011) 6.0 / 42.8 0.062 (0.005) 9.9 / 22.2 0.107 (0.006) 11.5 / 44.0 0.100 (0.026) 10.6 / 41.4
    burma 0.179 0.090 (0.008) 3.0 / 43.9 0.098 (0.006) 6.0 / 44.8 0.115 (0.040) 5.0 / 54.6 0.193 (0.046) 14.1 / 86.6
    dukakis 0.206 0.197 (0.014) 14.4 / 88.0 0.208 (0.020) 9.5 / 98.5 0.210 (0.039) 68.8 / 40.1 0.235 (0.036) 21.0 / 101.1
    ireland 0.225 0.188 (0.005) 4.8 / 93.1 0.189 (0.011) 16.2 / 81.9 0.220 (0.024) 12.4 / 101.8 0.228 (0.016) 13.8 / 104.7
    quayle 0.256 0.161 (0.009) 23.3 / 60.2 0.222 (0.012) 19.0 / 96.6 0.143 (0.010) 42.3 / 32.1 0.263 (0.035) 17.2 / 119.4
    budget 0.379 0.336 (0.010) 10.6 / 164.2 0.361 (0.009) 29.0 / 158.5 0.350 (0.014) 57.1 / 124.7 0.392 (0.016) 25.7 / 177.9
    hostages 0.439 0.415 (0.024) 30.1 / 185.6 0.360 (0.016) 44.7 / 142.6 0.466 (0.039) 78.3 / 164.3 0.431 (0.018) 25.3 / 199.0

    The data shows that C4.5 trained on 999 uncertainty instances achieves lower error rates than when trained on 10,000 random instances across 8 of 10 categories, while providing a substantially more balanced distribution of false positive and false negative errors than the non-learning Reject All baseline.

  9. Knowl 9 — Uncertainty Sampling Pathology Under Inherent Label Noise

    limitation

    Uncertainty sampling heuristics assume that posterior probabilities near P(C∣w)≈0.5P(C \mid \mathbf{w}) \approx 0.5 indicate lack of training evidence in the version space. However, when category labels are stochastic or contain human annotation noise, true underlying class probabilities may naturally be near 0.5.

    In such settings, uncertainty sampling iteratively queries these inherently ambiguous ("murky") instances in later iterations rather than informative boundary instances. Because these instances cannot resolve classifier uncertainty, active learning efficiency degrades unless the selection mechanism accounts for the variance of probability estimates or frames querying as regression/interpolation rather than boundary classification.

Coverage note — None was omitted; all key methods, algorithm specifications, experimental conditions, empirical findings, comparative tables, and analytical limitations were extracted.

References

  1. 1.Dana Angluin. Queries and concept learning. Machine Learning, 2:319–342, 1988.
  2. 2.I. Bratko, I. Mozetic, and N. Lavrac. KARDIO: a study in deep and qualitative knowledge for expert systems. MIT Press, Cambridge, Massachusetts, 1989.
  3. 3.Leo Breiman, Jerome H. Friedman, Richard A. Olshen, and Charles J. Stone. Classification and Regression Trees. Wadsworth, Belmont, CA, 1984.
  4. 4.J. Catlett. Megainduction: a test flight. In Machine Learning: Proceedings of the Eigth International Workshop, pages 596–599, San Mateo, CA, 1991. Morgan Kaufmann.
  5. 5.William G. Cochran. Sampling Techniques. John Wiley & Sons, New York, 3rd edition, 1977.
  6. 6.David Cohn, Les Atlas, and Richard Ladner. Improving generalization with self-directed learning, 1992. To appear in Machine Learning.
  7. 7.Stuart L. Crawford, Robert M. Fung, Lee A. Appelbaum, and Richard M. Tong. Classification trees for information retrieval. In Eighth International Workshop on Machine Learning, pages 245–249, 1991.
  8. 8.Daniel T. Davis and Jenq-Neng Hwang. Attentional focus training by boundary region data selection. In International Joint Conference on Neural Networks, pages I–676 to I–681, Baltimore, MD, June 7–11 1992.
  9. 9.James P. Egan. Signal Detection Theory and ROC Analysis. Academic Press, New York, 1975.
  10. 10.Y. Freund, H. S. Seung, E. Shamir, and N. Tishby. Information, prediction, and query by committee. In Advances in Neural Information Processing Systems 5, San Mateo, CA, 1992. Morgan Kaufmann.
  11. 11.William A. Gale, Kenneth W. Church, and David Yarowsky. A method for disambiguating word senses in a large corpus. Computers and the Humanities, 26:415–439, 1993.
  12. 12.B. K. Ghosh. A brief history of sequential analysis. In B. K. Ghosh and P. K. Sen, editors, Handbook of Sequential Analysis, chapter 1, pages 1–19. Marcel Dekker, New York, 1991.
  13. 13.Norm Goldstein, editor. The Associated Press Stylebook and Libel Manual. Addison-Wesley, Reading, MA, 1992.
  14. 14.Donna Harman. Ranking algorithms. In William B. Frakes and Ricardo Baeza-Yates, editors, Information Retrieval: Data Structures and Algorithms, pages 363–392. Prentice Hall, Englewood Cliffs, NJ, 1992.
  15. 15.Peter E. Hart. The condensed nearest neighbor rule. IEEE Transactions on Information Theory, IT-14:515–516, May 1968. Reprinted in Agrawala, Machine Recognition of Patterns, IEEE Press, New York, 1977.
  16. 16.Jenq-Neng Hwang, Jai J. Choi, Seho Oh, and Robert J. Marks II. Query-based learning applied to partially trained multilayer perceptrons. IEEE Transactions on Neural Networks, 2(1):131–136, January 1991.
  17. 17.Igor Kononerko, Ivan Bratko, and Esidija Roskar. Experiments in automatic learning of medical diagnostic rules. Technical report, Jozef Stefan Institute, Ljubljana, Slovenia, 1984.
  18. 18.David D. Lewis and William A. Gale. Training text classifiers by uncertainty sampling. In Seventeenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1994. To appear.
  19. 19.David D. Lewis and Philip J. Hayes. Editorial. ACM Transactions on Information Systems. Special Issue on Text Categorization, 1994. To appear.
  20. 20.David J. C. MacKay. The evidence framework applied to classification networks. Neural Computation, 4:720–736, 1992.
  21. 21.David J. C. MacKay. Information-based objective functions for active data selection. Neural Computation, 4(4):589–603, 1992.
  22. 22.Michel Manago. Knowledge intensive induction. In Machine Learning: Proceedings of the Sixth International Workshop, pages 151–155, 1989.
  23. 23.P. McCullagh and J. A. Nelder. Generalized Linear Models. Chapman & Hall, London, 2nd edition, 1989.
  24. 24.Tom M. Mitchell. Generalization as search. Artificial Intelligence, 18:203–226, 1982.
  25. 25.Mark Plutowski and Halbert White. Selecting concise training sets from clean data. IEEE Transactions on Neural Networks, 4(2):305–318, March 1993.
  26. 26.J. R. Quinlan. Discovering rules by induction from large collections of examples. In Expert systems in the micro-electronic age, Edinburgh, UK, 1979. Edinburgh University Press.
  27. 27.J. Ross Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA, 1993.
  28. 28.J.R. Quinlan. Decision trees as probabilistic classifiers. In Proceedings of the Fourth International Workshop on Machine Learning, pages 31–37, Irvine, California, 1987.
  29. 29.Gerard Salton. Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley, Reading, MA, 1989.
  30. 30.Claude Sammut, Scott Hurst, Dana Kedzier, and Donald Michie. Learning to fly. In Ninth International Workshop on Machine Learning, pages 385–393, 1992.
  31. 31.H. S. Seung, M. Opper, and H. Sompolinsky. Query by committee. In Proceedings of the Fifth Annual ACM Workshop on Computational Learning Theory, pages 287–294, 1992.
  32. 32.Bikas Kumar Sinha. Sequential methods for finite populations. In B. K. Ghosh and P. K. Sen, editors, Handbook of Sequential Analysis, chapter 1, pages 1–19. Marcel Dekker, New York, 1991.
  33. 33.Paul E. Utgoff. Improved training via incremental learning. In Sixth International Workshop on Machine Learning, pages 362–365, 1989.
  34. 34.Sholom M. Weiss, Robert S. Galen, and Prasad V. Tadepalli. Maximizing the predictive value of production rules. Artificial Intelligence, 45(1–2):47–71, September 1990.
  35. 35.P. H. Winston. Learning structural descriptions from examples. In P. H. Winston, editor, The Psychology of Computer Vision, pages 157–209. McGraw-Hill, New York, 1975.
  36. 36.J. Wirth and J. Catlett. Costs and benefits of windowing in ID3. In Proceedings of the Fifth International Conference on Machine Learning, Ann Arbor, Michigan, 1988. Morgan Kaufmann.

Citation

MLA
Lewis, D. D., and J. Catlett. “Heterogeneous Uncertainty Sampling for Supervised Learning”. Machine Learning Proceedings 1994, Elsevier, 1994, pp. 148–56, https://doi.org/10.1016/b978-1-55860-335-6.50026-x.
APA
Lewis, D. D., & Catlett, J. (1994). Heterogeneous Uncertainty Sampling for Supervised Learning. In Machine Learning Proceedings 1994 (pp. 148–156). Elsevier. https://doi.org/10.1016/b978-1-55860-335-6.50026-x
Chicago
Lewis, D. D., and J. Catlett. 1994. “Heterogeneous Uncertainty Sampling for Supervised Learning”. In Machine Learning Proceedings 1994. Elsevier. https://doi.org/10.1016/b978-1-55860-335-6.50026-x.
Harvard
Lewis, D.D. and Catlett, J. (1994) “Heterogeneous Uncertainty Sampling for Supervised Learning”, Machine Learning Proceedings 1994. Elsevier, pp. 148–156. Available at: https://doi.org/10.1016/b978-1-55860-335-6.50026-x.
Vancouver
1. Lewis DD, Catlett J (1994) Heterogeneous Uncertainty Sampling for Supervised Learning. In: Machine Learning Proceedings 1994. Elsevier, pp 148–156

BibTeX

@inbook{Lewis_1994, title={Heterogeneous Uncertainty Sampling for Supervised Learning}, ISBN={9781558603356}, url={http://dx.doi.org/10.1016/b978-1-55860-335-6.50026-x}, DOI={10.1016/b978-1-55860-335-6.50026-x}, booktitle={Machine Learning Proceedings 1994}, publisher={Elsevier}, author={Lewis, David D. and Catlett, Jason}, year={1994}, pages={148–156} }
Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF