BoosTexter: A Boosting-based System for Text Categorization

article2000Machine-mediated learning2,433 citations

Develops a boosting-based multiclass text categorization system and shows its effectiveness across text, speech, and automatic call-type identification tasks.

Listen

Organizations face growing operational challenges in accurately categorizing high volumes of unstructured digital text and spoken customer interactions. Real-world categorization tasks are frequently multiclass and multi-label, meaning documents can belong to several categories at once. However, many conventional machine learning methods struggle with multi-label structures or require cumbersome decomposition into separate binary systems, resulting in lower classification performance and high computational complexity.

The article evaluates a system called BoosTexter, which implements an improved family of boosting algorithms specifically engineered to handle multiclass, multi-label text and speech categorization tasks. Boosting sequentially combines simple, moderately accurate rules (called weak hypotheses, which test for the presence or absence of terms) into a single, highly accurate classification model that can both assign labels and rank them by relevance.

To demonstrate performance, the system was evaluated across multiple large-scale standard benchmarks: several subsets of the Reuters-21450 news collection (including a 93-class partition), over 200,000 headlines from the Associated Press Titles corpus, 20,000 Usenet newsgroup postings, and an 8,000-utterance customer service spoken call routing dataset. The evaluation assessed error rates, ranking quality (coverage and average precision), and operational training speed across four boosting variants against prominent alternatives, including RIPPER, Rocchio, Sleeping-experts, Naive-Bayes, nearest-neighbor, and linear least-squares methods.

The analysis produced several critical findings. First, BoosTexter with real-valued confidence predictions (real AdaBoost.MH) consistently outperformed traditional classification baselines across almost all text benchmarks, cutting classification error by nearly 50% compared to the best competitors on benchmark datasets. Second, allowing continuous confidence-rated predictions rather than simple binary choices dramatically accelerated convergence, achieving a more than 200-fold training speedup on large AP datasets (reaching the target error in 642 rounds versus 180,000 rounds). Third, BoosTexter reached state-of-the-art accuracy on the 93-class Reuters benchmark, achieving an 11-point average precision of 0.934 and outperforming nearest-neighbor and linear classifiers without exhibiting overfitting. Fourth, on automated speech categorization for customer call routing, BoosTexter matched or exceeded specialized grammar-based models, yielding near-perfect classification accuracy on roughly 40% of calls when rejecting low-confidence predictions.

These findings demonstrate that boosting simple term-matching rules is an effective, production-grade strategy for automated document routing and speech triage. The ability of the algorithm to resist overfitting even after thousands of training rounds reduces operational risk and deployment tuning. Furthermore, its reliable confidence scoring enables practical hybrid workflows where high-confidence items are automated directly and ambiguous cases are routed to human operators.

For operational implementation, organizations should adopt the real-valued AdaBoost.MH framework for large-scale text sorting and customer intent routing, while establishing confidence thresholds to manage automated processing versus human review. When training data is constrained (fewer than 1,000 examples per category), practitioners should prefer probabilistic models or collect more data, as boosting requires substantial training volume to overcome its lack of built-in domain priors. For speech applications, training models directly on output generated by automatic speech recognizers—rather than manual human transcripts—is recommended to prevent mismatches between training and runtime environments.

AT&T Labs—Research The Hebrew University of Jerusalem boosting document categorization document classification Text Categorization AdaBoost Weak

SCHAPIRE et al (2000).pdf

Paper: Experiments with a New Boosting Algorithm, Yoav Freund et al. (1996). This foundational paper introduces AdaBoost, the core algorithmic framework that the source adapts and extends for multiclass text categorization in BoosTexter.
Paper: A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting, Yoav Freund et al. (1997). Reading this paper provides the essential theoretical foundations of online learning and boosting bounds upon which the BoosTexter architecture is built.

Paper: Greedy function approximation: A gradient boosting machine, Jerome H. Friedman (2001). This seminal work generalizes boosting into a broader gradient descent framework in function space, directly extending the boosting approach explored in BoosTexter.
Paper: XGBoost: A Scalable Tree Boosting System, Tianqi Chen et al. (2016). This system-level advancement builds directly upon earlier text and tree boosting foundations like BoosTexter to deliver highly scalable gradient tree boosting.

Cover for BoosTexter: A Boosting-based System for Text Categorization

Abstract

This work focuses on algorithms which learn from examples to perform multiclass text and speech categorization tasks. Our approach is based on a new and improved family of boosting algorithms. We describe in detail an implementation, called BoosTexter, of the new boosting algorithms for text categorization tasks. We present results comparing the performance of BoosTexter and a number of other text-categorization algorithms on a variety of tasks. We conclude by describing the application of our system to automatic call-type identification from unconstrained spoken customer responses.

Topics

boosting document categorization document classification multi-class classification speech signals

Keywords

Text Categorization AdaBoost Weak Algorithms Labels Boosting-based System Set Training Boosting Classi

1. Introduction
2. Preliminaries
3. Boosting algorithms for multi-label multiclass problems
3.1. AdaBoost.MH
3.2. AdaBoost.MR
4. Weak hypotheses for text categorization
4.1. AdaBoost.MH with real-valued predictions
4.2. AdaBoost.MH with real-valued predictions and abstaining
4.3. AdaBoost.MH with discrete predictions
4.4. AdaBoost.MR with discrete predictions
5. Evaluation measures
6. Text categorization experiments
6.1. Test corpora
6.2. Other algorithms
6.3. Experiments using single-label corpora
6.4. Experiments using multi-label corpora
6.5. An experiment with a large number of classes
7. Speech categorization experiments
Appendix A: Description of text datasets and summary of results
Acknowledgments
Notes
References

Knowls

Knowl 1 — AdaBoost.MH Algorithm for Multi-Label Categorization
algorithm
Let $\mathcal{X}$ denote the document domain and $\mathcal{Y} = \{1, \dots, k\}$ denote a finite set of labels. In multi-label classification, each instance $x_i \in \mathcal{X}$ is associated with a subset $Y_i \subseteq \mathcal{Y}$ , represented by binary indicators $Y_i[\ell] = +1$ if $\ell \in Y_i$ and $Y_i[\ell] = -1$ if $\ell \notin Y_i$ .

AdaBoost.MH reduces multi-label learning to binary classification over document-label pairs $(i, \ell) \in \{1, \dots, m\} \times \mathcal{Y}$ , explicitly optimizing the empirical Hamming loss.
Input: Training examples $((x_1, Y_1), \dots, (x_m, Y_m))$ where $x_i \in \mathcal{X}$ , $Y_i \subseteq \mathcal{Y}$
Input: Number of boosting rounds $T$
Initialize $D_1(i, \ell) = \frac{1}{mk}$ for all $i \in \{1, \dots, m\}$ and $\ell \in \mathcal{Y}$
for $t = 1$ to $T$ do
    Pass distribution $D_t$ to weak learner
    Get weak hypothesis $h_t: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$
    Choose $\alpha_t \in \mathbb{R}$
    Compute normalization factor $Z_t = \sum_{i=1}^m \sum_{\ell \in \mathcal{Y}} D_t(i, \ell) \exp(-\alpha_t Y_i[\ell] h_t(x_i, \ell))$
    for each $i \in \{1, \dots, m\}$ and $\ell \in \mathcal{Y}$ do
         $D_{t+1}(i, \ell) = \frac{D_t(i, \ell) \exp(-\alpha_t Y_i[\ell] h_t(x_i, \ell))}{Z_t}$
    end for
end for
Output: Final scoring function $f(x, \ell) = \sum_{t=1}^T \alpha_t h_t(x, \ell)$
The space and per-round time complexity of AdaBoost.MH (excluding the weak learner) is $O(mk)$ . The empirical Hamming loss of the final classifier $f$ is upper bounded by $\prod_{t=1}^T Z_t$ .
Knowl 2 — AdaBoost.MR Algorithm and Linear-Time Implementation
algorithm
In multi-label ranking, the objective is to order labels such that relevant labels $\ell_1 \in Y_i$ rank above irrelevant labels $\ell_0 \in \mathcal{Y} \setminus Y_i$ . A pair $(\ell_0, \ell_1) \in (\mathcal{Y} \setminus Y_i) \times Y_i$ is termed a crucial pair. AdaBoost.MR minimizes the empirical ranking loss:

$\frac{1}{m} \sum_{i=1}^m \frac{1}{|Y_i| |\mathcal{Y} \setminus Y_i|} |\{(\ell_0, \ell_1) \in (\mathcal{Y} \setminus Y_i) \times Y_i : f(x_i, \ell_1) \le f(x_i, \ell_0)\}|$

A naive implementation maintaining weights over crucial pairs requires $\Theta(mk^2)$ space and time per round. AdaBoost.MR maintains instead factorized weights $v_t(i, \ell)$ such that $D_t(i, \ell_0, \ell_1) = v_t(i, \ell_0) \cdot v_t(i, \ell_1)$ , reducing space and per-round time complexity to $O(mk)$ .
Input: Training examples $((x_1, Y_1), \dots, (x_m, Y_m))$ where $x_i \in \mathcal{X}$ , $Y_i \subseteq \mathcal{Y}$ with $0 < |Y_i| < k$
Input: Number of boosting rounds $T$
Initialize $v_1(i, \ell) = (m \cdot |Y_i| \cdot |\mathcal{Y} \setminus Y_i|)^{-1/2}$ for all $i \in \{1, \dots, m\}$ and $\ell \in \mathcal{Y}$
for $t = 1$ to $T$ do
    Train weak learner using pair distribution $D_t(i, \ell_0, \ell_1) = v_t(i, \ell_0) v_t(i, \ell_1)$ for $\ell_0 \notin Y_i, \ell_1 \in Y_i$
    Get weak hypothesis $h_t: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$
    Choose $\alpha_t \in \mathbb{R}$
    Compute $Z_t = \sum_{i=1}^m \left[ \left( \sum_{\ell \notin Y_i} v_t(i, \ell) \exp(\frac{1}{2} \alpha_t h_t(x_i, \ell)) \right) \left( \sum_{\ell \in Y_i} v_t(i, \ell) \exp(-\frac{1}{2} \alpha_t h_t(x_i, \ell)) \right) \right]$
    for each $i \in \{1, \dots, m\}$ and $\ell \in \mathcal{Y}$ do
         $v_{t+1}(i, \ell) = \frac{v_t(i, \ell) \exp(-\frac{1}{2} \alpha_t Y_i[\ell] h_t(x_i, \ell))}{\sqrt{Z_t}}$
    end for
end for
Output: Final ranking function $f(x, \ell) = \sum_{t=1}^T \alpha_t h_t(x, \ell)$
The empirical ranking loss of $f$ is upper bounded by $\prod_{t=1}^T Z_t$ .
Knowl 3 — Decision Stump Weak Hypotheses for Multi-Label Text Categorization
model/method
In BoosTexter, base hypotheses $h: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$ are formulated as one-level decision trees (decision stumps) testing the presence or absence of a term $w$ (a word, word bigram, or sparse n-gram) in a document $x$ :

$h(x, \ell) = \begin{cases} c_{0\ell} & \text{if } w \notin x \\ c_{1\ell} & \text{if } w \in x \end{cases}$

where $c_{j\ell} \in \mathbb{R}$ is the prediction score for label $\ell \in \mathcal{Y}$ on partition $X_j$ (with $X_0 = \{x : w \notin x\}$ and $X_1 = \{x : w \in x\}$ ). The sign $\operatorname{sign}(c_{j\ell})$ predicts label membership ( $+1$ for relevant, $-1$ for irrelevant), while $|c_{j\ell}|$ represents prediction confidence.

To evaluate candidate weak hypotheses efficiently across large corpora:
1. An inverted index maps each term $w$ to the set of documents containing it.
2. On each round, partition totals $W_{j\ell} = \sum_{i: x_i \in X_j} D_t(i, \ell)$ are precomputed.
3. The learner iterates through terms in the inverted index, calculating $W_{1,+}^\ell = \sum_{i: x_i \in X_1, Y_i[\ell]=+1} D_t(i, \ell)$ and $W_{1,-}^\ell = W_{1\ell} - W_{1,+}^\ell$ .
4. Prediction scores $c_{j\ell}$ and the normalization score $Z_t$ are evaluated. The term minimizing $Z_t$ (or maximizing correlation $r_t$ ) is selected.
The time complexity per round to evaluate all candidate terms is linear in the total number of term occurrences across the training corpus.
Knowl 4 — Real and Real-Abstaining Weak Learners for AdaBoost.MH
model/method
For AdaBoost.MH, the optimal real-valued predictions $c_{j\ell}$ and terms $w$ are derived by directly minimizing the normalization factor $Z_t$ . Let $X_0 = \{x : w \notin x\}$ and $X_1 = \{x : w \in x\}$ , and define for $j \in \{0, 1\}$ and $b \in \{-1, +1\}$ :

$W_{j,+}^\ell = \sum_{i: x_i \in X_j, Y_i[\ell] = +1} D_t(i, \ell), \quad W_{j,-}^\ell = \sum_{i: x_i \in X_j, Y_i[\ell] = -1} D_t(i, \ell)$

With step size parameter $\alpha_t = 1$ :
1. Real AdaBoost.MH: Unconstrained real predictions are smoothed with $\epsilon = \frac{1}{mk}$ : $c_{j\ell} = \frac{1}{2} \ln\left(\frac{W_{j,+}^\ell + \epsilon}{W_{j,-}^\ell + \epsilon}\right)$ The term $w$ is chosen to minimize the exact score: $Z_t = 2 \sum_{j \in \{0, 1\}} \sum_{\ell \in \mathcal{Y}} \sqrt{W_{j,+}^\ell W_{j,-}^\ell}$
2. Real Abstaining AdaBoost.MH: To avoid assuming that the absence of a term carries topical information, the weak hypothesis abstains when $w \notin x$ by setting $c_{0\ell} = 0$ , while setting $c_{1\ell} = \frac{1}{2} \ln\left(\frac{W_{1,+}^\ell + \epsilon}{W_{1,-}^\ell + \epsilon}\right)$ . Defining $W_0 = \sum_{i: x_i \in X_0} \sum_{\ell \in \mathcal{Y}} D_t(i, \ell)$ , the term $w$ is chosen to minimize: $Z_t = W_0 + 2 \sum_{\ell \in \mathcal{Y}} \sqrt{W_{1,+}^\ell W_{1,-}^\ell}$
The real abstaining learner is approximately 15% faster per round because $Z_t$ only requires processing documents that contain $w$ , while achieving predictive accuracy comparable to unrestricted Real AdaBoost.MH.
Knowl 5 — Discrete Weak Learners for AdaBoost.MH and AdaBoost.MR
model/method
When weak hypothesis predictions are constrained to discrete values $c_{j\ell} \in \{-1, +1\}$ :
1. Discrete AdaBoost.MH: For partition $X_j$ ( $j \in \{0, 1\}$ ) and label $\ell \in \mathcal{Y}$ : $c_{j\ell} = \operatorname{sign}\left(W_{j,+}^\ell - W_{j,-}^\ell\right)$ where $W_{j,+}^\ell = \sum_{i: x_i \in X_j, Y_i[\ell]=+1} D_t(i, \ell)$ and $W_{j,-}^\ell = \sum_{i: x_i \in X_j, Y_i[\ell]=-1} D_t(i, \ell)$ . The correlation $r_t$ and step size $\alpha_t$ are: $r_t = \sum_{j \in \{0, 1\}} \sum_{\ell \in \mathcal{Y}} \left|W_{j,+}^\ell - W_{j,-}^\ell\right|, \quad \alpha_t = \frac{1}{2} \ln\left(\frac{1 + r_t}{1 - r_t}\right)$ yielding $Z_t = \sqrt{1 - r_t^2}$ . The term $w$ maximizing $r_t$ is selected.
2. Discrete AdaBoost.MR: Using factorized distribution weights $v_t(i, \ell)$ , define: $d_t(i, \ell) = \frac{1}{2} v_t(i, \ell) \sum_{\ell': Y_i[\ell'] \neq Y_i[\ell]} v_t(i, \ell')$ Predictions and correlation parameter $r_t$ are: $c_{j\ell} = \operatorname{sign}\left( \sum_{i: x_i \in X_j} d_t(i, \ell) Y_i[\ell] \right), \quad r_t = \sum_{j \in \{0, 1\}} \sum_{\ell \in \mathcal{Y}} \left| \sum_{i: x_i \in X_j} d_t(i, \ell) Y_i[\ell] \right|$ Setting $\alpha_t = \frac{1}{2} \ln\left(\frac{1 + r_t}{1 - r_t}\right)$ guarantees $Z_t \le \sqrt{1 - r_t^2}$ . AdaBoost.MR chooses the term $w$ maximizing $r_t$ to minimize this upper bound on $Z_t$ .
Knowl 6 — Evaluation Measures for Multi-Label Categorization
definition
Let $f: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$ be a learned ranking function on a dataset $S = \langle (x_1, Y_1), \dots, (x_m, Y_m) \rangle$ , with $|\mathcal{Y}| = k$ . Let $\operatorname{rank}_f(x, \ell) \in \{1, \dots, k\}$ denote the rank of label $\ell$ ordered descendingly by $f(x, \ell)$ , such that $f(x, \ell_1) > f(x, \ell_2) \implies \operatorname{rank}_f(x, \ell_1) < \operatorname{rank}_f(x, \ell_2)$ . Let $[[\pi]]$ be 1 if predicate $\pi$ holds and 0 otherwise.
1. One-Error: The fraction of instances where the top-ranked label is not in the true label set $Y_i$ : $\text{one-err}_S(f) = \frac{1}{m} \sum_{i=1}^m \left[\left[ \arg\max_{\ell \in \mathcal{Y}} f(x_i, \ell) \notin Y_i \right]\right]$ For single-label classification problems ( $|Y_i| = 1$ ), one-error is identical to conventional classification error.
2. Coverage: The average depth down the ranked list needed to cover all labels in $Y_i$ , minus 1: $\text{coverage}_S(f) = \frac{1}{m} \sum_{i=1}^m \left( \max_{\ell \in Y_i} \operatorname{rank}_f(x_i, \ell) - 1 \right)$ For single-label tasks, coverage represents the average zero-indexed rank of the true label.
3. Non-Interpolated Average Precision: The average fraction of labels ranked above a relevant label that are also relevant: $\text{avgprec}_S(f) = \frac{1}{m} \sum_{i=1}^m \frac{1}{|Y_i|} \sum_{\ell \in Y_i} \frac{|\{\ell' \in Y_i : \operatorname{rank}_f(x_i, \ell') \le \operatorname{rank}_f(x_i, \ell)\}|}{\operatorname{rank}_f(x_i, \ell)}$ The score satisfies $\text{avgprec}_S(f) \in [0, 1]$ , reaching 1 if and only if all true labels in $Y_i$ are ranked strictly above all labels in $\mathcal{Y} \setminus Y_i$ for all instances.

Knowl 7 — Benchmark Categorization Results on 93-Topic ModApte Reuters-21450

data/table

On the standard 93-class ModApte partition of Reuters-21450 (7,789 training documents and 3,309 test documents, averaging 1.24 labels per document), real AdaBoost.MH trained for 10,000 rounds was evaluated against the leading algorithms reported in prior benchmark surveys: $k$ -nearest neighbors ( $k\text{NN}$ ) and Linear Least Squares Fit ( $\text{LLSF}$ ).

Algorithm	11-pt Avg. precision	$F_1$ (Data-adjusted threshold)	$F_1$ (Threshold = 0)	Micro-avg. BEP
AdaBoost.MH	0.934	0.853	0.851	0.86
kNN	0.924	0.852	–	0.85
LLSF	0.901	0.855	–	0.85

AdaBoost.MH achieved the top performance in 11-point interpolated average precision (0.934) and micro-averaged break-even point (0.86), and matched LLSF on $F_1$ score (0.853 vs 0.855). Training one-error reached near zero after approximately 1,000 rounds, while test error continued to decrease through 10,000 rounds without overfitting.

Knowl 8 — Empirical Superiority and Convergence Speed-up of Real vs Discrete Boosting
empirical result
Across single-label and multi-label document corpora (Reuters-21450, AP Titles, UseNet 20 Newsgroups), Real AdaBoost.MH achieves superior classification accuracy and orders-of-magnitude faster convergence compared to discrete boosting variants:
- On the 6-class AP Titles dataset (142,727 train, 66,973 test headlines), Discrete AdaBoost.MH required 180,000 rounds to reach 32.22% training error, whereas Real AdaBoost.MH reached that same error rate in 642 rounds—representing an acceleration factor exceeding 200x.
- On test data for this 6-class task, Real AdaBoost.MH at 30,000 rounds reached 27.43% error, outperforming Discrete AdaBoost.MH at 180,000 rounds (32.22%), Sleeping-experts (29.44%), Rocchio (40.14%), and RIPPER (53.29%).
- Discrete AdaBoost.MR consistently achieved worse one-error and ranking metrics than Discrete AdaBoost.MH, reflecting the penalty of optimizing an upper bound on $Z_t$ rather than exact minimization.
Knowl 9 — Sample Complexity Limitations of BoosTexter Relative to Probabilistic Classifiers
limitation
On the UseNet 20-newsgroups dataset evaluated under 3-fold cross-validation across training set sizes ranging from 200 to 12,977 articles:
- With small training sets ( $< 1,000$ documents), Real AdaBoost.MH performs significantly worse than Naive Bayes and probabilistic TF-IDF (e.g., at 200 training examples, one-error is $\approx 72\%$ for AdaBoost.MH versus $\approx 55\%$ for Naive Bayes and $\approx 60\%$ for probabilistic TF-IDF).
- This performance gap arises because AdaBoost.MH incorporates minimal prior domain knowledge, optimizing training Hamming loss directly, which yields high generalization error when data is scarce.
- Once the training set size reaches $\ge 10,000$ examples, the domain prior embedded in TF-IDF weights becomes less critical, and Real AdaBoost.MH outperforms both Naive Bayes and probabilistic TF-IDF across one-error, coverage, and average precision.
Knowl 10 — Margin-Based Confidence Scoring for Spoken Call-Type Classification
empirical result
BoosTexter was applied to an automated call-type classification task ("How may I help you?" task, 8,000 train and 1,000 test utterances across 14 call categories plus an 'other' class) using sparse word trigrams as features over 300 boosting rounds.

To construct rejection ROC curves (routing uncertain calls to human operators), prediction confidence was computed from the margin between top two predicted label scores:

$\operatorname{conf}(x) = f(x, \ell_1) - f(x, \ell_2)$

where $\operatorname{rank}_f(x, \ell_1) = 1$ and $\operatorname{rank}_f(x, \ell_2) = 2$ .

On human-transcribed test text, AdaBoost.MH outperformed the salient grammar fragment baseline at reject rates $\ge 40\%$ , achieving near 100% accuracy on non-rejected calls. On automatic speech recognition (ASR) outputs, both models experienced comparable performance degradation due to acoustic mismatch and speech transcription errors.

Coverage note — Detailed per-class and subset breakdowns (Tables A.1-A.11) across intermediate subsets of Reuters and AP Titles are summarized within the algorithmic comparisons, convergence findings, and 93-class benchmark results.

References

1.Apt´e, C., Damerau, F., & Weiss, S. M. (1994). Towards language independent automated learning of text categorization models. Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 23–30).
2.Biebricher, P., Fuhr, N., Lustig, G., Schwantner, M., & Knorz, G. (1988). The automatic indexing system AIR/PHYS—from research to application. Proceedings of the 11th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 333–342).
3.Blum, A. (1997). Empirical support for winnow and weighted-majority based algorithms: results on a calendar scheduling domain. Machine Learning, 26, 5–23.
4.Breiman, L. (1998). Arcing classifiers. The Annals of Statistics, 26(3), 801–849.
5.Cohen, W. (1995). Fast effective rule induction. Proceedings of the Twelfth International Conference on Machine Learning (pp. 115–123).
6.Cohen, W.W. & Singer, Y. (1996). Context-sensitive learning methods for text categorization. Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 307–315).
7.Drucker, H. & Cortes, C. (1996). Boosting decision trees. In Advances in Neural Information Processing Systems 8 (pp. 479–485).
8.Field, B.J. (1975). Towards automatic indexing: automatic assignment of controlled-language indexing and classification from free indexing. Journal of Documentation, 31(4), 246–265.
9.Freund, Y. & Schapire, R.E. (1996). Experiments with a new boosting algorithm. Machine Learning: Proceedings of the Thirteenth International Conference (pp. 148–156).
10.Freund, Y. & Schapire, R.E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119–139.
11.Freund, Y., Schapire, R.E., Singer, Y., & Warmuth, M.K. (1997). Using and combining predictors that specialize. Proceedings of the Twenty-Ninth Annual ACM Symposium on the Theory of Computing (pp. 334–343).
12.Fuhr, N. & Pfeifer, U. (1994). Probabilistic information retrieval as a combination of abstraction, inductive learning, and probabilistic assumptions. ACM Transactions on Information Systems, 12(1), 92–115.
13.Gorin, A.L., Parker, B.A., Sachs, R.M., & Wilpon, J.G. (1996). How may I help you?. Proceedings Interactive Voice Technology for Telecommunications Applications (IVTTA) (pp. 57–60).
14.Gorin, A.L., Riccardi, G., & Wright, J.H. (1997). How may I help you?. Speech Communication, 23(1-2), 113–127.
15.Ittner, D.J., Lewis, D.D., & Ahn, D.D. (1995). Text categorization of low quality images. Symposium on Document Analysis and Information Retrieval (pp. 301–315). Las Vegas, NV. ISRI; Univ. of Nevada, Las Vegas.
16.Joachims, T. (1997). A probabilistic analysis of the Rochhio algorithm with TFIDF for text categorization. Machine Learning: Proceedings of the Fourteenth International Conference (pp. 143–151).
17.Koller, D. & Sahami, M. (1997). Hierarchically classifying documents using very few words. Machine Learning: Proceedings of the Fourteenth International Conference (pp. 171–178).
18.Lang, K. (1995). Newsweeder: Learning to filter netnews. Proceedings of the Twelfth International Conference on Machine Learning (pp. 331–339).
19.Lewis, D. (1992). Representation and learning in information retrieval. Technical Report 91-93, Computer Science Department, University of Massachusetts at Amherst. Ph.D. Thesis.
20.Lewis, D. & Catlett, J. (1994). Heterogeneous uncertainty sampling for supervised learning. Machine Learning: Proceedings of the Eleventh International Conference.
21.Lewis, D. & Gale, W. (1994). Training text classifiers by uncertainty sampling. Seventeenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.
22.Lewis, D.D. & Ringuette, M. (1994). A comparison of two learning algorithms for text categorization. Third Annual Symposium on Document Analysis and Information Retrieval (pp. 81–93).
23.Maclin, R. & Opitz, D. (1997). An empirical evaluation of bagging and boosting. Proceedings of the Fourteenth National Conference on Artificial Intelligence (pp. 546–551).
24.Margineantu, D.D. & Dietterich, T.G. (1997). Pruning adaptive boosting. Machine Learning: Proceedings of the Fourteenth International Conference (pp. 211–218).
25.Mitchell, T.M. (1997). Machine learning. McGraw Hill.
26.Moulinier, I., Raˇskinis, G., & Ganascia, J.-G. (1996). Text categorization: a symbolic approach. Fifth Annual Symposium on Document Analysis and Information Retrieval (pp. 87–99).
27.Ng, H.T., Goh, W.B., & Low, K.L. (1997). Feature selection, perceptron learning, and a usability case study for text categorization. Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 67–73).
28.Quinlan, J.R. (1996). Bagging, boosting, and C4.5. Proceedings of the Thirteenth National Conference on Artificial Intelligence (pp. 725–730).
29.Riccardi, G., Gorin, A.L., Ljolje, A., & Riley, M. (1997). Spoken language understanding for automated call routing. Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (pp. 1143–1146).
30.Rocchio, J. (1971). Relevance feedback information retrieval. In G. Salton, (Ed.), The Smart retrieval system—experiments in automatic document processing (pp. 313–323). Englewood Cliffs, NJ: Prentice-Hall.
31.Salton, G. (1991). Developments in automatic text retrieval. Science, 253, 974–980.
32.Salton, G. & McGill, M.J. (1983). Introduction to modern information retrieval. McGraw-Hill.
33.Schapire, R.E. (1997). Using output codes to boost multiclass learning problems. Machine Learning: Proceedings of the Fourteenth International Conference (pp. 313–321).
34.Schapire, R.E., Freund, Y., Bartlett, P., & Lee, W.S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics. 26(5), 1651–1686.
35.Schapire, R.E. & Singer, Y. (1998). Improved boosting algorithms using confidence-rated predictions. Proceedings of the Eleventh Annual Conference on Computational Learning Theory (pp. 80–91). To appear, Machine Learning.
36.van Rijsbergen, C.J. (1979). Information retrieval. London: Butterworths.
37.Weiss, S., Apte, C., Damerau, F., Johnson, D., Oles, F., Goetz, T., & Hampp, T. (1999). Maximizing text-mining performance. IEEE Intelligent Systems.
38.Wright, J.H., Gorin, A.L., & Riccardi, G. (1997). Automatic acquisition of salient grammar fragments for calltype classification. Proceedings of the 5th European Conference on Speech Communication and Technology (pp. 1419–1422).
39.Yang, Y. (1994). Expert network: effective and efficient learning from human decisions in text categorization and retrieval. Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 13–22).
40.Yang, Y. (1999). An evaluation of statistical approaches to text categorization. Information Retrieval, 1, 69–90.

Citation

MLA

Schapire, R. E., and Y. Singer. “BoosTexter: A Boosting-based System for Text Categorization”. Machine Learning, vol. 39, nos. 2-3, 2000, pp. 135–68, https://doi.org/10.1023/A:1007649029923.

APA

Schapire, R. E., & Singer, Y. (2000). BoosTexter: A Boosting-based System for Text Categorization. Machine Learning, 39(2-3), 135–168. https://doi.org/10.1023/A:1007649029923

Chicago

Schapire, R. E., and Y. Singer. 2000. “BoosTexter: A Boosting-based System for Text Categorization”. Machine Learning 39 (2-3): 135–68. https://doi.org/10.1023/A:1007649029923.

Harvard

Schapire, R.E. and Singer, Y. (2000) “BoosTexter: A Boosting-based System for Text Categorization”, Machine Learning, 39(2-3), pp. 135–168. Available at: https://doi.org/10.1023/A:1007649029923.

Vancouver

1. Schapire RE, Singer Y (2000) BoosTexter: A Boosting-based System for Text Categorization. Machine Learning 39:135–168

BibTeX

@article{Schapire_2000, title={BoosTexter: A Boosting-based System for Text Categorization}, volume={39}, ISSN={1573-0565}, url={http://dx.doi.org/10.1023/A:1007649029923}, DOI={10.1023/a:1007649029923}, number={2-3}, journal={Machine Learning}, publisher={Springer Science and Business Media LLC}, author={Schapire, Robert E. and Singer, Yoram}, year={2000}, month=May, pages={135–168} }

Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

BoosTexter: A Boosting-based System for Text Categorization

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — AdaBoost.MH Algorithm for Multi-Label Categorization

Knowl 2 — AdaBoost.MR Algorithm and Linear-Time Implementation

Knowl 3 — Decision Stump Weak Hypotheses for Multi-Label Text Categorization

Knowl 4 — Real and Real-Abstaining Weak Learners for AdaBoost.MH

Knowl 5 — Discrete Weak Learners for AdaBoost.MH and AdaBoost.MR

Knowl 6 — Evaluation Measures for Multi-Label Categorization

Knowl 7 — Benchmark Categorization Results on 93-Topic ModApte Reuters-21450

Knowl 8 — Empirical Superiority and Convergence Speed-up of Real vs Discrete Boosting

Knowl 9 — Sample Complexity Limitations of BoosTexter Relative to Probabilistic Classifiers

Knowl 10 — Margin-Based Confidence Scoring for Spoken Call-Type Classification

References

Citation

BibTeX

Access the Paper