Decision Combination in Multiple Classifier Systems

Tin Kam HoJonathan J. HullSargur N. Srihari

article1994TPAMI1,757 citations

Presents foundational methods for combining disparate pattern classifiers through rank-based class set reduction and reranking techniques to significantly boost recognition accuracy in large-scale, noisy classification tasks.

Listen

Real-world automated pattern recognition tasks—such as processing degraded mail images, reading complex scripts, or interpreting medical scans—face significant hurdles when dealing with large sets of candidate categories and noisy inputs. Single classification algorithms often fail to achieve acceptable accuracy under difficult conditions, yet different algorithms offer complementary capabilities. Combining their outputs has historically been hindered by mismatched numerical confidence scores and high computational complexity when dealing with large numbers of categories.

The main objective of the article is to demonstrate and evaluate a general framework for combining multiple pattern recognition algorithms by converting raw outputs into ranked lists of candidate categories. The article specifically evaluates methods designed to reduce the size of the candidate category set and reorder candidate categories to place the correct decision at the top.

To test these methods, the authors conducted empirical experiments on degraded machine-printed word and character recognition tasks using live mail images from postal services. The word recognition experiment evaluated four algorithms across 1,365 word categories using a dataset of 4,624 training images and 1,384 test images. The character recognition experiment evaluated six distinct algorithms across 48 merged alphanumeric categories using 19,151 training samples, 8,000 model-estimation samples, and 12,000 test samples. The combination approaches included set reduction via intersection and union, as well as category reordering using highest-rank selection, Borda count aggregation, and logistic regression modeling.

The findings show that combining ranked algorithm outputs substantially outperforms individual classifiers. First, in the 1,365-class word recognition task, a dynamically selected logistic regression combination achieved a top-choice accuracy of 93.9%, outperforming the best individual classifier's accuracy of 84.9% by 9.0 percentage points and the static regression model's 90.7%. Second, in character recognition, combining four complementary classifiers via logistic regression increased top-choice accuracy from 85.8% for the best standalone classifier to 88.8%. Third, the statistical regression framework effectively identified redundant classifiers whose weights were near zero, allowing system simplification without compromising accuracy. Finally, the highest-rank and union methods reliably isolated true classes within a small candidate subset, achieving up to 98.9% cumulative accuracy within the top ten choices for word recognition.

These results indicate that organizations deploying complex recognition systems can significantly improve accuracy, reduce operational failure rates, and eliminate redundant computing processes without requiring standardized confidence scoring scales across legacy systems. By separating classifier correlation from classification correctness through regression modeling, systems become more robust against conflicting decisions. This proves that high-level rank information is sufficient to achieve substantial gains even in high-dimensional classification problems.

Decision-makers should consider adopting rank-based combination and dynamic model selection when deploying large-scale pattern recognition systems. For practical implementations, engineering teams should first filter out redundant classifiers using statistical screening, apply set reduction to isolate small candidate pools, and use dynamic weighting based on classifier agreement levels. When near-term decisions involve complex multi-stage architectures, teams should conduct domain-specific pilot testing to establish the optimal balance between computational speed and classification accuracy.

The findings carry high confidence due to extensive testing on large, noisy real-world postal datasets, though certain boundary conditions apply. The linear assumptions in basic logistic regression can degrade if long rank tails are modeled without truncation, and dynamic selection relies on easily computable agreement rules specific to the chosen problem domain. Readers should validate classifier error correlations in their specific operational environments before finalizing system architectures.

AT&T Bell Laboratories Center of Excellence for Document Analysis and Recognition (CEDAR)State University of New York feature descriptors logistic regression logistics multiple classifier systems decision combination classifier combination

Ho et al (1994).pdf

Paper: Neural Network Ensembles, Lars Kai Hansen et al. (1990). Establishes foundational consensus voting models and error reduction principles for combining multiple classification models that precede and motivate multi-classifier system design.
Paper: Adaptive Mixtures of Local Experts, Robert A. Jacobs et al. (1991). Introduces the mixture-of-experts architecture for combining outputs from specialized sub-networks, providing core background on modular multi-classifier fusion.

Paper: On Combining Classifiers, Josef Kittler et al. (1998). Develops a unified Bayesian theoretical framework for classifier combination rules that generalizes and formalizes rank- and score-based multi-classifier fusion.
Paper: Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy, Ludmila I. Kuncheva et al. (2003). Analyzes the formal measures of diversity among ensemble members that underpin the performance improvements observed in multiple classifier combination schemes.
Paper: An Efficient Boosting Algorithm for Combining Preferences, Yoav Freund et al. (1998). Extends the principle of combining orderings and rank-based decisions by applying boosting algorithms directly to preference and ranking aggregation.
Paper: Rank aggregation methods for the Web, Cynthia Dwork et al. (2001). Formalizes and extends multi-source rank aggregation algorithms, including positional voting and Markov chain methods, to large-scale information retrieval settings.
Paper: Statistical Pattern Recognition: A Review, Anil K. Jain et al. (2000). Surveys the broader statistical pattern recognition landscape, reviewing classifier combination strategies as a primary paradigm for error reduction.
Paper: Popular Ensemble Methods: An Empirical Study, David Opitz et al. (1999). Provides a comprehensive empirical study of ensemble combination techniques, evaluating the operational conditions under which classifier combination succeeds.
Paper: Rotation Forest: A New Classifier Ensemble Method, Juan J. Rodríguez et al. (2006). Applies ensemble combination principles by introducing feature-space rotation to enforce diversity across combined base classifiers.

Cover for Decision Combination in Multiple Classifier Systems

Abstract

A multiple classifier system is a powerful solution to difficult pattern recognition problems involving large class sets and noisy input because it allows simultaneous use of arbitrary feature descriptors and classification procedures. Decisions by the classifiers can be represented as rankings of classes so that they are comparable across different types of classifiers and different instances of a problem. The rankings can be combined by methods that either reduce or rerank a given set of classes. An intersection method and a union method are proposed for class set reduction. Three methods based on the highest rank, the Borda count, and logistic regression are proposed for class set reranking. These methods have been tested in applications on degraded machine-printed characters and words from large lexicons, resulting in substantial improvement in overall correctness.

Topics

feature descriptors logistic regression logistics svm individual classifiers classification algorithm ensemble classifiers

Keywords

multiple classifier systems decision combination classifier combination classifier selection multistage combination voting methods class set reduction class rankings classifier correlation nearest-neighbor classifier

I. INTRODUCTION
II. INPUTS AND OUTPUTS OF DECISION COMBINATION FUNCTIONS
III. METHODS FOR CLASS SET REDUCTION
IV. METHODS FOR CLASS SET REORDERING
A. The Highest Rank Method
B. The Borda Count Method
C. Logistic Regression
V. ALTERNATIVES TO PARALLEL APPLICATION AND COMBINATION
A. Dynamic Classifier Selection
B. Multiple Stage Organization
VI. EXPERIMENTAL RESULTS
A. Machine-Printed Word Recognition
B. Identifying Redundant Classifiers
VII. CONCLUSION
ACKNOWLEDGMENT
REFERENCES

Knowls

Knowl 1 — Class Set Reduction via Intersection of Worst-Case Neighborhoods
model/method
Class set reduction aims to extract a subset of candidate classes from an initial set $\mathcal{C}$ such that the subset size is minimized while preserving the true class.

In the intersection method, each classifier $C_j$ ( $j = 1, \dots, m$ ) produces a complete ranking of all classes for an input pattern. During training on $N$ patterns $\{I_1, \dots, I_N\}$ , the worst-case (maximum) rank assigned by classifier $C_j$ to the true class across all training instances is identified: $T_j = \max_{i=1}^N \text{rank}(I_i, C_j)$ where rank 1 represents the top choice and larger numerical ranks represent lower choices.

During testing on an unseen input, each classifier $C_j$ selects a neighborhood consisting of all classes ranked from 1 up to $T_j$ . The final reduced class set $S$ is obtained by taking the intersection of these neighborhoods: $S = \bigcap_{j=1}^m \{c \in \mathcal{C} \mid \text{rank}_j(c) \le T_j\}$

A classifier $C_j$ is redundant in this combination if its threshold $T_j$ equals the total number of classes $|\mathcal{C}|$ , meaning it provides no pruning constraint. This approach requires all classifiers to exhibit moderate worst-case behavior; if any classifier assigns a very low rank to a true class on a difficult input, its threshold $T_j$ becomes large, expanding the neighborhood and reducing pruning efficiency.
Knowl 2 — Class Set Reduction via Union of Max-Min Neighborhoods
model/method
The union method for class set reduction is designed for ensembles of specialized classifiers, where each classifier excels on different types of inputs.

Given $m$ classifiers $C_1, \dots, C_m$ and $N$ training patterns $I_1, \dots, I_N$ with known true classes, the rank assigned to the true class of pattern $I_i$ by classifier $C_j$ is $\text{rank}(I_i, C_j)$ . The best (minimum) rank among all classifiers for pattern $I_i$ is determined as: $r_{\min}(i) = \min_{j=1}^m \text{rank}(I_i, C_j)$

This minimum rank is assigned to the specific classifier that achieved it: $r^*_{i,j} = \begin{cases} r_{\min}(i) & \text{if } j = \arg\min_k \text{rank}(I_i, C_k) \\ 0 & \text{otherwise} \end{cases}$

The rank threshold $T_j$ for classifier $C_j$ is the maximum of these minima across all training instances: $T_j = \max_{i=1}^N r^*_{i,j}$

For a test pattern, each classifier $C_j$ selects classes ranked from $1$ to $T_j$ , and the reduced candidate set is formed by their union: $S = \bigcup_{j=1}^m \{c \in \mathcal{C} \mid \text{rank}_j(c) \le T_j\}$

This construction guarantees that the true class is contained in $S$ for 100% of the training examples. A classifier $C_j$ with $T_j = 0$ is strictly redundant and can be omitted from the ensemble, as its decision is always inferior to that of another classifier for every training sample.
Knowl 3 — Class Set Reordering via the Highest Rank Method
model/method
The highest rank method combines $m$ classifiers to reorder a candidate class set $\mathcal{C}$ by emphasizing the best performance achieved on each class across the ensemble.

Let $r_j(c) \in \{1, \dots, |\mathcal{C}|\}$ be the rank assigned to class $c$ by classifier $C_j$ ( $j = 1, \dots, m$ ), where $r_j(c) = 1$ is the most preferred class. The score $S_{\text{high}}(c)$ assigned to class $c$ is the minimum numerical rank (i.e., the highest position) given by any classifier: $S_{\text{high}}(c) = \min_{j=1}^m r_j(c)$

The candidate classes are then sorted in ascending order of $S_{\text{high}}(c)$ to produce the combined ranking (with ties broken arbitrarily).

If at least one classifier places the true class at rank $k$ , the true class is guaranteed to appear at a position no worse than $k \times m$ from the top in the combined ranking, regardless of the rankings assigned by the remaining $m - 1$ classifiers.

A classifier is redundant under this method if the rank it assigns to a true class is always numerically greater than (lower than) the rank assigned by at least one other classifier. A limitation of this method is that when the number of classifiers $m$ is large relative to the number of classes, many classes share identical minimum ranks, resulting in extensive ties.
Knowl 4 — Class Set Reordering via the Borda Count Consensus Function
model/method
The Borda count is a group consensus ranking method adapted from social choice theory that combines individual classifier rankings without requiring parameter training.

For an input pattern evaluated over a set of $n = |\mathcal{C}|$ classes, let $r_j(c) \in \{1, \dots, n\}$ denote the rank assigned to class $c$ by classifier $C_j$ for $j = 1, \dots, m$ (with $r_j(c) = 1$ being the top choice). The Borda count $B(c)$ for class $c$ is the total number of candidate classes ranked below $c$ summed over all $m$ classifiers: $B(c) = \sum_{j=1}^m (n - r_j(c))$

The consensus ranking of the ensemble is obtained by ordering classes in descending order of $B(c)$ .

The Borda count measures the cumulative agreement among classifiers that the input belongs to class $c$ . For a two-class problem ( $n=2$ ), the Borda count ranking is equivalent to simple majority voting. The method assumes additive independence among the individual classifier outputs and treats all classifiers equally. A classifier is redundant under the Borda count if all classes it ranks above the true class are always contained within the set of classes ranked above the true class by other classifiers.
Knowl 5 — Class Set Reordering via Logistic Regression Analysis
model/method
Logistic regression provides a statistical framework to combine class rankings by weighting classifiers according to their reliability and modeling non-uniform classifier performance.

For each candidate class $c \in \mathcal{C}$ and a given input pattern, a binary response variable $Y_c \in \{0, 1\}$ is defined such that $Y_c = 1$ if $c$ is the true class and $Y_c = 0$ otherwise. Let $\mathbf{x}_c = (x_{c,1}, x_{c,2}, \dots, x_{c,m})$ denote the vector of rank scores assigned to class $c$ by classifiers $C_1, C_2, \dots, C_m$ , where higher scores represent higher ranks (e.g., $x_{c,j} = K$ for rank 1 down to $x_{c,j} = 0$ for ranks beyond $K$ ).

The probability $\pi(\mathbf{x}_c) = P(Y_c = 1 \mid \mathbf{x}_c)$ is modeled via the logistic response function: $\pi(\mathbf{x}_c) = \frac{\exp\left(\alpha + \sum_{j=1}^m \beta_j x_{c,j}\right)}{1 + \exp\left(\alpha + \sum_{j=1}^m \beta_j x_{c,j}\right)}$

Applying the logit transformation yields a linear relation: $L(\mathbf{x}_c) = \log\left(\frac{\pi(\mathbf{x}_c)}{1 - \pi(\mathbf{x}_c)}\right) = \alpha + \sum_{j=1}^m \beta_j x_{c,j}$ where $\alpha$ is the intercept and $\beta_j$ represents the regression coefficient (weight) reflecting the marginal contribution and relative importance of classifier $C_j$ .

Model parameters $(\alpha, \beta_1, \dots, \beta_m)$ are estimated on training data via maximum likelihood. At inference time, candidate classes are reordered in descending order of their predicted logits $L(\mathbf{x}_c)$ . Because $\sum_{c \in \mathcal{C}} Y_c = 1$ for each input pattern, the binary responses are correlated across classes of the same pattern, leading to overdispersion and underestimated standard errors, but the point estimates $\beta_j$ remain unbiased.
Knowl 6 — Dynamic Classifier Selection via Top-Choice Agreement Partitioning
model/method
Dynamic classifier selection adapts the combination function to the difficulty of each individual input pattern based on the level of consensus among the classifiers' top choices.

For an ensemble of $m$ classifiers, let $\hat{c}_j$ be the top-ranked class predicted by classifier $C_j$ ( $j = 1, \dots, m$ ). The agreement state of an input is defined by the subset of classifiers that produce identical top choices. For $m = 4$ classifiers, there are $\sum_{k=1}^4 \binom{4}{k} = 12$ possible mutually exclusive agreement configurations (ranging from all 4 classifiers agreeing on the top choice to all 4 predicting different classes).

The training dataset is partitioned into subsets corresponding to these agreement grades. For each partition $g \in \{1, \dots, 12\}$ , a distinct logistic regression combination model is estimated: $L_g(\mathbf{x}_c) = \alpha_g + \sum_{j=1}^m \beta_{g,j} x_{c,j}$

During inference on an unseen pattern:
1. The top choices $\hat{c}_1, \dots, \hat{c}_m$ are extracted to compute the pattern's agreement grade $g$ .
2. The specific parameter vector $(\alpha_g, \beta_{g,1}, \dots, \beta_{g,m})$ is retrieved.
3. The candidate classes are reranked using $L_g(\mathbf{x}_c)$ .
This agreement condition is computable without knowledge of the true class label and allows the combination model to adjust classifier weights dynamically (e.g., placing higher confidence on specialized classifiers during ambiguous or low-agreement states).
Knowl 7 — Identification of Redundant Classifiers via Logistic Regression Significance
model/method
Logistic regression can identify redundant classifiers in an ensemble by evaluating the statistical significance and magnitude of their estimated regression coefficients $\beta_j$ .

A classifier $C_j$ is redundant in an ensemble if its weight $\beta_j$ is close to zero, has a large standard error, or has an insignificant Wald Chi-Square test statistic ( $p > 0.05$ ). Setting insignificant parameters to zero or removing the corresponding classifiers yields an equivalent or superior combination model with fewer components.

When combining classifiers that use identical feature representations but different decision rules (e.g., 1-nearest-neighbor versus 2-nearest-neighbor), introducing the higher-performing classifier can cause the parameter of the correlated classifier to drop to near zero and lose statistical significance. Conversely, classifiers operating on complementary feature sets (e.g., raw pixel vectors versus structural stroke features) maintain positive, statistically significant coefficients when combined.

Knowl 8 — Empirical Comparison of Reordering Methods in Large-Lexicon Word Recognition

data/table

Four classifiers were applied to recognize postal machine-printed word images against a lexicon of 1,365 classes: a character recognition polynomial discriminant with postprocessing (poly), a segmentation-based binary pixel classifier (segb), a holistic stroke-direction word-shape classifier (sfv), and a holistic Baird-feature word-shape classifier (bfv). Evaluated on a test set of 1,384 images (trained on 4,624 images across 12 top-choice agreement grades), the individual and combined recognition rates across top $N$ choices are reported below:

Classifier / Combination Method	% Correct in Top $N$ Choices
Classifier / Combination Method	1	2	3	5	10
1) Character recognition (poly)	84.9	88.4	90.3	91.2	92.3
2) Segmentation-based method (segb)	86.1	90.0	90.9	91.8	92.8
3) Word-shape with stroke directions (sfv)	65.2	74.5	78.5	82.4	85.5
4) Word-shape with Baird features (bfv)	50.9	59.0	62.2	66.3	70.9
5) Combination by Highest Rank	50.9	84.7	96.2	98.6	98.9
6) Combination by Borda Count	87.4	95.8	97.2	98.2	99.0
7) Combination by Static Logistic Regression	90.7	96.2	97.5	98.5	99.0
8) Combination by Dynamically Selected Model	93.9	97.2	97.9	98.3	99.0
9) Oracle (best classifier per image)	98.1	98.8	99.0	99.1	99.3

The static logistic regression model achieved 90.7% top-1 accuracy, outperforming the best individual classifier (segb at 86.1%). Dynamically selecting the logistic regression model based on the top-choice agreement grade achieved 93.9% top-1 accuracy (a 7.8% absolute gain over segb). The highest rank method had low top-1 performance (50.9%) due to tie-breaking among unranked classes, but achieved 98.9% top-10 accuracy.

Knowl 9 — Empirical Results on Degraded Character Recognition and Classifier Pruning

data/table

Six classifiers were constructed for recognizing degraded machine-printed characters over 48 merged character classes: Bayesian independence (PBC), modified 1-nearest-neighbor (PNC), and modified 2-nearest-neighbor (P2N) using 576-dimensional pixel vectors; and Bayesian independence (BBC), modified 1-nearest-neighbor (BNC), and modified 2-nearest-neighbor (B2N) using 288-dimensional Baird feature vectors. Maximum likelihood logistic regression was trained on 8,000 samples and evaluated on 12,000 independent test samples.

In the full six-classifier regression model, PNC (estimate $\beta = -0.0579$ , Wald $p = 0.0085$ ) and BNC (estimate $\beta = -0.0232$ , Wald $p = 0.2741$ ) were rendered redundant by the presence of P2N ( $\beta = 0.2130$ ) and B2N ( $\beta = 0.2648$ ). Removing PNC and BNC produced an optimal 4-classifier ensemble (PBC, BBC, P2N, B2N).

Classifier(s) / Combination	Correct Rate (%) at Top $N$ Choices
Classifier(s) / Combination	1	2	3	4	5	10
PBC (Pixel Bayes)	79.3	87.8	91.2	92.8	94.3	97.6
PNC (Pixel 1-NN)	85.3	91.3	93.3	94.7	95.4	97.7
P2N (Pixel 2-NN)	85.8	91.9	93.9	94.9	95.7	97.9
BBC (Baird Bayes)	79.1	87.5	90.7	92.4	93.9	97.1
BNC (Baird 1-NN)	84.3	90.8	93.3	94.6	95.4	97.7
B2N (Baird 2-NN)	85.4	91.7	93.9	95.1	95.7	98.0
Regression: PBC, PNC	85.4	92.2	94.5	95.7	96.7	98.5
Regression: PBC, P2N	86.3	92.8	94.7	95.9	96.8	98.6
Regression: BBC, BNC	85.3	91.8	94.1	95.3	96.1	98.3
Regression: BBC, B2N	86.0	92.3	94.5	95.7	96.4	98.4
Regression: PBC, BBC	81.3	89.8	92.7	94.5	95.8	98.3
Regression: PNC, BNC	86.7	92.6	94.6	95.6	96.2	98.4
Regression: P2N, B2N	86.9	92.9	95.0	95.9	96.5	98.5
Regression: PBC, BBC, PNC, BNC	88.1	93.8	95.7	96.6	97.2	98.8
Regression: PBC, BBC, P2N, B2N	88.8	94.1	95.7	96.7	97.4	98.9

The pruned four-classifier combination (PBC, BBC, P2N, B2N) achieved 88.8% top-1 accuracy, representing a 3.0% net increase over the best individual classifier (P2N at 85.8%).

Knowl 10 — Multistage Classifier System Architecture
model/method
Multiple classifier systems can be organized into a multistage pipeline to manage computational cost and accuracy across large class sets.

In a multistage organization:
1. At each stage, a designated subset of classifiers operates in parallel on the currently active set of candidate classes.
2. Decisions at early stages are combined using class set reduction methods (the intersection or union thresholding procedures) to prune the candidate set from $| \mathcal{C}|$ to a smaller subset $\mathcal{C}'$ .
3. Subsequent stages apply more computationally demanding feature extractors or classifiers solely to $\mathcal{C}'$ .
4. Decisions at terminal stages are combined using class set reordering methods (Borda count or logistic regression models) to produce the final consensus ranking.
5. Dynamic selectors can be placed between stages to determine which downstream classifiers are activated based on intermediate confidence or top-choice agreement measures.

Coverage note — No substantial contributed material was omitted; all reduction algorithms, reordering functions, dynamic combination methods, statistical regression formulations, redundancy analyses, and experimental data are fully covered.

References

1.A. Agresti, Categorical Data Analysis. New York: Wiley, 1990.
2.J. A. Anderson, "Regression and ordered categorical variables," J. Roy. Statist. Soc., Ser. B, vol. 46, no. 1, pp. 1-30, 1984.
3.H. S. Baird, H. P. Graf, L. D. Jackel, and W. E. Hubbard, "A VLSI architecture for binary image classification," in From Pixels to Features, J. C. Simon, Ed. Amsterdam: North-Holland, 1989, pp. 275-286.
4.D. Black, The Theory of Committees and Elections, 2nd ed. London: Cambridge University Press, 1958, 1963.
5.R. Bradford and T. Nartker, "Error correlation in contemporary OCR systems," in Proc. 1st Int. Conf. Document Analysis and Recognition, Saint-Malo, France, 1991, pp. 516-523.
6.D. R. Cox and E. J. Snell, Analysis of Binary Data, 2nd ed. Burlington, VT: Chapman and Hall, 1989.
7.R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Addison-Wesley, 1973.
8.B. Efron, "The efficiency of logistic regression compared to normal discriminant analysis," J. Amer. Statist. Ass., vol. 70, no. 352, pp. 892-898, Dec. 1975.
9.R. M. Haralick, "The table look-up rule," Commun. Statist.—Theory and Methods, vol. A5, no. 12, pp. 1163-1191, 1976.
10.T. Hastie, personal communication, 1992.
11.T. P. Hettmansperger, Statistical Inference Based on Ranks. New York: Wiley, 1984.
12.T. K. Ho, J. J. Hull, and S. N. Srihari, "Combination of structural classifiers," in Pre-Proc. IAPR Syntactic and Structural Pattern Recognition Workshop, Murray Hill, NJ, June 1990, pp. 123-136.
13.T. K. Ho, A Theory of Multiple Classifier Systems and Its Application to Visual Word Recognition, Ph.D. dissertation, Dept. of Computer Science, SUNY at Buffalo, 1992.
14.T. K. Ho, J. J. Hull and S. N. Srihari, "A regression approach to combination of decisions by multiple character recognition algorithms," in SPIE Proc. Vol. 1661, Machine Vision Applications in Character Recognition and Industrial Inspection, San Jose, CA, Feb. 10-12, 1992.
15.------, "On multiple classifier systems for pattern recognition," in Proc. 11th Int. Conf. Pattern Recognition, The Hague, Netherlands, Aug. 30-Sept. 3, 1992, pp. 84-87.
16.------, "A word shape analysis approach to lexicon based word recognition," Patt. Recogn. Lett., vol. 13, pp. 821-826, 1992.
17.------, "Combination of decisions by multiple classifiers," in Structured Document Image Analysis, H. Baird, H. Bunke, and K. Yamamoto, Eds. New York: Springer-Verlag, 1992, pp. 188-202.
18.------, "A computational model for recognition of multifont word images," Machine Vision and Applications, vol. 5, pp. 157-168, 1992.
19.D. W. Hosmer and S. Lemeshow, Applied Logistic Regression. New York: Wiley, 1989.
20.J. J. Hull, A. Commike, and T. K. Ho, "Multiple algorithms for handwritten character recognition," in Proc. 1st Int. Workshop on Frontiers in Handwriting Recognition, Montreal, Apr. 1990, pp. 117-124.
21.F. Kimura and M. Shridhar, "Handwritten numerical recognition based on multiple algorithms," Patt. Recogn., vol. 24, no. 10, pp. 969-983, 1991.
22.E. M. Kleinberg, "Stochastic discrimination," Ann. Math. Artificial Intell., vol. 1, pp. 207-239, 1990.
23.------, "The theory of stochastic modeling in pattern recognition," in preparation.
24.E. M. Kleinberg and T. K. Ho, "Pattern recognition by stochastic modeling," in Proc. 3rd Int. Workshop on Frontiers in Handwriting Recognition, Buffalo, NY, May 1993, p. 175.
25.J. M. Landwehr, D. Pregibon, and A. C. Shoemaker, "Graphical methods for assessing logistic regression models," J. Amer. Statist. Ass., vol. 79, no. 385, pp. 61-71, Mar. 1984.
26.S. le Cessie, "Model building techniques for logistic regression, with applications to medical data," Ph.D dissertation, Univ. of Leiden, The Netherlands, 1991.
27.E. Mandler and J. Schuermann, "Combining the classification results of independent classifiers based on the Dempster/Shafer theory of evidence," in Pattern Recognition and Artificial Intelligence, E. S. Gelsema and L. N. Kanal, Eds. Amsterdam: North-Holland, 1988, pp. 381-393.
28.V. D. Mazurov, A. I. Krivonogov, and V. L. Kazantsev, "Solving of optimization and identification problems by the committee methods," Patt. Recogn., vol. 20, no. 4, pp. 371-378, 1987.
29.R. Meddis, Statistics Using Ranks: A Unified Approach. Philadelphia: Basil Blackwell, 1984.
30.C. Nadal, R. Legault, and C. Y. Suen, "Complementary algorithms for the recognition of totally unconstrained handwritten numerals," in Proc. 10th Int. Conf. Pattern Recognition, Atlantic City, NJ, 1990, pp. 443-449.
31.S. J. Press and S. Wilson, "Choosing between logistic regression and discriminant analysis," J. Amer. Statist. Ass., vol. 73, no. 364, pp. 699-705, Dec. 1978.
32.SAS Institute Inc., SAS/STAT User's Guide, version 6, 4th ed., vol. 2. Cary, NC: SAS Institute Inc., 1989.
33.K.-D. Wernecke, "A coupling procedure for the discrimination of mixed data," Biometrics, vol. 48, pp. 497-506, June 1992.
34.L. Xu, A. Krzyzak, and C. Y. Suen, "Methods of combining multiple classifiers and their applications to handwriting recognition," IEEE Trans. Syst., Man, Cybern., vol. SMC-22, no. 3, pp. 418-435, May/June 1992.

Citation

MLA

Tin Kam Ho, et al. “Decision Combination in Multiple Classifier Systems”. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 1, 1994, pp. 66–75, https://doi.org/10.1109/34.273716.

APA

Tin Kam Ho, Hull, J. J., & Srihari, S. N. (1994). Decision combination in multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(1), 66–75. https://doi.org/10.1109/34.273716

Chicago

Tin Kam Ho, J. J. Hull, and S. N. Srihari. 1994. “Decision Combination in Multiple Classifier Systems”. IEEE Transactions on Pattern Analysis and Machine Intelligence 16 (1): 66–75. https://doi.org/10.1109/34.273716.

Harvard

Tin Kam Ho, Hull, J.J. and Srihari, S.N. (1994) “Decision combination in multiple classifier systems”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(1), pp. 66–75. Available at: https://doi.org/10.1109/34.273716.

Vancouver

1. Tin Kam Ho, Hull JJ, Srihari SN (1994) Decision combination in multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 16:66–75

BibTeX

@article{Tin_Kam_Ho_1994, title={Decision combination in multiple classifier systems}, volume={16}, ISSN={0162-8828}, url={http://dx.doi.org/10.1109/34.273716}, DOI={10.1109/34.273716}, number={1}, journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, publisher={Institute of Electrical and Electronics Engineers (IEEE)}, author={Tin Kam Ho and Hull, J.J. and Srihari, S.N.}, year={1994}, pages={66–75} }

Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Decision Combination in Multiple Classifier Systems

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Class Set Reduction via Intersection of Worst-Case Neighborhoods

Knowl 2 — Class Set Reduction via Union of Max-Min Neighborhoods

Knowl 3 — Class Set Reordering via the Highest Rank Method

Knowl 4 — Class Set Reordering via the Borda Count Consensus Function

Knowl 5 — Class Set Reordering via Logistic Regression Analysis

Knowl 6 — Dynamic Classifier Selection via Top-Choice Agreement Partitioning

Knowl 7 — Identification of Redundant Classifiers via Logistic Regression Significance