Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning

Mark A. Hall

article1999ICML2,056 citations

Introduces Correlation-based Feature Selection (CFS), a fast filter algorithm for discrete and continuous learning tasks that evaluates feature subsets by balancing class correlation against inter-feature redundancy to cut data dimensionality in half and improve model compactness.

Listen

High-dimensional datasets containing irrelevant or redundant information present a major operational challenge in machine learning, slowing down model training and increasing the risk of overfitting. While wrapper-based selection methods are computationally expensive and traditional filter algorithms largely focus on discrete classification problems, there is a strong need for efficient, universal preprocessing techniques that scale to large datasets across both discrete and continuous prediction tasks.

The article sets out to develop and evaluate Correlation-based Feature Selection (CFS), a fast filter method designed to assess feature subsets rather than individual features, and to demonstrate its effectiveness across both discrete classification and continuous regression problems.

The analysis evaluated CFS across 16 discrete and 19 continuous benchmark datasets from the UCI repository, comparing its performance against the established ReliefF algorithm. Evaluated over multiple ten-fold cross-validation runs, the filtered datasets were processed using diverse learning algorithms, including probabilistic methods, tree-based models, and nearest-neighbor approaches.

The primary findings show that CFS is an aggressive and effective feature selector. First, CFS reduced data dimensionality by approximately 47% on discrete datasets and 54% on continuous datasets, systematically outperforming ReliefF in pruning unneeded variables. Second, CFS maintained or significantly improved the predictive accuracy of downstream learning algorithms in the majority of cases, enhancing classification accuracy on up to six datasets and lowering regression error across eight to nine datasets. Third, preprocessing with CFS produced substantially smaller decision trees in 56% of discrete cases and reduced the number of linear models in 42% of regression tree models, without ever increasing tree size in classification tasks.

These results demonstrate that organizations can lower computational costs and streamline machine learning pipelines by stripping away roughly half of their data features without sacrificing predictive power. By generating smaller, less complex decision trees, CFS improves model interpretability, which reduces deployment risk and simplifies compliance in regulated environments.

Technical leaders and data science teams should consider adopting CFS as a standard preprocessing filter for large-scale classification and regression workflows, particularly when wrapper methods are too slow. Practitioners should, however, evaluate whether their specific domain relies heavily on complex multi-feature interactions, as CFS evaluates features based on individual and pairwise correlations and cannot detect strongly interacting attributes such as parity problems. Given the strong evidence base across diverse standard benchmarks, stakeholders can have high confidence in using CFS for general tabular data tasks.

University of Waikato database systems genetic selection naive bayes correlation-based feature selection feature subset selection numeric class machine learning

Hall (1999).pdf

Paper: Irrelevant Features and the Subset Selection Problem, George H. John et al. (1994). It establishes the foundational distinction between filter and wrapper models of feature selection and formalizes concepts of attribute relevance that motivate fast correlation-based filters.
Paper: Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning, Usama M. Fayyad et al. (1993). It develops the standard minimum description length discretization method for continuous attributes that enables discrete-oriented feature selection metrics to evaluate continuous data.
Paper: Supervised and Unsupervised Discretization of Continuous Features, James Dougherty et al. (1995). It provides the foundational study on supervised entropy-based discretization heuristics for continuous features in classifiers like Naive Bayes and decision trees.
Paper: Feature Selection: Evaluation, Application, and Small Sample Performance, Anil K. Jain et al. (1997). It systematically analyzes feature subset search algorithms and evaluations across varying sample sizes, framing the computational trade-offs addressed by heuristic filters.
Paper: Instance-based learning algorithms, D. Aha et al. (1991). It introduces standard instance-based learning algorithms (IB1–IB3), which serve as key evaluation benchmarks in the source paper's experiments.
Paper: On the Optimality of the Simple Bayesian Classifier under Zero-One Loss, Pedro M. Domingos et al. (1997). It analyzes the behavior and optimality of simple Bayesian classifiers under feature dependencies, directly motivating the removal of redundant attributes.

Paper: Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution, Lei Yu et al. (2003). It advances correlation-based filtering to very high-dimensional settings by introducing predominant correlation and the Fast Correlation-Based Filter (FCBF) algorithm based on symmetrical uncertainty.
Paper: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy, Hanchuan Peng et al. (2003). It formulates the minimal-redundancy-maximal-relevance (mRMR) framework using mutual information, extending the concept of balancing feature-target relevance with inter-feature redundancy.
Paper: An Introduction to Variable and Feature Selection, Isabelle M Guyon et al. (2003). It provides a comprehensive post-2000 taxonomy and practical methodology synthesizing correlation ranking, filter approaches, wrappers, and embedded selection techniques.
Paper: Toward integrating feature selection algorithms for classification and clustering, Huan Liu et al. (2005). It synthesizes filter and wrapper algorithms, including correlation-based criteria, into a unified evaluation framework spanning classification and clustering tasks.
Paper: Theoretical and Empirical Analysis of ReliefF and RReliefF, M. Robnik-Sikonja et al. (2003). It provides theoretical and empirical analysis of the ReliefF and RReliefF attribute estimation family, the primary baseline against which the source algorithm is compared.
Paper: Gene Selection for Cancer Classification using Support Vector Machines, ISABELLE GUYON et al. (2002). It introduces SVM Recursive Feature Elimination, establishing an embedded alternative to correlation-based filtering for high-dimensional feature selection.

Cover for Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning

Abstract

Algorithms for feature selection fall into two broad categories: wrappers that use the learning algorithm itself to evaluate the usefulness of features and filters that evaluate features according to heuristics based on general characteristics of the data. For application to large databases, filters have proven to be more practical than wrappers because they are much faster. However, most existing filter algorithms only work with discrete classification problems. This paper describes a fast, correlation-based filter algorithm that can be applied to continuous and discrete problems. The algorithm often outperforms the well-known ReliefF attribute estimator when used as a preprocessing step for naive Bayes, instance-based learning, decision trees, locally weighted regression, and model trees. It performs more feature selection than ReliefF does—reducing the data dimensionality by fifty percent in most cases. Also, decision and model trees built from the preprocessed data are often significantly smaller.

Topics

database systems genetic selection naive bayes relieff classification models adaptive algorithms optimization problems

Keywords

correlation-based feature selection feature subset selection numeric class machine learning filter algorithms naive Bayes decision trees model trees nearest neighbour

1. Introduction
2. Related Work
3. CFS: Correlation-based Feature Selection
3.1 Feature Evaluation
3.2 Searching the Feature Subset Space
3.3 Locally Predictive Features
4. Applying CFS to Discrete Class Data
5. Applying CFS to Continuous Class Data
6. Conclusions
References

Knowls

Knowl 1 — Heuristic Merit Function for Feature Subset Evaluation in CFS
equation
The core heuristic of Correlation-based Feature Selection (CFS) evaluates the quality ("merit") of a feature subset by favoring subsets whose features correlate strongly with the class label while exhibiting low intercorrelation among themselves. For a subset $S$ containing $k$ features, the subset merit $\text{Merit}_S$ is defined as:

$\text{Merit}_S = \frac{k \overline{r_{cf}}}{\sqrt{k + k(k-1)\overline{r_{ff}}}}$

where:
- $k \in \mathbb{N}_{\ge 1}$ is the number of features in subset $S$ .
- $\overline{r_{cf}} \in [-1, 1]$ is the mean feature-class correlation across all features in $S$ .
- $\overline{r_{ff}} \in [-1, 1]$ is the mean pairwise feature-feature intercorrelation across all distinct pairs in $S$ .
This formula represents Pearson's correlation coefficient between a standardized composite variable (formed by summing the standardized features) and the standardized target variable. The numerator reflects the aggregate predictive power of the feature subset, while the denominator penalizes feature redundancy. Features that are poor predictors of the class yield low $\overline{r_{cf}}$ , while redundant features inflate $\overline{r_{ff}}$ and decrease the overall merit score.
Knowl 2 — Best-First Search and Locally Predictive Feature Inclusion in CFS
model/method
Correlation-based Feature Selection (CFS) explores the $2^n$ feature subset space using a best-first heuristic search, followed by a post-search step to capture locally predictive features:
1. Pre-computation: CFS computes a matrix of all pairwise feature-class correlations and feature-feature intercorrelations over the training data.
2. Best-First Search: The search initializes with the empty subset ( $\emptyset$ ) and generates all possible single-feature additions. The subset with the highest evaluation according to the CFS merit heuristic is chosen and expanded in the same manner. If an expansion yields no improvement in merit, the search backtracks to the next best unexpanded subset. The search terminates when five consecutive fully expanded subsets fail to improve upon the best merit score found so far.
3. Locally Predictive Feature Inclusion: Because global correlations computed across the full dataset can exclude subsidiary features that are predictive only within localized regions of instance space, unselected features are inspected sequentially after the subset search. An unselected feature $X$ is added to the selected subset $S_{\text{selected}}$ if its correlation with the target class $c$ exceeds its maximum correlation with any already selected attribute $Y$ :
$r_{Xc} > \max_{Y \in S_{\text{selected}}} r_{XY}$

This rule integrates locally predictive attributes while preventing the inclusion of highly redundant variables.
Knowl 3 — Discrete Attribute Correlation via Symmetrical Uncertainty in CFS
equation
To evaluate feature-class and feature-feature associations on discrete classification tasks, CFS first discretizes continuous numeric features using the Fayyad and Irani entropy-based multi-interval discretization method. Pairwise correlations are then computed using Symmetrical Uncertainty ( $SU$ ):

$SU(X, Y) = 2.0 \times \left[ \frac{H(X) + H(Y) - H(X, Y)}{H(X) + H(Y)} \right]$

where:
- $X$ and $Y$ are discrete attributes or the class variable.
- $H(X) = -\sum_{x} p(x) \log_2 p(x)$ is the Shannon entropy of $X$ .
- $H(Y) = -\sum_{y} p(y) \log_2 p(y)$ is the Shannon entropy of $Y$ .
- $H(X, Y) = -\sum_{x} \sum_{y} p(x, y) \log_2 p(x, y)$ is the joint entropy of $X$ and $Y$ .
The numerator corresponds to mutual information (information gain) $I(X; Y) = H(X) + H(Y) - H(X, Y)$ . Symmetrical uncertainty normalizes information gain to the range $[0, 1]$ and is symmetric ( $SU(X, Y) = SU(Y, X)$ ), allowing its use for feature-feature dependencies where neither variable is a target class. Missing attribute values are distributed across represented nominal categories proportionally to their marginal frequencies.
Knowl 4 — Correlation Formulations for Continuous and Mixed-Type Data in CFS
equation
When evaluating feature-class and feature-feature dependencies on continuous regression tasks or datasets with mixed continuous and discrete attributes, CFS calculates correlations using standard and weighted Pearson correlation measures:
1. Continuous-Continuous Pair ( $X$ and $Y$ both continuous): $r_{XY} = \frac{\sum x y}{n \sigma_X \sigma_Y}$ where $x$ and $y$ are values expressed as deviations from their respective sample means, $n$ is the number of instances, and $\sigma_X, \sigma_Y$ are standard deviations.
2. Discrete-Continuous Pair ( $X$ discrete with $k$ values, $Y$ continuous): $r_{XY} = \sum_{i=1}^{k} p(X = x_i) \, r_{X_{bi} Y}$ where $X_{bi} \in \{0, 1\}$ is a binary indicator variable taking value $1$ when $X = x_i$ and $0$ otherwise, $p(X = x_i)$ is the prior probability of category $x_i$ , and $r_{X_{bi} Y}$ is the Pearson correlation between $X_{bi}$ and $Y$ .
3. Discrete-Discrete Pair ( $X$ discrete with $k$ values, $Y$ discrete with $l$ values): $r_{XY} = \sum_{i=1}^{k} \sum_{j=1}^{l} p(X = x_i, Y = y_j) \, r_{X_{bi} Y_{bj}}$ where $p(X = x_i, Y = y_j)$ is the joint probability and $r_{X_{bi} Y_{bj}}$ is the Pearson correlation between indicator variables $X_{bi}$ and $Y_{bj}$ .
Missing values are imputed with the attribute mean for continuous features and the mode for discrete features.
Knowl 5 — Inability of CFS to Detect Strongly Interacting Attributes
limitation
Because Correlation-based Feature Selection (CFS) constructs a pairwise correlation matrix and treats attributes independently during evaluation, it cannot detect feature combinations whose predictive utility depends strictly on higher-order multi-attribute interactions (such as parity problems). CFS is, however, capable of identifying relevant features under moderate degrees of interaction.
Knowl 6 — Experimental Setup for Evaluating CFS Across Discrete and Numeric Benchmarks
experimental setup
CFS was evaluated across 16 discrete classification datasets and 19 continuous regression datasets from standard benchmarks (including the UCI repository):
- Cross-Validation & Preprocessing: Performance was measured using 10 runs of 10-fold cross-validation. Feature selection was conducted on each training split prior to model fitting. Statistical significance was determined using a paired two-sided $t$ -test at the $1\%$ significance level ( $p < 0.01$ ).
- Discrete Class Learners: Naive Bayes (NB), C4.5 decision trees (release 8), and instance-based learning ( $k\text{NN}$ / $\text{IB}k$ , where $k$ is selected via cross-validation on training data). Discretized copies of training splits using the Fayyad-Irani method were provided to CFS.
- Continuous Class Learners: Naive Bayes for Regression (NBR, utilizing Gaussian kernel density estimators), $\text{M5}'$ model trees (decision trees with linear models at the leaves), and Locally Weighted Regression (LWR). The evaluation metric was the Relative Root Mean Squared Error (RRSE), defined as the root mean squared error of the learner divided by the root mean squared error of the test sample mean.
- ReliefF Baseline: Evaluated with sample size $m = 250$ . On discrete datasets, $k = 10$ nearest neighbours and relevance thresholds $\{0.0, 0.01, 0.05, 0.1\}$ were tested. On continuous datasets, RReliefF was run with $m = 250$ , $k = 200$ , $\sigma = 20$ , and a relevance threshold of $0.0$ .
Knowl 7 — Empirical Performance and Dimensionality Reduction of CFS on Discrete Classification Tasks
empirical result
On the 16 discrete UCI benchmark datasets evaluated using 10 runs of 10-fold cross-validation:
- Dimensionality Reduction: CFS reduces data dimensionality by an average of $47\%$ . In contrast, ReliefF with a threshold of $0.0$ achieves only a $2.6\%$ reduction on average (outside of three datasets). CFS selects significantly fewer features than ReliefF with threshold $0.01$ across 15 of the 16 datasets ( $p < 0.01$ ).
- Classification Accuracy Relative to Full Features:
  - Naive Bayes: Accuracy significantly improves on 6 datasets and degrades on 4.
  - C4.5: Accuracy significantly improves on 4 datasets and degrades on 3.
  - $\text{IB}k$ : Accuracy significantly improves on 4 datasets and degrades on 3.
- Comparison with ReliefF (Threshold 0.01): CFS wins against ReliefF on 4 datasets and loses on 5 for Naive Bayes; wins on 2 and loses on 2 for C4.5; and wins on 4 with 0 losses for $\text{IB}k$ .
- Decision Tree Size: Feature selection via CFS never increases the size of C4.5 decision trees and produces significantly smaller trees on 9 of the 16 datasets compared to training on all features.
Knowl 8 — Empirical Performance and Model Complexity Reduction of CFS on Numeric Regression Tasks
empirical result
On the 19 continuous regression benchmark datasets evaluated using 10-fold cross-validation:
- Dimensionality Reduction: CFS achieves an average feature set reduction of $54\%$ , compared to $42\%$ achieved by ReliefF (threshold $0.0$ ). The difference in selected feature counts between CFS and ReliefF is statistically significant ( $p < 0.01$ ) on 15 of the 19 datasets.
- Relative Root Mean Squared Error (RRSE) Relative to Full Features:
  - Naive Bayes for Regression (NBR): CFS significantly improves performance (lowers RRSE) on 9 datasets and degrades performance on 1 dataset.
  - $\text{M5}'$ Model Trees: CFS significantly improves performance on 5 datasets and degrades performance on 4 datasets.
  - Locally Weighted Regression (LWR): CFS significantly improves performance on 8 datasets and degrades performance on 4 datasets.
- Comparison with ReliefF: CFS outperforms ReliefF on 4 datasets and loses on 2 for NBR; each method wins on 3 datasets for $\text{M5}'$ ; and CFS wins on 4 datasets and loses on 6 for LWR.
- Model Tree Complexity: CFS significantly reduces the number of linear models generated by $\text{M5}'$ on 8 of the 19 datasets (increasing tree complexity on 2 datasets).

Knowl 9 — Classification Accuracy Benchmark on 16 Discrete UCI Datasets

data/table

The following table details the classification accuracy (percentage of correct classifications) averaged over 10 runs of 10-fold cross-validation across 16 discrete UCI datasets. Performance is shown for Naive Bayes (NB), C4.5, and instance-based learning ( $\text{IB}k$ ) using all features, feature selection with CFS, and feature selection with ReliefF at threshold 0.01 (Rlf0.01). Statistically significant improvements ( $\circ$ ) and degradations ( $\bullet$ ) relative to the full feature baseline at the $1\%$ level ( $p < 0.01$ , paired two-sided $t$ -test) are indicated.

Data Set	NB	CFS	Rlf0.01	C4.5	CFS	Rlf0.01	IBk	CFS	Rlf0.01
glass-2	62.36	63.85 $\circ$	63.75 $\circ$	75.73	80.63 $\circ$	78.77	77.31	86.59 $\circ$	82.41 $\circ$
anneal	86.49	86.11	86.10 $\bullet$	98.68	98.78	98.77	99.04	98.69	98.85
breast-cancer	72.81	72.90	72.79	73.46	73.76	73.03	72.62	72.18	72.88
credit-g	74.88	74.42	74.39	70.70	72.67 $\circ$	71.60 $\circ$	73.42	73.28	73.09
diabetes	75.68	75.04	75.53	71.84	71.75	71.89	73.66	75.41 $\circ$	74.54 $\circ$
horse colic	81.04	86.46 $\circ$	82.35 $\circ$	85.35	85.09	85.10	83.64	86.32 $\circ$	84.27
heart-c	83.63	84.04	83.38	74.56	75.83	75.24	82.04	81.84	82.10
heart-statlog	84.00	83.81	84.22	76.88	79.17 $\circ$	77.13	81.70	79.89 $\bullet$	80.74 $\bullet$
ionosphere	82.61	88.63 $\circ$	82.61	90.71	90.99	90.16	89.63	89.56	89.60
labor	94.57	86.98 $\bullet$	93.37	84.16	86.90 $\circ$	84.16	90.13	82.30 $\bullet$	93.10
lymph	83.27	80.95 $\bullet$	83.08	75.34	74.28	74.84	80.78	80.92	81.67
segment	80.01	81.28 $\circ$	83.32 $\circ$	96.46	96.63	96.63	97.20	97.19	97.22
sick	92.70	94.48 $\circ$	94.18 $\circ$	98.73	97.49 $\bullet$	97.87 $\bullet$	96.09	96.12	96.47 $\circ$
soybean	92.84	91.94 $\bullet$	92.72	91.89	90.66 $\bullet$	91.80	91.41	90.52 $\bullet$	91.37
vote	90.04	94.00 $\circ$	90.04	96.37	95.52 $\bullet$	96.37	92.52	94.67 $\circ$	92.25
zoo	95.58	94.59 $\bullet$	95.58	92.83	93.17	92.24	95.07	95.38	95.07

Knowl 10 — Regression Error Benchmark on 19 Numeric Datasets

data/table

The following table presents the Relative Root Mean Squared Error (RRSE, where lower values denote superior accuracy) across 19 continuous regression datasets. Comparisons are made for Naive Bayes for Regression (NBR), $\text{M5}'$ model trees, and Locally Weighted Regression (LWR) under the unpruned full feature set, CFS feature selection, and ReliefF feature selection with threshold 0.0 (Rlf). Statistically significant improvements ( $\circ$ ) and degradations ( $\bullet$ ) relative to the full dataset baseline at the $1\%$ level ( $p < 0.01$ , paired two-sided $t$ -test) are indicated.

Data Set	NBR	CFS	Rlf	M5 $'$	CFS	Rlf	LWR	CFS	Rlf
autoHorse	39.17	40.88	39.78 $\bullet$	33.32	33.31	31.95	24.79	27.46	24.02
autoMpg	45.26	45.26	45.19	35.67	35.67	35.66	33.28	33.28	33.65
autoPrice	43.53	40.59	43.57	39.82	37.38 $\circ$	38.67	40.69	40.17	40.76
bodyfat	26.08	13.61 $\circ$	21.52 $\circ$	11.15	10.72 $\circ$	11.24	11.91	10.66 $\circ$	11.22 $\circ$
breastTumor	131.10	128.31 $\circ$	122.63 $\circ$	97.29	97.76	99.03 $\bullet$	103.06	102.19 $\circ$	99.23 $\circ$
cholesterol	114.45	109.91	112.71	101.62	100.15 $\circ$	100.15	103.89	99.34 $\circ$	100.45 $\circ$
cloud	54.22	48.59 $\circ$	50.72 $\circ$	38.36	37.55	38.16	41.09	39.23 $\circ$	40.00 $\circ$
cpu	34.58	29.33 $\circ$	29.33 $\circ$	21.23	16.12 $\circ$	16.35 $\circ$	21.99	19.53 $\circ$	19.53 $\circ$
echoMonths	96.93	81.54 $\circ$	75.34 $\circ$	72.15	72.11	71.94	68.04	69.82	70.10 $\bullet$
fishcatch	32.16	29.15 $\circ$	32.31	16.23	18.77 $\bullet$	16.45	22.45	30.69 $\bullet$	23.51 $\bullet$
housing	64.46	48.30 $\circ$	58.30 $\circ$	39.84	46.20 $\bullet$	41.11	39.93	48.29 $\bullet$	43.40 $\bullet$
hungarian	79.51	81.75	80.61	73.79	73.24	73.22	68.60	72.10 $\bullet$	70.74 $\bullet$
lowbwt	73.31	71.88	73.06	62.00	61.27	60.95 $\circ$	62.66	63.36	61.75 $\circ$
meta	152.87	141.41 $\circ$	176.00	150.68	180.39	172.71	160.32	141.39 $\circ$	135.76 $\circ$
pbc	97.76	100.10	97.47	80.83	85.26 $\bullet$	82.28	81.38	86.60 $\bullet$	83.23 $\bullet$
pharynx	98.39	93.34	92.77 $\circ$	105.87	71.53 $\circ$	82.64 $\circ$	118.05	74.25 $\circ$	81.99 $\circ$
quake	140.07	136.99 $\circ$	137.76 $\circ$	99.96	99.87	100.02	99.76	99.79	99.93 $\bullet$
servo	93.60	97.40 $\bullet$	104.84 $\bullet$	37.92	41.15 $\bullet$	65.15 $\bullet$	38.81	39.04	65.19 $\bullet$
veteran	94.21	95.20	104.28	90.53	90.86	90.66	97.77	93.66 $\circ$	95.66

Coverage note — Summary tables listing raw UCI dataset metadata (Tables 1 and 5) and pairwise t-test matrix comparisons (Tables 2, 3, 4, 6, 7, 8) were condensed into the experimental setup and empirical result knowls to prevent redundant tabulation.

References

1.Almuallim, H. & Dietterich, T. G. (1991). Learning with many irrelevant features. In Proceedings of the Ninth National Conference on Artificial Intelligence (pp. 547–552). AAAI Press.
2.Atkeson, C. G., Moore, A. W. & Schaal, S. (1997). Locally weighted learning. Artificial Intelligence Review, 11, 11–73.
3.Blake, C., Keogh, E. & Merz, C. J. (1998). UCI Repository of Machine Learning Data Bases. Irvine, CA: University of California, Department of Information and Computer Science. [http://www.ics.uci.edu/~mlearn/MLRepository.html].
4.Fayyad, U. M. & Irani, K. B. (1993). Multi-interval discretisation of continuous-valued attributes. In Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence (pp. 1022–1027). Morgan Kaufmann.
5.Frank, E., Trigg, L., Holmes, G. & Witten, I. H. (in press). Naive bayes for regression. Machine Learning.
6.Ghiselli, E. E. (1964). Theory of psychological measurement. McGraw-Hill.
7.Hall, M. A. (1998). Correlation-based feature selection for machine learning. PhD thesis, Department of Computer Science, University of Waikato, Hamilton, New Zealand.
8.Kira, K. & Rendell, L. (1992). A practical approach to feature selection. In Proceedings of the Ninth International Conference on Machine Learning (pp. 249–256). Morgan Kaufmann.
9.Kohavi, R. & John, G. H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97, 273–324.
10.Koller, D. & Sahami, M. (1996). Toward optimal feature selection. In Proceedings of the Thirteenth International Conference on Machine Learning (pp. 284–292). Morgan Kaufmann.
11.Kononenko, I. (1994). Estimating attributes: Analysis and extensions of relief. In Proceedings of the Seventh European Conference on Machine Learning (pp. 171–182). Springer-Verlag.
12.Press, W. H., Flannery, B. P., Teukolski, S. A. & Vetterling, W. T. (1988). Numerical recipies in C. Cambridge University Press.
13.Robnik-Šikonja, M. & Kononenko, I. (1997). An adaption of relief for attribute estimation in regression. In Proceedings of the Fourteenth International Conference on Machine Learning (pp. 296–304). Morgan Kaufmann.
14.Wang, Y. & Witten, I. H. (1997). Induction of model trees for predicting continuous classes. In Proceedings of the poster papers of the European Conference on Machine Learning (pp. 128–137). Prague, Czech Republic.

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors

Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Heuristic Merit Function for Feature Subset Evaluation in CFS

Knowl 2 — Best-First Search and Locally Predictive Feature Inclusion in CFS

Knowl 3 — Discrete Attribute Correlation via Symmetrical Uncertainty in CFS

Knowl 4 — Correlation Formulations for Continuous and Mixed-Type Data in CFS

Knowl 5 — Inability of CFS to Detect Strongly Interacting Attributes

Knowl 6 — Experimental Setup for Evaluating CFS Across Discrete and Numeric Benchmarks

Knowl 7 — Empirical Performance and Dimensionality Reduction of CFS on Discrete Classification Tasks

Knowl 8 — Empirical Performance and Model Complexity Reduction of CFS on Numeric Regression Tasks

Knowl 9 — Classification Accuracy Benchmark on 16 Discrete UCI Datasets

Knowl 10 — Regression Error Benchmark on 19 Numeric Datasets

References

Access the Paper