Feature selection with harmony search and its applications
Abstract
Feature selection is a term given to the problem of selecting important domain
attributes which are most predictive of a given outcome. Unlike other dimensionality
reduction methods, feature selection approaches seek to preserve the semantics of
the original data following reduction. Many strategies have been exploited for this
task in an effort to identify more compact and better quality feature subsets. A
number of group-based feature subset evaluation measures have been developed,
which have the ability to judge the quality of a given feature subset as a whole, rather
than assessing the qualities of individual features. Techniques of stochastic nature
have also emerged, which are inspired by nature phenomena or social behaviour,
allowing good solutions to be discovered without resorting to exhaustive search.
In this thesis, a novel feature subset search algorithm termed ?feature selection
with harmony search? is presented. The proposed approach utilises a recently
developed meta-heuristic: harmony search, that is inspired by the improvisation
process of musical players. The proposed approach is general, and can be employed
in conjunction with many feature subset evaluation measures. The simplicity of
harmony search is exploited to reduce the overall complexity of the search process.
The stochastic nature of the resultant technique also allows the search process to
escape from local optima, while identifying multiple, distinctive candidate solutions.
Additional parameter control schemes are introduced to reduce the effort and impact
of static parameter configuration of HS, which are further combined with iterative
refinement, in order to enforce the discovery of more compact feature subsets.
The flexibility of the proposed approach, and its powerful performance in selecting
multiple, good quality feature subsets lead to a number of further theoretical developments.
These include the generation and reduction of feature subset-based
classifier ensembles; feature selection and adaptive classifier ensemble for dynamic
data; hybrid rule induction on the basis of fuzzy-rough set theory; and antecedent
selection for fuzzy rule interpolation. The resultant techniques are experimentally
evaluated using data sets drawn from real-world problem domains, and systematically
compared with leading methodologies in their respective areas,