title: How can feature selection methods be evaluated?
body: <p><strong>Hello,</strong></p>
  
  <p>How can I choose the best fit feature selection method for a given dataset (textual data) ?</p>
  
  <p>In Weka for example, there are several attribute selection methods (CfsSubsetEval, ChiSquaredAttributeEval, ... etc), and several search methods (bestfirst, greedy, ranker ... etc).</p>
  
  <p><strong>My Question:</strong> How can I know which attribute selection method and search method is best for a given dataset ?!</p>
  
  <p><strong>My Guess:</strong> Should I use cross validation to test the dataset after applying the feature selection filter ? so for example, that means if i have 10 attribute selection methods and 10 search methods, I will need to perform 100 cross validation test then pick the configuration with the highest accuracy !!!!!!! and I am assuming here that I am testing against one classifier only. So what if i have 2 classifiers (SMO and J48), will i need to perform 200 cross validation test ?!</p>
  
  <p>Please correct me if I misunderstood something ...</p>