A project team performed a feature selection procedure on the full data set and reduced their large feature set to a smaller set. Then they split the data into test and training portions. They built their model on training data using several different model settings, and report the best test error they achieved. Which of the following is TRUE about the given experimental setup?
A. Best setup
B. Problematic setup
C. Invalid setup
D. Cannot be decided
Answer:B
A project team performed a feature selection procedure on the full data set and reduced their large feature set to a smaller set. Then they split the data into test and training portions. They built their model on training data using several different model settings, and report the best test error they achieved. Which of the following is TRUE about the given experimental setup?
A. Best setup B. Problematic setup C. Invalid setup D. Cannot be decided