title: Predictive Performance of Random Forest: Is there a difference between permuting or run the model without the permuted variable?
body: <p>I'm using the R package "ModelMap" to produce Random Forest (RF) models (regression model). It provides a plot showing relative importance of input variables, by measuring the decrease of MSE in % of the overall accuracy of the model for input variable xy after permuting = eliminating its predictive ability. Another plot shows the RMSE of the model, based on model validation using the test data (predicted values vs. observed values).</p>
  
  <p>Now lets produce another RF model excluding the mentioned input variable xy and lets compare the RMSE with the RMSE of the first model. The result <strong>almost 0</strong> (the unit is for each ca 300 kg/ha ) even though the decrease of MSE of the previous model showed a decrease of 70% for variable xy and should lead to a decrease in performance by excluding it.</p>
  
  <p>Does this approach make sense? And if yes, why does the 2nd model has the same predictive performance?</p>
  
  <p>Thanks for help</p>