title: Avoiding overfitting with random forest
body: <p>I am training a random forest model for the first time and I find this situation.</p>
  
  <ol>
  <li><p>My accuracy on the training set, with the default parameters (as in 
  <a href="https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html" rel="nofollow noreferrer">https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html</a> ) is very high, 0.95 or more , which looks a lot like overfitting. On the test set, accuracy goes to 0.66. My goal would be to make the model less overfitting, hoping to improve performance on the test set.</p></li>
  <li><p>I tried to perform 5-fold cross validation, using random grid search like here ( <a href="https://towardsdatascience.com/hyperparameter-tuning-the-random-forest-in-python-using-scikit-learn-28d2aa77dd74" rel="nofollow noreferrer">https://towardsdatascience.com/hyperparameter-tuning-the-random-forest-in-python-using-scikit-learn-28d2aa77dd74</a> ) with the following grid:  </p></li>
  </ol>
  
  <pre class="lang-py prettyprint-override"><code>n_estimators = [16,32,64,128]
  max_features = ['auto', 'sqrt']
  max_depth = [int(x) for x in np.linspace(10, 110, num = 11)]
  max_depth.append(None)
  min_samples_split = [2, 5, 10]
  min_samples_leaf = [1, 2, 4]
  bootstrap = [True, False]
  random_grid = {'n_estimators': n_estimators,
                 'max_features': max_features,
                 'max_depth': max_depth,
                 'min_samples_split': min_samples_split,
                 'min_samples_leaf': min_samples_leaf,
                 'bootstrap': bootstrap}
  </code></pre>
  
  <p>The best model had an accuracy of 0.7 across the folds.</p>
  
  <ol start="3">
  <li>I used the best selected parameters in step 2 on the training set and test set, but again accuracy on training set was 0.95 and test set 0.66.</li>
  </ol>
  
  <p>Any suggestion ? What do you think is going on here ? How can I reach the result to avoid overfitting ( and maybe improve model performance ) ?</p>