title: What is the Optimal Parameter Distribution for tuning Hyperparameters using random search
body: <p>The machine learning algorithms from SK-learn take a huge amount of parameters, and I am having a real hard time finding out which interval I should assign for the Random Search tuning algorithm. I am currently using 7 different ML algorithms (SVM, LDA, LR, DT, GBC, KNN), but let us take Random Forest as an example.</p>
  
  <pre><code>X, y = df.something().....
  from sklearn.ensemble import RandomForestClassifier
      param_dist = {"criterion": ["gini", "entropy"],
                     "max_features": ['auto', 'sqrt', 'log2'],
                     "max_depth": randint(1, 100),
                     "min_samples_split": randint(2, 150),
                     "min_samples_leaf": randint(1, 150),
                     "max_leaf_nodes": randint(2, 100),
                     "n_estimators": randint(2, 200)}
      model = RandomForestClassifier()
      ts_rs = run_randomsearch(X, y, model, param_dist, cv=5, n_iter_search=100)
  </code></pre>
  
  <p><strong>Overall</strong>: How do I assign the best parameter distribution to search within?
  <strong>More specifically</strong>: How do I find out whether its better to search for the max-depth in [1:10000], rather than, lets say; [1:10]? Can anybody recommend any optimum? (maybe even with a source) Currently, this is the closest i get: <a href="https://stackoverflow.com/questions/36107820/how-to-tune-parameters-in-random-forest-using-scikit-learn">How to tune parameters in Random Forest, using Scikit Learn?</a></p>