<p>Based on the recent <a href="https://blog.dask.org/2019/09/30/dask-hyperparam-opt" rel="nofollow noreferrer">https://blog.dask.org/2019/09/30/dask-hyperparam-opt</a>, <code>HyperbandSearchCV</code> does require models implementing <code>partial_fit</code> because the point of using HyperbandSearchCV is to avoid training on the entire data in order to make a decision whether the model is good. This is where HyperbandSearchCV's speed advantage comes from. The way I interpret the blog post is that once a model is fully trained, HyperbandSearchCV cannot do anything more, there's no early-stopping left to do. However, this might be true for the Dask implementation and not necessarily for the Hyperband algorithm described in the original <a href="https://arxiv.org/pdf/1603.06560.pdf" rel="nofollow noreferrer">paper</a> which I should re-read.</p>
Does HyperbandCV and other incremental search algorithms work for models without partial_fit and fir pipelines?
<p>I have been deep diving on the github pages and reading the documentation, but I am not fully understanding whether HyperbandCV will be useful to speed up hyperparameter optimization in my case.</p>

<p>I am using SKLearn's pipeline functionality. And I am also testing models like LinearRegression() which doesn't support partial_fit; it has to use all the data to fit the parameters all at once. In this case, can HyperbandCV still be used? If it is used, what exactly is it optimizing if from my understanding neither Pipeline nor said models have partial fit implemented. In Hyperband's api, it reads that it needs to have partial_fit implemented in order to use it. However, in another documentation it reads it can be a drop-in replacement for RandomizedSearchCV since it just spends less time training low performing models. </p>

<p>If anyone can clarify this for me, this will be great. </p>