<p>In <code>scikit-learn</code> there are 3 classes that share interface: <em>Estimators, Transformers</em> and <em>Predictors</em></p>
<p><strong>Estimators</strong> have <code>fit()</code> function, which serves always the same purpose. It estimates parameters based on the dataset.</p>
<p><strong>Transformers</strong> have <code>transform()</code> function. It returns the transformed dataset. Some Estimators are also Transformers, e.g. <code>MinMaxScaler()</code></p>
<p><strong>Predictors</strong> have <code>predict()</code> function, which returns predictions on new instances, e.g. <code>KNeighborsClassifier()</code></p>
<p>Both <code>MinMaxScaler()</code> and <code>KNeighborClassifier()</code> contain <code>fit()</code> method, because they share interface of an <strong>Estimator</strong>.</p>
<blockquote>
<p>However, there are cases when there is no 'learning' involved with <code>fit()</code></p>
</blockquote>
<p>There is 'learning' involved. Transformer, <code>MinMaxScaler()</code> has to 'learn' min and max values for each numerical feature.
When you call <code>min_max_scaler.fit(X_train)</code> your scaler estimates values for each numerical column in your train set. <code>min_max_scaler.transform(X_train)</code> scales your train set based on the estimations. <code>min_max_scaler.transform(X_test)</code> scales the test set with the estimations learned for train set. This is important to scale both train and test set with the same estimations.</p>
<p>For further reading, you can check this: <a href="https://arxiv.org/abs/1309.0238" rel="nofollow noreferrer">https://arxiv.org/abs/1309.0238</a></p>
Sklearn - fit, scale and transform
<p>The <code>fit()</code> method in <code>sklearn</code> appears to be serving different purposes in same interface.</p>
<p>When applied to the training set, like so:</p>
<pre><code>model.fit(X_train, y_train)
</code></pre>
<p><code>fit()</code> is used to learn parameters that will later be used on the test set with <code>predict(X_test)</code></p>
<hr />
<p>However, there are cases when there is no 'learning' involved with <code>fit()</code>, but only some normalization to transform the data, like so:</p>
<pre><code>min_max_scaler = preprocessing.MinMaxScaler()
min_max_scaler.fit(X_train)
</code></pre>
<p>which will simply scale feature values between, say, 0 and 1, to avoid some features with higher variance to have a disproportional influence on the model.</p>
<hr />
<p>To make things even less intuitive, sometimes the <code>fit()</code> method that scales (and already appears to be transforming) needs to be followed by further <code>transform()</code> method, before being called again with the <code>fit()</code> that actually learns and builds the model, like so:</p>
<pre><code>X_train2 = min_max_scaler.transform(X_train)
X_test2 = min_max_scaler.transform(X_test)

# the model being used
knn = KNeighborsClassifier(n_neighbors=3,metric=&quot;euclidean&quot;)
# learn parameters
knn.fit(X_train2, y_train)
# predict
y_pred = knn.predict(X_test2)
</code></pre>
<hr />
<p>Could someone please clarify the use, or multiple uses, of <code>fit()</code>, as well as the difference of scaling and transforming the data?</p>