Q:

Choosing right range for data while using scikit-learn

I have a dataset with 1175 examples and 21 features which are in the range of [-1, +1], and two class labels 1 and 0. As I read in the most of the resources, it is good to have data in the range of [-1, +1] or [0, 1]. So I thought I don't need any preprocessing. But when I run SVM and decision tree classifiers from scikit-learn. I got 100% accuracy using cross-validation with 10 folds.
Afterwards, I tried to multiply the data by 10, 100, 1000 etc. Every time I increase the number, I get less accuracy. For example I got 91% for multiplying with 100. On the other hand, the decision tree classifier stays the same for all multiplications.

A:

From the sklearn docs:

Support Vector Machine algorithms are not scale invariant, so it is
  highly   recommended to scale your data. For example, scale each
  attribute on the input vector X to [0,1] or [-1,+1], or standardize it
  to have mean 0 and variance 1. Note that the same scaling must be
  applied to the test vector to obtain meaningful results. See section
  Preprocessing data for more details on scaling and normalization.

Decision trees though, are scale invariant, and in fact, are invariant under any monotonic transformation of the features.
To see why, notice that the default kernel, the rbf kernel, contains a scale parameter $\gamma$:
$$ exp( -\lambda |y - x|^2) $$
The $\lambda$ parameter controls the width of the kernel, so the scale is built in right there.  You can pass a $\lambda$ into the svm as a parameter to the fit function, and by choosing it appropriately you can probably recover your initial results with all the features re-scaled.
On the other hand, the decision tree is just looking for optimal splits between data points, and the concept of a split is not dependent on scale, only ordering.

