Take the 2-minute tour ×

Let's say I have a list of 100 MLB pitchers and 5 statistics for each of them. The difference between, for example, an ERA of 3.5 and 3.1 might not look like a lot to a naive similarity algorithm, but is a lot in baseball. Given that a lot of the player statistics that I'm looking at have this little variance, a lot of a little variance like this, what is the best way to calculate similarity between two players?

An example of the data might look like this:

Player | ERA | Wins | Strikeouts
A      | 3.5 | 15   | 180
B      | 3.1 | 12   | 210
C      | 3.4 | 13   | 150

I've used cosine similarity, and the results are all too similar, down to a thousandth of a decimal place.

share|improve this question
You might want some normalized Euclidean distance. What does ERA represent? Is it Gaussian, multinomial, ...? –  larsmans Jan 16 '12 at 19:05
You might get better answers at statistics stackexchnage –  amit Jan 16 '12 at 19:43
I don't believe you have enough info to start making comparisons. I believe that 'phs' is on the right track by saying 'normalize' but I might start with Ks per inning or game, wins vs loses, etc. then look at variance, skew, kurtosis and after that the type of distribution, and, Oh, find some original data. –  oaxacamatt Jan 18 '12 at 1:41
add comment

2 Answers

up vote 2 down vote accepted

Normalize each feature before comparing examples.

So for each column in your dataset, calculate the mean and range (width). Then subtract the mean off and divide by the range. If you have a lot of outliers, divide by the standard deviation instead.

share|improve this answer
add comment

I would use some probability based statistics to compare. Best places to start are:

http://en.wikipedia.org/wiki/Analysis_of_variance : Most of the methods here are parametric.

http://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test : Example of a non-parametric method

share|improve this answer
add comment

Your Answer


