user224: After decades away from tournament play, I decided to play in an open tourney my child entered. Trying to gauge how my game might have changed during my time away, I found an online rating estimator at Chess Maniac. I was surprised to find that it estimated me 500 points stronger than my (USCF) rating - which actually went down about 60 points at the tournament. So, how accurate is this estimator?
Edit: I realize the Chess Maniac page in question is just an estimator. I'm looking for answers indicating that it tends to overrate or underrate players and by how much. If it doesn't have a tendency one way or another, I'd like to know that.

user11629: As I wrote in a comment above, I also believe that it is unrealistic for many reasons.
You could also think about it from a technical point of view. How would you program the evaluation function of that website (No idea whether that code is public or not.)?
The best you could do I believe is that you have each of the ten positions analyzed by an engine. Then you match the entered moves to the engine evaluation.

In the simplest case you just accept the best possible move and assign something like 200 ELO points to each correct answer (so as to cover  roughly 2000 ELO point range from beginner level to World Champion). So you have a huge error just from the method used (not including all the other factors I mentioned in the comment).
If you want to do it a bit more fancy you could also accept second, third, etc. best moves and give ELO points for them, perhaps weighted (with an arbitrary weight). As you can see, all of this seems a bit random.


In summary a few points not related to the technical difficulties mentioned above:

The ELO system is made to measure the relative strength of players, not the absolute strength (which might not even exist) as this site is trying to achieve. So by design, the ELO rating only tells you how strong you are relative to all other ELO rated players, not how strong you are overall.
Players might have the same rating but completely different strength and weaknesses. How do you expect to measure all the different factors with just 10 puzzles?


user825: Absolutely inaccurate. ELO rating is determined by playing a whole game of chess, not by solving a few positions without any pressure.
Please play some online games for better estimate of your playing strength.

224: <p>After decades away from tournament play, I decided to play in an open tourney my child entered. Trying to gauge how my game might have changed during my time away, I found an <a href="http://www.chessmaniac.com/ELORating/ELO_Chess_Rating.shtml" rel="nofollow noreferrer">online rating estimator</a> at <a href="http://www.chessmaniac.com/" rel="nofollow noreferrer">Chess Maniac</a>. I was surprised to find that it estimated me 500 points stronger than my (USCF) rating - which actually went down about 60 points at the tournament. So, how accurate is this estimator?</p>

<p>Edit: I realize the Chess Maniac page in question is just an estimator. I'm looking for answers indicating that it tends to overrate or underrate players and by how much. If it doesn't have a tendency one way or another, I'd like to know that.</p>
 11629: <p>As I wrote in a comment above, I also believe that it is unrealistic for many reasons.</p>

<p>You could also think about it from a technical point of view. How would you program the evaluation function of that website (No idea whether that code is public or not.)?</p>

<p>The best you could do I believe is that you have each of the ten positions analyzed by an engine. Then you match the entered moves to the engine evaluation.</p>

<ol>
<li><p>In the simplest case you just accept the best possible move and assign something like 200 ELO points to each correct answer (so as to cover  roughly 2000 ELO point range from beginner level to World Champion). So you have a huge error just from the method used (not including all the other factors I mentioned in the comment).</p></li>
<li><p>If you want to do it a bit more fancy you could also accept second, third, etc. best moves and give ELO points for them, perhaps weighted (with an <em>arbitrary</em> weight). As you can see, all of this seems a bit random.</p></li>
</ol>

<hr>

<p>In summary a few points not related to the technical difficulties mentioned above:</p>

<ul>
<li><p>The ELO system is made to measure the <em>relative strength of players</em>, not the absolute strength (which might not even exist) as this site is trying to achieve. So by design, the ELO rating only tells you how strong you are relative to all other ELO rated players, not how strong you are overall.</p></li>
<li><p>Players might have the same rating but completely different strength and weaknesses. How do you expect to measure all the different factors with just 10 puzzles?</p></li>
</ul>
 825: <p>Absolutely inaccurate. ELO rating is determined by playing a whole game of chess, not by solving a few positions without any pressure.</p>

<p>Please play some online games for better estimate of your playing strength.</p>
