user606: I'm basically fishing for suggestions here... there is an AI contest to figure out a person's rating from a single game. The contest's description mentions Ken Regan's approach that makes use of engine evaluations of players' moves (and the contest's accompanying game data is pre-equipped with engine evaluations for such use). What other kinds of factors do you believe would be useful for such a task?
I could imagine things like:

Ability to follow recent opening theory (for strong players)
Total number of blunders (for weak players)
At what point in the game did resignation occur? (for strong players, it often evaluates to numbers closer to a draw than for weak players)
Average depth of tactical traps avoided

Other ideas?

user4362: I also try to take part in that competition, although I'm not sure I will be able to spare the time and computational effort to actually make a submission. 
Unfortunately I think the Kaggle data set of engine evaluations isn't suitable to get the best possible result. I think it is crucial to not only have the evaluation of the best move, but also of the inferior choices. 
The reason is that you need to calculate some kind of measure of the complexity of the position to weight the accuracy shown by the players. 
In short: It is hard to be accurate in a complex position and easy to be accurate in a simple position.
Another problem is that at a runtime of one second per move, stockfish probably isn't much stronger than the average player in the dataset (which would be around 2200 Elo) …
So to create a decent dataset one would have to invest a lot of cpu-hours. 50000 games x 40 moves x 1second (or more) >= 555 hours.
To create an opening tree that allows you to see when the players deviate from already played games and how strong the players of the played games were, is another idea to give an initial guess of the players strength.
So my rough outline is to create an opening tree to get an initial guess of the strength, ignore the moves played in the opening, weight the accuracy of the moves after the opening phase with the complexity of the position and do some kind of linear regression analysis. Maybe look out not to overvalue time trouble blunders.
I think the key is to get a good "complexity number". The other possibilities you mention … "number of blunders" just reduces "accuracy of play" to something similar with less information. "Depth of tactical traps avoided" is tricky and may computationally be even more expensive.

606: <p>I'm basically fishing for suggestions here... there is <a href="http://www.kaggle.com/c/finding-elo" rel="nofollow">an AI contest to figure out a person's rating from a single game</a>. The contest's description mentions Ken Regan's approach that makes use of engine evaluations of players' moves (and the contest's accompanying game data is pre-equipped with engine evaluations for such use). What other kinds of factors do you believe would be useful for such a task?</p>

<p>I could imagine things like:</p>

<ol>
<li>Ability to follow recent opening theory (for strong players)</li>
<li>Total number of blunders (for weak players)</li>
<li>At what point in the game did resignation occur? (for strong players, it often evaluates to numbers closer to a draw than for weak players)</li>
<li>Average depth of tactical traps avoided</li>
</ol>

<p>Other ideas?</p>
 4362: <p>I also try to take part in that competition, although I'm not sure I will be able to spare the time and computational effort to actually make a submission. </p>

<p>Unfortunately I think the Kaggle data set of engine evaluations isn't suitable to get the best possible result. I think it is crucial to not only have the evaluation of the best move, but also of the inferior choices. 
The reason is that you need to calculate some kind of measure of the complexity of the position to weight the accuracy shown by the players. </p>

<p>In short: It is hard to be accurate in a complex position and easy to be accurate in a simple position.</p>

<p>Another problem is that at a runtime of one second per move, stockfish probably isn't much stronger than the average player in the dataset (which would be around 2200 Elo) …</p>

<p>So to create a decent dataset one would have to invest a lot of cpu-hours. 50000 games x 40 moves x 1second (or more) >= 555 hours.</p>

<p>To create an opening tree that allows you to see when the players deviate from already played games and how strong the players of the played games were, is another idea to give an initial guess of the players strength.</p>

<p>So my rough outline is to create an opening tree to get an initial guess of the strength, ignore the moves played in the opening, weight the accuracy of the moves after the opening phase with the complexity of the position and do some kind of linear regression analysis. Maybe look out not to overvalue time trouble blunders.</p>

<p>I think the key is to get a good "complexity number". The other possibilities you mention … "number of blunders" just reduces "accuracy of play" to something similar with less information. "Depth of tactical traps avoided" is tricky and may computationally be even more expensive.</p>
