{Arrest This Man, He Talks In Maths } spacer

Blog : Archives : Homepage

With your feet in the air, and your head on the ground . . .


{Wednesday, April 26, 2006}

For all the Bill James, Billy Beane, Moneyball talk about optimally predictive baseball statistics, I've still never heard of this conceptually simple, but computationally intensive statistic:

First, consider all the at-bats in all the major league games ever played: at the time of a given at-bat, there's a game condition, which can be neatly summarized by 3 simple measures: Current difference in score, current inning & number of outs, and current positioning of runners on base (e.g. leading by 2, 4th inning with 1 out, a runner on second.) You can add more variables if you want (also increasing degrees-of-freedom) but I think this is a good minimal set of measures. Okay, now let's call these measures D (difference), O (outs), and R (runners). For each triple {D,O,R}, there's an associated probability that the team will win the game, in the end; we'll call this p(W|{D,O,R}). To create a 'look-up table' for this stat, we need to crunch through every at-bat in MLB history (or the last 50 years, or whatever) and calculate the probabilities.

Okay, so: When the batter comes up to the plate, the team's chance of winning is p(W|{D,O,R}i), with 'i' denoting the fact that this is the 'initial' state. We can look the value up in our table. The result of the player's at-bat is p(W|{D,O,R}f); the team's chance of winning has changed, and taken on some new value. The player's contribution to the team, therefore, has been p(W|{D,O,R}f)-p(W|{D,O,R}i). Simple!

The two best things about this stat are:

(1) It measures 'clutch peformance' in a really meaningful, and nicely continuous way. If A-Rod hits lots of HRs, but really does actually tend to hit them in the late innings of blowouts, he will get a lower 'score' than you'd expect from the stats we usually look at; if a guy is an awesome situational hitter, drawing walks when he should (to best help the team), getting down the sacrifice bunt, etc, he'll get a higher score.

(2) It's directly interpretable in terms of a player's expected impact on his team, in terms of wins & losses. Really, what more could you want from a stat?

The way I described it is really a slight over-simplification. Players should get 'points' for anything attributable to them as an 'action'; if a guy steals a base, he's changed the {D,O,R} state, and should get credit for it. Like, Dave Roberts would have gotten a massive p(W|DOR) jolt for stealing that base off of Rivera & Posada in game 4 a few years ago, even though he didn't even have an AB in the game. Better yet, you can apply the stat equally well to pitchers: each batter a guy faces results in a change in p(W|DOR). Using this stat, in fact, might be a cool way to resolve the debate about how important closers are relative to starters, and who the best closers are; it would be ideal, since it provides such a direct measure of 'clutch' performance, and closer is the position where that matters most consistently.

One cool potential addition: add measures for 'games left in the season' and 'number of games in or out of the playoffs', to get a handle on clutch performance on a longer timescale. You wouldn't build these factors into the main look-up table; instead you'd just use them to weight the player's d(p(W|DOR)): if you hit a game-winning HR for a team 25 games out, in August, that's less meaningful than bunting a guy over to third for the first out in the 9th when you're down by one, for a team just a half game up in the wildcard race, in mid-september.

The only disadvantage of this stat, that I see, is the fact that it gives so much weight to clutch performance. If you think psychology has no impact on the game at all, and situational hitting ability is homogeneous across the leage, then you'd take an A-Rod HR late in a blow-out as just as predictive of his chances of helping the team in a close game, regardless of the situation. Turned around, you don't want to make the Type 1 error of thinking apparently 'clutch' contributions are more predictive than they actually are. However, players have so many ABs in a season, and in a career, than with a continuous stat like this, I don't think 'spurious' clutch performances will affect things too much: there's not enough noise.

I'd love to actually compute this stat. I wonder who has the necessary database, coded the right way. I figure it's got to exist.

posted by Miles 3:34 PM

You are a mad friggin' scientist, my man. Genius! How often do you update the database (and thusly, the probability distribution)? Once a year, once a month, continuously? Do retired players get their performance retroactively graded against the current distribution, or against the one that existed when they played? I can see some really funky numbers for the early seasons of pro ball just due smaller sample size. I think you'd want to look at the evolution of the distribution over smaller hunks of time (2, 5, 10 years at a time) to see if there are significant differences between eras. This might tell you if you want to keep the giant database of all at-bats ever as your source data, or use something like a sliding 5-year database to "keep up with the times."
Post a Comment