Wednesday, February 27, 2013

EXP Podcast #213: Data-driven Reviews

Image from Wikipedia

- Subscribe to the EXP Podcast via iTunes
- Find the show on Stitcher
- Here's the show's stand-alone feed

Show notes:

- Runtime: 32 min 22 sec
- KnobFeel: Reviews of receiver knobs
- Music by: Brad Sucks


  1. There's already been some use of metrics in this way. Off the top of my head I can recall an IGF nominee who stirred up a minor internet fuss who mentioned that his submission has no plays. I'm almost certain that this has happened with a publisher rebuking reviews this way but I can't recall the reviewer or the game.

    I think it's important for a reviewer to be open about their methodology, but too often not finishing a game or playing in a certain way is used to silence critics unfairly.

    If we're talking about consumer reviews then we need to consider the reality of the consumers relation with video games. The review should reflect how players will interact with the game. This includes the fact that most people don't finish games. Many games are very long and if a game fails to engage a player early then that lowers its value to a consumer (and thus its rating).

    Maybe a game does improve significantly based on how (or how long) you play, but that's only relevant academically (or what you referred to as critic vs review).

  2. Well said. As always, data cuts both ways: everyone is afraid of big brother, but it's also empowering to the little guys. If that IGF nominee's story is true, I'm all for keeping the judging system honest.

    Ultimately, regardless of how reviews change, transparency regarding the process is going to be paramount.