|Board Game Geek: Why So Narrow?
||[Jul. 9th, 2011|12:20 pm]
Suit up, son! You're going to Mars!
boardgamegeek.com is the go-to site for information about board games—the IMDB analogue, if you will. So, as you probably know. |
Like many sites of this ilk, this site invites users to rate games on a scale of 0 to 10 stars, and shows the average rating for each game. Take a look at the most voted-on games published in 2010. Note that these are the games are not necessarily the best games; they are the games with the largest "rating sample size." Presumably "rating sample size" has a lot to do with popularity, so we would expect the ratings to skew high here.
But the thing that strikes me about this list is the density of the user ratings. The ratings for the first 10 range from 7.14 to 8.00. The first 100 range from 6.20 to 8.47. That seems like a pretty narrow band to me. It's also striking that the rankings don't get above 9.0. (A search of all games with 100 or more votes shows that the highest all-time average rating for a standalone game is 8.86.)
An interesting question in my mind is "What's the variance?".
That said, I think my BGG ratings (few in number as they may be) skew high (6-10) because I don't bother rating stuff like Munchkin that I hate with a passion.
I think the ratings density probably has to do with something I've noticed in video game reviews and a variety of other places as well (like Yelp, etc.), which is that people almost never use the entirety of a ratings scale.
The video game rating community seems to have informally come to a consensus that a 7-8 is the bar for a "good" game, just like Yelp has a 4 star average as a bar for being a "good" local business.
What's interesting to me is that on Board Game Geek and in at least some video game review venues, the ratings are largely informed, but they're constrained to a very narrow band of the scale (10s are mostly forbidden, because that's "too much" praise, while anything less than 6 is anathema), while in the context of less-informed reviews, like Yelp and Amazon, ratings only seem to exist on the extremes of the scale - either something is Teh Best EVAR! or it's irredeemable crap (0 or 1 stars).
I keep meaning to post about this at more length, but it's interesting to me to observe the lower bound on game ratings as well - you have to get to stuff like Candyland and Snakes and Ladders to drop much below 5 on BoardGameGeek.
I think there's at least two different things going on here. First, even if the average for a game is only 7 or 8, that doesn't mean nobody gave it a 10; the average will always be lower than the max except for the uniformly belovedly perfect game.
Second, not all distributions are necessarily uniform. There's that "how much pain are you in?" scale that people get asked in the ER - if you polled everyone in the world, the average amount of pain they were in would probably be pretty low. So if it's "rate this game, from the worst game you can imagine, to the best game you can imagine" quite possibly most games that get published are closer to the latter than the former.
(I did think once I should figure out what my own distribution for book reviews was, out of curiosity.)
First, I'll note that if you do the same analysis for all games in their database, not just those published in 2010, still sorting by number of voters, the first 100 range from 4.50 (Monopoly) to 8.31 (Through the Ages). Even if I insist on a more recent cutoff date, the lowest is still pretty low, with Fluxx (1997) at 5.72 and Munchkin (2001) at 6.03.
If you look on the pages for individual games, they have a histogram of how many people gave it each rating. The distributions for the most reviewed games have a pretty consistent pattern that markedly deviates from a bell curve shape. I don't know what the mechanism is, but it looks almost as if the majority of voters are actually rating on a scale from 0 to 8, and a substantial minority are using a scale from 6 to 10. The first group's effect is quite pronounced on the very good games, in that the bars at 9 and 10 are way too short, and the other group becomes more important on the bad games, where the 6 bar is way too tall.
Obviously there are other stories that could just as accurately describe the distortions; I don't especially believe in the actual existence of those two groups. Whatever it is, the low tail is usually much fatter than the high tail.
The story for your 2010 games is different, though. There are some of them with that pattern with the fat low tail and skinny high tail, like Forbidden Island, Fresco, or Space Hulk Death Angel Food Cake: The Card Game, but some of them are a lot closer to bell curve shapes (7 Wonders, Troyes) or even have fat high tails and skinny low tails, like Dominant Species and Runewars.
What would be interesting to track would be how the rating of a game has evolved since its publication. I've noticed on IMDB a tendency for movies ratings to spread out a lot once they've been out for a few years. It seems like the people who go see a movie in the theatre always give it an 8, and only people who watch it on DVD are willing to give 4s or 10s.