The Use of Metacritic

Metacritic is becoming more and more the barometer of determining game quality. Is it a great scale to use?

Yes and No.

Using an average does give you a statistically better score than relying on one particular source. However, the review process metacritic does have issues:

* It is suspected by many that review scores are subject to many factors including advertising dollars spent with the publication by the publisher, or even how many free lunches the journalist got. Of course many journos can also get caught up in the hype machine and be swayed simply by that.
* Some reviewers like to be purposely harsh to get noticed, throwing in a contradictary opinion.
* Small scores like out of 5 skew results so 3/5 which is a fairly decent score when considered in that sense become 60% as a score out of 100, which is considered to be bad.
* Reviews without a score are attributed a score by metacritic themselves based on the content of the review, which is extremely subjective.
* Review scores are weighted according to the publication by metacritic themselves. This process is not transparent, and has no feedback to the public lending the sense that this is somewhat arbitrary.

So - is it reliable? Again - I think it does actually produce fairly decent results. The main problem is not from metacritic themselves, but the whole reviewing process. However, for all the corruption and one-up manship it generally does reflect the quality of a game.

Some publishers are now tying developer bonuses to metacritic scores. In a way I like this. It is a way of preserving the chances of a developer being paid that bonus - old style dirty tricks in this case (like purposely reducing the metacritic score) would simply be shooting themselves in the foot.

It's interesting to see that people have been analysing metacritic scores by publisher to show their quality bar:

http://kotaku.com/393104/the-big-publishers-metacritic-averages