We evaluate the rating system of "Heroes of Newerth" (HoN), a multiplayer online action role-playing game, by using statistical analysis and comparison of a player's in-game performance metrics and the player rating assigned by the rating system. The datasets for the analysis have been extracted from the web sites that record the players' ratings and a number of empirical metrics. Results suggest that the HoN's Matchmaking rating algorithm, while generally capturing the skill level of the player well, also has weaknesses, which have been exploited by players to achieve a higher placement on the ranking ladder than deserved by actual skill. In addition, we also illustrate the effects of the choice of the business model (from pay-to-play to free-to-play) on player population.