esports
Curious to see how the rankings are done? Here's the full step-by-step process.
Determining the SSBMRank nominees
The list of SSBMRank nominees comprised initially of players who made the top 100 last year and players who placed a certain placing in the larger events. In addition, community members could nominate players with an application process showcasing their 2017 achievement. In total, 295 players qualified into the initial funnel.
From here, a small pre-panel of seven panelists flagged players that either didn’t have enough 2017 activity or didn’t have enough solid wins/results. Any player with three or more flags were removed from the list and certain players were further audited to see whether they made the activity requirement. In the end, this left roughly 126 players on the ballot for the larger panel to rate.
Panelists
For this year’s rankings, we selected roughly 60 panelists to rate the players. The panelists were picked for their activity in their community and we aimed to achieve a healthy balance of top players, community leaders, tournament seeders and regional power rankers. Furthermore, a strong effort was made to ensure that all regions had fair representation in panelists. The list of panelists can be found here.
The voting algorithm
Here are the set of instructions given to panelists:
Based on quality and quantity of results in 2017, rate each player on a scale of 1-10. Imagine if everyone entered a tournament, one hundred times. Who would perform the best?
- 10 = Best Player on the list, 1 = Worst Player on the list (decimals are okay)
- Please fill out a minimum of 100 players, if you don't know a player well enough, then leave blank
- Start Period: Eden, Jan. 9, 2016
- End Period: General Rankings Dec. 10, 2017; Last edits to Top 30, Dec. 17, 2017
Post-ballot audits
Since some panelists don’t always follow instructions, we audit the ballots for errors (Is PPMD a 5.9 or 9.5?) and renormalize the ballots just in case they do not scale the ballot properly (ex. Rating the worst player in the list as a 6 instead of a 1). Lastly, we remove the 3 worst scores and 3 best scores from a player’s ratings and then take the average of the remaining scores for a player’s overall rating.
