Interview with ratetheplayers.com's Christopher May

I can tell you a great deal about the Chicago Fire and how individual players perform but sometimes it's difficult to get an accurate snapshot for players on other teams.  In addition less sources and media coverage existing for MLS players than other American sports and other soccer leagues, soccer has a very limited stat sheet.  For those reasons and more, my interest was piqued when I saw a new website pop up on the radar - http://ratetheplayers.com.  Rate the Players was created by Christopher May with the idea that the fans know the players the best and if a large amount of fans were rating the players on a regular basis, we all could get a much more accurate idea of how players were currently performing around the league if not around the world.  I was fortunate enough to ask Christopher some questions and obtain more insight into the website.  Our interview is below with my questions in bold and his answers following. 

1. Could you tell us a little bit about yourself and provide a brief personal soccer history?

I never played soccer beyond AYSO focusing on other sports. My younger brother played at a competitive level, which got my dad very interested in the professional game. He had convinced me to go to some games when the Fire was playing in Naperville, and we ended up taking a trip to London to see some matches. But I didn't get hooked until Comcast screwed up my cable box and I started getting Fox Soccer for free. I started recording the games and highlight shows and when they finally got my account updated and took away the channel I added it back to my package. I've been a Fire STH now for 4 years and regularly follow the European leagues as well.

2. What was your inspiration behind starting http://ratetheplayers.com/?

I think soccer is sport that is inherently impossible to evaluate through objective statistics. Things like positioning, creativity, defense, making good runs, tracking back etc. don't show up in any players stat sheet. A diving finger-tip save is the same as picking up slow roller. They are both a Shot On Goal for the player taking the shot and a save for the keeper. But in reality, they were very different events.

More with Christopher May of Rate the Players after the break

Even if the match you are watching has detailed stats (which I don't see for MLS games) like passes attempted vs. passes completed, that doesn't really tell the story. 20 square passes across the back are not as valuable as 1 through ball to create a chance on goal.

I believe it's because of this inherent inability to "tell the story" of a game via traditional statistics that caused the 1-10 ratings system to start getting used for soccer matches by the writers. I'd see people posting their player ratings on the various forums, and it felt very limited and awkward to me. Based on these ratings, people get into discussions: This guy was a 6.5... no he sucked he as a 4 at best etc. I thought, wouldn't it be nice to be able to say "Ok, YOU thought he was a 6.5, but on average your fellow fans thought he was only a 4.8."

Aside from that I have always found it very frustrating that when talking about goals scored/conceded it seems like the conversation begins and ends with the goal scorer and keeper respectively. A keeper's "goals against average" is almost always used as a metric for how good a goal keeper is, but I would guarantee you that if you took the top GAA keeper and had him swap teams with the bottom GAA keeper, their respective stats would change drastically. That's because bad defensive teams tend to give up lots of goals, and it often isn't the keepers fault.

Think of this situation: a keeper starts a quick counter attack with a perfect 30 yard throw to a midfielder making a run. After beating a few defenders, he gets fouled in the box. A striker steps up and converts the PK. Does the striker really deserve most of the credit for that goal? Does the keeper really deserve most of the blame? Of course not. This is why we have our users assign credit/blame for each goal, so the attacking team fans could say 30% credit to the keeper, 60% credit to the midfielder, and 10% to the PK taker. And the defending team can assign 20% blame to the guy who missed the first tackle, 10% to the keeper and 70% to the player who committed the foul in the box.

3. What was your biggest challenge in getting the website setup?

There were 2 major challenges.

1) Finding time to work on it. I've rarely been as busy with my professional career as I have been these last few months, and trying to divert time away from client work to focus on advancing RateThePlayers.com, with a quickly approaching MLS First Kick, was extremely difficult. The amount of hours I was (and still am) putting in each was crazy. Couple that with the fact that I have a 15 month old baby boy at home, and I'm trying to take care of him as well and it's added up to a very hectic, busy and stressful period.

2) Getting data integration to work correctly. It's impossible for me to enter all the data manually, so I'm paying out of pocket for a 3rd party data service where I can programmatically access real time information about players, teams, matches etc. The problem has been that their data has not been extremely accurate in some respects. The 2 new MLS teams were missing. Players who retired are still shown. Players who switched teams are still with their old team. Players with the same, or similar name, are assigned to teams instead of the correct player. For example, for a while the Josh Wolff who was assigned to DC United was a guy from Europe, not the American Josh Wolff. So once I point that out and they correct it, now I end up with duplicates in my database as DC United now has 2 Josh Wolffs. Aside from these errors, just getting the system to the point where it's automatically updating everything as new data becomes available was more challenging than I had anticipated.

4. Player ratings can be fairly complex for the uninitiated. What steps do you recommend for individuals who are completely new to player ratings? Is this something anyone can do in a short amount of time?

There is nothing too complicated about it. I think the main thing that users might not understand is something like "What exactly is an 8.0 performance"? If you had no prior knowledge you might think, well 8/10 is like a "B" in school so maybe a few of the better players deserve an 8? But that isn't how the 1-10 scale typically works. It's setup more like a bell curve. 1s and 10s are unheard of, extremely rare, and even 2s and 9s are so infrequent that if a player received one of these it would be very rare occasion.

Typically a 5 is "average", 7s are usually candidates for Man of the Match, and 3s are usually the "Failure of the Fixture".

For more detailed explanation of the ratings I've created this page to help users understand what we are looking for:

http://ratetheplayers.com/Explainratings.aspx?x=all

5. The site has been up for about a week now.  What is the most unexpected thing you are finding in the short time people have been rating players?

Without question the most interesting thing in my mind is that, while the average player rating has been a 5.6, the average referee rating has been a 5.9 and on top of that the average rating for the assistant referees (linesmen) has been a 6.3!

I think this is both amazing and phenomenal! It's amazing because I fully expected that fans would take out their frustration on the ref and blame a call that didn't go their way for their bad result. It's phenomenal because it means that our users being fantastically objective in their ratings, which is great! It also might indicate that our site has attracted a more sophisticated, experienced fan who can understand the situations refs find themselves in. I'm really excited about this.

6. All sites launch with the founder thinking in the back of their head 'I wish I could have added this...', what do raters have to look forward to as the site evolves?

Oh there are so many additional things I want to do but I knew I wouldn't have time to get them in place before the start of MLS. In the near term we are adding the ability to view individual rating submissions and also we are adding more stats to the match page, as well as adding team, player and league stats.

The first step has been to get the average ratings and stats for each match. Now, we want the users to be able to page through all the ratings and see how each user felt about each player, and also of course read their comments and maybe comment on them as well.

In terms of adding additional stats there are lot of things we are collecting data for now in order to show these stats later. For example, at the player level, how has player X's rating changed from week to week with a line graph. Or how does his average player rating change when he's a starter vs. a sub? Home vs. away? When he scores vs. not scoring? Etc. At the team level, people want to see who are the top rated players (on average) on my team? Who's been credited (or blamed) the most for goals so far? Which players on my team put in the most consistent performances each week, and which players fluctuate between a great match and a terrible match? At the league level: who's been credited with the most goals? Which team feels the ref has been hardest on them? How do team ratings correlate to table position? Which team hates their manager the most?

In the future we are also going to be covering the major European leagues and international competitions. I'm going to try to make the Champions League available starting with the next round, and lots of people have asked about the Gold Cup, so I'm going to try to get that available as well.

7. Is there any piece of advice you have for expert and novice raters alike when it comes to using the site?

My 1 suggestion is to just be fair in your ratings. Don't give the ref a 1 just because he missed a call or 2. Don't give Beckham a 10 because he's dreamy. Don't give Sean Johnson a 2.0 because you really miss Andrew Dykstra. J You get the idea.

And also, this site has only been online for 1 week: there are going to be some bugs/issues along the way. So if you run into any problems please send me an email at chris@ratetheplayers.com and I'll try to resolve the problems as soon as possible.

Finally, of course, tell your friends who are soccer fans. The Fire is well represented but some of the less popular teams need some supporters (I'm talking about Portland, Chivas USA and Colorado Rapids). The more people submit ratings the more information we'll have to crunch.

Thanks again to Christopher for all that great information.  I just signed up myself and you can follow Rate the Players on Facebook and Twitter.  I personally am very excited to see this concept unfold and we wish all the best.