The Rally: Ranking the Rankings

This week in the Rally, Kamakshi Tandon and I talk rankings. What’s their significance, and which system is best? Kamaskshi starts us off.

Hi Steve,

Kim Clijsters seemed to set the rankings world right side up again last week when she finally got back to No. 1 again. But don’t look now, as Caroline Wozniacki could grab the top spot back this week by reaching the semifinals in Dubai, sending Clijsters back down to No. 2 despite her two Slams, year-end championships and Miami titles.

I always feel a bit uncomfortable in these situations, because on one hand the bandwagon complaining is getting tiring and it’s often misplaced. Wozniacki is No. 1 without a Slam. How ridiculous. There must be something wrong with the system.  

On the other hand, there actually is something wrong with the system, and this is the only time people really notice—when they disagree which who’s No. 1. So it’s tempting to pile on, hoping that it will result in the system being changed.  

But these No. 1 debates aren't really about rankings, they’re about who’s the best player. Wozniacki isn’t the best player, yet she was/is/will be No. 1. Yet the issue of who’s the best player is separate from the issue of what the best ranking system is.  

The ranking system isn’t really there to anoint ‘the best.’ It’s there to measure the players’ results and rank them to decide who should get into which tournaments, which means calculating who should be No. 67 every bit as much as who should be No. 1. But people don’t have opinions on who should be No. 67. They have opinions about who should be No. 1.  

This discussion is going to be about the right ranking system, not the right No. 1, which isn’t going to do wonders for our popularity since the discussion of the right No. 1 would definitely be more interesting and entertaining to most people. But who’s in charge here anyway? Besides, the two questions are related in the end, so have patience.  

For starters, we have to accept that when there’s parity, there are always going to be disputes about who should be No. 1. Every system looks a little suspect when you have Wozniacki, Jankovic, Safina—or Kafelnikov, Moya, Rios—sitting at No. 1. And you can have a system that’s completely crazy but as long as the right player is No. 1, it will hardly be noticed.  

As I said, I really do think the women’s system has problems, it’s just that the fact that a player can be No. 1 without winning a Slam isn’t one of them. And I think the men now have in place a system that is similar but actually even a little worse, only no one’s noticed because Roger Federer and Rafael Nadal have been so dominant. One of them was going to be No. 1 under any system during the past few years, but that doesn’t mean this one is the best for sorting things out in less clear situations, or that it’s as good at ranking No. 67.  

In the mid-90s, the women had a system that took an average of a player’s results, with a minimum of 14 tournaments (if you played two less, for example, you got two zeros in points). Women also got bonus points for beating top players. Clearly, this emphasized quality, though the 14-tournament minimum made players play a reasonable schedule.  

Then there was a year when someone took leave of their senses, and the WTA changed the system so players were ranked based on how many points they had accumulated during the past year, no matter how many tournaments they had played. It was called the Rolling Race, though I believe the institutional memory has pretty much been wiped clean on this fine period. Essentially, Steffi Graf’s total over 12 tournaments was measured alongside Amanda Coetzer’s total in 30 tournaments. This was a pure quantity system. The money list is another good example of this idea, and sometimes people suggest that as an alternative ranking system.  

Since then, the system has taken the total of a player’s best [14/18/16] results, meaning that you can choose to play the minimum tournaments without being at a disadvantage (as you would be under a quantity system), or you can choose to play a lot without getting hit for early losses (as you would under an averaging system).  

I think both averaging and Best XX work OK in theory, as long as the XX is a reasonable figure. But Best XX encourages a bigger schedule than the average system, which doesn’t matter much in rankings terms but is helpful for the tour in trying to get players to play more tournaments. It also means that some losses don’t count, which affects the results again. Every little change to the system affects the rankings that it produces. Admittedly in practice, the differences tend to be fairly minimal. But sometimes the difference is between No. 1 and No. 2, so it’s important to agree that whatever causes that difference is legitimate.  

These two systems get to the heart of the most basic issues in designing a system—do you measure the whole picture, or the high points? It’s like the difference between your average track time and your best track time—which says more about your running ability?  

A quick quiz for you to see where you are on the spectrum:  

1. One title = how many finals?  

2. One title = how many semifinals?   

3. One title = how many quarterfinals?  

4. One Slam title = how many Masters? (in achievement terms, not public value)  

I’ll give my tentative answers next, and talk about what the current systems measure and the results this produces.  

Kamakshi

<em>*</em>  

Hi Kamakshi,  

I’m glad I let you lead the way on this one, as you’re much better versed in the rankings system and its recent history than I am. When Caroline Wozniacki began at No. 1, I likened that accomplishment not to being the best player in the world, but being the best at her job. That's what the women's No. 1 has reflected much of the time recently. "Employee of the Year" is not quite as grand as being the called "The Greatest," but it's a mark of distinction nonetheless.  

Before I talk about which system I think is best, let me back up and say a few general things that come to mind when I think about rankings—like, how weird are they, anyway, how brutally definitive and hierarchical? We take them for granted by now, but what a strange thing it must be to, as Ilie Nastase said, “walk around with a number over your head.” Nasty thought some of the camaraderie went out of the game when the weekly computer rankings came in and separated the players one by one from each other—they really had been reduced to a number. But a lot of players thought otherwise. Instituting the ATP computer system in 1973 marked a shift in power away from the sport’s federations, who had always done the rankings with a degree of subjectivity, and toward the players themselves. Politics was out; meritocracy was in. You no longer had to be in the favor of any amateur official to, say, be put on the Davis Cup team or get invited to play Wimbledon. It was all about how you did on the court.  

I thought it was interesting that Andy Murray said a few years ago that he thought the ranking system was one reason the game itself had improved so much over the years. It made guys keep trying to climb over each other. It’s given players like Sampras and Federer and Nadal a clear goal. But it’s still kind of weird. Like the above commercial Federer did for ESPN a few years ago, where one of the anchors says, “You know what I like about tennis, are the rankings. I like to know exactly where you stand at all times.” See what I mean? It sounds a little ridiculous, and a little nerve-wracking, when you put it like that.  

But as you pointed out, Kamakshi, the computer is only as useful as the system that’s programmed into it. The one that seems to me to be the truest reflection of a player’s season is taking the average of their results. It doesn’t encourage them to play as many tournaments are the Best Of does, but just as important, it doesn’t encourage as many tanked matches or half-hearted efforts—everything you do counts. It’s not good for tennis if players don’t enter many events, but it’s just as bad if they show up and give less then their best.  

You asked how many finals, how many semifinals, how many quarterfinals I thought a tournament title was worth, and how many Masters a Slam was worth. The idea, I think, was to gauge what we believe is more important, winning it all or steadily doing well form week to week and over the course of a season. Which should be rewarded more in rankings?  

That's another tough question. Some people believe it’s titles or bust; only the winner’s trophies matter. And it’s true, while Federer’s 23 straight Slam semis were amazing, it’s the 16 titles that are even more impressive. What if, say, he hadn’t come through and won <em>any</em> of the 23 tournaments where he reached the semis? He would have been, rightfully, a laughingstock, not a legend. After bombing out in the final of the Aussie Open this year, Murray groped with how to think of those two weeks and the six wins he recorded as a positive. And they <em>were</em> a positive—those wins counted and shouldn’t be tossed aside as garbage. Virtually every player would rather reach the final than go down in the first round, and not because of the money.  

I’m not sure how many finals a title is worth: 4? 5? You see, it's a tough question, because while I believe Murray's six wins to the get to the final in Australia were something to be very proud of, I've just speculated that Novak Djokovic's one win in the final was four or five times more important than those six Murray wins combined.

Slams vs. Masters: Probably in most players’ eyes, because they don’t have a chance at either, one major would be worth an infinite number of Masters. It’s something that historically can never be taken away from you—except, maybe, if you’re Thomas Johannson. His Aussie win almost doesn’t count in my mind. But that's public valiue, as you said. Purely as an accomplishment, stripping away the prestige difference, I'd say a Slam is worth 2.5 Masters titles. Is that currently reflected in the rankings? A Slam win is worth twice as much as a Masters, correct? That may be a slight devaluation of the major, but then the Masters are the ATP's core events, so they want to promote them where they can, within reason.

But the rankings aren’t about a player's place in history. They’re about the week to week and how you measure you up over 52 of them. In this sense, the average of all results is the fairest assessment.  

My question to you, Kamakshi: In the old WTA system, would Safina, Jankovic, and Wozniacki still have reached No. 1?  

Steve