What Do We Want?

I'm writing this as a sort-of rebuttal to Scott Van Pelt's excellent take on why the RPI should be retired, and why predictive models such as KenPom and Sagarin should have a larger influence on Selection Sunday. And, I should mention, I agree with those larger points so this isn't much of a rebuttal at all.

SVP does mention however, there are a couple problems that at least some coaches have identified in those predictive models. First is the fact that KenPom considers the margin of victory (and I'll stick with that for the purposes of this post, as it is the gold standard. My standard response to all things BPI remains the following image):
not happening

Here's the rub, however—margin of victory matters. Specifically,

And, whether one likes it or not, there is predictive information in whether a team won the first game by 20 or 30 or 40.

We can yell all we want at the clouds, but it does not change the fact that the best evidence we have suggests that Team A that beats Cupcake U by 40 is better than Team B that beats Cupcake U by 15. We cannot eliminate facts simply because we find them uncomfortable, or because the facts do not respect our cultural norms around good sportsmanship.

The good news is that this also suggests that beating a team by 40 points is hard. So, when SVP quotes VCU coach Will Wade for his concern that "you're going to have some people trying to blow some people out to make your numbers look as good as possible," well, so what? When it comes to things with some value, while "trying" is usually a necessary prerequisite to "getting," it is rarely sufficient (I'd have a much nicer car, if that were not the case). So, Wade can try all he wants to beat Buffalo or some other middling mid-major team by 40, but he probably won't be able to.

The second problem, which is very familiar to SVP as a Maryland fan, is that predictive models don't take into account what actually happened. As SVP put it

If the outcomes of the games don't matter, then what's the point of keeping score?

Of course I agree with that sentiment, but I don't accept the premise that KenPom doesn't digest and account for actual events. Of course it does. If a team beats a top KenPom team on the road by 20, you can expect the winning team's ranking to skyrocket.

What SVP is actually talking about are wins. Of course, KenPom does account for those things as well—a margin of +2 is greater than a margin of -2. So to be more precise, SVP wants to see bonus points for wins for any ranking system that is relied upon by the Selection Committee, because Wins Matter.

And this is the part where I (finally) explain the title of this post. What do we want our ranking systems to deliver? Anyone can rank teams by any number of methodologies to arrive at a specific result. More importantly, we can devise all sorts of ranking systems to be used in various ways to create incentives for teams, which I think is the more interesting possibility.

And frankly, through that lens, I don't have a huge problem with the RPI, philosophically, from "who's in and who's out" perspective. Remember, the RPI is looking at, broadly speaking, who teams beat and how good those teams are. Of course it's not perfect. It doesn't account for home and away (some helpful readers reminded me that, starting in 2004, RPI now accounts for home and away with a 40% effect. This doesn't change the fact that how RPI accounts for home court advantage is not all that scientific), and winning percentage is a particularly blunt instrument for measuring how good a team is in any event.

But those are problems with the math. Entirely fixable. But what I like about RPI is that it creates incentives for teams to craft at least somewhat-challenging out-of-conference schedules. Because one thing I'm sure that We Want is good games. We want Villanova to play Purdue. The November 2006 matchup between Ohio State and North Carolina remains some of the best early season basketball you're likely to see under the current system.

I want more of that. And, short of the Chairman Mao approach to scheduling (which I do not oppose in the abstract, but I remain skeptical will happen because it would entail that someone not currently charged with Doing Something must now be paid money to Do Something), I think the best way to do this is have the NCAA Tournament ticket entry come with a price of beating good teams in places where it's difficult to beat them. In other words, if we had a system where we picked teams based on the number of wins they had in Difficult Games (as measured by KenPom standards), that would be all well and good with me.

Sure, there might be some unpleasantness. If you're a coach in the MAC, it means you'll have to travel to hostile arenas of power-5 conferences to try and win some games. In an extreme example, such a team might capture a handful of scalps but with the price tag of a terrible out-of-conference record. So, as a part of this system we would have to look past raw win and loss records, and focus more on the records in the games that mattered. Rutgers played exactly two teams ranked within the KenPom top 150, and it lost both. Its non-conference record the Committee would consider on Selection Sunday? 0-2.

Now, to the second part, which is fixing the real crime of the RPI. Sure, there are always one or two teams that can file a legitimate gripe about being left outside of the field of 68. But we can be reasonably sure that the Committee has never excluded a team that would have otherwise won the whole thing. There's a reason these teams were on the Bubble in the first place, and in my mind no great crime occurs when one fails to receive an invite.

But I do take issue when a KenPom #13 team is seeded on the 11-line. Or when two top-20 teams face each other in the play-in game. These things aren't fair to the underrated team or their opponent. And the overrated teams get a relative pass at the expense of other teams. Again, What Do We Want? I'd argue that in the Tournament, we want the best basketball possible at every round. So while we could make the best two teams play each other in the first round, as well as #3 vs. #4, and so forth, that would make for mild basketball in the subsequent rounds.

So, I would propose that while Wins Matter when it comes to whether or not a team gets into the Tournament, Quality Matters when it comes to where teams get seeded. Now, there might be some teams (like, say, Maryland) that might get upset about being on a hypothetical 9-seed line after finishing a season in which they won 26 games. "All we do is win!", they would complain, "and what happens on the court should matter more than what happens on the spreadsheet!"

Well, good news, hypothetical fans—the beauty of college basketball is that all you need to do to prove up such a case is demonstrate this very nature of Just Winning, and all wrongs will be righted.

But the good news is that the Selection Committee is opening up a dialogue, and with some very smart people. I just hope they know what they want.