Catching up: A response to a Monday post from Doc Saturday

I meant to post this when I saw it on Tuesday but holiday plans and all interfered.

On Monday, the esteemed Dr. Saturday went off on the BCS computers a little, citing the ranking of James Madison, the highest rated team in the FCS in most of the computers. His point, in brief, was that if James Madison can lose to 4-8 Duke by 24 points they have no business being ranked in the top 40, much less the top 25, as they were in the Sagarin ratings as of Monday. As someone who has spent a fair amount of time the last couple of years working on a ranking system, I felt the need to stick up for the computer geeks.

While Doc does mention that eliminating margin of victory does distort the ranking somewhat, he still chides the rankings for missing differences in teams that appear to be blatantly obvious to anyone who has watched more than a minute of college football this year

What he fails to mention, and I think he probably knows, is that the purpose of the computer rankings is different than the purpose of the polls. While it is not clearly stated anywhere, I would guess that most human poll voters vote based on who they think is a better team. Computer rankings, on the other hand, are a mathematical measure of which team has had the better season. They are the ultimate resume rankers. Just as the resume rankers produced rankings early in the season that made little sense (East Carolina at #1) to people expecting rankings to be a measure of team strength, computer rankings don't make sense if you expect them to be a measure of team strength, instead of season strength.

As with most resume rankers, the more data input into the rankings, the more they tend to converge with the general consensus of which teams are best. The problem that most of the computer ranking systems have with FCS teams is that there are limited opportunities to measure FCS teams against FBS teams, and most of those opportunities are slanted towards the FBS teams (i.e. FCS teams never get home games). Most would agree that if James Madison played more than one FBS team they wouldn't be ranked in the top 25. But until we have a system where FCS teams get multiple games against FBS teams, the computer rankings will be imperfect.

The computer rankings are fine for what they set out to do, measure teams based on what they have accomplished in the season, as opposed to some sort of intangible assessment of which teams are best.