As our loyal readers (all three of them) may remember, we ended last season by taking a look at the effectiveness our ranking system as opposed to Golden's original system. As you may recall, the two systems differ in how they deal with FCS teams. My system ranked all Division I teams, FBS and FCS, and treated wins and losses over FBS teams equally with wins or losses over FCS teams. Golden's system does not include games with FCS teams unless the FCS team beats the FBS teams.
We looked at how the two rankings systems did in predicting the bowls:
Was there a difference in how the two ranking sets did in predicting the outcome of the bowl games?At the heart of it, a power ranking system like these two is really designed to help predict future outcomes. So which system was better? Mine, but not by much. Golden's system went 16-16, mine went 18-14, with one match-up, West Virginia/Oklahoma, a virtual dead heat before the game. We differed on four games: The aforementioned Fiesta bowl, the Motor City Purdue/CMU clash, the Insight Bowl's Indiana/Oklahoma State game, and the USC/Illinois Rose Bowl. The Rose Bowl went to Golden, the rest to me.
and came to the following conclusion:
So what does this all mean? Is it worth the time and effort to include FCS teams in the rankings?I would say yes, but I do feel that the system needs to be refined to count wins over FCS teams less than wins over FBS teams (say 0.75
wins) and losses to FCS teams extra.
So I started to think about how would could quantify the difference between FBS and FCS teams in a way that could applied in a logical fashion in the rankings. At first I asked myself, "What is it that makes FBS teams better than FCS teams?" Scholarships? Most FBS teams have the max number of 85 players on scholarship, whereas FCS teams vary in the number of scholarships they offer. Nice idea, but it would be difficult to get an accurate read on the number of players on scholarship at most of the FCS teams. Athletic budget? Again, certainly a contributing factor to the disparity between the two levels of Division I football, but next to impossible to accurately measure.
Upon reflection, I realized the key question is not why are FCS and FBs teams, but how are they different. I took a look at the winning percentages of FBS and FCS teams from last year, and tried to apply that as a factor to wins and losses. With games against lower division teams eliminated, FCS teams had an overall winning percentage of about 47.5%, while FBS teams were at 52.5%. I awarded one point for every win or loss against an FBS team, and approximately 0.9 for every win over an FCS team (the percentage difference between the two overall winning percentages), and 1.1 for every loss to an FCS team, and applied this to the 2007 results. What I found that it made little difference in the final rankings. The bowl predictions didn't change at all. The top FCS teams fell a few spots, but we still had a number of them in the top 30, including North Dakota State, which played an incredibly weak schedule, even by FCS standards.
So I thought about taking the head to head winning percentage, which was about 85% for the FBS teams and using that as a factor. Too big a difference if we applied that directly. I then realized that I still wasn't asking myself the right question. The real question was, "What is the difference in winning probabilities when an FBS team plays another FBS team or plays an FCS team?" That is to say, if FBS teams win at a 85% clip over FCS teams, there is a 35% better chance they will win than if they were playing another FBS team, where the winning probability is 50% by definition.
So this is how we are going forward with the rankings this year. Every win or loss against an FBS team will count for one point (plus other bonuses or penalties). Each win or loss against an FCS team will be multiplied by a factor (or the reciprocal of that factor) that measures the difference of FBS winning percentage versus FCS and FBS internal winning percentage (50%). When I applied this system to last years results, I found our bowl predictions improved by two games to 20-12, and several of our top FCS teams made significant drops including the aforementioned ND State. The top FCS team Appalachian State finished in the low 20s, two spots below Michigan, which seems reasonable.
|