Which recruiting service is the best - An analysis of the ESPN, Rivals and Scout rankings
Around this time every year, hard-core college football fans scour the Internet for the latest recruiting news, hoping their team will reel in the next class of superstars.
Every year, more recruitniks appear, to the point where seemingly everyone who follows college football is buzzing about where their team's commits are ranked in this recruiting service or that one.
Inevitably, opposing fans begin to argue which recruiting service is more reliable - more trustworthy - in order to prop up their recruiting haul even more.
But which service is the best? Is there any definitive data to prove one way or the other that one service gets it right more often than the others?
To attempt answer that question, I devised a formula to quantify the success each service has at accurately ranking prospects.
Like recruiting, it's a bit of an inexact science. Simply put, there are too many factors to have a black-and-white answer about which recruiting service is the best.
With that in mind, I'll explain my methodology for ranking the services and provide some results that I've found based on my research of the Top 100 prospects in each recruiting service from 2005-2007.
There are a couple reasons I chose this date range. First, ESPN did not start doing recruiting rankings until 2006. I wanted to include ESPN in the study, so it made sense to incorporate the 2006 and 2007 classes.
I stopped at 2007 because there are still a handful of players in the 2008 class who have not reached their potential based on transfers, injuries, etc.
I would like to go back all the way to 2002 to compare Rivals and Scout. However, I was unable to find Scout's Hot 100 for 2003 or 2004 (not available on their site). Since those two years were unavailable, I decided not to rank the 2002 class because I wouldn't be able to identify trends as easily as if the data were contiguous.
If and when I can get my hands on a copy of Scout's Hot 100 from those two years, I will extend the study to include 2002-04. It may not be immediate due to the immense amount of time it takes to research each of these players, but I will do it at some point in the future. If you can provide me with a copy of one of these lists, please inform me on Twitter at @ThomasGoldkamp.
Analysis of the 2005, 2006 and 2007 recruiting rankings
During the course of the three-year span I investigated, one service performed better than the other two.
However, the differences were minimal between the top two recruiting services. While many fans claim that one service is significantly better than another, actual results showed that the top two are similar, while the third is not far behind.
So which service actually came out on top for this three-year period?
According to my research on each prospect ranked in the Top 100 from 2005-07, Rivals.com provided the most accurate rankings.
In each of the three years, Rivals finished first in the weighted calculation from my formula. Rivals also finished first in raw score in two of the three years, with Scout edging Rivals for first in raw score in 2005 by one point.
This means that Rivals was the best at getting players that finished with a great college career high on its board.
Rivals finished with one more All-American on its board than Scout in each of the three years, while finishing with more All-Conference players in its Top 100 ranking in two of the three years.
On the other end of the spectum, ESPN finished with less All-Americans and more busts (Grade 0) than both Rivals and Scout in its first two years ranking prospects. It will be interesting to see how Rivals and Scout fared in their first two years ranking prospects (2002 and 2003) in comparison to ESPN's first years. Naturally, one would assume there is a learning curve and the services will improve with more experience ranking prospects from year to year.
Thus, ESPN's finish in these rankings is not altogether surprising.
Perhaps the most concrete piece of information to come out of my study is that Rivals has done an exceptional job of avoiding ranking future busts. Rivals averaged 3.0 less busts per year than Scout and 5.7 less than ESPN over the three-year span.
When you take into account the fact that several of these busts have been the result of career-ending injuries or off-field situations that are completely unpredictable, Rivals' track record of avoiding these players in its rankings is even more impressive. More research from 2002-04 and in future recruiting classes will be particularly interesting given that this success on Rivals' part could be due to the small sample size.
In terms of the actual year-to-year differences in recruiting classes, there are some interesting notes to be gleaned from my research.
First, there's a different makeup in each class as a whole. In 2006, there were more All-Americans but less All-Conference players than in the other two years.
At the same time, there were significantly less busts in 2006 than there were in either of the other two years. Rivals was the only service over the three-year period to avoid double-digit busts (players who never contributed) in each year.
You can visually compare how each service stacked up by sorting the tables I have provided for each year. I have color coded each grade level so that it is easy to see how each recruiting service fared in its Top 100 ranking.
The more green and less red you see near the top of the table for each service, the better the service did in ranking top prospects near the top of its board.
Information for sorting these tables by recruiting service Top 100, individual player grades or alphabetically by player can be found on the links provided to each year's chart at the bottom of this page.
If you notice any problems with any of my individual player rankings or calculations, please contact me on Twitter at @ThomasGoldkamp. I would like this to be completely accurate and will update it if I have made any mistakes.
General comments should also be directed to my Twitter account.