Thomas Goldkamp Media

  • Increase font size
  • Default font size
  • Decrease font size

Which recruiting service is the best - An analysis of the ESPN, Rivals and Scout rankings

Thomas Goldkamp Bio

A beat reporter covering the Florida Gators for GatorCountry.com. Previous experience as an intern at Rivals.com and as a freelancer for the Palm Beach Post. Extensive background in mathematics, having spent two years studying electrical engineering at UF.


Methodology

To come up with a way to objectively determine how well the recruiting services fared each year in their rankings, it was necessary to come up with a concrete system and formula to rank the recruits themselves.

GRADING SCALE
5All-American Player
4All-Conference Player
3Full-time Starter
2Occasional Starter
1Rare Starter / Career Backup
0Never Contributed

What I've done here is create a grading scale for each college prospect and given them a grade from 0-5 based on their performance in college. These grade levels are shown on the table to the right.

After assigning a grade to each player that was ranked by one of the major services, I took two steps to rank the services.

First, I compiled a raw score. This is the total points the recruiting service earned based on the performance of the players it had ranked in its Top 100.

The second calculation is a weighted score. This takes into account where a recruiting service ranked a particular prospect. The higher a player was ranked in the Top 100, the more points he earns in this calculation. The top-ranked prospect in each class earns whatever he graded out at times a multiplier of 1. Players further down the Top 100 have their grade multiplied by a decreasing fraction of 1. (For example: No. 1 Prospect's grade is multiplied by 1.00, No. 2 Prospect's grade is multiplied by 0.99, No. 3 Prospect's grade is multiplied by 0.98, etc.).

This rewards the services for ranking players who went on to have great careers higher on their board. The better the weighted score, the more players the service had high in its ranking who turned out to be stars.

Around this time every year, hard-core college football fans scour the Internet for the latest recruiting news, hoping their team will reel in the next class of superstars.

Every year, more recruitniks appear, to the point where seemingly everyone who follows college football is buzzing about where their team's commits are ranked in this recruiting service or that one.

Inevitably, opposing fans begin to argue which recruiting service is more reliable - more trustworthy - in order to prop up their recruiting haul even more.

But which service is the best? Is there any definitive data to prove one way or the other that one service gets it right more often than the others?

To attempt answer that question, I devised a formula to quantify the success each service has at accurately ranking prospects.

Like recruiting, it's a bit of an inexact science. Simply put, there are too many factors to have a black-and-white answer about which recruiting service is the best.

With that in mind, I'll explain my methodology for ranking the services and provide some results that I've found based on my research of the Top 100 prospects in each recruiting service from 2005-2007.

There are a couple reasons I chose this date range. First, ESPN did not start doing recruiting rankings until 2006. I wanted to include ESPN in the study, so it made sense to incorporate the 2006 and 2007 classes.

I stopped at 2007 because there are still a handful of players in the 2008 class who have not reached their potential based on transfers, injuries, etc.

I would like to go back all the way to 2002 to compare Rivals and Scout. However, I was unable to find Scout's Hot 100 for 2003 or 2004 (not available on their site). Since those two years were unavailable, I decided not to rank the 2002 class because I wouldn't be able to identify trends as easily as if the data were contiguous.

If and when I can get my hands on a copy of Scout's Hot 100 from those two years, I will extend the study to include 2002-04. It may not be immediate due to the immense amount of time it takes to research each of these players, but I will do it at some point in the future. If you can provide me with a copy of one of these lists, please inform me on Twitter at @ThomasGoldkamp.


2005 ACCURACY RESULTS
Service Raw Score Weighted Score
Scout298158.66
Rivals297159.25

2005 RECRUITING RESULTS
RivalsScout
All-American1312
All-Conference3031
Full-time Starter2729
Occasional Starter108
Rare Starter / Career Backup1111
Never Contributed99

2006 ACCURACY RESULTS
Service Raw Score Weighted Score
Rivals294163.27
Scout289158.44
ESPN280155.72

2006 RECRUITING RESULTS
ESPNRivalsScout
All-American141716
All-Conference151514
Full-time Starter323133
Occasional Starter202120
Rare Starter / Career Backup141414
Never Contributed523

2007 ACCURACY RESULTS
Service Raw Score Weighted Score
Rivals291153.61
Scout262141.70
ESPN244130.33

2007 RECRUITING RESULTS
ESPNRivalsScout
All-American6109
All-Conference192521
Full-time Starter303128
Occasional Starter161719
Rare Starter / Career Backup161613
Never Contributed13210

AVERAGE ACCURACY RESULTS
RawWeighted
Rivals294158.71
Scout283153.10
ESPN262143.03

AVERAGE RECRUITING RESULTS
ESPNRivalsScout
All-American1013.312.3
All-Conference1723.322
Full-time Starter3129.730
Occasional Starter181615.7
Rare Starter / Career Backup1513.712.7
Never Contributed94.37.3

Analysis of the 2005, 2006 and 2007 recruiting rankings

During the course of the three-year span I investigated, one service performed better than the other two.

However, the differences were minimal between the top two recruiting services. While many fans claim that one service is significantly better than another, actual results showed that the top two are similar, while the third is not far behind.

So which service actually came out on top for this three-year period?

According to my research on each prospect ranked in the Top 100 from 2005-07, Rivals.com provided the most accurate rankings.

In each of the three years, Rivals finished first in the weighted calculation from my formula. Rivals also finished first in raw score in two of the three years, with Scout edging Rivals for first in raw score in 2005 by one point.

This means that Rivals was the best at getting players that finished with a great college career high on its board.

Rivals finished with one more All-American on its board than Scout in each of the three years, while finishing with more All-Conference players in its Top 100 ranking in two of the three years.

On the other end of the spectum, ESPN finished with less All-Americans and more busts (Grade 0) than both Rivals and Scout in its first two years ranking prospects. It will be interesting to see how Rivals and Scout fared in their first two years ranking prospects (2002 and 2003) in comparison to ESPN's first years. Naturally, one would assume there is a learning curve and the services will improve with more experience ranking prospects from year to year.

Thus, ESPN's finish in these rankings is not altogether surprising.

Perhaps the most concrete piece of information to come out of my study is that Rivals has done an exceptional job of avoiding ranking future busts. Rivals averaged 3.0 less busts per year than Scout and 5.7 less than ESPN over the three-year span.

When you take into account the fact that several of these busts have been the result of career-ending injuries or off-field situations that are completely unpredictable, Rivals' track record of avoiding these players in its rankings is even more impressive. More research from 2002-04 and in future recruiting classes will be particularly interesting given that this success on Rivals' part could be due to the small sample size.

In terms of the actual year-to-year differences in recruiting classes, there are some interesting notes to be gleaned from my research.

First, there's a different makeup in each class as a whole. In 2006, there were more All-Americans but less All-Conference players than in the other two years.

At the same time, there were significantly less busts in 2006 than there were in either of the other two years. Rivals was the only service over the three-year period to avoid double-digit busts (players who never contributed) in each year.

You can visually compare how each service stacked up by sorting the tables I have provided for each year. I have color coded each grade level so that it is easy to see how each recruiting service fared in its Top 100 ranking.

The more green and less red you see near the top of the table for each service, the better the service did in ranking top prospects near the top of its board.

Information for sorting these tables by recruiting service Top 100, individual player grades or alphabetically by player can be found on the links provided to each year's chart at the bottom of this page.

If you notice any problems with any of my individual player rankings or calculations, please contact me on Twitter at @ThomasGoldkamp. I would like this to be completely accurate and will update it if I have made any mistakes.

General comments should also be directed to my Twitter account.

2005 Recruiting Chart | 2006 Recruiting Chart | 2007 Recruiting Chart