Grading the recruiting sites on the NFL Draft

[ellitor 33,094]

...through the 1st 100 picks. Interesting read

Quote

 

At 247Sports, the NFL Draft has always been our compass. Our 32 five-stars every year is reflective of the 32 first round draft picks every year. Our four-star rankings represent guys we think are NFL-bound. The scouts and personnel departments in the NFL are looking for talent, regardless of offensive system, team politics, depth charts and all the other factors that can impact a college career. They're just looking for the best players and that's what we want our rankings to reflect.

With the completion of the NFL draft, it's time to grade our work as it compares to the other major recruiting networks and rankings systems.

First, a note about our formula:

We used five sets of rankings: 247sports, 247Sports Composite, Rivals, Scout and ESPN. For each of the Top 100 picks in the NFL draft we assigned a point value to the team 1-5, depending on which rankings was the closest to the player's actual draft position. The lower the score the better. Strictly in the first round, we assigned bonus points – in this case bonus deductions – for each ranking that had a player ranked within its top 32 prospects for that cycle. For all 100 picks, we added a bonus deduction for any ranking that was a positive outlier (defined as 100 spots or more higher than any of the other 3 sites, excluding the composite) as well as a penalty point for a negative outlier (defined as 100 spots or more lower than any of the other 3 sites, excluding the composite). A three-star ranking also merited a penalty point if any of the other four sites had the prospect as a four-star or better.

We limited the rankings to the top 100 picks because the deeper you get into the draft the harder it is to measure the accuracy of each pick due to the variety of ways each site ranks 3-star prospects. We also didn't assign any points to prospects that every site missed and ranked below the four-star threshold.

This is how the rankings shook out.

1. 247Sports (129 total points, 47 first round points)

Highlights:

- 247Sports was the only site to have Myles Garrett ranked as the No. 1 prospect in the country and ran away with the first round accuracy.

- Mitch Trubisky was a major outlier, ranked as the No. 1 dual-threat quarterback and 79th overall on 247Sports and no higher than 207 on any other site.

- 247Sports had Takkarist McKinley ranked as the nation's No. 6 weakside defensive end coming out of high school while he was a three-star or unranked by the other sites.

- 247Sports was the only site to have a four-star grade on former Washington cornerback Kevin King.

- While every site had Deshone Kizer ranked as a four-star, 247Sports was an outlier in how high he was ranked, barely making the cut at 101 spots higher than Scout's ranking of 263.

- Misses included a three-star grade on former Florida corner Quincy Wilson, a three-star grade on wide receiver John Ross and a much lower grade on Jamal Adams than the rest of the industry.

2. 247Sports Composite (151 total points, 59 first round points)

Highlights:

- The 247Composite has got Alabama pegged. Remarkably it was the most accurate ranking for Tim Williams, ArDarius Stewart, Ryan Anderson, O.J. Howard and Jonathan Allen. LSU's Ethan Pocic is the non-Alabama prospect that the Composite beat the others on.

- The nature of the Composite as a consensus ranking doesn't allow for many bad misses relative to the rest of the industry but a three-star grade on Malik Hooker and Justin Evans were the only notable clanks.

3. Scout (158 total points, 68 first round points)

Highlights:

- Scout did a great job in Texas. They went the highest with Texas native Solomon Thomas at 20th overall and Jamal Adams at 8th overall. They were also the only site with the onions to throw a four-star grade on Patrick Mahomes.

- John Ross was an outlier for Scout. They ranked the Long Beach, Calif. native No. 118 nationally and no other site had him inside their top 300.

- Scout was the only site with the common sense to put a four-star grade on the brother of J.J. Watt in Wisconsin.

- Misses included a 3-star grade on Quincy Wilson and Justin Evans but otherwise Scout didn't make many dramatic mistakes.

4. ESPN (161 total points, 65 first round points)

Highlights:

- ESPN really closed strong in the back end of the Top100 with the most accurate ranking for guys like Cordrea Tankersley, Jourdan Lewis and Alex Anzalone.

- There were a couple of strong hits at quarterback for ESPN. They had Deshaun Watson the highest of any site and they were the only site to rank Davis Webb as a four-star.

- LSU's Duke Riley was another outlier. He was a three-star everywhere but ESPN.

- There were a few of bad misses for ESPN. It was a negative outlier for Christian McCaffrey, ranking him outside its Top200 while others had him significantly higher. It failed to put a ranking on Takkarist McKinley. It was a negative outlier for Tredavious White, ranking him No. 186 nationally and it was the only site that had a 3-star grade on Delano Hill.

5. Rivals (164 total points, 68 first round points)

Highlights:

- While a couple of others had Quincy Wilson as a three-star, Rivals had him as a four-star and ranked higher than any other site.

- While there were no positive outlier for Rivals, it claimed the most accurate ranking for Montravious Adams, Eddie Vanderdoes, Chris Godwin, Dalvin Tomlinson, Marcus Maye, Cam Robinson and Leonard Fournette.

- Rivals' score took a hit because it was alone on a few negative outliers. Specifically, it was the only non-Composite ranking that had Mitch Trubisky, Malik Hooker and Gareon Conley as three-stars. John Ross and Justin Evans were also three-stars on Rivals but those picks had company from other sites as well.

 

 

[The Plainsman 948]

Scouts did pretty good,. But are now apart of 247?

[ellitor 33,094]

4 minutes ago, The Plainsman said:

Scouts did pretty good,. But are now apart of 247?

Umbrellaed with but still do their own evals & what not.

[The Plainsman 948]

1 minute ago, ellitor said:

Umbrellaed with but still do their own evals & what not.

Thanks E. 

[Rednilla 5,391]

Interesting, but a bit suspect considering the top three scorers are all managed by the same company.

[ellitor 33,094]

4 hours ago, Rednilla said:

Interesting, but a bit suspect considering the top three scorers are all managed by the same company.

Scout isn't technically managed by 247. They are financed by them but looks like Shannon Terry lets them still do their own work as in evals, have its own staff,  & have their own creative control. What I find funny is in most articles by 247 they use the composite rankings instead of their own. lol

[McLoofus 35,182]

33 minutes ago, ellitor said:

What I find funny is in most articles by 247 they use the composite rankings instead of their own. lol

I'm impressed by that. They 1) beat the rest and 2) still include the rest to present a comprehensive picture. 

[ClaytonAU 4,606]

247 has been the best and will continue to be the best! 

[ArgoEagle 1,788]

2 hours ago, ellitor said:

Scout isn't technically managed by 247. They are financed by them but looks like Shannon Terry lets them still do their own work as in evals, have its own staff,  & have their own creative control. What I find funny is in most articles by 247 they use the composite rankings instead of their own. lol

So would you say you are the lesser of the two evals? haha

[lionheartkc 6,150]

Anyone could have guessed that 247 was the best, but I'm surprised that Scout was better than ESPN and Rivals. Not that I follow recruiting as much as E and some others, but when I look into players, more often than not, Scout has our recruits ranked much lower than the other sites.

[Rednilla 5,391]

4 hours ago, ellitor said:

Scout isn't technically managed by 247. They are financed by them but looks like Shannon Terry lets them still do their own work as in evals, have its own staff,  & have their own creative control. What I find funny is in most articles by 247 they use the composite rankings instead of their own. lol

Okay, financed then. I understand they do not interfere in evaluations, but nevertheless, they are connected, and they were at the top of these rankings. That makes it all a bit suspect, if you ask me.

[ellitor 33,094]

27 minutes ago, lionheartkc said:

Anyone could have guessed that 247 was the best, but I'm surprised that Scout was better than ESPN and Rivals. Not that I follow recruiting as much as E and some others, but when I look into players, more often than not, Scout has our recruits ranked much lower than the other sites.

Look at production and where they were drafted. Unfortunately they have been more right on AU recruits than the other sites.

[ellitor 33,094]

15 minutes ago, Rednilla said:

Okay, financed then. I understand they do not interfere in evaluations, but nevertheless, they are connected, and they were at the top of these rankings. That makes it all a bit suspect, if you ask me.

3rd out of 5 is in the middle, not the top of the rankings. And it's too early to tell on 1 year if it's suspect based on 1 article in 1 year. We have to wait and see if they do future articles in future years, having rank order variances among the sites.

[Rednilla 5,391]

Just now, ellitor said:

3rd out of 5 is in the middle, not the top of the rankings. And it's too early to tell on 1 year if it's suspect based on 1 article in 1 year. We have to wait and see if they do future articles in future years, having rank order variances among the sites.

Yes, but the top two were 247 and 247 Composite. It's a bit suspicious for the other site under their umbrella, as you put it, to finish just behind them, while the two that are unconnected finished in the last two places. That's my point.

[ellitor 33,094]

15 minutes ago, Rednilla said:

Yes, but the top two were 247 and 247 Composite. It's a bit suspicious for the other site under their umbrella, as you put it, to finish just behind them, while the two that are unconnected finished in the last two places. That's my point.

Again. If it happens more than a year or 2 then yes but 1 year alone then no. If they do the grades article each year with similar rank order results or randomly don't have a grades article in some years then yes it becomes suspicious. 1 article in 1 year alone does not make data suspicious statistically speaking.

[Rednilla 5,391]

1 minute ago, ellitor said:

Again. If it happens more than a year or 2 then yes but 1 year alone then no. If they do the grades article each year with similar rank order results or randomly don't have a grades article in some years then yes it becomes suspicious. 1 article in 1 year alone does not make data suspicious statistically speaking.

If you say so ell.

[ellitor 33,094]

1 minute ago, Rednilla said:

If you say so ell.

I'm a stats guy. Trust me on this for the time being. Now that this article has been done we'll track to see if it's done in future years & what the results are.

[Rednilla 5,391]

7 minutes ago, ellitor said:

I'm a stats guy. Trust me on this for the time being. Now that this article has been done we'll track to see if it's done in future years & what the results are.

I understand that, but the saying is that there are lies, damn lies, and statistics. Statistics can be manipulated. I'm not saying that's what they did, I'm merely pointing out that having the site who publishes results like this finish in the top two spots, with a connected company finishing third and competitors finishing in the last two spots, it's suspicious. It doesn't mean it's wrong, but it COULD mean that they set the parameters in their own favor somehow. I mean, do you really think they would have published this article if rivals finished first?

Again, however, it's an interesting read. You'll just have to forgive me if I don't put too much stock in it at this point.

[ellitor 33,094]

46 minutes ago, Rednilla said:

I understand that, but the saying is that there are lies, damn lies, and statistics. Statistics can be manipulated. I'm not saying that's what they did, I'm merely pointing out that having the site who publishes results like this finish in the top two spots, with a connected company finishing third and competitors finishing in the last two spots, it's suspicious. It doesn't mean it's wrong, but it COULD mean that they set the parameters in their own favor somehow. I mean, do you really think they would have published this article if rivals finished first?

Again, however, it's an interesting read. You'll just have to forgive me if I don't put too much stock in it at this point.

And again. That has been my whole point from the first moment of our disagreement. Let's wait and see what they do. If they grade the sites in future years with a different rank order, particularly Rivals & ESPN being higher then it's not suspicious. If they randomly not do articles in future years & only do them when 247, & Scout are in the top 3 then yes that would be suspicious. And that's why a few years needs to go by before this article can truly be deemed suspicious. As for right now the more appropriate category for this article alone is curious.

[Rednilla 5,391]

1 minute ago, ellitor said:

And again. That has been my whole point from the first moment of our disagreement. If they grade the sites in future years with a different rank order, particularly Rivals & ESPN being higher then it's not suspicious. If they randomly not do articles in future years & only do them when 247, & Scout are in the top 3 then yes that would be suspicious. And that's why a few years needs to go by before this article can truly be deemed suspicious. As for right now the more appropriate category for this article alone is curious.

We can agree to disagree on the meaning of words, then.