FanPost

A Quantitative Approach to handing out draft grades

Al Bello

To me, evaluating draft grades has always felt like some sort of voodoo; sports writers and other simply assign a grade to a team’s draft haul based on how they feel about the players drafted…it’s not very scientific or rigorous. Seeing all these grades got me thinking: can we make draft grading rigorous, quantitative, driven by data? I think the answer is ‘yes’.

I started with Pro Football Reference’s Career Approximate Value, or CarAV. CarAV was created by PFR founder Doug Drinen, and the method is an attempt to "put a single number on the seasonal value of a player at any position from any year (since 1950)". This article isn’t meant to be a discussion on the merits of CarAV, so you can explore it yourself if you wish – most analysts accept it as a valid statistic that can be applied to all positions, including offensive linemen.

To begin the analysis, I looked at every player drafted in the 2010 NFL draft by draft position and CarAV. I then created a logarithmic curve fit for the relationship – its well accepted that this relationship is logarithmic. Here is the 2010 fit -- the outlier at the top of the chart is NaVorro Bowman:

10402878_1421743878097150_8308212359140980786_n_medium

via scontent-b-dfw.xx.fbcdn.net

Next, I calculated expected CarAV, or expCarAV, based on a player’s draft position. From there, I can calculate net CarAV, or netCarAV. I can then sum up the netCarAVs for each team that year to determine how well that team did drafting. Alternate approaches can include simply summing up CarAV for that team in that draft, but if you do that, teams that drafted high automatically come out on top unless you blew that. But with this approach, if you take a player at #7 overall, he better play like a #7 overall pick, or you’ll get negative netCarAV.

The next step is to convert team netCarAV into some kind of grade. I decided early on that I’d grade on a curve; this makes sense because an NFL draft is zero-sum. So, first thing I did is determine a percentile score for each team based on a normal distribution. I then created a grading curve where the 50th percentile team received a 75, a C. Results are below for 2010:

Team netCarAV Z-score percentile grade
SFO 46.48 2.06 98.01 99.01
NWE 39.15 1.73 95.86 97.93
DEN 37.53 1.66 95.19 97.59
PIT 30.71 1.36 91.37 95.68
GNB 27.68 1.23 89.08 94.54
MIA 20.48 0.91 81.98 90.99
ARI 19.41 0.87 80.71 90.35
SEA 17.99 0.81 78.96 89.48
NOR 11.85 0.54 70.38 85.19
DAL 11.7 0.53 70.15 85.07
CHI 10.25 0.47 67.91 83.95
SDG 7.03 0.32 62.69 81.35
CIN 6.48 0.3 61.77 80.88
ATL 0.73 0.05 51.87 75.93
DET -2.27 -0.08 46.63 73.31
OAK -2.62 -0.1 46 73
CAR -3.32 -0.13 44.79 72.39
KAN -5.85 -0.24 40.44 70.22
HOU -6.05 -0.25 40.1 70.05
WAS -6.74 -0.28 38.94 69.47
IND -8.3 -0.35 36.33 68.16
CLE -11.85 -0.51 30.65 65.32
BAL -14.2 -0.61 27.13 63.56
TEN -16.36 -0.7 24.08 62.04
NYG -16.74 -0.72 23.56 61.78
PHI -20.83 -0.9 18.41 59.2
JAX -21.08 -0.91 18.12 59.06
TAM -21.22 -0.92 17.95 58.98
NYJ -25.88 -1.12 13.1 56.55
MIN -29.24 -1.27 10.21 55.11
BUF -38.55 -1.68 4.66 52.33
STL -47.21 -2.06 1.98 50.99

As you can see, the 49ers come out on top, followed closely by New England and Denver. If everything above 96.66% is an A+, then the 49ers, Patriots, and Broncos all got an A+. The Jaguars, Bucs, Jets, Vikings, Bills, and Rams all get Fs. Notice how poor of a score the Rams got; if we went with the other method, they would have scored well because they had the #1 overall pick.

A final step I took is to assess draft performance over the 2010-2013 time period. Since CarAV goes up with number of seasons, CarAV is already weighted for a time-lag: the 2010 draft matters much more for the 4-year assessment period than 2013.

Team 2,010.00 2,011.00 2,012.00 2,013.00 Total netCarAV
SEA 17.99 37.79 40.57 -5.96 90.38
NWE 39.15 -10.06 4.74 6.17 40.01
DEN 37.53 3.07 0.74 -2.67 38.68
PIT 30.71 2.98 -12 0.85 22.54
MIA 20.48 12.03 4.78 -15 22.29
SFO 46.48 4.5 -24 -4.91 22.07
ATL 0.73 15.13 7.49 -3.71 19.64
CIN 6.48 17.18 -2.23 -4.7 16.73
GNB 27.68 -5.96 -15.32 8.92 15.32
CHI 10.25 -2.25 -7.6 14.59 15
DAL 11.7 15.76 -20.51 5.83 12.78
NOR 11.85 -8.53 1.88 5.69 10.89
ARI 19.41 6.12 -10.45 -6.32 8.76
HOU -6.05 22.15 -8.88 -1.01 6.22
CLE -11.85 14.97 2.91 -5.74 0.29
SDG 7.03 -11.87 -8.57 13.22 -0.19
WAS -6.74 3.68 6.36 -4.52 -1.21
NYJ -25.88 23.51 -2.42 2.59 -2.19
TEN -16.36 8.75 3.31 0.11 -4.19
CAR -3.32 -12.86 8.93 0.22 -7.03
TAM -21.22 -7.83 11.97 9.99 -7.08
OAK -2.62 -1.74 -3.12 -3.09 -10.57
PHI -20.83 -10.85 17.35 3.74 -10.59
BAL -14.2 1.18 6.25 -4.51 -11.27
IND -8.3 -18.3 11.58 -3.08 -18.1
DET -2.27 -19.82 -8.4 1.53 -28.95
KAN -5.85 -17.78 -1.63 -7.61 -32.87
MIN -29.24 -6.74 3.62 -4.93 -37.3
STL -47.21 -17.58 9.2 8.93 -46.65
JAX -21.08 -14.37 -11.19 -3.08 -49.72
NYG -16.74 -18.72 -10.75 -5.69 -51.91
BUF -38.55 -11.45 -7.01 3.75 -53.26

These are raw scores; for the 4-year period the 49ers get a B+/A-:

Team 4-year grade
SEA 99.93
NWE 95.58
DEN 95.22
PIT 89.05
MIA 88.92
SFO 88.82
ATL 87.58
CIN 86.01
GNB 85.22
CHI 85.04
DAL 83.75
NOR 82.61
ARI 81.3
HOU 79.69
CLE 75.84
SDG 75.52
WAS 74.85
NYJ 74.21
TEN 72.89
CAR 71.05
TAM 71.02
OAK 68.8
PHI 68.79
BAL 68.36
IND 64.32
DET 58.92
KAN 57.33
MIN 55.79
STL 53.31
JAX 52.71
NYG 52.33
BUF 52.12

Here are draft grades for each team by year – feel free to make them letter grades if you want.

Team 2010 2011 2012 2013
SEA 89.48 99.78 99.98 59.29
NWE 97.93 62.5 83.02 91.18
DEN 97.59 79.51 76.76 67.25
PIT 95.68 79.38 58.59 77.58
MIA 90.99 90.02 83.07 50.61
SFO 99.01 81.39 51.37 61.55
ATL 75.93 92.73 86.89 64.48
CIN 80.88 94.22 71.96 62.04
GNB 94.54 67.36 55.6 95.51
CHI 83.95 72.27 63.9 99.29
DAL 85.07 93.22 52.54 90.48
NOR 85.19 64.24 78.6 90.2
ARI 90.35 83.46 60.29 58.59
HOU 70.05 96.9 62.21 72.03
CLE 65.32 92.61 80.22 59.74
SDG 81.35 60.61 62.61 98.83
WAS 69.47 80.32 87.05 62.47
NYJ 56.55 97.44 71.66 82.61
TEN 62.04 86.58 80.84 75.37
CAR 72.39 59.66 88.73 75.69
TAM 58.98 65.06 92.08 96.68
OAK 73 72.95 70.54 66.11
PHI 59.2 61.66 96.23 85.67
BAL 63.56 76.95 85.2 62.5
IND 68.16 55.37 91.69 66.13
DET 73.31 54.47 62.83 79.58
KAN 70.22 55.71 72.93 56.34
MIN 55.11 66.39 81.32 61.5
STL 50.99 55.84 89.05 95.52
JAX 59.06 58.3 59.47 66.12
NYG 61.78 55.11 59.95 59.84
BUF 52.33 61.04 64.71 85.69

Note: teams get credit for drafting a player even if they trade him away. So, even though Tayler Mays is a bust in the big picture, he still has a CarAV of 7, significantly below expected CarAV of 14.17, but still something…even though it was with another team. The philosophy behind this is that even if a player didn’t have room on a team, the GM still evaluated the talent level effectively…we also received credit for Marcus Cooper last year.

A final consideration is undrafted free agents. Seahawks WR Doug Baldwin has performed like a first round draft pick (18 CarAV) despite being undrafted. This makes the 2011 class for them even better…but you won’t see it here.

Questions?

This is a FanPost and does not necessarily reflect the views of Niners Nation's writers or editors. It does reflect the views of this particular fan though, which is as important as the views of Niners Nation's writers or editors.

X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Niners Nation

You must be a member of Niners Nation to participate.

We have our own Community Guidelines at Niners Nation. You should read them.

Join Niners Nation

You must be a member of Niners Nation to participate.

We have our own Community Guidelines at Niners Nation. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9341_tracker