2006 NFL Draft Grades: Overall Team Value and Efficiency Ratings

Welcome back for Part 2 of my series on draft grading. Yesterday, I introduced an objective, stat-based system for evaluating draft picks that involves comparing their actual career performances vs. an estimate of the career performances we should have expected given the selection at which they were drafted. Although the potential applications of this system are numerous, the most obvious ones involve evaluating entire draft classes. Today, I'm going to look back at the 2006 NFL Draft, and grade each team based on 2 stats I talked about yesterday:

1. Value Above Expectation (VAE)
2. Return on Investment (ROI)

Just to remind everyone how the system works, what I do is first calculate a draft pick's Career Approximate Value per Year (AV/Yr). For example, J.J. Stokes amassed a Career AV of 41 over a 9-year career, which translates to 4.56 AV/Yr. Next, I calculate a draft pick's Expected AV/Yr (Exp AV/Yr) by plugging his pick number into the equation:

Exp AV/Yr = 7.82 - 1.29*ln(Pick)

For instance, J.J. Stokes was the 10th pick in the 1995 NFL Draft, so 7.82 - 1.29*ln(10) = 4.85 Exp AV/Yr.

Once I know a player's AV/Yr and Exp AV/Yr, I can then calculate VAE and ROI.

TEAM VAE

VAE is the simple difference between the player's actual career and his expected career (i.e., VAE = AV/Yr - Exp AV/Yr). If he's outperformed expectations, VAE is positive, and vice versa. Fundamentally, VAE tells you how far above or below expectations that a draft pick performed during his NFL career. For instance, Stokes' VAE was -0.29, meaning he underachieved expectations.

In a team context, the goal of a draft is to maximize VAE with the picks you have at your disposal. Obviously, the more overall value a team gets in the draft, the better. However, the point of VAE in the context of a draft class is to tell us how much total value a team added to their roster above and beyond what was expected given the picks at their disposal. For instance, due to the picks they traded away to acquire Stokes, the 49ers only had 4 picks in the 1995 draft: #s 10, 127, 201, and 238. Based on these pick numbers, the Niners should have gotten a total of 8.16 AV/Yr of career value in that draft. Unfortunately, the actual career value of the 4 players they selected ended up being only 5.31 AV/Yr, meaning the 49ers actually lost 2.85 AV/Yr-worth of value with the selections they made. That's negative team VAE, and it tells us that they did a bad job of maximizing value in the 1995 NFL Draft. If, say, the 49ers would have taken Stokes with the 127th pick instead of the 10th, they would have ended up doing a better job of maximizing value. And if they took him with the 238th pick, even better.

The whole idea here is that - as I'm sure most of you understand in the context of fantasy football drafts - it's not enough to draft good players. What matters more is that you pick players who end up being better than what you'd expect based on where you drafted them. A draft consisting of 7 massively overachieving Tom Bradys is better than one consisting of 7 mildly overachieving Peyton Mannings, and is a lot better than one consisting of 7 mildly underachieving Alex Smiths. The difference between these 3 levels of "adding value in the draft above expectation" is exactly what Team VAE is measuring.

After the jump, I'll explain the ROI concept in English, present Team VAE & Team ROI stats for the 2006 NFL Draft, and go into some detail about the stats for specific teams...

TEAM ROI

ROI, a statistic similar to VAE, is the simple ratio of actual career performance to expected career performance minus 1 (i.e., ROI = [AV/Yr divided by Exp AV/Yr] - 1), and is expressed as a percentage. If a player has outperformed expectations, ROI is greater than 0%, and vice versa. Akin to its meaning in finance, ROI in an NFL draft context tells you a player's career value relative to the pick value a team invested in selecting him. For instance, Stokes' ROI was -6.1%. As was the case with his VAE, Stokes' ROI means he underachieved expectations. Unlike VAE, however, Stokes' ROI also specifically means the 49ers lost 6.1% - or, alternatively, recovered 93.9% - of the 10th pick's value by selecting Stokes.

As was the case with VAE, teams are trying to maximize ROI given the picks at their disposal. However, the way that ROI differs from VAE in a team context is analogous to the way it differs in a player context. Specifically, in addition to knowing the total value a team added with their selections given pick expectations, it's useful to know how efficiently they added it. To understand the distinction, consider that the Colts had a Team VAE of -2.53 AV/Yr in the 1995 NFL Draft, which was very similar to the 49ers' -2.86. However, because the total expectation of IND's 7 picks was nearly double that of SF's 4 picks (14.33 AV/Yr vs. 8.16 AV/Yr), the Colts losing 2.53 AV/Yr was far more efficient than the 49ers losing 2.86 (-11.8% Team ROI for IND vs. -35.0% for SF). In other words, whereas both teams lost similar value in the aggregate, SF's loss represented a 23.2% worse return on their investment. Or, to look at it another way, losing $2 on a $14 bet - although not a good investment overall - is a better investment than losing $2 on an $8 bet, and is a much better investment than losing $2 on a $4 bet. The difference between these 3 levels of "beating value expectation given the size of your pick investment" is exactly what Team ROI is measuring.

THE 2006 NFL DRAFT - OVERALL TEAM AV/Yrs and EXPECTED AV/Yrs

Below are the basic team stats used to calculate Team VAEs and Team ROIs for the 2006 NFL Draft:

Tm	Picks	Tot AV/Yr	Rk	Tot Exp AV/Yr	Rk
ARI	6	14.80	23	14.55	19
ATL	6	8.15	31	9.61	30
BAL	10	26.10	9	17.95	9
BUF	9	28.15	7	18.56	8
CAR	8	21.00	16	14.72	17
CHI	7	22.00	15	12.82	26
CIN	8	17.60	18	14.04	23
CLE	10	19.23	17	19.30	5
DAL	8	11.25	29	14.70	18
DEN	7	36.80	1	14.29	21
DET	7	11.03	30	14.47	20
GB	11	35.85	2	23.89	1
HOU	7	35.80	3	19.77	4
IND	7	29.90	5	11.84	27
JAX	6	24.90	11	11.18	28
KC	7	16.07	21	12.99	25
MIA	5	7.33	32	8.91	31
MIN	7	23.80	13	16.22	13
NE	10	14.77	24	17.87	10
NO	8	35.40	4	17.13	11
NYG	7	14.80	22	13.46	24
NYJ	10	29.00	6	23.26	2
OAK	7	14.65	25	15.38	15
PHI	8	24.80	12	17.05	12
PIT	9	16.25	20	14.97	16
SD	8	23.60	14	14.09	22
SEA	6	13.00	28	10.14	29
SF	9	26.20	8	18.65	7
STL	10	13.08	27	19.24	6
TB	10	14.00	26	15.53	14
TEN	11	25.87	10	20.76	3
WAS	6	16.50	19	8.25	32
NFL	255	671.68	--	495.57	--

The Packers and Titans had the most picks in the draft that season, whereas the Dolphins had the fewest. Not surprisingly then, GB and TEN are near the top of the rankings for total expected pick value, and MIA's near the bottom. However, you'll notice that although GB and TEN had the same number of picks, GB's picks were worth slightly more than TEN's (23.89 Exp AV/Yr vs. 20.76). That's because GB had 5 picks in the first 3 rounds (i.e., highly valuable ones), while TEN only had 2. Therefore, although both teams stockpiled picks in the 2006 NFL Draft, the Packers stockpiled picks that were more valuable than the Titans'.

In addition to having the most pick value of any team in that draft, the Packers also ended up taking the 2nd-best group of players. In other words, they invested valuable picks on valuable players. A little later, I'll show you whether that translated into a high-ranking Team ROI. Contrast GB's use of their picks with that of the Broncos, who ended up picking the single best group of players in the 2006 Draft (AV/Yr = 36.80) despite only having 7 picks that were middling in aggregate value (Exp AV/Yr = 14.29).

Focusing in on the Niners, they were near the top of the league in total picks, total value, and total pick value. In other words, they amassed a large number of valuable picks, and turned those picks into valuable players. I guess McNolan wasn't such a disaster after all. Well, at least in the context of the 2006 draft they weren't.

One other thing I'd like to mention before presenting the Team VAEs and Team ROIs is how the 32 NFL teams did as a group in the 2006 Draft. As you can see at the bottom of the table, this was a draft characterized by immense value, to the point where the average NFL team added 5.5 AV/Yr of player value, and got a 35.5% return on their pick investment. An interesting implication of this is that, as I'll show you in a minute, very few teams had really bad drafts in 2006, and their missteps seem to have put them severaly behind the eight ball during subsequent seasons.

THE 2006 NFL DRAFT - OVERALL TEAM VAEs

Below is a chart of Team VAEs for the 2006 NFL Draft (click to enlarge...notice the color-coding!!!):

By far, the Broncos added the most player value above expectation in 2006. Of their 7 picks, DEN got Pro-Bowl QB Jay Cutler at #11, starting TE Tony Scheffler at #61, Pro-Bowl WR Brandon Marshall at #119 (!!!), Pro Bowl OLB Elvis Dumervil at #121 (!!!), and 4-year starting G Chris Kuper at #161. Not to mention that they used the 130th pick to draft Domenik Hixon, who's gone on to have some decent seasons with the Giants. To put the sickness of DEN's 2006 draft class into perspective, the Lions had nearly an identical amount of total pick value (Exp AV/Yr = 14.47 vs. DEN's 14.29), yet they're 3rd-from-the-bottom in terms of the caliber of players they got with those picks. Seeing how much Mike Shanahan et al. pwned the rest of the league in the 2006 draft, it makes you wonder 2 things:

1. How did the Broncos not win more from 2006-2008?
2. Is Josh McDaniels the biggest, talent-destroying, epic-failing, reverse supernova of a personnel manager in NFL history?

OK, maybe that's a little harsh. But, nevertheless, I have an incredible amount of sympathy for Bronco fans now that I see just how much talent McDaniels squandered in his 1+ years with the team. It's like Shanahan struck gold in 2006, and then McDaniels sold it all after seeing a commercial during an episode of Glenn Beck.

On the other end of the VAE spectrum is the Rams, who it seems would have been better off having a bingo caller or Auctioneer Dan from Storage Wars making their selections. Indeed, of STL's ten picks in 2006, only two - 4th-round DE Victor Adeyanju and 7th-round G Mark Setterstrom - exceeded expectations, and even both of those players have already washed out of the league. When arguably the best player you select is children's TV character, Joe "H.R." Klofnstof (or whatever his name is), and you spent a 2nd-round pick on said player, you know you had a bad draft. The Ram's VAE for the 2006 draft bears this out poignantly.

Speaking of the Rams, what you've probably already noticed from the chart is that the cellar-dwelling NFL teams of the late 2000s are all in the Rams' neighborhood. Recall that, thanks in part to their abysmal 2006 draft, STL went on to win a total of 6 games from 2007-2009. Similarly, DET had a winless season in 2008, the Dolphins had a 1-win season in 2007, the Buccaneers had a 3-win season in 2009, the Raiders only won 2 games in 2006, and the Browns had 4-win seasons in both 2006 and 2008. Aside from the Cowboys, Falcons, and Cardinals, whose demises were mostly the result of sudden QB (or QB-on-dog) trauma, and the Patriots, who seem immune to bad drafts, every team with a negative VAE in the 2006 draft ended up being a perennial laughing stock for the next 3-4 seasons.

Now, I don't want to go too far out on a limb in terms of linking VAE to winning because we're only talking about 1 draft here. So, any statements I make linking VAE (or ROI) to winning should be interpreted as mere observations rather than definitive declarations. For instance, here's another observation: 3 of the 4 Conference Championship game participants in 2006 had a Top 8 VAE in the 2006 NFL Draft (Saints, Colts, and Bears).

Indeed, it's tempting to say that these teams probably would not have gotten as far as they did without significant contributions from high-value rookies in their 2006 draft classes. NO's high-powered offense got a big boost from Pick #108, T Jahri Evans, and Pick #252 (!!!), WR Marques Colston, both of whom vastly exceeded expectations. IND seamlessly transitioned from prolific RB Edgerrin James (Pick #4 in 1999) to overachieving RB Joseph Addai (Pick #30 in 2006), and got 14 starts out of Pick #207, S Antoine Bethea, who played a major role in limiting the pass defense's decline during All-Pro S Bob Sanders' 12 games missed due to injury. Finally, it's almost certain that the Bears wouldn't have advanced to Super Bowl XLI without an otherworldly, All-Pro rookie season from PR/KR Devin Hester, whom CHI acquired through grand larseny at the 57th pick.

In terms of the 49ers, the chart shows that our beloved team had a VAE that was slightly above average, with 5 of their 9 picks having exceeded expectations (TE Vernon Davis, OLB Manny Lawson, OLB Parys Haralson, TE Delanie Walker, and DE Melvin Oliver). I'll discuss specific Niner picks over the next few days, so I won't go into too much detail here. However, I do think it's important to point out that SF had the best VAE of any team in their division, and was the only NFC West team to have an above-average VAE in the 2006 NFL Draft.

Before moving on to Team ROI, let me just make one final point about Team VAE. Because these are objective stats, we can now assign objective draft grades if we so desire. For example, we could give VAEs ranked 1-6 an A, those ranked 7-12 a B, those ranked 27-32 an F, those ranked 21-26 a D, and those ranked 13-20 a C. If we did that, you'd end up with the following objective draft grades according to VAE:

As - DEN, NO, IND, HOU, JAX, GB

Bs - BUF, SD, CHI, WAS, BAL, PHI

Cs - MIN, SF, CAR, NYJ, TEN, CIN, KC, SEA

Ds - NYG, PIT, ARI, CLE, OAK, ATL

Fs - TB, MIA, NE, DET, DAL, STL

THE 2006 NFL DRAFT - OVERALL TEAM ROIs

Below is a chart of Team ROIs for the 2006 NFL Draft (click to enlarge):

As you can see, the ordering of teams in this chart is very similar to the one for Team VAE. For instance, the Broncos' ROI confirms that, not only did their picks add value above expectation, they more than doubled their pick investment. Similarly, the Rams' ROI confirms that, were the NFL Draft a stock market, STL would have lost their shirts.

Nevertheless, there are subtle differences between the 2 charts that give us some information about how efficiently some teams used their picks. First, remember that STL had 11 picks in 2006. So, in terms of how much bang they got for their buck, I think TS Eliot is instructive: "This is the way the Rams draft, not with a bang but with a whimper." The same can be said about the team that had the fewest picks in the 2006 NFL Draft, MIA. Rather than overcoming that disadvantage, and parlaying their 5 picks into, at the very least, a small profit, they ended up with a -17.7% ROI. So not only did they sacrifice value by trading away picks, they ended up sacrificing even more value with the picks they actually made.

On the other end of the spectrum, we find 2 teams, the Jaguars and Redskins, who succeeded where the Dolphins failed. Specifically, JAX only had 6 picks with a total expectation of 11.18 AV/Yr, but turned that small haul into a group of players representing 24.90 AV/Yr of value, which translates to an ROI of 122.8%! Similarly, WAS earned a 100.0% profit on their 6 picks. What we can say about JAX and WAS in the 2006 draft was that, unlike MIA, when the NFL gave them lemons, they actually made lemonade.

Moving on to the 49ers, we can see that their middle-of-the-road ROI ranking for the 2006 draft was essentially the same as their VAE ranking. Nevertheless, as a measure of drafting efficiency, the Niners' ROI suggests that the front office did a good job of efficiently deploying their 9 picks. I mean, a 40% ROI is nothing to scoff at, even if there were 14 teams that profited more than they did.

Finally, converting Team ROIs into draft grades using the same procedure I used for Team VAE, we arrive at the following:

As - DEN, IND, JAX, NO, WAS, HOU

Bs - CHI, SD, BUF, GB, MIN, PHI

Cs - BAL, CAR, SF, SEA, CIN, NYJ, TEN, KC

Ds - NYG, PIT, ARI, CLE, OAK, TB

Fs - ATL, NE, MIA, DAL, DET, STL

If we compare these grades to the Team VAE grades, we see several differences that represent how Team ROI and Team VAE differ in what they're measuring. For instance, WAS earned a B for adding value above expectation, but adding that value with only 5 picks earned them an A for efficiency. GB, on the other hand, earned an A for adding value above expectation, but because their 11 picks were associated with a high total expectation, they only got a B in terms of ROI.

BOTTOM LINE

So, from today's post, we've learned the following:

• The Broncos were the clear winner of the 2006 NFL Draft, and, therefore, Josh McDaniels has been rightly maligned for squandering that victory.
• The Rams were the clear loser of the 2006 NFL Draft, and that might have had something to do with their 6 total wins from 2007-2009.
• The Jaguars and Redskins turned lemons into lemonade.
• The Dolphins turned lemons into compost.
• The 49ers added value and got a good return on investment in the 2006 NFL Draft, but those successes only placed them in the middle of the pack due to how well most teams drafted that year.

Tomorrow, I'll focus in on the top of the 2006 NFL Draft by presenting each team's VAE and ROI for only the first 2 rounds, and using this actual-vs.-expectation framework to identify specific players who we can objectively label as draft busts.