2006 NFL Draft Grades: Value and Efficiency Ratings for Rounds 1 & 2
Today, in Part 3 of this series, I'm going to focus on how well NFL teams did in the first 2 rounds of the 2006 NFL draft. At this point, I've explained the Value Above Expectation (VAE) and Return on Investment (ROI) stats that form the basis of my objective draft evaluations. If this is the first installment you've read, or if you want to refresh your memory, see Part 1 and Part 2. The only things I'm adding to the mix today are my definitions of a "full-on" draft bust and a "mild" draft bust, which I originally developed last year:
- A "full-on" draft bust is a player who was selected in the 1st or 2nd round and played 67% or more below the expected performance of his specific pick number.
- A "mild" draft bust is a player who was selected in the 1st or 2nd round and played 50% or more below the expected performance of his specific pick number.
In other words, a full-on draft bust is a high pick whose ROI stat is less than or equal to -67.0%, and a mild draft bust has an ROI stat from -50.0% to -66.9%. I'm going to use these definitions at the end of the post to present how many draft busts each team drafted, as well as to list the 2006 draft picks who can objectively be considered busts to this point. Other than that, things will proceed very similarly to yesterday.
THE 2006 NFL DRAFT - TEAM VAEs FOR ROUNDS 1 & 2
So, without further ado, below is a chart of Team VAEs for the first 2 rounds of the 2006 NFL Draft (click to enlarge):
After the jump, I'll discuss the chart, present and discuss Team ROIs, and list the busts...
According to VAE, the Packers added the most value above and beyond expectation in the first 2 rounds of the 2006 NFL Draft. They had 3 picks (#s 5, 47, and 52) worth a total Exp AV/YR of 11.32, and turned those picks into LB A.J. Hawk (VAE = 1.26 AV/Yr), T Daryn Colledge (VAE = 4.35 AV/Yr), and WR Greg Jennings (VAE = 6.28 AV/Yr), all of whom have been 5-year starters for the team. What I find interesting about GB's draft performance in the first 2 rounds is that their VAE for the entire draft was 11.96 AV/Yr, meaning that they basically got all of their 2006 draft value from their first 3 picks. Close behind GB was the Jaguars, thanks predominately to having stolen Pro-Bowl RB Maurice Jones-Drew at the 60th pick (VAE = 8.66 AV/Yr).
On the other end of the spectrum, we once again have the Rams. Oh, those poor Rams! Just above the Rams, though, are the Bills, who are an interesting case to consider. Although I didn't really discuss them yesterday, the Bills actually had a pretty good draft overall, ranking in the Top 10 at both VAE and ROI. Focusing only on the first 2 rounds, however, they were only hundredths of VAE better than STL. Why? Well, in 2006, BUF had the good fortune of holding two 1st-round picks (#s 8 and 26). The first of those picks was DB Donte Whitner, who has basically played up to expectation for a #8 pick. The second pick, however, was the disaster known as DT John McCargo (VAE = -2.82 AV/Yr), a player whose best season to date was as a backup to Larry Tripplett in 2007. What makes things worse is that they traded the 42nd and 73rd picks to get him. You could say BUF got "Stokesed."
Speaking of the 49ers, once again their ranking is right around average despite actually adding positive value early in the draft. Like BUF, they had two 1st-round picks (#s 6 and 22). Unlike BUF, they avoided a catastrophe. Both their first pick, Vernon Davis (VAE = 0.69 AV/Yr), and their second pick, Manny Lawson (VAE = 1.77 AV/Yr), have slightly beaten objective expectations. The problem for the Niners in the 2006 draft was that, even though they got good value at the top of the draft, 19 teams did a better job of it.
Based on the above graph, here are my objective team grades for the first 2 rounds of the 2006 NFL Draft:
As - GB, JAX, CHI, SD, BAL, DEN
Bs - IND, CAR, KC, CLE, CIN, NYJ
Cs - TB, PIT, MIN, HOU, NO, WAS, SEA, SF
Ds - OAK, PHI, TEN, ARI, DET, ATL
Fs - DAL, NYG, NE, MIA, BUF, STL
THE 2006 NFL DRAFT - TEAM ROIs FOR ROUNDS 1 & 2
Below is a chart of Team ROIs for the first 2 rounds of the 2006 NFL Draft (click to enlarge):
Thanks to MJD, JAX moves to the head of class when we look at value added on a percentage basis. Because they were able to spend such a low pick on a player of MJD's caliber, the Jags have gotten a 341.2% return on their investment. Meanwhile, their 1st-round pick, TE Marcedes Lewis, has also been a hit in retrospect (ROI = 76.1%), although not a home run like Lewis's diminutive UCLA teammate.
One team that creeps up towards the top despite only being in the middle of the pack according to VAE is the Steelers. Not surprisingly, the reason for this has to do with the fact that they only had 1 pick in the first 2 rounds; and a low 1st-rounder at that. With that long pick, however, they took WR Santonio Holmes, who's been 107.2% better than what you would expect from a #25 pick.
There was also a change at the bottom of the rankings as well. Specifically, STL finally made its way out of the cellar, having been replaced by the Dolphins, who took the disadvantage of having only 1 pick in the first 2 rounds, and made things worse by drafting DB Jason Allen, a player whom they released in the middle of last season after 4 years-worth of backup duty.
With respect to the Niners, their ROI ranking for the first 2 rounds (21st) was very similar to their VAE ranking (20th). However, according to ROI, they ended up much farther below the NFL average, which was mostly because - as I mentioned earlier - their picks were good, just not good enough in the context of a solid draft leaguewide. In addition, whereas they were essentially tied with the Seahawks for best Team VAE in the first 2 rounds among NFC West teams, SEA's picks yielded a much better ROI; placing them well in front of the 49ers, Cardinals, and Rams.
Finally, converting Team ROIs for the first 2 rounds into draft grades, we arrive at the following:
As - JAX, CHI, SD, IND, PIT, GB
Bs - BAL, DEN, CAR, WAS, KC, CIN
Cs - TB, CLE, SEA, NYJ, NO, MIN, HOU, OAK
Ds - SF, PHI, TEN, ARI, DET, DAL
Fs - NYG, NE, ATL, BUF, STL, MIA
If we compare these grades for Rounds 1-2 to the Team VAE grades, we see several differences. First and foremost, the Niners went from a C in value added to a D in efficiency. The Broncos, whom I talked at length about yesterday, and the Ravens, who I have yet to even mention this week, both dropped from As in value added to Bs in efficiency. In contrast, the Colts went from a B to an A, and the Steelers skyrocketed from a C in VAE to an A in ROI. Finally, the archrival Cowboys flunked the ROI test after having barely passed the VAE test.
THE 2006 NFL DRAFT - THE BUSTS (SO FAR)
To remind everyone of what I said in the opening, full-on draft busts are picks in the first 2 rounds who have an ROI of -67.0% or worse; mild draft busts have an ROI of -50.0% to -66.9%. According to these definitions, it turns out that, as of this evaluation, there were only 3 full-on busts and 3 mild busts selected in the first 64 picks of the 2006 NFL draft. That works out to a league-wide bust percentage of about 10%.
Because there were so few, I'll show you the worst 10 picks of the first 2 rounds, the worst 6 of which were the busts:
|
Pick |
Tm |
Player |
Pos |
AV/Yr |
Exp AV/Yr |
VAE |
ROI |
|
36 |
NE |
WR |
0.67 |
3.20 |
-2.53 |
-79.15% |
|
|
49 |
NYJ |
QB |
0.60 |
2.80 |
-2.20 |
-78.57% |
|
|
26 |
BUF |
John McCargo |
DT |
0.80 |
3.62 |
-2.82 |
-77.88% |
|
44 |
NYG |
WR |
1.00 |
2.94 |
-1.94 |
-65.97% |
|
|
18 |
DAL |
LB |
1.80 |
4.09 |
-2.29 |
-56.01% |
|
|
15 |
STL |
DB |
2.00 |
4.33 |
-2.33 |
-53.78% |
|
|
16 |
MIA |
Jason Allen |
DB |
2.40 |
4.24 |
-1.84 |
-43.45% |
|
10 |
ARI |
QB |
3.00 |
4.85 |
-1.85 |
-38.14% |
|
|
40 |
DET |
DB |
2.33 |
3.06 |
-0.73 |
-23.79% |
|
|
46 |
STL |
TE |
2.25 |
2.88 |
-0.63 |
-21.91% |
Three of these names should be familiar, as I've already mentioned them either yesterday or today. The Bills' disastrous #26 pick ended up being 1 of the only 3 full-on busts in the draft, along with the Patriots' #36 pick, WR Chad Jackson, and the Jets' #49 pick, QB Kellen Clemens. I also already mentioned Dolphin ROI-killer, Jason Allen, who just missed mild-bust status.
Finally, yesterday I brought up the fact that the Rams' draft was so bad that Joe "H.R" Klopfnstof was their most valuable pick. Well, it turns out that their best pick was very nearly a bust, and their other pick in the first 2 rounds ended up being 1 of the 3 mild busts, along with the Giants' #44 pick, WR Sinorice Moss, and the Cowboys' #18 pick, LB Bobby Carpenter. Seriously, there was enough epic fail in the Rams' war room that fateful Spring weekend to fill an entire stadium for the next 4 seasons. Oh wait, it actually did.
One thing I kind of brushed over yesterday that seems more relevant to talk about now is the lousy drafting of the Patriots in 2006. According to yesterday's overall draft grades, NE got an F in terms of both VAE and ROI. Today, we see that they were the not-so-proud recipients of the biggest bust in that draft. And yet, despite all this failure in the 2006 draft, they were only a monumental choke-job away from going to the Super Bowl that year, they famously went 16-0 and lost the Super Bowl just one year later, and have won 63 (!!!) games over the last 5 seasons. How?
Well, free agency and player trades of course, at least from 2007 on, anyway. You may recall that the offseason between 2006 and 2007 was when the Patriots landed Randy Moss, Wes Welker, and Donte Stallworth to help their offense, and Adalius Thomas to help their defense. Who needs a draft when you can add perennial Pro-Bowlers through free agency and trades?
As far as their heroics during the 2006 season go, the only things I can point to in lieu of their abysmal 2006 draft class are that (a) they still had Tom Brady on offense, (b) their offensive line was largely unaffected by injury, (c) they still had Richard Seymour and Rodney Harrison on defense, (d) Vince Wilfork and Asante Samuel were just entering their prime, and (d) they signed 5-day retiree Junior Seau. Essentially, they had built such a good roster prior to 2006 that their strikeout-out-swinging in the draft had little effect. Or, to put it another way, they were the Dave Kingman of the 2006 calendar year.
BOTTOM LINE
So, from today's post, here's what we can say about the first 2 rounds of the 2006 NFL Draft:
- The Jaguars and Packers hit home runs.
- The Rams struck out.
- The Bills grounded into a double play on a 3-1 count.
- The Steelers got a triple on an 0-2 count.
- The Dolphins got ejected for arguing a called 3rd strike.
- The Patriots had already homered in their first 3 ABs, so striking out in their 4th didn't matter.
- The 49ers went 2-4 with 2 RBIs (RBI? RsBI? Ugh!), but it wasn't enough to win.
Tomorrow, I'll look at the winners and losers of Rounds 3-7, and identify the diamonds in the rough.
34 comments
|
0 recs |
Do you like this story?
Comments
Love the advanced stats, terrified of the reading
If you have firefox download Text to Voice :)
49er fanhood up in the air with Kaepernick aboard and there not being an QB worth rooting for on the niner roster
for future reference...
i put that “bottom line” part in for those who want the “look at pretty charts + read 5 lines of text” experience. i’m aware of many people on here who do just that with my posts.
by (Florida) Danny Tuccitto on May 11, 2011 5:15 PM PDT up reply actions
actually..
minus the pretty tables and charts, this is one of my shortest posts ever
by (Florida) Danny Tuccitto on May 11, 2011 5:13 PM PDT up reply actions
hey florida danny..uhh
howabout a post on 49er’s past drafts like this one except grade the players on how well they do now starting w/ the 2006 draft
"Winning means being unafraid to lose." – Fran Tarkenton
i'll get to that...
at some point.
by (Florida) Danny Tuccitto on May 11, 2011 8:01 PM PDT up reply actions
Can't do it.
I’ll have to go look at the film again to understand all this. LOL jk
"Winning means being unafraid to lose." – Fran Tarkenton
actually read it. i want more
I think the bust label is too conservative. Lots of good info.
by mcwagner on May 11, 2011 4:50 PM PDT via mobile reply actions
i might...
tend to agree with you about the conservative bust definition. that was why i created the mild bust category. what i’ll say is that one of the reasons that there weren’t many “busts” in 2006 has nothing to do with the definition. the farther you go back in time, the more busts you find according to my definition. it’s just that teams have gotten better at avoiding them in the past 10 years or so.
by (Florida) Danny Tuccitto on May 11, 2011 7:35 PM PDT up reply actions
Is that because the use of high draft picks changed in the last 10-15 years?
There was a time where even 1st round picks sat the bench the first season and maybe multiple seasons before getting a sniff of playing time. A bad pick these days gets on the field for a season or two before the team moves on. In the past the bad pick might not ever had cracked the lineup. These days 1st round picks are guaranteed to play because of rookie money the ramifications of the salary cap limiting the use of veteran players.
sd377 wants to ban me for unleashing the Kaepernick Supernova Gamma Ray
i'll make the same point...
i made to ougdas yesterday. it’s just not true that “1st round picks” are guaranteed to play after the middle of the first round. look at the mid-to-late-1st busts in the table. bobby carpenter has basically had no career, and wasn’t thrust into the lineup as a rookie. allen didn’t start as a rookie, only started as a rookie because MIA had nothing at S in 2007 (see 1-15), and then basically fell off the face of the earth once coaches realized he sucked. hill is pretty much the same except, yeah, he did start his rookie year, but on STL for god’s sake.
as i said, the decision to play a mid-to-late first-rounder ahead of someone else, and his long-term (read career) fate as a player/starter is based on whether or not he’s better than the guy that’s next on the depth chart, not simply on the fact that he was picked at some magic number below 32. not to mention that i’m including 2nd-rounders as having bust potential. one can’t seriously argue that 2nd-rounders are “guaranteed to play.” i know you didn’t say that, but it’s just an extension of your argument vis-a-vis the context of busts in this post.
also, something i didn’t bring up to ougdas, but which directly refutes the alternate idea that it’s all about money, i’d ask you to look at the graph i posted in my original post on this stuff last year (link’s near the top of this post). that graph shows the trajectory of rookie salaries as the draft proceeds. just as performance drops off logarithmically, so to do rookie salaries. in that graph, it shows that, in 2005 (particularly relevant for this discussion of 2006), teams only spent about $1-1.5M per team on rookie salaries for picks 16-32, and then only about $500k for 2nd-rounders. when coach’s were already invariably millionaires, and the NFL was already a multi-billion dollar business, i don’t think any of these figures would provide some compelling reason for any owner (save al davis) to compel a coach to give playing time to a player who sucks.
by (Florida) Danny Tuccitto on May 11, 2011 9:22 PM PDT up reply actions
bignerd...
see below. i’m just making arguments here, not being defensive. i know this isn’t a foolproof method. there’s just no way to improve it without exploring things in a discussion.
by (Florida) Danny Tuccitto on May 11, 2011 9:26 PM PDT up reply actions
two places..
that seem ripe for improvement are my bust and diamond in the rough definitions. as you’ll see tomorrow, there were faaaaaaaaaar more DIRs in 2006 than there were busts. i’m just wondering, because i haven’t been able to look back at it yet, if this isn’t just a byproduct of the “teams getting better at drafting” thing i’ve mentioned a couple of times. or also if perhaps it’s just idiosyncratic to 2006.
by (Florida) Danny Tuccitto on May 11, 2011 9:30 PM PDT up reply actions
I am in-agreement that scouting has probably got better over time. I’m guessing (yes guessing) there are probably two separate eras inside your sample data, pre-salary cap and salary cap with some kind of notable difference. I am hypothesizing salary cap busts might have rosier rating than their equivalent pre-salary cap busts. Mainly because pre-salary cap rookies usually had to overcome an existing veterans to reach a lineup where even the worst salary cap era busts got the benefit of a little playing time with no veteran to overcome to boost their rating/score on your scale.
I think you can probably split the data into those two groups (pre salary cap, salary cap) and probably see the same upward trend in scouting performance on each. However maybe there is a slightly different threshold for full bust, partial bust between the two groups.
sd377 wants to ban me for unleashing the Kaepernick Supernova Gamma Ray
well...
i used a random sample to get around the era problem. the years that ended up in the sample were 74-79, 81-83, 85, 88, 94, 97-98, and 01. you think this wasn’t sufficient?
by (Florida) Danny Tuccitto on May 11, 2011 10:04 PM PDT up reply actions
the population...
was 70-04, so i guess perhaps if the 05-10 era is qualitatively different in terms of vets not standing in the way of youngsters, that could explain part of the issue. or even 01-10 since 01 was the latest draft that ended up in the sample. although, if that’s the case, salary cap wouldn’t be an issue because 1/3 of the sample were in the cap era.
by (Florida) Danny Tuccitto on May 11, 2011 10:08 PM PDT up reply actions
check out...
the first graph in part 1, and tell me if you think it provides any hints here.
by (Florida) Danny Tuccitto on May 11, 2011 10:09 PM PDT up reply actions
the one...
that tracks logarithmic r^2 over time.
by (Florida) Danny Tuccitto on May 11, 2011 10:18 PM PDT up reply actions
It's hard to tell
Cause if you believe (which I do) scouting has gotten better over time than it’s expected the low points on your graph would be lower in 70’s and 80’s than in the late 90’s and 00’s.
Maybe the more wild spikes in results between the 70’s and 80’s compared to the 90’s and 00’s provides the clue. It’s really the low points that are different between the eras. Wouldn’t expect a big difference in the high points because Pro Bowl / HOF players are generally going to establish their greatness early in their career and remain on the field at a high level for a long time no matter what era.
The difference is really those bust picks because the veteran is no longer there to overcome. Yes the ultra bad Vernon Gholston’s and Rashaun Woods will still produce nothing (not even good enough to beat out below average players for PT). However maybe guys like Matt Leinart and Laurence Maroney who were thrust into the lineup quickly because of necessity never see any playing time in the 70’s and 80’s but teams got some early return on investment because they did force them into the lineup, thus dulling the downward drops.
Of course other factors come into play. Certain injuries in the 70’s and 80’s would end a career early where now they can be overcome (less hazards). Teams had more picks and less resources 20-30 years ago which can account for the improvement. Maybe there is a bigger crop of more talented players coming out in the 90’s and 00’s.
sd377 wants to ban me for unleashing the Kaepernick Supernova Gamma Ray
substantive typo...
allen didn’t start as a rookie, only started as a rookie his second year because MIA had nothing at S in 2007 (see 1-15), and then basically fell off the face of the earth once coaches realized he sucked.
by (Florida) Danny Tuccitto on May 11, 2011 9:32 PM PDT up reply actions
people need to stop complaining about how long these posts are.
Florida Danny obviously puts a lot of work into these posts with all these advanced statistics so if you have time to comment “too long so i didn’t read it” then take the time to read the dam post, otherwise don’t even bother posting anything. On another note, although I did read through all of it, I think it would be helpful in your bottom line post if you could add in parenthesis or something the notable players that caused those ratings. Like for example, the Jaguars hit the home (MJD, Lewis etc.) just as a final reminder. Excellent post once again! :)
Niners, Lakers and the Warriors... when Lakers have the night off.
"Change your sig." - bluexfalcon
"no more gun and kitty pics plox." - bluexfalcon
"lol I just noticed I'm in your sig. epic." -bluexfalcon
"are you a tree trunk?" -bluexfalcon
by afrikabamboodle on May 11, 2011 5:18 PM PDT reply actions
Great post
your stats seem to suggest that bad teams have a higher likelihood of drafting a bust (based on how VAE is calculated anyway). Is it the chicken or the egg? In other words, does being drafted to a bad team increase the likelihood of ending up a bust? I know we always say that great players exceed anway, but your average players might not. Just curious if the present stats would offer insite to that question
I'm calling shennanigans...
Joe Klopfenstein is not a real person.
Currently stifling the bacon, the world.
actually...
he’s a 60s-70s children’s tv character…as my link clearly demonstrated.
by (Florida) Danny Tuccitto on May 11, 2011 7:31 PM PDT up reply actions
I think I'm seeing the same issues as Oudgas had
I’m some areas this measure is a good approximate cause I agree Denver had a great draft and Rams did horrible but then measures like this popup that don’t pass the mustard.
LB A.J. Hawk (VAE = 1.26 AV/Yr) (#5 pick)
TE Vernon Davis (VAE = 0.69 AV/Yr) (#6 pick)
Vernon Davis is a much better player than AJ Hawk. Vernon was also a pick later so his pick # shouldn’t even work against Vernon when compared to Hawk. At the beginning of the 2010 season the 49ers made Vernon the highest paid TE in the league while Hawk was presumed to be weeks away from being cut until the Packers had a rash of injuries that forced Hawk back into the lineup. This measure is more favorable to backup/role LB than a Pro Bowl TE.
I’m not here to dismiss the work cause I think you are onto something but I think a few areas need a bit of cleanup.
sd377 wants to ban me for unleashing the Kaepernick Supernova Gamma Ray
i agree...
that this is definitely not the end of the road in terms of cleaning things up, but re the exact comparison you made between hawk and VD, it’s just factually wrong that hawk has been a “backup/role LB.” the guy’s started 77 out of a possible 80 games in his career. also, it’s not like his standard stats have been even close to bad. plus, let’s not forget VD was on the verge of bust status before mike singletary whipped him into shape. 2006 to mid-2008 does count towards VD’s overall career.
to the extent that your conclusion may be right despite your characterization of hawk being wrong is because, at this point, it’s apparent that when VD’s career’s over, he’ll probably be looked at as a much more valuable player than hawk. as i’m obviously only looking at what they’ve done through 2010, VD’s current VAE doesn’t capture this idea whatsoever.
oh, and one other thing. the distinction you draw is instructive in that, according to VAE, hawk looks like he’s given double the value-add to GB that VD’s given to SF. but if you look instead at the ROIs, hawk’s is about 21.5% and VD’s is about 12.5%. that’s only 9% better return. i mean, we’re talking about a very, very small difference here.
by (Florida) Danny Tuccitto on May 11, 2011 9:07 PM PDT up reply actions 1 recs
Again, I'm trying to emphasis
It’s good work. You have many, many more hits than misses. To me the misses just indicate there are a few holes in the approach. The fact that hit on Jennings and MJD illustrates the approach mostly works.
sd377 wants to ban me for unleashing the Kaepernick Supernova Gamma Ray
oh yeah...
i understand you’re not throwing around “GIGO” like ougdas did yesterday. i was just making a couple of arguments in rebuttal. i know you’re more reasonable, so i was just continuing the discussion more than anything. sorry if it came out the wrong way. yesterday, i was just responding to unnecessary condescension and dismissal.
by (Florida) Danny Tuccitto on May 11, 2011 9:25 PM PDT up reply actions
I apologize if I came off as condescending and dismissive
But I try to take care to talk harshly only about the facts on the table, and never about the people involved.
I am just still trying to make the point that for any measure to accurately describe the value of a player it must not only be inclusive of high value but exclusive of low value and most of all, actually measure value, and not some other thing which is only a very loose measure of value.
I don’t see AV being exclusive of lower value players at all since it measures no performance on the field and gives points just for playing. I think it’s absurd that the only difference between one tackle who started 16 games and another tackle who started 16 games on the same team is bonus points for subjectively voted awards.
Your analysis is great, I just don’t see how we can trust the accuracy or resolution of the results since the input metric of AV is so heavily dependent on playing time, and with talent not being evenly distributed in the league playing time does not measure player value in a way that we can use for comparison.
If we assume that the best players in the NFL are playing then it all comes out in the wash and AV is great.
But my big hangup is that this assumption is not true because talent is not spread evenly around the league.
To me that makes AV a great metric for capturing the value of the very successful players but it is very poor at excluding the midrange and poor performers in the league who manage to get playing time.
For example.
Manny Lawson and Elvis Dumervil : Same AV
AJ Hawk, Kamerion Wimbley, and Tamba Hali : Separated by 1 point
Cortland Finnegan, Santonio Holmes, D’Brickshaw Ferguson and Kyle Williams are all equally valued.
A single point above them are Mario Williams and Brandon Marshall.
15 points above Williams and Marshall are Jahri Evans and Devin Hester.
And those are just the most obvious “WTFs” in just the 2006 draft.
That just doesn’t pass the smell test. The order and magnitudes don’t match up with what we can see on the field and the wide ranges between those grouping make it obvious that some of these fudge factors that PFR used to try to adjust their “playing time percentage” statistic are just simply wrong if the goal is to measure player value.
Yeah wait
After looking over my examples again I’m not sure AV is even a good metric for capturing the value of the very successful players.
I still think Big Number = Good Player, but when you start to compare them it’s pretty obvious that the values aren’t right.
thanks...
for the apology.
first, let me say that i understand how AV is a less-than-ideal metric. doug acknowledges same. i’m using it here because, until further notice, there’s nothing else out there that even comes close to “approximating” a value for a player’s season (or player’s value given his team’s season). even if i were to use a more “statistical” measure like, individual DVOA, there would be several criticisms based on the choice of that metric as well. i mean, as of right now, there’s no way you can tease out the individual’s contribution frorm the whole of his team. so, whil i get (and have said before on NN) that AV is flawed, at this point in time, this is just one of those “criticize without offering any alternative” situations. if there was a better measure available, i’d use it. and i think AV does a decent enough job “approximating” value, that those flaws aren’t fatal for the purposes of what i’ve done here.
second, i’m not seeing what’s so WTF about the examples you list. lawson and dumervil only have the same AV because dumervil missed all of 2010. in my database, dumervil’s AV/Yr is 7, whereas lawson’s is 5.6. wimbley and hali have very similar sack totals, and hawk plays an entirely different position. granted, part of the problem in football stats is grading defenders and comparing positions, but i wouldn’t say that hawk’s career thus far has been significantly worse at LB than wimbley’s and hali’s have been at DE, especially considering the relative success of GB, CLE, and KC. granted, i also realize, again, that extrapolating player value from team performance is an issue to consider, but i wouldn’t be the first one to conflate the 2 (see almost every MVP in every pro league ever). the same can be said about the williams/marshall comparison with evans/hester. evans has been a starting T on a SB champ, and a consensus top 5 pass offense since he’s been in the league. hester is potentially the greatest return man of all-time (showing it from day 1), and has been a big part of a SB team and an NFC conf game team. meanwhile, marshall had a quasi-nothing rookie season, is not a prolific TD-scorer, and can’t seem to shake mediocre teams. williams also had a quasi-nothing rookie season, and has been stuck on mediocre teams for 5 years.
all of that is just to say that, contrary to playing time being the biggest issue with using AV in this context, i think the much bigger issue is teasing out player value from the caliber of team the player’s on. although i think you’re making decent statistical and methodological points about AV as a flawed metric, you still haven’t convinced me that playing time is the main problem in the context of what i’m doing here. i don’t think it’s much of a problem at all relative to the larger issue of “winning teams = higher player value.” but again, even there, i don’t see much of an alternative.
my question to you is, “what would you do to make this better?” i’m open to suggestions.
by (Florida) Danny Tuccitto on May 12, 2011 10:43 AM PDT up reply actions
Well
I’m normally a big proponent of offering better suggestions, but like you are hitting on near the end of this comment it’s a herculean task to do a perfect job for the reasons you have listed about teasing out player value. I’ve really bit into what I see as the dominance of playing time because that is the part which offends my football sense the most but you are right that statistically it is not the largest problem. Like you say there is also a large issue of playing on better teams influencing AV that I’ve only mentioned in passing.
There are issues any which way you want to go about it, and the ideas I would turn to first, expected points, value over average, success rates, all have weaknesses that I can rip into just as much as AV. And simpler metrics such at TDs and yardage break so much context that it’s impossible to evaluate them without a ton of metadata.
But in my experience when you hit a point where you have so many issues you have to step back and think " Am I even asking the right question?"
Maybe if it’s just an intellectual exercise we don’t care and we use what we have at hand, flawed as it is, because we are having fun with numbers and they might show something interesting. If they are perfectly accurate or not, or if they have good resolution or not, maybe we don’t care so much.
If that’s the case, it’s still much better than the subjective.
But if it wasn’t just fun with numbers, I would probably try to modify AV to try to address some of the concerns.
First, the issue with the Total Team points. A team with better players, on average, is going to have more total team points to distribute. But there is also some contribution of coaching ability, strength of opponent defense, and of course a random factor that we can’t control. I don’t see how coaching ability can be measured apart from player talent, since to some extent that ability is manifest in the development of players. But the coaching skills of game-planning, creating matchups, and play calling are a huge part of creating those team points. So the only thing I would do to make that better is to try to normalize the team points against the “average defense” for each year. It’s not perfect, but it’s a step in the right direction.
Second, the playing time issue which I still believe is significant. I think the only way to try to account for this is to modify the points with some other metric before they are split among the players of each of the groups with the chosen metric being specific to each group. For offensive skill players maybe AV does a good enough job already given the impossibility of separating QB and WR skill, but for the O-line and Defensive players I think it could be much better.
That too is almost Herculean unless it can be automated, and brings with it it’s own issues.
If we believe the studies that show that the most consistent rate that follows a QB from team to team is sack rate, how does that effect our metrics for pass rushing positions? Do we have to track every sack ever made and normalize it out to trying to sack the avg QB? If every team played every team every year it should wash out, but get 3 high sack rate QB’s in a division with the way the NFL does the scheduling and we have problems.
Even if we could do that, we also have to take into account the sack rates per attempted pressure, and not just aggregates, which creates another layer of calculated weighing.
And since we are trying to compare players of every position against each other, shouldn’t we take into account that some positions are more important than others?
PFR tries to do this by weighting the positions according to frequency of draft, but I think it could be done much better by correlating team level statistics with winning and seeing which ones are the strongest and attempting to modify those draft frequencies with that information. That will help us answer the important question— which teams are drafting players that are contributing to the bottom line.
I have many more ideas, but as I’m sure we both can see, each level of accuracy seems to create exponential amounts of work that isn’t feasible for a human being to complete without heavy automation and play context specific information stored in a database. And even with that, it’s still a considerable amount of work to draw up the queries.
And, it’s getting much later, but not much darker (unfortunately), and I still have more comments I’d like to get to and I’ve already written a short novel here.
I’m not sure any of these ideas are good ones, or even feasible, and tomorrow I could probably poke a half dozen holes in each, but they are the first thoughts that I would entertain and run a test case or two to see how the modifications changed things.
once again great job danny....
2011 season can't come quick enough..
by AzNiner on May 11, 2011 9:24 PM PDT via mobile reply actions

by 














































