AUTHOR'S NOTE: This is Part 3 of a 5-part series on predicting the career performance of NFL QBs. For the rest of the week, I'll be posting a new installment every day at 4 p.m. PDT.
Yesterday, in Part 2 of this series, I did some deep thinking about the purpose of NFL-outsider QB projection and the various methodological decisions that follow from that purpose. I'm still finalizing the details of my QB prediction model, which builds off of my thoughts in Part 2, so today I'm going to just throw the floor open for discussion about some preliminary things I've found in my analyses.
The first thing I did was to go back to the original data I used for my Lewin Career Forecast re-analysis of 2 years ago, and collected data on a few more variables to see if there were any meaningful correlations that hadn't been included previously. As I alluded to yesterday, this is all made possible by the fact that we're not so focused anymore on projecting QBs before the draft. So, without further delay, here are some of the new predictors I looked at:
- Whether or not the QB graduated from 1 of the 6 major BCS conferences
- Whether or not there was a new, incoming head coach to the NFL team that drafted them
- If drafted onto a team with a returning head coach, how long had he been with the team
- Whether or not their NFL head coach -- whether incoming or returning -- had an offensive coaching background
- How many wins the team that drafted them had 1, 2, and 3 seasons before they were picked, along with the average of those 3 seasons
- The Total, Offense, Pass Offense, Rush Offense, and Defense DVOA the team that drafted them had 1, 2, and 3 seasons before they were picked, along with the average of those 3 seeasons for each of the 4 types of DVOA
- What number they were picked
- Where they were in the QB order of selection (e.g., 1st QB taken, 2nd QB taken, etc.)
- How many NFL-eligible games passed before they became their team's primary starter
After the jump, I'll tell you what I found, and offer up some questions for discussion...
Before I talk about the results, let me just mention that the only difference between the QB sample I used 2 years ago, and the one I used for this analysis was that the prior one included 35 QBs drafted in the first 2 rounds from 1997-2006, whereas this one includes the 53 QBs drafted in the first 2 rounds from 1993-2008. The main reason for this was that, since my post 2 years ago, Football Outsiders has taken DVOA all the way back to 1992, so I could now look at "Year N-1 team performance" going all the way back to the 1993 draft. Of course, it doesn't hurt that it also increases the sample size.
As I'm sure you'd expect if you read yesterday's post, fantasy football points per game FFPts/G is what I'm using to measure NFL performance. Without further ado, here's what I found to be meaningfully related to FFPts/G among the new variables I listed before the jump (correlation coefficients in parentheses):
- QBs taken by a team whose head coach had a defensive coaching background scored more FFPts/G during their careers (r = -.199).
- The fewer wins a QB's team had the season before they were drafted, the more FFPts/G he scored during his career (r = -.255).
- The worse Total DVOA a QB's team had the season before they were drafted, the more FFPts/G he scored during his career (r = -.229).
- The worse Defense DVOA a QB's team had the season before they were drafted, the more FFPts/G he scored during his career (r = .232, the correlation is positive because a higher number for Defense DVOA means worse defense).
- The higher a QB was picked in the draft, the more FFPts/G he scored during his career (r = .444).
- The higher a QB was picked in the QB order of selection, the more FFPts/G he scored during his career (r = .353).
Before I open it up for discussion, let me just make 3 quick observations. First, except for those regarding how high the QB was picked, the other 4 meaningful correlations aren't mind-bogglingly large. In fact, if we weren't talking about football anaysis, where high correlations are as rare as a smart gameday decision by recent Niner head coaches, these would be considered "small." Nevertheless, we are talking about football analysis, and even the smaller correlations are statistically signicant with 95% confidence.
Second, it seems to me that findings 2, 3, and 5 are all related to the fact that worse teams have higher picks, and therefore are in a more advantageous position for selecting the most talented QBs. Total DVOA is very highly correlated with Ws, and draft picks are awarded based on reverse order of wins. What's a little less clear to me is how findings 4 and 6 fit into this. Although worse teams have worse defenses more often than not, they don't necessarily do (See San Francisco 49ers teams of recent vintage); as the correlation between Total DVOA and Defense DVOA is closer to 0.50 than it is to 1.00. Also, given that the purpose of using" QB order of selection" instead of "pick number" is to adjust for the phenomenon whereby a QB goes higher than he otherwise would have simply because a specific high-pick team needs a QB, I'm not sure the relationship between QB order of selection and FFPts/G is related to the "bad teams get high picks, and that's where the best QBs are" phenomenon.
Finally, I'm kind of at a loss for why QBs do better when drafted to defensive-minded head coaches. As I said earlier, the correlation isn't that big, so I'm not going to get carried away with this. Nevertheless, it does seem interesting, and worth figuring out. I have a couple of theories that I'll leave for the comment thread (e.g., is it just because Peyton Manning, by far the best QB in the sample, was drafted by former LB coach, Jim Mora, Sr.)?