The Impact of 1st-Round Offensive Linemen: III. What About (a Lack of) Continuity?

Welcome back for Part 3 of my series on how the drafting of Anthony Davis and Mike Iupati might impact the 49ers' statistics in 2010 (and beyond). Just to recap, Part 1 and Part 2 basically came to the same conclusion: Davis and Iupati are likely to be develop into good NFL players fairly quickly, but their impact on team statistics, and therefore winning, won't necessarily show up until 2011 at the earliest.


Today, in Part 3, I'm going to address a specific issue raised by Doug Farrar in the 49ers chapter of Football Outsiders Almanac 2010 (p. 244; emphasis mine):

Drafting two offensive linemen in the first round is a rare and highly proactive strategy designed to fix the team’s weakest link, but the importance of line continuity means that major overhauls rarely bear fruit in the first season.

Essentially, Doug's argument was that, even though the 49ers injected better talent along the OL, this increase in talent will likely be offset in 2010 by the lack of OL continuity associated with starting the new addition(s). Of course, he wasn't just making this up out of thin air. As Aaron Schatz, FO's Editor in Chief, was kind enough to inform me via e-mail, FO's research shows that the addition of rookie starters at OT has a positive statistical influence on team performance from one year to the next; and so too does continuity among OL starters. Therefore, because the 49ers added 1 rookie starter at OT, but will have 2 OL positions changing starters (LG & RT), the net effect of these changes is negative; thereby lowering their performance projection.

I don't doubt the integrity of their OL-countinuity findings whatsoever, but I do think that the 2010 49ers OFF is not an ideal context in which to apply them. Here are the 2 primary reasons why not:

After the jump, 2 reasons, 2 remedies, 2 models, 2 analyses, and 2 disclaimers...

1. Although I'm not 100% certain about this, it's almost certainly the case that the data set FO used in their OL-continuity analysis included both (a) teams that started rookie OLs, and (b) teams that did not. The 2010 49ers will clearly be in the former group of teams, so the findings of an analysis based on both groups might not apply to one or the other individually. And the main reason why I'm suspicous that they would apply is because...
2. Drafting and starting rookie OLs disrupts OL continuity by definition. As I showed in Part 1, the talent level of a rookie OL starter has no bearing on team stats. Combine these 2 facts, and you realize that such teams were low performance and low continuity simply because they happened to start a rookie OL. In other words, what looks like a continuity effect in FO's anaysis is quite likely to have been a rookie-OL-starter effect in disguise.

To adequately address these 2 issues in a way that makes an analysis of OL continuity more applicable to the 2010 49ers, we can introduce the following adjustments:

1. Control for the inherent differences between teams that do and do not start rookie OLs by examining the continuity effect for each type of team seprately. In other words, analyze data from the "started rookie OLs" group in one analysis, and analyze data from the "did not start rookie OLs" group in a separate analysis.
2. Control for the lack of continuity necessarily associated with starting rookie OLs by measuring continuity on a team's OL outside of turnover in the position at which the rookie OL starts. In other words, if a rookie starts at LT, only look at that team's lack of continuity at LG, C, RG, and RT.

In the analysis I'm about to present, I made both of these adjustments. Namely, I limited my data to include only those teams that started rookie OLs, and I measured continuity for the rest of the OL. The specific continuity measure I used was "total number of games started in Year 1 by a given team's primary OL starters the previous year." As an illustration, the 49ers' primary OL starters in 2006 were Jonas Jennings, Larry Allen, Eric Heitmann, Justin Smiley, and Kwame Harris. The following season, Joe Staley started as a rookie, replacing Harris at RT. Therefore, the Niners' continuity score in Staley's rookie year equals the total number of games started by Jennings, Allen, Heitmann, and Smiley in 2007: 5 + 16 + 16 +8 = 45 (Note: The results of my analysis end up being the same even if I just measured continuity as total number of primary OL starters a team carried over from one year to the next, ignoring the rookie OL starter. For example, measured this way, the 2007 49ers' continuity score would be Jennings + Allen + Heitmann + Smiley = 4. I chose the total-games-started route because there's more information underlying games started than bodies).

TWO MODELS

Continuing in the vein of Part 2, finding that OL continuity predicts winnning doesn't go far enough. There's nothing inherent in continuity that should mean more wins. If it did, football scoreboards would show continuity score rather than points. So, we have to answer that "why?" question. And just like before, my answer involves what OLs actually do, i.e., block for runners and protect passers. And as before, I'll put this in the form of a theory diagram:

And again, let's add the real world along with our actual measures (same substitions as Part 2 for the running model apply here as well) :

Unlike what I did in Part 2, however, today I'm only going to test this theory in Year 1 of the OL's career. That's simply because FO's argument that lack of OL continuity will have a negative impact on the 49ers' OFF was in FOA 2010. I'm not Nostradamus -- at least not today -- so I have no idea whether that will be their argument in FOA 2011 or FOA 2012. After all, I can always revisit the issue to test OL continuity in future years.

TWO ANALYSES

I have a little more space available today, so I'll go into a little more detail about the structural equation modelling (SEM) analysis itself. Most important of these details is that the math behind SEM is matrix algebra, as SEM analyses assess the fit of theoretical models to a correlation matrix (Note: Actually, it's the covariance matrix, which can be transformed mathematically into a correlation matrix; but I'm using "correlation matrix" because only the hardcore nerds here will know what a covariance matrix is.). Here's the correlation matrix being fit to my passing game model:

And the correlation matrix being fit to my running game model looks like this:

In each of these matrices, the correlations relevant to the respective models are in bold. Taking a quick look at these correlations, you can already see that OL continuity in a starting OL's rookie year is not related to blocking, neither in the passing game nor in the running game. Furthermore, even if we ignore the nonsignificant magnitudes of the continuity-blocking correlations, they're actually in the opposite direction of the theoretical expectation. In other words, when a team starts a rookie OL, higher continuity along the rest of the OL is actually related to worse pass protection (i.e., higher ASR) and worse run blocking (i.e., lower ALY).

In SEM analysis, this survey of relevant correlations is a preliminary step to actually testing the model. If I weren't writing an article for NN right now, I'd stop at this step because it's pretty obvious the continuity-to-blocking path in both models is not going to be significant, and therefore the models are going to fit suboptimally. However, I ran the model tests anyway just for the sake of illustration. My results were as follows:

1. The passing model did not fit the data well. Therefore, the theory that OL continuity from year to year indirectly improves Ws through improved pass protection was not supported in the context of teams who start a rookie OL.
2. The running model did not fit the data well. Therefore, the theory that OL continuity from year to year indirectly improves Ws through improved run blocking was not supported in the context of teams who start a rookie OL.

Shocking, I know. What might be a little more shocking, however, is just how badly the models fit the matrices. For a point of reference, the best model in Part 2, the Year-3 running game model, had non-sample-size-dependent fit indices of .955 and .929 (anything better than .90 is good, and 1.00 is essentially the upper limit). In comparison, the running game continuity model had fit indices of .885 and .794; and the passing game continuity model had fit indices of .845 and .722. So, it's plainly evident that, at least among teams who start rookie OLs, there's no support for the theory that OL continuity along the rest of the line has a positive impact on wins because of improved blocking.

If these results are to be believed, then Doug's contention in the 49ers chapter of FOA 2010 that lack of OL continuity will more than offset the addition of starting Anthony Davis is not supported by the past 16 years of NFL football. Of course, that's not to say that FOA's research regarding the importance of OL continuity is wrong. It could very well be -- and is actually highly likely given the nonsignificant results here -- that continuity does positively impact team wins from one season to the next for teams who do not have a rookie OL as a full-time starter. But, again, my results clearly show that continuity doesn't matter for those who do start a rookie OL. Because the 2010 49ers exemplify the latter group, I don't believe Doug's argument holds up in this specific context.

TWO DISCLAIMERS

I'll once again offer the same caveats that I offered in Part 2: low sample size (n = 57) and the likelihood of other factors that were not included in my models. And just as was the case in Part 2, I'll leave it for the comments section to discuss this if any of you wish to. I will say this, though. It could be the case that the next 16 years will see an explosion of continuity-related improvements in blocking -- and thereby wins. However, even with a doubling of the sample size, and the entire addition of sample data reflecting continuity-based improvements, the magnitude of just how poorly my 2 models fit, and just how nonsignificant the prediction of blocking improvements from OL continuity was, I'm skeptical that the bottom line would change here. Speaking of bottom lines...

BOTTOM LINE

From this, the final installment of my OL series, the major things to remember are as follows:

1. The concept of OL continuity is inherently flawed in any analysis unless you treat teams who start rookie OLs differently from teams who don't. Starting a rookie OL is lack of continuity by definition.
2. For teams like the 2010 49ers, who are going to be starting (at least) 1 rookie OL, continuity along the rest of the line has no impact on win improvement from the previous season. Because of this statistical finding, part of the reasoning offered in FOA 2010 for an offensive decline by the 49ers this season is not supported by the available evidence.

If you stuck around for all 3 parts of the series, I really appreciate your attention. Now that training camp is about to start, I'll be posting as regularly as I have the past few seasons. I'm pretty sure I'll have a post on offensive coordinator continuity, and I'm sure I'll have other little statistical tidbits to pass along between now and Week 1. Stay tuned. As for now, comment away!

*DVOA, ALY, and ASR statistics used in this article were provided by Football Outsiders.