Efficiency metrics say Chip Kelly’s offense had no effect on his defense

There’s an abundance of narratives that follow Chip Kelly. Reporters inquired about many of these during his opening press conference but one of the narratives grew wings and took off since the San Francisco 49ers introduced Kelly as their new head coach.

On Friday, Chris Biderman posted an article saying "There is statistical evidence that proves Chip Kelly should consider slowing things down on offense," generally capturing the argument many are making about the stress Kelly’s offense puts on his defense. The logic of the article is straight forward and intuitive: Chip Kelly’s offense played so fast that the Eagles defense played an outsized number of snaps, directly contributing to poor defensive performance. Had Kelly’s offenses played slower, the defense would have played better.

Biderman falls into a trap that I don't blame him for, because it is one that trips up many people. In order for this argument to work we must ignore that Kelly’s offensive success is in many ways predicated on tempo. If you remove the tempo, you also remove a key driver of the offense’s success. An ineffective offense would affect the defense in ways people don’t account for when they advance the "slow down" argument.

The bigger problem arises when people try and use statistics like total yards, yards per play, and points per play as key points of evidence. These numbers are not the best or most complete indicators of efficiency so they often paint an incomplete or incorrect picture.

By and large, total yards is one of the least effective measures of efficiency. If you need 20 yards for a first down, and you get 15, it looks good on the stat sheet but the drive still stalls. Yardage gains in garbage time further exacerbate the issue by inflating numbers when the outcome of the game is decided. Yards per play is certainly better than total yards, but important contextual variables like down and distance make this an incomplete measure. In the example above, the yards per play is still high, but not high enough to get a first down, which was kind of the point of the play.

Points per play is essentially a proxy for explosiveness and not much else. Much like the previous numbers, the statistic is not wholly unimportant. These metrics can be directionally indicative of offensive efficiency, but there are better and more complete measures for evaluating offensive and defensive efficiency.

Football Outsiders’ Defense-Adjusted Value Over Average, more commonly referred to as DVOA, solves many of the problems that plague looking at total yardage stats, and add variables to something like points per plat to make it more complete. I’d highly recommend reading the full explanation, but the too-long-didn’t-read version goes a little something like this.

Each play gets a success number based on several factors including yardage gained, down and distance, field location, and results like turnovers and touchdowns. That number is compared to average success numbers for plays from all teams in similar situations, and as the name suggests, adjusted for opponent. The resulting output is presented as a percentage score above or below zero, or average. For offenses a positive percentage is best and for defenses, because you want to hold teams below average, a negative score is best.

We see the value of this statistic when applied to a real game. This season, the 49ers posted their best yardage total versus the Pittsburgh Steelers. The 49ers had more first downs than the Steelers, averaged more yards per rush, and won the time of possession battle. Yet, the 49ers still ended up on the business end of a 43 to 18 blow out. Their offensive DVOA for the game? -21.7 percent, meaning their offensive efficiency was 21.7 percent below how we would expect average teams to perform.

What, then, does DVOA say about Philadelphia’s defensive performance under Chip Kelly? If the defense’s time on the field indeed impacts performance we would expect to see a dip not only in the second half of games, but also in the latter half of the season. The numbers, however, indicate something different than the prevailing narrative.

In each season, Kelly’s defense actually performed better in the second half of games than they did in the first. And in Kelly’s first two years his defense also performed better over the final 8 games, indicating that time on the field was not really to blame for the defensive performance. The only season where we see what’s "expected" is 2015. The data simply does not support the prevailing narrative that Chip Kelly’s offense negatively impacts his defense.

The New England Patriots are another team that expertly manipulates tempo to their advantage. In 2013 and 2014 they ended the season 3rd and 2nd respectively in overall pace. They are also playing in their 5th consecutive AFC Championship game, meaning they have been playing 18 to 19 game seasons for half a decade. Yet, their defensive failings are not placed at the feet of tempo.

Kelly’s offensive philosophy is an easy scapegoat. It’s much easier to blame the offense’s pace than it is to identify deficiencies in talent, scheme, and play calling. It’s human to point the finger and blame something strange and new. But in this case there is simply little evidence that Kelly’s tempo causes deficiencies in defensive performance.