Jim Harbaugh, 49ers Hire Vic Fangio: A Brief Statistical Evaluation of the New DC

Now that Vic Fangio's been hired, this seems downright prescient. In yesterday's Golden Nuggets, Fooch linked to Matt Barrows' Twitter Tuesday post, wherein reader, Anthony1642, asked the following question:

is there a concern that Vic F's defenses seemed to regress statistically each year he was DC?

Unless you've been under a rock for the past few days, "Vic F" is Vic Fangio, the freshly hired defensive coordinator (DC) of the San Francisco 49ers. Given that this hiring was the worst kept secret since "Baalke Hired as GM," I'm assuming many of you are already familiar with Fangio's biography. For those who aren't, here's his pre-Niner coaching résumé:

So, Vic "The Fang" Fangio spent 24 years as an NFL assistant coach prior to being hired at Stanford. More importantly, my conveniently bolded excerpt shows that Fangio was a defensive coordinator for 11 of those 24 years: twice serving 4 years with an expansion team led by Dom Capers, along with a 3-year stint for Jim Mora's Colts. These are the "Vic F defenses" that Barrows' reader was expressing his concern about. (Aside: Interesting how Fangio followed Jim Mora around [Saints and Colts], followed Dom Capers around [Panthers and Texans], and now seems to be following members of the Harbaugh family around [Ravens and 49ers]. The NFL's not an ol' boys network...at all!)

When you look at the total yardage rankings related to Fangio's previous DC tenures, the underlying premise of the Twitter question seems to check out: In CAR, his defense went from 7th-best in 1995 to 10th in 1996 to15th in 1997 to 30th in his (and Capers') last season. In IND, Fangio's unit finished 15th in 1999, then 21st in 2000, then 29th in 2001. Finally, after reuniting with Capers in HOU, the Fang's Texan defense ended 2002 at #16, 2003 at #31, 2004 at #23, and 2005 at #31 again.

Fooch likes to constantly remind us that he hates the saying, ""There are three kinds of lies: lies, damned lies, and statistics." In his honor, the question today is, "Are these stats lying?" Of course, we don't want to stroke his ego too much, so this post will also be a general statistical evaluation of Vic Fangio, Defensive Coordinator of your San Francisco 49ers.

After the jump, Vic’s résumé in a statistical Fanguage...

PREVAR-VIC-ATION

First, let's examine whether the theory of the incredible shrinking Vic Fangio defense is reality or a lying-stat-driven myth. This one's pretty easy. Assuming someone's not using a statistic to lie to you on purpose, the lying statistic scenario basically results from either (a) the person using the statistic isn't very good at using statistics, (b) the statistic itself isn't very good, or (c) a combination of the two.

Clearly, (b) is applicable here. That's because the results of just about every bit of research that's ever been done on the topic -and just plain old football common sense - tells us that yards are not created equally. This should really go without saying, but the yard between the opponent's 1 and goal line on a 4th-and-1 play with 5 seconds left in the 4th quarter is far more valuable than the yard between a team's own 41 and 42 on a 1st-and-10 play with 5 minutes left in the 1st quarter. Therefore, any statistic like total yardage, which treats each yard equally, is a bad statistic. Now, add on top that "counting stats" like total yardage, which simply tally up a bunch equally weighted numbers, are meaningless because some teams have more opportunities to accumulate yards than others (i.e., they run or defend against more plays), and you have a reeeeeeeeeally bad statistic.

Of course, that fact that there's a mountain of readily available evidence proving the worthlessness of total yardage implies that anyone who cites total yardage as evidence in any argument ever isn't very good at using statistics. That doesn't mean people who do so are idiots. Total yardage has been the ranking statistic-of-choice for eons in the NFL, and it's a very easy number for the average fan to comprehend; hence its rhetorical omnipresence on NFL telecasts and water cooler arguments of all kinds. Expecting to change this state of affairs is like Sisyphus expecting to get that boulder over the top of the hill; but suffice it to say that, if you learn anything from me during our time here on Niners Nation, just remember to never believe an NFL-related argument based on total yardage.

You say you're sick of my pontificating, and want me to show you Fang's stats already? Great! Because my little diatribe dovetails nicely with a more general discussion of Vic's NFL track record. Now that he's been hired, that's probably a good discussion to have.

FANGS AND VICURES

Alright, let's start with the Vic F's overall rankings based on some better stats than the now-discredited total yardage:

• Defense Simple Rating System (SRS) - Pro Football Reference's (P-F-R) user-friendly team rating statistic based on margin of victory and strength of schedule.
• Defense DVOA - my statistic of choice from Football Outsiders (FO), which is based on play success rate and strength of schedule.
• Defense Yards per Play (Yds/P) - your typical total yardage statistic that adjusts for the number of plays each team defends against, thereby eliminating at least one of the major problems with total yardage.

And for the sake of comparison, I've included total yardage (Yds) rankings in the table, which I present to you below:

As I mentioned earlier, yes, Fang's defenses have seemed to get worse over time at each of his 3 previous DC stops according to the bad statistic known as total yardage. And, unsurprisingly, the same seems to be true according to total yardage's cousin, Yds/P.

The same can't be said, however, when we look instead at better statistics like SRS and DVOA. Except for Fang's first season in CAR according to SRS, both stats tell the same basic story:

• In CAR, the defense was really good for 2 years and bad for 2 years.
• In IND, the defense actually got better in Fang's second season.
• In HOU, Fang's defense got worse, then better, then worse again.

That's a pretty clear picture of how bad stats have the potential lie to you, but good stats don't. According to the good stats, SRS and DVOA, it's apparent that Fang's defenses didn't show the tidy year-by-year regression suggested by the inferior stats, Yds and Yds/P. I'm going to go ahead and believe the good stats, and deem the bad stats liars.

OK...that's overall defense. Of course, a defense's plays can be broken down into defending passes and defending runs. Furthermore, given the proliferation of aerial attacks in today's NFL - not to mention the fact that the passing game performance is most predictive of winning - I'm guessing it'd be useful to see whether Fang's pass defenses regressed during his previous 3 DC gigs.

Because P-F-R's SRS stat is based on points rather than yards, there's no such thing as Pass Defense SRS or Run Defense SRS. Therefore, the next 2 tables replace SRS with the following efficiency-based, relatively advanced statistics advocated by both P-F-R and Brian Burke of Advanced NFL Stats:

• Net Yards per Pass Attempt (nYPA) - this is your basic yards-per-attempt stat, except it includes sack yards in the "yards" part, and includes sacks in the "attempts" part.
• Rushing Yards per Carry (YPC) - I'm assuming that everyone's familiar with this one, and, if not, it's pretty self-explanatory.

Below is the table showing the rankings of Fang-led NFL defenses against opposing pass offenses (Pa = Pass):

Well, this is interesting. Our bad stat (PaYds) doesn't show any kind of discernable pattern for any of Fang's 3 tenures. However, now there is, in fact, a pattern of PaDVOA regression during Fang's tenure in CAR, as well as a nYPA regression when he was in IND (and perhaps CAR if you loosen your definition of "gets worse over time"). But before you go thinking this proves the theory, remember that, with 2 stats and 3 tenures, there were 6 opportunities to see the regression pattern. Yet, it only shows up in 2 (or perhaps 3) of them. That's by no means enough evidence to say Fango's defense, in general, have a tendency to get worse over time. If the Niners win 2 or 3 of their first 6 games, you wouldn't say they have a tendency to win. In fact, you'd probably say the opposite.

Now for Fang's run defense; below is the table showing the rankings (Ru = Run):

Here, RuYds maybe - maybe! - shows a regression pattern in IND. On the other hand, RuDVOA shows no regression pattern, and YPC only shows regression in HOU. So, again, although there are a couple of specific instances of Fang's squads getting worse over time according to good statistics, there's certainly not enough evidence to consider it a defining feature of Fang's NFL defenses. And looking forward, there's no reason to believe any kind of regression pattern will show up during his tenure with the Niners.

THE TWO ELEFANGS IN THE ROOM

So I think I've demonstrated that there's no statistical reason to worry about a Fang-led 49ers defense getting worse with age. However, let's say that I had demonstrated the opposite. Let's say I had shown that, even according to good stats like DVOA, SRS, etc., there was indeed a clearly discernible pattern of regression in CAR, IND, and HOU. Well, even in that case, there are 2 huge elephants in the room that would still caution us against predicting that the 49ers' defense will get worse over time too:

1. Sample size
2. Context

With regards to sample size, I don't know many statisticians who would make bold predictions based on 12 observations, nor would they do so based on 3 groups of 3 or 4 observations. I'm not going to turn this into a statistics class, but suffice it to say that there's a reason why political polls don't survey only 3 people. The probability that the opinions of these 3 people are representative of the opinions of the entire US population is ridiculously small. Analogously, the probability that Fang's DC tenures in CAR, IND, and HOU are representative of the entire universe of possible Fang defenses is also ridiculously small. Therefore, to extrapolate a pattern of regression - if it existed - to Fang's new job in SF based on a sample of 3 would be pretty laughable statistically speaking.

One reason why small sample sizes are especially pernicious in making predictions about football is because performance is highly reliant on context, particular with respect to the symbiotic relationship between coaching and player personnel. Jon Gruden couldn't win a Super Bowl without Brad Johnson and Tony Dungy's former defensive players. Brian Billick couldn't win a Super Bowl without a great defense and Tony Dungy's former quarterback. Tony Dungy couldn't win a Super Bowl with the guys he groomed for Gruden and Billick, but he could with a QB originally groomed by Jim Mora. Huh? Untangling these contextual webs is something you simply have to do when evaluating a coach or player statistically; the difficulty of doing so is one reason why people are so skeptical of using stats to talk about football.

The context of Fangio's previous DC gigs is instructive here. He's been the DC for 2 expansion teams (CAR & HOU) and a 3rd team (IND) that was ranked 28th in Defense DVOA and 27th in Defensive SRS the season before he got there. So really, we're talking about an incredibly bare cupboard of talent for The Fang when he arrived at each stop. In that context, shouldn't we instead be focusing on how the 1995 Panthers, 1999 Colts, and 2002 Texans actually overachieved in Fang's first season as DC?

Furthermore, when you look at the context of Fang's best defensive units according to the stats, the 1995 &1996 Panthers, you notice a veteran 3-4 defense (average age about 30), with Sam Mills at LILB, Carlton Bailey at RILB, Kevin Greene at LOLB, and Lamar Lathon at ROLB. As I and others have reiterated a million times on here, the 3-4 is all about the LBs. That was a hell of a LB unit that unsurprisingly started to decline when Greene left for SF in 1997. Oh, and did I mention Fang's gig as LB coach for the Dome Patrol Saints of the 1980s, one of the best LB corps of all time?

In contrast to those successful LB-rich defenses, Fang actually had to run a 4-3 in IND because he only had 3 good LBs. The 4-3 revolves around the DL, and he didn't have a good one...at all. Check out Pro Football Reference's team pages for the 1999-2001 Colts, and you'll see what I mean. Interestingly enough, the LBs on those defenses were by far the best unit talent-wise (Cornelius Bennett or Marcus Washington at LLB, Dwight Hollier or Rob Morris at MLB, and Mike Peterson at RLB), as well as performance-wise.

When he went to HOU, he apparently had enough of the 4-3 because he installed a 3-4 defense...expansion roster be damned! Unlike his previous 3-4 defenses, the LBs on the 2002-2005 Texans were basically a bunch of nobodies. Is it any wonder, then, that Fang's 3-4 defense was pretty bad in HOU?

So, in my view, you can't divorce the performance of Fang's previous defenses to the context in which he coached them. He was positional coach for one of the greatest LB corps of all time as part of a successful 3-4 defense in NO, and he ran a Top-Five 3-4 defense with great, veteran LBs in CAR. At his less successful stops, Fang didn't even use the 3-4 in IND because he didn't have enough good LBs, and used the 3-4 by default in HOU because he didn't have enough talent, LB or otherwise. Given this contextual analysis, and the inevitability that Fang will be running the 3-4 in SF, the question we should be asking is not, "Is there a concern that Fangio's defenses seemed to regress statistically each year he was DC?" Rather, the question should be, "Is the 49ers' talent at LB more like the talent in NO during the 80s and CAR during the mid-90s or more like HOU's during the early-00s?" When it comes to the 49ers' performance on defense over the next few seasons, the answer to the latter question will prove to be far more meaningful than the answer to the former.

BOTTOM LINE

1. The total yardage stats and rankings for Fang's previous NFL defenses are meaningless.
2. According to better stats like DVOA, SRS, nYPA, and YPC, there's no pattern of Fang's NFL defenses getting worse every subsequent season.
3. Even if there was a pattern, it would still be meaningless because it's based on an incredibly small, unrepresentative sample. I bet I could find a pattern of shoddier and shoddier work over the course of 3 years in your employment past.
4. Even if there was a pattern, who cares? This is the NFL. You have to put any statistical analysis into the context of Vic's previous DC gigs. Going forward, the most important thing for the 49ers is whether or not he's got the talent at LB to run a successful 3-4 defense in SF. If he does, I bet you'd forget all about that pattern you saw.