Statistical Review of the 2009 49ers: I. Overall

With the Saints (!!!) crowned champions this past Sunday, the time has arrived for my yearly statistical review of the 49ers' season. This year's review will be an 8-part series, with a new episode airing each Monday, Wednesday, and Friday from now until the end of February. Barring unforeseen circumstances, here's the schedule:

1. Overall - Thursday, 2/10/10
2. Offense - Friday, 2/12/10
3. Defense - Monday, 2/15/10
4. Special Teams - Wednesday, 2/17/10
5. QBs & RBs - Friday, 2/19/10
6. WRs & TEs - Monday, 2/22/10
7. OL - Wednesday, 2/24/10
8. DF7 & DBs - Friday, 2/26/10

For those that hadn't yet discovered the internet oasis known as Niners Nation prior to last April, the main purpose of these posts is to identify the 49ers' strengths and weaknesses based on their 2009 stats so that we can then identify the specific positions to target in free agency and the draft. Obviously, you're likely to read sentences like, "The 49ers' run DEF was awesome in 2009," but sentences like these are simply meant to convey the more-important idea that "improving their run DEF" is not an urgent need in free agency or the draft.

Nearly all of the stats you'll see in these season review articles come from Football Outsiders (FO). I'm really trying to keep these posts as short and sweet as humanly possible, so if you're unfamiliar with a stat I mention, click here to find out what it means and how it was developed. Rest assured, though, I take pride in being able to explain this stuff to people with non-statistical backgrounds. If a stat - or its interpretation - is new or particularly complicated, I'll do what I can to prevent brain implosion.

One last thing before I get started. Most of you know already that the vast majority of FO's stats are interpreted as, "Team 1/Player A is x % better/worse than the league average." Although the league average is the correct reference point from a statistical methodology perspective, I don't think many of you are statistical methodologists. You (and I) are probably more interested in knowing how the 2009 49ers compared with specific teams of interest, not the league average. From my perspective, the Niners are striving to be a playoff team, and eventually a championship team; not an average team. So, one feature I've added to this year's review is that I'll present all of the 49ers' 2009 stats alongside the corresponding stats for the average playoff team in 2009, as well as for the Super Bowl Champion New Orleans Saints.

Without further ado - sorry for all that ado I've already a-done - let's move on to my review of the 49ers' overall stats in 2009.

After the jump, much more interesting ado...

OVERALL PERFORMANCE

In 2009, the 49ers were as average in team efficiency as their 8-8 record indicated (Top-8 ranking in bold; Bottom-8 ranking in italics):

As the table shows, the Niners were merely the 19^th-most efficient team in the NFL during the 2009 season. However, this is in and of itself an accomplishment considering they were 25^th in 2008, and had not been above-average in team efficiency since 2002.

Comparing the 2009 49ers to the Saints and the 12 playoff teams, a few key points emerge. First, the 49ers are still a long way from joining the upper echelon of NFL teams. If the 2010 season is like the 2009 season, then it looks like the Niners will need to be approximately 20% more efficient to make the playoffs next season. While that's certainly doable, this year's Super Bowl champion could only muster a 15% improvement from 2008, and they started their ascent from a higher baseline and ranking than will the Niners.

Nevertheless, the fact that the Niners ended 2009 with a positive TOT DVOA gives them a nice head start going into 2010. That's because 11 of the 12 playoff teams this season finished last season with a positive TOT DVOA. The only exception was the Cincinnati Bengals, who were 19.3% below average in 2008. This season, CIN's TOT DVOA was 0.4% (20^th), which means that, low and behold, they improved about 20%, just like the Niners need to next season. So, like I said, it's certainly doable (dare I say a-doable?).

One last point I'll make - and I know this is editorializing a little bit - but the comparison between the 2009 49ers and the 2009 playoff teams suggests that a significant impediment to the Niners' playoff run this season was the fact that they started from such a low level of baseline efficiency thanks to last season. By ending 2008 with such a low TOT DVOA, what we as fans were really asking the 49ers to do this season was to somehow become about 35% more efficient in one offseason. Like the old NFL saying goes, it's much easier to go from 6-10 to 8-8 than it is to go from 8-8 to 10-6. In the spirit of this saying, what we were asking, essentially, was for the Niners to go from 6-10 to 10-6. It turns out, unfortunately, that the 2008 Dolphins and teams like them are the exception, not the rule. The good news, however, is that the Niners' 15.0% improvement from 2008 to 2009 places them halfway home, and in a better position for a similar improvement next season to actually pay playoff dividends.

OVERALL PERFORMANCE CONSISTENCY

In 2009, the 49ers were in the top quartile of the NFL in terms of week-to-week consistency, but played better towards the end of the season:

To me, the focal point of this table is that, according to Weighted (Wtd) TOT DVOA, the Niners were playing nearly as efficiently as the Super Bowl Champion Saints were towards the end of the season. Granted, part of this had to do with the Saints sitting starters during the final few weeks of the season; which, I might add, is supported by the fact that NO's Wtd TOT DVOA was over 5% worse than the average 2009 playoff team. Nevertheless, when talking about TOT DVOA of any kind, the mere mention of "Super Bowl champions" and "49ers" in the same sentence is something to behold in its own right. Indeed, this year's team had the highest Wtd TOT DVOA of any Niner team going back to 2001, when they finished the season ranked #1 at 27.3%.

With respect to TOT DVOA Variance, a picture tells a thousand words:

As the white trendline illustrates, the 49ers consistently performed in the -20% to +20% range. Indeed, only 4 of their 16 games were considerably below -20% efficiency or considerably above +20% efficiency. The trendline also reiterates the conclusion already indicated by Wtd TOT DVOA: SF performed better towards the end of the season. Essentially, the 49ers became an above-average NFL team right around the Bears game and remained one for the remainder of 2009.

Overall, then, the graph suggests that the 49ers improved their efficiency by about 20% over the course of the season, and that this improvement was gradual rather than abrupt. However, just because their improvement was gradual, it's not necessarily the case that specific events during the season didn't matter. The obvious one that comes to mind is the addition of Alex Smith and Michael Crabtree to the Niners OFF during the Texans game. Although I'll have to save a more detailed discussion for Part 2 of the season review, it's interesting to note that, between the Falcons game and the end of the season, SF's weekly performance - as indicated by the trendline - actually improved by about 40%. In other words, about half of the team's improvement beginning @ HOU was spent overcoming the decline from Games 1-5.

TRANSLATING OVERALL PERFORMANCE INTO WINS

One of the constant complaints of recent years has been that, for the 49ers, improvement doesn't seem to translate into winning. The case appears to have been the same in 2009: Despite improving their TOT DVOA by 15% over 2008, their win-loss record remained practically identical. This begs the question, "Why?"

One of the first places to look for an answer is the luck factor. And the first place to look for a general indicator of luck is a tried-and-true advanced NFL stat, Pythagorean Wins (Pyth Ws). Basically, Pyth Ws tells you how many wins a team should have, based on the number of points they scored and the number they gave up (Click here for a detailed discussion). Subtract a team's Pyth Ws from their number of actual wins, and you arrive at a value that represents how many wins a team had that weren't attributable to the scoreboard. In statistics, this leftover, unexplained number of wins is called random variation; and, in the real world, random variation is called luck. Among FO's stats, Estimated Wins (Est Ws) operates in the same exact way, except that FO uses various aspects of DVOA, rather than points for and points against, to calculate it.

So, thanks to Pyth Ws and Est Ws, we can now identify lucky teams in the NFL: lucky teams win more games than they should have, whereas unlucky teams win less than they should have. With this in mind, here are the relevant stats that give a general answer to the question, "Were the 2009 49ers unlucky?"

Typically, Est Ws agrees with Pyth Ws when it comes to showing how many games a team should have won. In the case of the 2009 49ers, however, the difference between actual wins and Pyth Ws suggests they were very unlucky, whereas the difference between actual wins and Est Ws suggests they were neither lucky nor unlucky.

The same thing holds true when we compare the Niners' fortune to the fortunes of the 12 playoff teams. Comparing the Niners Est Ws difference to that of the average playoff team, the 49ers were neither more nor less fortunate to a meaningful degree. However, relative to the average playoff team, the Niners Pyth Ws difference suggests they were considerably less fortunate.

One thing these stats clearly show is that, when measured by Pyth Ws difference and Est Ws difference, the Saints were pretty damn lucky in 2009, and were much much luckier than the 2009 49ers. In fact, by simply substracting the Niners' Pyth Ws difference and Est Ws difference from those of the Saints, we find that NO was 1.5 to 2.9 wins luckier than the 49ers. Put a different way, we can say that, out of the actual 5-win difference between the Saints and 49ers this season, about 2 to 3 of those wins can be attributed to luck of some sort.

I bet that really leaves you dissatisfied. You might be saying to yourself, "That's too general, Danny. What exactly do you mean by luck? What does luck look like in an NFL game?" Well, based on the research to date (Click here, here, & here for some of the actual research), 3 phenomena -- all of which are statistically random and generally out of a team's control - have come to represent luck in the NFL:

1. Injury
2. Fumble recovery rate (FR Rate)
3. Strength of schedule (SOS)

Very lucky teams go relatively unscathed by injuries, they recover fumbles - both theirs and their opponents' - far more than 50% of the time, and they play a relatively easy schedule. The opposite is true for very unlucky teams. So was the 49ers stagnant win total, despite their improving efficiency, due to being unlucky in these specific ways? Here are the relevant stats:

The stat I'm using for injuries here is FO's Adjusted Games Lost (AGL), which measures how many total games a team's starters lost due to injury; after adjusting for various factors. As you can see, the 49ers were by no means unlucky with respect to injuries this season. In fact, they were healthier than the average playoff team, and were  an outpatient rehab clinic compared to NO's trauma center.

Regarding SOS, the 49ers once again were actually pretty lucky having faced the 3^rd-easiest DVOA schedule in the NFL. Compared with other teams, the Niners' SOS was easier than the average playoff team, as well as this year's best playoff team. However, it is interesting to note that the average playoff team faced opponents who were below-average overall.

Finally, with respect to FR Rate, the Niners recovered just about their fair share of fumbles. For you stat geeks out there, the difference between 54.7% and random chance (aka 50.0%) is not statistically significant. Neither, of course, is the FR Rate for the average playoff team.

So, taken together, the Niners were in the Top 11 of every luck measure I presented in this section; except for Pyth Ws difference. Therefore, it's safe to say that bad luck was not the reason why SF's win improvement didn't match its TOT DVOA improvement. This mystery remains unsolved; as does the one involving a specific explanation for the Saints general good fortune, which I described earlier. I guess further research will just have to find other sources of "random variation" outside of injuries, FR Rate, and SOS.

Nevertheless, this little exercise was by no means a waste of blog space. That's because luck has this predictable tendency to average out over time. In fact, FO's research has found that each luck indicator in this section can itself predict what happens the following season. Specifically, lucky teams one season are likely to become unlucky teams the next season, and their winning percentage ends up showing the effect of bad luck. So, the fact that the 49ers seem to have been pretty lucky in 2009 should make you pause before offering any wild proclamations about the certainty of a playoff-contending Niner team in 2010.

OVERALL PERFORMANCE BY VENUE

Throughout the 2009 season, the Niners seemed to display a peculiar pattern of home/road performance. Just by eye, it sure seemed like the OFF played much better on the road, whereas the DEF played much better at home. Before I get into the specifics during future episodes of this series, here's a table showing the 49ers' overall team efficiency by venue:

As it turns out, the 49ers were among the worst teams in the NFL this season with respect to home-field advantage. Obviously, most teams play better at home than on the road (22 of the 32 teams did this season). What's astonishing, however, is that the Niners were 1 of only 10 teams to actually play below average at home. Who are the other 9 home-averse teams? From #24 to #32, they are the Jaguars, Browns, Redskins, Seahawks, Raiders, Buccaneers, Chiefs, Rams, and Lions. As these 9 teams had a combined 36-108 record in 2009, that's not company the 49ers need to be keeping.

It's one thing to say the 49ers were a bad home team. It's another thing to say that they were a better road team than home team. However, it's different altogether to say they were a really good road team, which is what you'd hope for with the first two statements being true. Unfortunately, as the above table suggests, the 2009 49ers were not, in fact, a really good road team. I sure wish they had been. It'd make me less concerned about their deficiencies at home because - switches into sport psychology hat - road success is a sign of mental toughness (or a high motivation to get away from one's family), and mental toughness is a good thing to have (See road-warrior-esque Niner teams of the 1980s).

This point is made even more clearly when we compare the 2009 49ers to the other teams in the table. Specifically, the average playoff team in 2009 was both a really good road team and a really good home team. Even the Saints, who were slightly worse than the average playoff team on the road, were nevertheless a good team on the road that played really, really good at home. So, if the 49ers are going to contend next season, the stats suggest that they need to be a much more efficient team at home.

 PERFORMANCE BY HALF

Another dead horse I beat into the ground during the season was the 49ers' penchant for starting games really inefficiently. By the end of the season, I could set my watch to it. Well, it's nice when the stats confirm what my eyes have seen; although I'd rather they confirm something a little more positive. Below is the 49ers' overall team performance by half:

As you can see, the Niners were a well-below-average team during 1^st halves, but an elite team during 2^nd halves. In fact, they were so much better during 2^nd halves that the magnitude of that difference was the 2^nd-largest in the NFL. If we consider that the #1 team in this regard, the Redskins, went from atrociously bad during 1^st halves to just plain bad during 2^nd halves, then the Niners stood alone at the top of the league in terms of being a really-bad-1^st-half/really-good-2^nd-half type of team.

As was the case with the home/road discussion earlier, the 49ers showed a superficial similarity to the Saints in terms of half-by-half efficiency. Like the 49ers, the Saints played much better during the 2^nd halves of games than during the 1^st. However, there are two subtle differences that distinguish the two teams. First, NO was by no means bad during the 1^st half. Rather, they were actually pretty good; just not as good as the 12 really-good teams ahead of them. So, analogous to their home/road split, the Saints were a good team during 1^st halves who played really, really good during 2^nd halves. Furthermore, and just as analogously, the average playoff team in 2009 was both a really good 1^st-half team and a really good 2^nd-half team.

Second - and this is probably more important - even though the 49ers resembled the Saints as 2^nd-half mavens, a major reason why the Saints just had a parade while the Niners just scheduled a tee time is that the Saints were mavens during the 2^nd halves of close games in addition to being mavens during the 2^nd halves of all games. In contrast, SF was great during 2^nd halves when the game wasn't close, but really bad during 2^nd halves when the game was close. The distinction here doesn't just apply to the Saints, either. Looking at the average playoff team's Late & Close TOT DVOA, you see that they were about 40% more efficient than the Niners in such situations.

So, taken together, if the 49ers are going to contend in 2010, the stats suggest they need to be a much more efficient team during the 1^st halves of games, as well as during the 2^nd halves of close games. It's simply not good enough to sleepwalk your way through the 1^st half, and save your playoff-caliber football for when you're getting blown out during the 2^nd half (See 2009 games @ HOU & @ GB).

BOTTOM LINE

Based on the stats I've presented in Part 1 of the season review, here are the things the Niners need to do in order to seriously contend in 2010:

1. Improve overall efficiency by about 20%
2. Build on the gradual game-to-game improvement they showed over the last 11 games this season
3. Improve efficiency at home by about 30%
4. Improve efficiency during the 1^st halves of games by about 30%
5. Improve efficiency during the 2^nd halves of close games by about 40%

Tomorrow, I'll review the OFF's performance. See you then...

*DVOA, Est Ws, AGL, and SOS statistics used to produce this article were provided by Football Outsiders.