Optimism and the 2009 San Francisco 49ers: Will They Make the Quantum Leap?

AUTHOR'S NOTE: I haven't posted in over a month, and we have a fair amount of newcomers to Niners Nation, so I figure a quick (re)introduction is in order. As you can see, my name's Danny. I'm currently getting my PhD in sport psychology at the University of Florida (but actually reside in Fort Lauderdale, FL...don't ask). I've been a 49er fan since the team's Super Bowl XIX victory over the hometown Dolphins on my 7^th birthday in 1985. My job title on Niners Nation is "Resident Stats Nerd" or something to that effect. During the season, I do a statistical preview and review of each 49ers' game. In addition to these game-specific posts, I also write up general stat-based articles on 49er-related news of the moment. You can find my previous posts here. Hopefully, you enjoy reading my articles, or, at the very least, learn something 49er-related from them. Two warnings, though: (1) My articles tend to be on the longish side; and (2) I fully realize that stats can only go so far in explaining and predicting football performance. If you don't like what the stats seem to suggest, feel free to ignore them. Oh, and one last thing...I mostly rely on defense-adjusted value over average (DVOA) and other alternative statistics developed by Football Outsiders (FO) because they're the best measures of NFL performance currently available to the public. For an explanation of FO's stats, see here.

For the second time in 3 years, there's a good bit of optimism surrounding our beloved 49ers (not here, of course). Many fans (including this one) feel a real sense that 2009 will be the season that ends SF's 6-year playoff drought. The reasons for such optimism are numerous. First, there's the whole, "they finally have a competent head coach" thing. Second, there's the fact that they finally have a real WR corps (thank you Al Davis). Third, they actually seem to be - gasp - tailoring their offensive and defensive schemes around the players' strengths and weaknesses. Fourth, rather than just putting up a words-not-deeds-style banner, they've actually enumerated a "Formula for Success" that tells players and fans exactly how the team plans to "Win the West."

So the 49ers' goal this upcoming season - in the minds of players, coaches, and fans alike - is to make the playoffs. As ARI showed last season - and as PIT showed in 2005 - anything can happen once a team gets in the postseason tourney. In this article, I'm going to address the general question, "Is our optimism justified from a statistical perspective?" More specifically, I'm going to look at the major statistical characteristics of teams that make the quantum leap into playoff participation from one season to the next, and evaluate whether or not the 2009 49ers appear to fit the profile.

After the jump, I'll try to stay optimistic...

IDENTIFYING GROUPS

In the current NFL, there are two groups of teams each season: A group of 12 who make the playoffs and a group of 20 who don't. But what happens to these teams the following season? Do the teams who make the playoffs one season, follow it up with another playoff appearance the next? Conversely, do the teams who miss the playoffs one season, follow it up with another January chock-full of tee times? If we take into account each team's playoff status in Season A, as well as its playoff status the next season, Season B, then we get 4 groups of teams:

1. Repeaters - teams that made the playoffs in Seasons A and B.
2. Fallers - teams that missed the playoffs in Season B after making them in Season A.
3. Leapers - teams that made the playoffs in Season B after missing them in Season A.
4. No-Shows - teams that missed the playoffs in Seasons A and B.

Using this categorization, there have been 39 Repeaters, 45 Fallers, 45 Leapers, and 94 No-Shows since 2002 (The 2002 Texans aren't included because they didn't play in 2001). As Don Banks of SI noted, these totals indicate that about 6 teams per season make the quantum leap into playoff participation. Based on the fact that the 2008 49ers didn't make the playoffs, this season's incarnation has the potential to be one of these Leapers. Of course, they also have the potential to be a No-Show. Therefore, it's useful to identify what differentiates these two groups. In other words, what changes occur from one season to the next that distinguish teams who make the playoff leap from those who have consecutive seasons ending in Week 17?

Obviously, there are a whole host of usual suspects here. Perhaps the Leapers change coaches, perhaps the Leapers change QBs, perhaps the Leapers get substantially better on offense and/or defense, or perhaps the Leapers simply end up with an easier schedule than the one they played the previous season. Not coincidentally, these are exactly the factors I'll be looking at for the rest of this artricle.

HEAD COACHING CHANGES

It's not news to anyone that the Niners' head coach in Week 1 of this upcoming season will be different from the dapper fellow who manned the sidelines in Week 1 of 2008. In my statistical world, that qualifies as a coaching change. We can quibble about how Mike Singletary was a mid-season replacement last season, but the value of a full offseason and maximum flexibility over assistant coach hires can't be overstated.

Since 2002, 36 of the 139 non-playoff teams had a new head coach in Week 1 of the following season. Not surprisingly, these 36 teams were generally the worst of the worst before the coaching change, especially when it came to passing offense. Specifically, among these 139 non-playoff teams, the 36 teams that changed head coaches averaged about 2 fewer wins (4.8 vs. 6.6), and had an average pass offense DVOA* ranking 4 spots lower, than the 103 non-playoff teams that didn't change coaches. So lesson #1 is that, for the most part, non-playoff teams change head coaches when they (a) suck reeeeeeeeally bad, and (b) can't pass the ball to save their lives. Looks like the Niners were a year late on firing Mike Nolan after all.

So how have these coach-changers fared in the season after making the switch? Well, 10 of the 36, or 27.8%, made the playoffs the next season. Three examples that fit the bill in 2008 were the Dolphins, Falcons, and Ravens. Considering that these new head coaches were taking the helm of sinking ships, these seem like Jack Sparrow-esque performances (without the grog-induced slurred speech). However, to get a valid read on this "leap rate," we need to compare it to the performance of the 103 non-playoff teams who didn't change coaches. In other words, how much did a coaching change matter among non-playoff teams?

It turns out not all that much. The leap rate for coach-keepers was 34.0%, or 35 out of 103. The 6.2% leap rate difference between the coach-changers and the coach-keepers is not even close to being statistically significant, which means that the coaching change had no meaningful impact on making the playoff leap above and beyond what would have been expected from dumb luck. Basically, about 30% of non-playoff teams have made the playoff leap the following season regardless of whether they changed coaches or not, a percentage that corresponds perfectly with the average playoff turnover of about 6 teams per season (i.e., 6 out of 20 is 30%).

Obviously, each specific coaching change occurs in its own context, and the move from Nolan to Singletary is no exception. All the stats tell us here is that, based on the last 7 years of the NFL, our optimism about a 49er playoff appearance in 2009 should not rest on the shoulders of Samurai Mike.

QUARTERBACK CHANGES

Another major change (for the better) this offseason has been the jettisoning of J.T. O'Mulligan as the 49ers' Week 1 starting QB. As was the case with the coaching change, a full offseason of first team reps in OTAs and training camp trumps "he actually became the starter last season" when it comes to classifying "Shaun Hill, 2009 Week 1 starter" as a QB change. Also, in figuring out the "starter" for a given team in a given season, the preferred measure is whether or not he started a majority of the team's 16 games. When there's no majority, the tiebreaker goes to the QB who was the originally designated starter that season. As this relates to the 49ers, neither Shaun Hill (8 starts) nor O'Mulligan (8 starts) reached the 9-start threshold. Therefore, I gave O'Mulligan the "2008 SF starter" classification because he was the originally designated starter last year, and classified "Shaun Hill, 2009 Week 1 starter" as a QB change because, well, he's not O'Mulligan (thankfully).

Since 2002, a whopping 75 of the 139 non-playoff teams had a different starting QB the next season. Not surprisingly, the non-playoff teams that made a QB change were coming off a particularly offensive season on offense (pun intended) when compared with non-playoff teams that didn't end up changing QBs. Specifically, the QB-changers ranked, on average, 5 spots lower than QB-keepers in offense DVOA. What might come as a surprise to some is that these QB-changers were statistically significantly worse in both pass and rush offense. What this says to me is that non-playoff teams change QBs when they experience systematic offensive ineptitude, not necessarily when the passing game is all that's rotten.

So has a QB change made any difference when it comes to non-playoff teams making the quantum leap? Surprisingly, no. Although Chad Pennington, Matt Ryan, and Joe Flacco brought sexy back for their teams in 2008, the last 7 years of NFL football seem to suggest that these were exceptions rather than the rule. That's because, as was the case with coaching changes, about 30% of non-playoff teams made the quantum leap a year later regardless of whether they changed QBs (33.3%) or not (31.2%).

Now, you might be saying, "Danny, not all QB changes are created equally!" You'd be right, so I classified QB changes into 4 groups based on the type of change. Based on this classification, 50 of the 75 non-playoff QB-changers simply got rid of the previous year's starter in favor of a new one (See "2008 New York Jets"), 11 had to replace their previous year's starter due to a major injury (See "2008 Detroit Lions"), 9 benched their previous year's starter in favor of his backup (See "2008 Minnesota Vikings"), and 5 had an established starter return from major injury to reclaim his earlier role (See "2008 Carolina Panthers"). For our purposes, the move from O'Mulligan to Shaun Hill qualifies as your run-of-the-mill "kick ‘em to the curb" type of QB change.

So is there an impact if we look at type of QB change? Once again, the answer is no, at least in terms of statistical significance. However, it's of practical importance to note that the "kick ‘em to the curb" club, which the 2009 49ers will become a member of in September, actually had the worst leap rate of the 4 types (28.0%). Therefore, taken together, the stats I've presented in this section seem to suggest that our optimism about a 49er playoff appearance in 2009 should not necessarily rest on the shoulders of Shaun Hill. You'll see in a moment why I've thrown that "necessarily" in there.

The next two potential differences between Leapers and No-Shows have to do with changes on the field; namely, offensive and defensive performance. I'll tackle offense in the next section (pun intended), and then attack the defense in the section after that (pun also intended).

OFFENSIVE PERFORMANCE CHANGES

So far, I'm probably raining on your playoff parade. Don't fret. Stick with me here. I promise the clouds are going to part very shortly...OK, how about right now?

It shouldn't come as a shock that, since 2002, the 140 non-playoff teams have been worse on offense than their 84 playoff-bound counterparts. Here's a table showing statistically significant differences with respect to offense DVOA:

Category	Playoff Teams	Non-Playoff Teams	Difference
Avg. Pass Offense DVOA	17.1%	-6.0%	23.1%
Avg. Rush Offense DVOA	3.0%	-5.5%	8.5%
Avg. Offense DVOA	10.4%	-5.5%	15.9%

Obviously, playoff teams have been better on offense than non-playoff teams. However, breaking down offensive performance into passing and rushing, the stats suggest that the vast majority of the difference between these two groups is in the passing game. Given that playoff teams are better in the passing game, it stands to reason that the main offensive avenue through which a non-playoff team makes the quantum leap is by vastly improving its aerial attack the next season. To investigate this, let's only focus on non-playoff teams, and examine differences between Leapers and No-Shows with respect to how their respective offensive stats change from Season A to Season B. Here are the relevant stats:

Category	Leapers	No-Shows	Difference
Avg. Pass Offense DVOA Improvement in Season B	+15.8%	+0.5%	15.3%
Avg. Rush Offense DVOA Improvement in Season B	+5.4%	+1.7%	3.7%
Avg. Offense DVOA Improvement in Season B	+10.7%	+1.0%	9.7%

Although the average non-playoff team has improved on offense the following season, those that go from playoff spectator one season to playoff participant the next season (i.e., Leapers) improve almost 11 times more. Furthermore, the vast majority of offensive improvement occurs - as expected - in the passing game. If we compare the differences in pass offense and rush offense improvement, we can conclude the following: If a non-playoff team hopes to make a quantum leap the following season, it's somewhere around 4 times more important for that team to improve its pass offense than its rush offense.

Relating these stats back to the 49ers, their pass offense DVOA last season was a pathetic -18.5%. The average improvement for Leapers, as shown in the table, has been 15.8%. So in order for us to get optimistic about a Niner playoff berth in 2009, we have to ask ourselves, "Do I think the 49ers' passing offense is going to be about as good as the league average (i.e., approximately 0.0%) next season?"

At the moment, I tend to believe that the answer to this question is yes. First, there's the "look what I found" addition of Michael Crabtree. Second, there's the maturation of Jason Hill and Josh Morgan. Third, there's the apparent (and much-needed) change in VD's role. Fourth, there's the return to a run-based offense, which is likely to help the passing game by proxy. Finally, and most importantly, there's the simple fact that the Niners' pass offense DVOA was about 30% better with Shaun Hill starting the final 8 games of 2008.

In total, then, based on the last 7 years of the NFL, our optimism about a 49er playoff appearance in 2009 might indeed rest on the shoulders of Shaun Hill after all. At the very least, it rests on the shoulders of Hill and his WRs.

DEFENSIVE PERFORMANCE CHANGES

As was the case on offense, playoff teams since 2002 have been better on defense than non-playoff teams (Remember, negative DVOA numbers on defense indicate above average performance):

Category	Playoff Teams	Non-Playoff Teams	Difference
Avg. Pass Defense DVOA	-4.8%	7.9%	12.7%
Avg. Rush Defense DVOA	-5.9%	-0.1%	5.8%
Avg. Defense DVOA	-5.3%	4.1%	9.4%

Again, we see here that the passing game - this time on defense - is primarily what separates playoff and non-playoff teams. It stands to reason, then, that, if we focus on the non-playoff teams, Leapers tend to improve their pass defense considerably in Season B, right? Here are the relevant stats (for the sake of clarity, I've made it so that plus signs indicate improvement):

Category	Leapers	No-Shows	Difference
Avg. Pass Defense DVOA Improvement in Season B	+8.2%	-1.5%	9.7%
Avg. Rush Defense DVOA Improvement in Season B	+3.8%	-1.1%	4.9%
Avg. Defense DVOA Improvement in Season B	+6.3%	-1.4%	7.7%

As expected, Leapers improve all aspects of their defense, while No-Shows don't improve any aspects of theirs. Also as expected, the majority of the difference in season-to-season defense DVOA change is in the passing game. The conclusion here is that it's about twice as important for a non-playoff team to improve its pass defense as it is for them to improve their rush defense.

Going back to the 49ers, their pass defense DVOA in 2008 was ranked 20^th at 15.7%. The average improvement for Leapers, as shown in the table, has been 8.2%. So, if we want to determine an appropriate level of optimism, the question becomes, "Do I think the 2009 49ers' pass defense will be half as bad as it was in 2008?"

This one's a little tougher to answer. On the positive side, there's the "it's about freaking time" demotion of Mark Roman, the apparent (and much-needed) change in Manny Lawson's pass rush role, the return of Shawntae Spencer, and the overall establishment of an attacking defensive identity. However, on the negative side, Walt Harris is done for the year, and Singletary's pass defense in the second half of 2008 actually was 23.2% worse than Nolan's during the first half of last season. So I don't know. Based on the stats, it's a tough call. If you want to err on the side of optimism here, be my guest.

STRENGTH OF SCHEDULE CHANGES

The final factor I'll look at that might distinguish Leapers from No-Shows has to do with changes in strength of schedule (SOS) from one season to the next. Specifically, do non-playoff teams make the quantum leap because their schedules get considerably easier?

The answer to this one is a resounding, "Yes!" Of all the comparisons I did in preparation for this article, SOS change had the strongest impact on playoff turnover from season to season.

First, though, it's useful to point out that playoff teams have had statistically significantly easier schedules than non-playoff teams over the past 7 seasons: opponents of playoff teams have had an average winning percentage of 48.0%, whereas opponents of non-playoff teams have had an average winning percentage of 51.2%. You might think that a 3.2% difference isn't that big. Remember, though, that the entire range of SOSs for the 224 teams from 2002-2008 is 41.4% to 59.0%, with 95% of these teams having an SOS between 43.2% and 56.8%. So that 3.2% difference is actually huge, representing a 9-spot difference in SOS ranking (13^th-toughest vs. 22^nd-toughest).

In terms of season-to-season change, the SOS difference between Leapers and No-Shows is almost perfectly symmetrical with that of the playoff vs. non-playoff difference. Specifically, whereas No-Shows' SOSs do not get any easier in Season B, Leapers' schedules get 3.4% easier. Again, that's not a big difference until you look at rankings: a 3.4% easier schedule translates to about a 10-spot difference in the SOS rankings. So the moral of the story here is that, if you want to be optimistic about the 49ers in 2009, hope that their schedule gets a lot easier. But how likely is that?

Well, the Niners' SOS in 2008 was an unbelievably easy 44.7%, meaning that an average Leaper improvement in SOS would take their 2009 SOS into uncharted territory. As I said a minute ago, the easiest schedule over the past 7 years was 41.4%, which the 2007 Seattle Seahawks happily converted into a playoff berth. In this context, a 3.4% easier schedule for the Niners in 2009 means that their SOS for next season would be 41.3%, or, in other words, the easiest schedule of any team since division realignment. So basically, we're banking on one hell of a statistical anomaly here.

Now, you might say, "Well, NFL.com tells me that the 49ers have the 7^th-easiest schedule next year, so maybe they'll pull off that miracle." My response to this is that, as I said in a previous post, a team's SOS based on opponents' records from last season bears no resemblance to their actual SOS based on opponents' records from this season. OK, let me rephrase that. Around 95% of a team's actual SOS has absolutely nothing to do with its before-the-season projected SOS, and projected SOS doesn't exhibit any of the huge statistical relationships that are consistently demonstrated by actual SOS with performance measures like, you know, wins, DVOA, and - particularly relevant to this discussion - playoff turnover. Essentially, you should treat projected SOS like Paris Hilton. It's fun to talk about, but totally meaningless.

BOTTOM LINE

The goal here was to determine whether our optimism about the 49ers in 2009 is justified based on the statistical characteristics of previous teams who made the quantum leap into playoff participation. Here's what the stats suggest:

1. Despite the MIA-ATL-BAL trifecta in 2008, changing head coaches doesn't seem to forebode an immediate quantum leap into NFL playoff participation. Therefore, don't rest your hopes on the 49ers' change from Nolan to Singletary.
2. Despite the Pennington-Ryan-Flacco trifecta in 2008, changing QBs also doesn't seem to forebode an immediate quantum leap into NFL playoff participation. Therefore, don't rest your hopes on the 49ers' change from O'Mulligan to Shaun Hill.
3. On offense, the majority of performance improvement for Leapers is in the passing game. The addition of Crabtree, the maturation of Jason Hill and Josh Morgan, the proper use of VD, the return to offensive balance, and the demonstrated passing game improvement with Shaun Hill at QB over the last half of 2008 all bode well for the Niners. Therefore, optimism seems justified when it comes to pass offense improvement.
4. On defense, the majority of performance improvement for Leapers is also in the passing game. For the 2009 49ers, some offseason changes bode well (e.g., Dashon Goldson replacing Roman, and Lawson rushing the passer more), while other changes don't bode so well (e.g., Harris blowing out his knee, and the pass defense sucking the big one during the last half of 2008). Therefore, optimism remains in the eye of the beholder when it comes to pass defense improvement.
5. The primary identifying characteristic of Leapers is that their SOS gets much easier the next season. The problem is that we have no way of knowing just how much easier (or more difficult) the 49ers' schedule is going to be in 2009. Therefore, break out your voodoo dolls and get to work on Kurt Warner et al. Interestingly enough, optimism is directly proportional to how talented you are in Black Magic.

**DVOA statistics used to produce this article were obtained from Football Outsiders.