FanPost

Does a terrible pre-season portend anything?

We've all seen knee-jerk "sky is falling" reactions to the 49ers' two heavy preseason defeats, and we've also seen (and possibly made) similarly knee-jerk "don't be ridiculous - it's only preseason" responses to those reactions.

The "it's only preseason" responses have some support from statistical analyses such as this one by Homie C. However, analyses correlating preseason and regular season wins are influenced by things that have nothing to do with those "sky is falling" reactions, such as how well a 4-0 or 3-1 preseason does compared with a 2-2 preseason team, or even how well a winning preseason team does compared with a team that lost their preseason games by small margins.

With this in mind, I decided to look at the regular season performances of all of the teams I could find after truly terrible preseasons, defined in terms of points difference. The hypothesis is that these teams tend to do worse than average in the regular season.

The straw man argument for this hypothesis is that a good team that is playing backups more their opponents, or playing more vanilla than their opponents, might be expected to lose, but should at least be competitive (if the Niners were 0-2 with a points difference of -6, I don't think even the most fragile of fans would be panicking). When I call the argument a straw man, I mean it. I don't really believe in it myself. However, when I tested the hypothesis, I didn't get the answer I was hoping for....

Null hypothesis

The null hypothesis is that regular season performance is no different among teams following a truly terrible preseason than among teams in general. The null hypothesis can be rejected if the mean record of such teams is significantly different from 8-8. I've defined "truly terrible" as having a points difference of -50 or worse.

The data

Below are all of the teams that I found (it's possible that I missed a couple) since 2000 that had a preseason points difference of -50 or worse. I could not find preseason records any further back.

2013 Jaguars: preseason PD -51; regular season record 4-12

2012 Dolphins: preseason PD -53; regular season record 7-9

2012 Chiefs: preseason PD -55; regular season record 2-14

2012 Jets: preseason PD -57; regular season record 6-10

2012 Bills: preseason PD -60; regular season record 6-10

2011 Raiders: preseason PD -57; regular season record 8-8

2010 Colts: preseason PD -70; regular season record 10-6

2006 Washington: preseason PD -77; regular season record 5-11

2003 Texans: preseason PD -69; regular season record 5-11

2003 Raiders: preseason PD -57; regular season record 4-12

2002 Seahawks: preseason PD -56; regular season record 7-9

2001 Washington: preseason PD -59; regular season record 8-8

2000 Chiefs: preseason PD -54; regular season record 7-9

2000 Cowboys: preseason PD -60; regular season record 5-11

2000 Panthers: preseason PD -53; regular season record 7-9


So that's a sample size of 15. The mean regular season record of those teams was 6-10, with a distribution standard deviation of 2 wins. The standard error of the data set (i.e. the error in the prediction of the mean of a much larger sample, due to the small sample size), is only 0.5 wins. An 8-8 record is about 4 standard errors from the mean.

Interpretation

The consistently poor performance of these teams means that we can comfortably reject the null hypothesis at the 99.9% confidence level, despite the small sample size. Teams with a truly terrible preseason do tend to do worse than average in the regular season.

What does this mean for the 2014 49ers? Is the sky falling after all? I'm not convinced. First of all, preseason isn't over yet. More importantly, most of the teams listed above had done badly the previous regular season and/or were expected to struggle before preseason even started. The 49ers, on the other hand, were (and still are) expected to be good in 2014. If we're looking for a good recent analogy for the 2014 49ers from the above list, we're not going to do any better than the 2010 Colts, who were the only one of the 15 teams to post a winning record in the regular season.

Nevertheless, I'm not going to be quite so dismissive any more of people who are panicking about the last two games.

This is a FanPost and does not necessarily reflect the views of Niners Nation's writers or editors. It does reflect the views of this particular fan though, which is as important as the views of Niners Nation's writers or editors.