Singletary’s Formula for Success: Introducing the 49ers’ Adjusted BS Rate

AUTHOR'S NOTE: Sorry for the delay in getting this posted. Below, you'll understand why it's a day late.

Welcome back for this week's look at how the 49ers are doing according to Mike Singletary's Formula for Success. For those that don't remember Singletary's Formula for Success, here it is:

1. Total Ball Security
2. Execute
3. Dominate the Trenches
4. Create Good Field Position
5. Finish

The major takeaways form last week's post were that (a) the 49ers are an average team based on the formula, and (b) the formula itself is a pretty reliable indicator of the 49ers' record. Given that the play-by-play sample sizes are so large now, we shouldn't expect much statistical fluctuation from game to game. Nevertheless, the directions in which these stats change (i.e., for better or worse) can tell us something about the progress of our beloved team when it comes to doing the things that their coach views as most crucial for winning. But I'll get to that stuff later.

The focus of today's post emerged from some thinking I did about the two stats I used last week to measure Total Ball Security: Fumbles and Interceptions. In my haste to put the post together, I didn't really sit down and fully contemplate whether these were the best stats I could use. From reading Football Outsiders and other sources, I already knew that Fumbles is a much better stat than Fumbles Lost because the recovery of a fumble is more about luck and less about skill. But honestly, aside from relying on this basic knowledge, I used Fumbles and Interceptions because they were the most (and only ones) readily available.

Since then, I've searched around the internet trying to find more - I guess what you'd call - advanced stats related to the ability of an NFL team to hold onto the ball. Or, in the language of Singletary's Formula for Success, the ability of an NFL team to achieve total ball security. Not surprisingly, this was a fruitless search. As far as I can tell, no one's made a public attempt to develop such a statistic.

Well, consider yourselves lucky. Based on the premises that (a) Total Ball Security is an ingredient in Singletary's Formula for Success, (b) I'm writing a weekly post about this formula, and (c) no one else seems to have come up with a good measure for this specific ingredient, I've come to the conclusion that I'm the person I've been waiting for.

Therefore, in today's Formula for Success post, I'm going to introduce a new stat, which I'll call Adjusted Ball Security Rate, aka Adjusted BS Rate. Don't worry. I've reported the rest of the Formula's stats at the end of the post. But the major purposes of this post will be to give you my rationale behind Adjusted BR rate, tell you the methods I used to develop Adjusted BS Rate, and present some evidence showing you that Total Ball Security, as measured by Adjusted BS rate, is - as Singletary clearly believes - directly associated with winning in the NFL.

After the jump, I'll unveil my newly minted Niners Nation special, Adjusted BS Rate...

RATIONALE FOR ADJUSTED BS RATE

The first problem that emerged when I thought about Fumbles and Interceptions as measures of Total Ball Security is something very basic to statistics: rates are better than totals when sample units differ in their ability to aggregate the total. Using an NFL example, this essentially means that it's wrong to compare the 49ers' total number of fumbles (16) to that of the Steelers (16) because the 49ers' have had fewer "opportunities" to actually fumble the ball (SF = 701; PIT = 779). The better stat to use than Total Fumbles is, therefore, Fumble Rate, i.e., "fumbles per fumbling opportunity."

Similarly, it's wrong to compare the 49ers' total number of interceptions (9) to that of the Seahawks (9) because the 49ers' have had fewer "opportunities" to actually throw an interception (SF = 352; SEA = 424). The better stat to use than Total Interceptions is, therefore, Interception Rate, i.e., "interceptions per interception opportunity."

However, determining that it's better to use Fumble Rate and Interception Rate as measures of Total Ball Security only gets you so far. Given my definitions for each, the question becomes, "What constitutes a fumbling or interception opportunity?" Well, in the case of an "interception opportunity," the answer is pretty simple. An interception can only occur if an offense attempts a pass. Therefore, an "interception opportunity" is the same thing as a pass attempt, and so Interception Rate is simply Total Interceptions divided by Total Pass Attempts. Not surprisingly, this exact statistic - presumably because it's so easily calculated and understood - is one of the 4 components of the NFL QB Rating formula.

A "fumbling opportunity," on the other hand, is a much tougher nut to crack. First, going back to my original question, what constitutes a fumbling opportunity? Second, and just as important, even if we can define a fumbling opportunity, are there stats available to quickly and appropriately measure it?

To determine the qualities of a "fumbling opportunity," a good place to start is the NFL rule book, which defines a fumble as "the loss of player possession of the ball." In addition, the rules distinguish between a fumble and a muff - let's keep our minds out of the gutter here, guys - which is defined as "the touching of a loose ball by a player in an unsuccessful attempt to obtain possession." Nearly all muffs occur on punts and kickoffs. However, when a KR/PR muffs a kick/punt, it's an indication of his inability to maintain Total Ball Security, so, for our purposes, such muffs can also be considered fumbles. Therefore, taking these definitions and formula-related purposes into account, the only time a player has the opportunity to fumble the ball is if he (a) is currently in possession of the ball; or (b) is attempting to obtain possession of a kick/punt.

So the key here is to figure out all of the circumstances in which a player "is currently in possession of the ball" or "is attempting to obtain possession of a kick/punt." Well, it turns out that, when you think about it, the options are pretty limited. They basically boil down to the following:

1. A QB has possession after he takes the snap from center
2. A RB/TE/WR has possession after he receives a handoff
3. A RB/TE/WR has possession after he receives a pass
4. A defensive player has possession after he recovers a fumble
5. A defensive player has possession after he intercepts a pass
6. A PR is attempting to obtain possession when he fields a punt
7. A PR has possession while he returns a punt
8. A KR is attempting to obtain possession when he fields a kick
9. A KR has possession while he returns a kick

Therefore, all of these constitute a "fumbling opportunity."

Now, with a "fumbling opportunity" defined, and the exhaustive list of fumbling opportunities identified, we move on to the next question: "Can we obtain stats to quickly and appropriately measure fumbling opportunities?" Thankfully, the answer here is "yes." That's because there are a whole host of official NFL stats that aggregate the various types of player possession (and attempts at obtaining possession of punts/kicks) listed above. For instance, the stat, Rushing Attempts, tells you how many times a team's RBs/TEs/WRs were in possession of the ball after receiving a handoff. In the next section, I'll list out all of the stats I used to measure "fumbling opportunities," but, at this point, just understand that we now know (a) what constitutes a fumbling opportunity, and (b) that we can measure it quickly and appropriately.

So, just to sum up, Total Fumbles and Total Interceptions are inadequate for measuring Total Ball Security because NFL teams differ with respect to the number of opportunities they have to fumble the ball or throw an interception. Better stats would be Fumble Rate and Interception Rate, but, before calculating them, we have to figure out what constitutes a "fumbling opportunity," and an "interception opportunity." In this section, I did just that.

METHODS FOR CALCULATING ADJUSTED BS RATE

Before we can calculate Adjusted BS Rate, we first have to calculate BS Rate. However, even before we can calculate BS Rate, we have to calculate Fumble Rate and Interception Rate, which I'll abbreviate as FR and IR from now on. As I mentioned earlier, IR is pretty straightforward. Here's the equation:

IR = Offensive Interceptions / Pass Attempts

Calculating FR eventually becomes straightforward as well, but only after first calculating "fumbling opportunities." Here's a list of the stats I've concluded best measure the number of opportunities a team has had to fumble the ball, as defined by either having possession of the ball or attempting to obtain it while fielding a kick/punt:

1. Fumbling opportunities during QB possession = Sacks Allowed + Incompletions*
2. Fumbling opportunities during a RB/TE/WR run = Rushing Attempts
3. Fumbling opportunities during a RB/TE/WR run after catch = Receptions
4. Fumbling opportunities during a fumble return = Defensive Fumble Recoveries
5. Fumbling opportunities during an interception return = Defensive Interceptions
6. Fumbling opportunities during the fielding or returning of a punt = Punt Returns
7. Fumbling opportunities during the fielding or returning of a kick = Kick Returns

Therefore, here's the equation for "fumbling opportunities," which I'll abbreviate FOPPS:

FOPPS = Sacks Allowed + Incompletions + Rushing Attempts + Receptions + Defensive Fumble Recoveries + Defensive Interceptions + Punt Returns + Kick Returns

And that makes the equation for FR as follows:

FR = Fumbles / FOPPS

So now we have our two core stats, IR and FR, which measure "interceptions per interception opportunity," and, "fumbles per fumbling opportunity," respectively.

The next step is to combine IR and FR so that we come up with our BS Rate. Because we want to measure Total Ball Security, rather than Total Ball Insecurity, the first thing we have to do is subtract each of these from 100%. Then, in order to get BS Rate, we have to calculate what's called a weighted average of IR and FR because Pass Attempts and FOPPS are not equal. You can click here for a thorough discussion of weighted averages, but the bottom line is that, if we simply added IR and FR, then IR would be make up 50% of BS Rate, despite the fact that Pass Attempts only makes up about 30% of all ball insecurity opportunities. In other words, we'd be giving IR 20% more of an impact on BS Rate than it deserves. Just to make things as clear as possible, here's the equation for BS Rate:

BS Rate = {[(1 - IR) * Pass Attempts] + [(1 - FR) * FOPPS)]} / (Pass Attempts + FOPPS)

Finally, we can't ignore the fact that different teams play schedules of differing strengths when it comes to (a) their opponents' ability to force fumbles, and (b) their opponents' ability to intercept the ball. Therefore, in ultimate satisfaction of measuring Total Ball Security, we have to adjust BS Rate for strength of schedule (SOS). I did some hardcore stats on this and it turns out that a team's FR is not related at all to their FR SOS (correlation = .072, which is not statistically different from 0). In other words, how often a team fumbles isn't related to how often their opponents force fumbles, at least not in 2009. Therefore, the only thing to adjust for is IR SOS, which does have a statistically meaningful impact on IR (correlation = .486). After adjusting for IR SOS, we end up with this equation for Adjusted BS Rate:

Adjusted BS Rate = {[(1 - Adjusted IR) * Pass Attempts] + [(1 - FR) * FOPPS)]} / (Pass Attempts + FOPPS)

Notice that the only manner in which the equation for Adjusted BS Rate differs from that of Unadjusted BS Rate is the use of Adjusted IR, rather than IR, in the numerator. I fully understand that, unless you have statistical software handy and know how to "adjust for" IR SOS, you can't directly calculate Adjusted BS Rate at home (assuming you'd even want to do so). As you'll see below, however, Adjusted BS Rate doesn't differ all that much from Unadjusted BS Rate, which you can, in fact, calculate without the need for statistical expertise.

TOTAL BALL SECURITY STATS

Below you'll find Adjusted BS Rates, Unadjusted BS Rates, FRs, IRs, and Adjusted IRs for each NFL team so far this season (49ers and top 8 for each category in bold; bottom 8 in italics):

Before I get into the 49ers' stats, I'd just like to point out some evidence supporting the validity of using Adjusted and Unadjusted BS Rate as measures of Total Ball Security. A pretty standard thing to do when you develop a so-called advanced NFL stat, is to find out how related the stat is to winning. Given that the context of my endeavor here is Singletary's Formula for Success, it makes extra sense to see whether being good at Total Ball Security, as measured by Adjusted and Unadjusted BS Rate, is associated with being good at, you know, winning games.  Of course, you probably suspect that I wouldn't be taking up all this time and space if they weren't. If so, you'd be suspecting correctly. Here's a table showing the correlations between the Total Ball Security stats above and Team Wins in 2009 (all are highly statistically different from 0), along with the percentage of "winning" that's explained by each stat (aka R-squared):

If you don't know how to interpret a correlation or R-squared, click here. Basically, what this table is telling you is that each stat is highly associated with winning in 2009, and that each stat explains a considerable amount of the "winning" phenomenon.** For instance, the correlation between FR and Team Wins (-.547) means that, beyond the longest shadows of statistical doubt, a team has won more games this season if they have a lower FR. Furthermore, the R-squared value for Adjusted BS Rate means that 43.45% of "winning" is due to Adjusted BS Rate. Or, alternatively, only 43.45% 56.55% of "what it takes to win" has nothing to do with Adjusted BS Rate. Given the complexity of figuring out "what it takes to win" in the NFL, I'd say these correlations clearly suggest that Adjusted BS Rate is a useful measure of Total Ball Security, and that Mike Singletary is clearly justified for including Total Ball Security as an ingredient in his Formula for Success.

THE REST OF THE 49ERS' FORMULA STATS

Now that I have a useful stat for measuring Total Ball Security, I can add it to the following table, which displays the Niners stats according to Singletary's Formula for Success, what those stats were last week, and what the extent of change was between this week's stats and last week's stats (top-8 in bold; bottom-8 in italics):

In terms of Total Ball Security, we now see that the 49ers a below-average team when it comes to their overall ability to keep possession of the ball, but an above-average team when it comes to keeping possession of the ball when they throw a pass. To this I say, enough already with the "shotgun offense equals more interceptions" crap.  Even their current Adjusted IR is almost twice as good as the team who has the #12 Rush OFF DVOA and a much-hyped wunderkind taking 82.5% of his snaps from under center (aka the New York Jets).

With respect to the other formula ingredients, like I said, not much has changed. The two biggest changes from Week 11 were defensive front 7 (DF7) ALY getting a lot worse and 4th Quarter DEF DVOA getting much better. Given that Maurice Jones-Drew was able to run the ball with reasonable success against the league's #5 run DEF, and the spectacular 4th-quarter display we saw from the Niners' #11 pass DEF, these changes make perfect sense.

BOTTOM LINE

Based on my introduction to Adjusted BS Rate and the 49ers' stats through 11 games, we can draw the following conclusions:

1. Adjusted and Unadjusted BS Rate are good composite measures of Total Ball Security.
2. FR is a good indicator of a team's ability to avoid fumbling the ball when they have the opportunity to do so.
3. Adjusted and Unadjusted IR are good indicators of a team's ability to avoid throwing an interception when they have the opportunity to do so.
4. A team's Adjusted BS Rate, Unadjusted BS Rate, FR, IR, and Adjusted IR are all highly associated with winning.
5. Singletary is right to include Total Ball Security in his Formula for Success.
6. The 49ers are a better team with respect to IR than FR and BS Rate (Adjusted or Unadjusted).
7. The 49ers are still an average team with respect to Singletary's Formula for Success.

Coming up tomorrow... team DVOA stats and rankings.

*Incidentally, some of you might quibble with my use of Sacks Allowed + Incompletions as a measure of QB FOPPS. My reasoning is fairly simple. The overwhelming majority of a QB's fumbles in the pocket occur when he's sacked, with the remainder occurring while he's maneuvering in the pocket. In other words, a FOPP for a QB is when he's dropped back to pass. However, I can't use a handy stat like Dropbacks, which equals Sacks + Pass Attempts, because about 60% of all pass attempts result in either a reception or an interception, and I'm already using both Receptions and Defensive Interceptions in the equation. From a statistical perspective, Pass Attempts are what's called multicollinear with Receptions and Defensive Interceptions, which basically means they're providing the same information multiple times in one equation. That's a big no-no in statistics.

Given the need to avoid multicollinearity, I can only use those pass attempts that don't result in a reception or interception. Such pass attempts are otherwise known as "Incompletions." Given that Pass Attempts is part of what makes up Dropbacks, the byproduct of this is that I can only use those dropbacks that don't result in a reception or interception. Such dropbacks are otherwise known as Sacks + Incompletions.

**In case you're wondering why the unadjusted stats explain more of the "winning" phenomenon than the adjusted stats, the reason is pretty simple. Given that IR is itself highly associated with winning, when you adjust for IR SOS, you're essentially removing part of the reason for team wins and losses. In other words, if having a lower IR means winning more games, and part of the reason for having a lower IR is having an easier IR SOS, then that means part of the reason for having more wins also because of having an easier IR SOS. Therefore, when you remove IR SOS as a factor in IR (i.e., when you adjust for it), you're taking out one of the things that explains why a team wins. Necessarily, the result of this "factoring out" is that you've just decreased the stat's ability to explain "winning," shows up via a lower R-squared. Remember, the goal here is to find out how good a team is in Total Ball Security, not how easy an IR SOS they've had. A minor decrease in R-squred is a price that's worth paying in pursuit of a more informative stat.

***DVOA, ALY, and ASR statistics used to produce this article were obtained from Football Outsiders.