ESPN has deemed 2011 the year of the quarterback and as part of their analysis of the position, they decided it was time to develop a new statistic for analyzing quarterback play. Quarterback rating has long been used to analyze quarterbacks, but it's a fairly flawed statistic for a variety of reasons. The quarterback rating does not provide sufficient context for a given QB's performance beyond what the basic counting statistics say, which often isn't nearly enough.
And so, ESPN's Total Quarterback Rating (Total QBR) has been created. ESPN ran a special on it last night, but the linked article provides a nice little rundown. Additionally, I thought I'd note some of the more interesting aspects of it, as well as some basic criticisms.
Part of the idea behind the Total QBR is to provide a better idea of performance based on overall context of a quarterback's play. In order to formulate the statistic, ESPN's stats folks broke down three years of plays dating back to 2008. In looking over thousands and thousands of plays, they were able to assign point values to the quarterbacks involvement in every offensive play. The QB contributions are weighted based on the importance of the situation in determining a game's outcome. For example, a short screen pass that a running back breaks open for a 20 yard gain will be of less value to the QB than a straight 20 yard completion to a wide receiver.
The statistic intends to reward some risk-taking. For example, the rating will apparently not decrease on an interception where the quarterback threw a Hail Mary into the end zone as time expired in the second quarter. At the same time, an interception with five minutes left in the fourth quarter of a tight game means the Total QBR will take a hit.
All in all, it definitely seems to be headed down the right path. Too many statistics in all sports do not provide nearly enough context. Not all eight-yard completions are created equal after all. An eight yard completion on 3rd and 17 is not nearly as valuable as an eight yard touchdown completion on 3rd and goal from the eight. And yet, it's marked as eight yards either way.
At the same time, the new formula does miss out on some easy ways it could be improved. Naturally Football Outsiders had a few comments on the subject. Aaron Schatz commented on the subject and had two primary concerns with Total QBR:
1) The lack of opponent adjustments, although I understand why they aren't including them. Remember, they are trying to produce ratings on the fly, whereas I produce ratings after all the games of a certain week are over. And you know even we don't include opponent adjustments early in the season, and we slowly ramp up the strength of those adjustments. So I understand their arguments against including them.
2) The "clutch" thing. I still think this gives a bonus to quarterbacks who happen to play with bad defenses. But they say they have worked it out so that no quarterback gets a bonus for BEING in more "clutch" situations. They only get a bonus for playing well in those high-leverage situations. Still, we've found that offensive DVOA in first quarter correlates best with winning, not other quarters. There is an advantage to getting off to an early lead, dictating the pace of the game, and an even bigger advantage to blowing your opponent away out of the gate.
All in all, Aaron does not seem to have a major problem with the new statistic. Does anybody else have any particular critiques of it? It's hard for us to fully wrap our heads around it as I'd imagine we'll potentially be pointed to specific ratings and that will be that.
While I was watching the show last night debuting the stat, they provided a breakdown of what the various levels of the stat mean for quality of QB play:
75+ = MVP Candidate
65-75 = Pro Bowl Consideration
50 = Average
Less Than 35 = Below Average
They've produced a ranking of 2010 Total QBR and our own Alex Smith ranks 30th with a Total QBR of 40.0. Matt Hasselbeck led the NFC West with a Total QBR of 42.4, good for 26th. Sam Bradford was 29th with a 41.0 and Derek Anderson was 31st with a 35.9. Not exactly a fantastic group of quarterbacks in the division.
Thoughts on this new statistic?