Pro Football Focus has grown more prominent in recent years, and has become a mainstay of a lot of media folks. Cris Collinsworth’s investment in it has gotten it a lot of publicity on NBC’s Sunday Night Football broadcast, but even beyond that, Peter King, ESPN, and others have taken to using their grades and statistics.
Over the past couple years, media members will mention a particular PFF stat or grade in a press conference, and sometimes coaches and players raise issue with it. On Thursday, Chip Kelly spoke to 49ers media, and the issue came up in discussing Marcus Martin and Andrew Tiller.
The media person started out asking why doesn’t Andrew Tiller get more playing time. He said the statistics seem to suggest Tiller is better, pointing to quarterback hurries, sacks, and the like. Kelly asked where it came from, and was told Pro Football Focus. He suggested he was not a fan of it, and went into specifics as it related to offensive linemen:
“No, I mean I’ve said it all along, how can they grade an offensive lineman when they don’t know what the play is? I’ve had it before, our left tackle gave up a sack. He didn’t give up a sack because the guy slanted in the ‘B’ gap. The guard had ‘B’ gap and that’s not the left tackle’s responsibility, but it’s written down as a sack on the left tackle because you don’t know what play was called. We called slide protection. We didn’t call man protection. If someone can look at a film and figure out what we call for a play and know what our scheme is and then give a guy a grade on it, I think there’s a lot of players and coaches that feel the same way. I don’t know. You can do whatever you want with it.”
It got particularly entertaining when he offered up an analogy, saying, “It’s like me going into a bank and grading a teller because they gave me a lollipop. I gave them a 94.3.”
Someone responded that they would give the teller a 100 for giving him a lollipop. Kelly suggested why he might grade lower:
“I mean, it depends on what lollipop. It was purple and I wanted red, but they did give you a lollipop. But, I didn’t know they weren’t allowed to give purple out that day. So, if I did know that I probably would have given them a higher grade, but I didn’t know the information going in.”
So there you have it. Chip might sometimes prefer a red lollipop over a purple one. Keep that in mind if you run into him.
But seriously, there is certainly validity to what he says. I think PFF’s grades, like any other potential statistic are just one part of a bigger picture. I personally think their signature stats and the charting data they track holds tremendous value. Their grading is fun for debating the potential value of players, but there is a subjective nature to it that impacts the analytical value. I don’t think they’re completely worthless, but as with any statistic, you have to take it with a grain of salt. I don’t think numbers lie, but I do think people can use numbers in a potentially misleading or incorrect way. And so, it’s just a matter of knowing what you are looking at, and assessing the proper value for it.