49ers Winning the Future With QB Alex Smith: A Statistical Profile?

Greetings from FO-land, everyone. Just thought I'd drop in to share a blog post I came across while scouring the internets for Extra Points fodder. This one pertains to Alex Smith, so I figured NN would be a better home for the discussion.

In case you're unaware, Ben Alamar is a Professor of Management at Menlo College, Editor of the Journal of Quantitative Analysis in Sports, coauthor of Football Outsiders Almanac (which, by the way, will be hitting the virtual shelves sometime shortly after the lockout ends), and author of the Analytic Football blog. On Analytic Football, he did a post last Friday identifying 5th-year QBs who put up numbers at age 26 similar to Smith's 2010 stats. Money quote:

On Yds/Att, QB rating, TD% (indexed by PFR), Int% (indexed by PFR), and Completion Percentage, Smith fits right in. He is better than Big Ben in TD%, QB rating, and Compl%, better than Brees in Int%, and just behind [Roethlisberger, Brees, and Peyton Manning] on Yds/Att. The other three of course all of [sic] Super Bowl rings and multiple All-Star game appearances on their resumes. So what is going on? How can the much maligned Alex Smith, playing for multiple head coaches and offensive coordinators, have delivered a similar performance as these shining examples of QB play?

I'll leave it to you to read the rest of Alamar's post, in which he does a more sophisticated stat analysis to find out "what gives."

After the jump, my take...

I think the fundamental thing to understand about how two different conclusions can arise out of the same stats is that there's a big difference, conceptually speaking, between working your way backward from an outcome and working your way forward from a precondition. In the first analysis (i.e., the one that shows Smith is a Super Bowl MVP waiting to happen), Alamar started with the idea, "given Smith's stats, let's see who was similar at the same age and experience." In the second analysis (you'll have to read his post to find out what that one shows), he started with the idea, "given Smith's age and experience, let's see who had similar stats."

Obviously, age and experience going into a season occurs chronologically before the statistical results of that season, so the second analysis is the correct one to believe. In other words, Smith didn't throw for his stats in 2010 first, and then enter his 5th season.

I highlight this concept because much of what I see in terms of popular football "analysis" (not at FO, of course) focuses on backward timelines. A term I use for this is "profiling," and I think it's one of the biggest reasons why fans think stats wear burning pants. In criminology, profilers encounter serial killers, and try to find out what those serial killers had in common before they started killing. They say, "the vast majority of serial killers abused animals as children." You hear this, and -- presuming you're not actually a serial killer -- think, "When I was a kid, I had a blast burning ants on the sidewalk with my magnifying glass! How come I'm not a serial killer, Mr. Profiler Guy!" It's because you're (correctly) thinking about your childhood, and then looking at how you turned out later.

In football, profiling is when someone notices a group of players/teams with a similar achievement, and then see what earlier characteristics they had in common. Basically, the profiling sentence construction goes, "Of the X (players/teams) who achieved Y, A of them did B earlier." My personal favorite example is the 3rd-year WR profile. If you click on that link from the Dark Ages of the internet, you'll read the profiling with your own eyes:

73.5% of the WRs in the sample (26 out of 36) had a 150-point breakout year when they had at least 55 receptions and 890 yards the previous year.

See how they did that there? 73.5%! Shockingly accurate! You too can be a maven at predicting breakout seasons by a 3rd-year WR! Sadly, no you can't. And that's because the correct thing to ask is, "Of the WRs who had at least 55 receptions and 890 yards one year, how many broke out with their first 150-point fantasy season the next year?" 

So, in closing, Alamar did things the right way. Profiling led him to find that Alex Smith has some select Super-Bowl-MVP company. He knew that couldn't be correct, so he flipped the analysis around. Lo and behold, it wasn't. When you see, hear, or read commentary that takes the form of profiling, ignore it. When it's done the way Alamar's second analysis was done, believe it. As a football fan, knowing how to discern between good stats and bad stats is an important skill. Ignoring profilers is Lesson #1.

Now, back to your regularly scheduled Alex Smith argument!