A look at Statistical Analysis, Part Two: David Friedman style
A few weeks ago, I posted some thoughts about statistical analysis and their importance to fans of sports, as well as it’s application. One of my favorite writers about the NBA is David Friedman, whose blog, 20 Second Timeout, has some of the best comprehensive analysis and opinion out there on the Internet. Here are a couple of articles regarding statistical analysis, and it’s application to basketball.
After the jump I’ve posted a couple of quotes for preview and I highly suggest reading the linked articles. Enjoy.
Economics is Not a Science, Nor is Basketball Statistical Analysis
Economics is Not a Science, Nor is Basketball Statistical Analysis, Part II
"Stats gurus" plainly do not want to discuss or consider the fact that some of their most precious numbers--the raw data that they plug into their formulas, stats like assists, steals, blocked shots and turnovers--are subjectively recorded. During last season's playoffs, I did a detailed post demonstrating that Chris Paul's supposedly record setting playoff assist totals were in fact inflated by generous scorekeeping. Shouldn't that be of interest to the "stats gurus"? Isn't that claim something that they seriously need to investigate on their own to either confirm or reject? I provided very specific information so that anyone could watch a tape of the game and find the exact plays that I described and thus judge for themselves whether or not each of those assists should have been awarded. Yet I see no indication that the "stats gurus" are the slightest bit concerned about the fact that a lot of their basic data is seriously flawed. A lot of these guys spent a good portion of the season pumping up Chris Paul as the MVP and it is highly likely that they did so on the basis of bogus assist numbers. Based on a skill-set evaluation of Paul's game, I consider him to be the best point guard in the NBA and a top five MVP candidate but that is not the point; the point is that if you are basing your whole analysis of the NBA purely on numbers and some of the basic numbers you are using are not right then your whole analysis is bogus. If a real scientist finds out that the raw data he has gathered is flawed then he understands that he has to gather new, accurate data. Unfortunately, many of the basketball "stats gurus" are not scientists; they are "mad scientists" at best.
In an insightful post titled Using statistics in basketball: the bar is higher, Rosenbaum writes, "Statistical analysis can play a critical role in basketball decision-making, but it can also be misleading if the complexities of the game of basketball (and the statistical issues generated by those complexities) are not well understood. In other words, the bar is higher for statistical analysis in basketball than it is in baseball. Ultimately this will greatly benefit the teams that incorporate skilled statistical analysts in the right way, because the greater complexities in basketball will mean that it will be harder for other teams to ever catch up with the first teams that get this right. It will be fascinating seeing how this all plays out over the next few years."
In The Difference Between Measuring Defense in Basketball and Baseball , I made the important point that basketball statistical analysis is pseudoscience because its practitioners do not base their research on the scientific method:
1. Ask a Question
2. Do Background Research
3. Construct a Hypothesis
4. Test Your Hypothesis by Doing an Experiment
5. Analyze Your Data and Draw a Conclusion
6. Communicate Your Results
14 comments
|
0 recs |
Do you like this story?
Comments
Friedman is great...
I found this little nugget in his Knicks article:
Jerry Krause once infamously stated that organizations win championships—a senseless swipe at the talents of Michael Jordan, Scottie Pippen and the other Chicago Bulls players who captured six NBA titles—but it is certainly true that organization wins championships—in other words, a franchise must be committed to doing things the right way from top to bottom in order to achieve the highest level of success. Just look at the New England Patriots, L.A. Lakers and San Antonio Spurs for three examples of 21st century sports franchises that have owners, talent evaluators and coaches who are intelligent and dedicated.
Hey 49ers FO, did you get this?
Well, we're waiting....
Even though I am 49ers fan I’m extremely skeptical about the greatness of an organization concept.
I doubt the Patriots knew what they were doing when they drafted Tom Brady in the 6th round.
What was so magnificent about the Spurs losing David Robinson to injury for the entire season which resulted in winning the lottery and coinciding with the year Tim Duncan declared pro?
Even the Lakers are more blessed than F.O. gifted. They had no post Magic Johnson plan but Shaq was swooned by Hollywood and insisted on playing in LA. The Pau Gasol trade still needs to be investigated by the SEC. The one brilliant move the Lakers have made the past two decades was targeting Kobe in the draft.
The 49ers were lucky in getting a true genius in Walsh and Eddie D. pretty much signed off on all of Walsh’s ideas/plans. The 49ers weren’t so genius in the post-Walsh era. Carmen Policy relied upon the talent Walsh left over and re-enforced the team by spending gobs of money in free agency. The plan worked for most of the decade until Walsh’s left over talent finally thinned out.
Even the Steelers owners are heavily praised without scrutiny. These are same owners using the same methods and philosophy that had the Steelers perennial bottom feeders until Chuck Noll was hired. Same plan for 6 decades: 3 decades the Steelers were rotten, 3 decades the Steelers were good/great.
Now do I believe in stupid franchises? You bet!
I found these comments very interesting and very illuminating
“Stats gurus” plainly do not want to discuss or consider the fact that some of their most precious numbers—the raw data that they plug into their formulas, stats like assists, steals, blocked shots and turnovers—are subjectively recorded.
Trying to gather data in football is particularly hard because of how hard it is to separate individual effort. If a QB throws the ball and it bounces off his receiver’s hands it still goes down as an incomplete pass in his stats. OTOH if he throws a 2 yard quick out and the receiver makes a couple guys miss and takes it 80 yards to the house he’s credited with an 80 yard TD pass.
I like how FO tackles this with the running stats separating it into the line and the RB effort, but it’s still an issue even there.
Yes Drew K, Tim Tebow will probably get picked in the first round.
It sounds like stats just need to be expanded then for things like WR drops in to a QB stat line and all of things that aren’t measured as cold hard stats need to be considered.
A computer once beat me at chess, but it was no match for me at kick boxing.
I think so too
but the NFL won’t do it because they’re way behind the times. Sites like Football Outsiders and PFF might take a look at it, or someone else make take a stab at it.
Yes Drew K, Tim Tebow will probably get picked in the first round.
it's even more difficult than that
When does it count as a drop? Is it a drop if the pass is way behind the WR but he can still get a hand on it in a futile attempt?
Likewise, if a QB leads a receiver perfectly and “throws him open” for a long RAC gain, shouldn’t the passer get some credit?
by microwave donut on Mar 4, 2010 10:08 AM PST up reply actions
This is like saying...
…that science is subjective because it depends on fallible people to record observations. It’s true, but totally misleading, because it implies that science is therefore incapable of generating reliable predictions.
I really think you should stop writing...
…about a subject you don’t seem to understand. I’m sorry if that’s rude, but give me a break. Quoting some articles by a critic of statistical analysis of basketball is not a substitute for comprehension of statistical analysis generally. And no offense, but nothing you’ve ever said on the subject gives me any confidence that you actually understand what you’re criticizing. Rather, your comments remind me of people who go onto physics forums claiming to have debunked General Relativity.
PS
The fact that Friedman would fault economists for failing to predict the behavior of the market reflects a rather basic misunderstanding of economics. You can’t predict the behavior of the market. How do we know this? Because statistical analysis — that’s right, statistical analysis — tells us that the behavior of stocks is random. It’s just betting. To use a famous analogy, monkeys throwing darts at the stock page have as much chance of being right as the most respected market forecasters.
But wait...
…it gets better! Friedman actually quotes Taleb’s Black Swan Theory to justify his position that statistical analysis is flawed. Now, in case you haven’t actually read the book, let me clarify that, by Black Swans, Taleb means events that literally change the course of history — e.g., the invention of the personal computer or the start of WWI. Please tell me, what Black Swan events have there been in football or sports generally in the past 100 years?
PPS
I’m sorry to be so hard on you, drummer. I don’t mean to question you knowledge of football, which is deep. I just don’t get why you keep harping on a subject that’s obviously outside your area of expertise. It reminds me of Joe Morgan ripping on Moneyball without even reading the book.
Thanks for your response...
If you will please allow me to clarify a few things…
First, if you look back at my first post on analysis, you would see that I already have addressed my level of understanding of the subject matter. I also addressed that I don’t discount the value of it. I think most of the writers who are “stat guys” here know that I hold them with high regard, and that I understand where they are coming from. I don’t slam the work. In fact, the very reason why this is my favorite fan site, even above the other traditional media outlets is because this site recognizes the cutting edges of modern football. My first post on the subject addressed how I discovered Football Outsiders before way before I became a member here. I didn’t call in to slam Beane on Moneyball. I called in to ask him how it applies to football. The original post was really meant for the people who discount the subject matter with impunity, not even giving them any value at all. The basic premise of the original post was how data has been and is a part of daily life, because in this “Brave New World”, data is being collected every minute of the day. I ended the premise with choice, not slamming the messenger’s who bring us their results, but by choosing on how you value it, and where it applies to your interest.
The second post on this is really for people like yourself who have a deeper involvement with statistical analysis. Again, going back to my original post I wrote “people who endeavor in this science”, whereas Friedman’s premise is that it isn’t a science, so right there we have a fundamental difference of opinion. Of course, he, like yourself and others here, have a greater understanding of the subject, which is why I posted his articles. Basically, I’m just a messenger here. I find his blog fascinating because it’s more pragmatic, and guess what? He still has to use stats as part of his message.
You had a comment in the original post about “statistical entrepreneurs”. Well, part of Friedman’s article may touch on that here:
If I make a skill set-based comparison of two players based on my informed opinion—i.e., an opinion based not only on watching a lot of basketball but also on interacting with professionals who make their living evaluating basketball players—every Tom, Dick and Harry thinks that his opinion is just as valid and informed but if some guy invents a formula, gives it a catchy name and says that player x is worth 30.2 but player y is worth 28.7 then Tom, Dick and Harry are ready to bow down to those numbers as if they are the Golden Calf.
Going back to my original post, I touched lightly on the misuse of stats when it is combined with opinion, and how given to one’s personal belief, stats become subjective. I think everybody can agree to a point that through all this “statistical noise”, statistical analysis with various formulas can lend to it being subjective. Here at Niners Nation, there already is 2 different formula’s posted from Florida Danny and smileyman. Even a “stable” formula like DVOA is constantly being tweaked by others to present their particular results. This stuff is great, and it helps us understand things a bit better. But it’s still not perfected, nor may it never really be. Whether it’s “noise” or “music” to one’s ears is up to that person. I don’t have a particular stake in any of it. I’m just trying to present a different side to it, and not forcing an opinion on anyone.
Again, thanks for the passionate response. I get where you’re coming from. I just want to let you know that I respect statistical analysis, the people who work at it, and I never discount it.
Well, we're waiting....
No, thank YOU...
..for your thoughtful and eloquent reply, drummer. Now I feel like even more of a douche. Thanks as well for the Friedman links — they guy actually makes some interesting points about the limits of statistical analysis in basketball specifically.

by 





































