Navigation: Jump to content areas:


Pro Quality. Fan Perspective.
Login-facebook
Around SBN: Kobe Bryant Will Never Top Michael Jordan

A look at Statistical Analysis, Part Two: David Friedman style

A few weeks ago, I posted some thoughts about statistical analysis and their importance to fans of sports, as well as it’s application. One of my favorite writers about the NBA is David Friedman, whose blog, 20 Second Timeout, has some of the best comprehensive analysis and opinion out there on the Internet. Here are a couple of articles regarding statistical analysis, and it’s application to basketball.

After the jump I’ve posted a couple of quotes for preview and I highly suggest reading the linked articles. Enjoy. 

Economics is Not a Science, Nor is Basketball Statistical Analysis

Economics is Not a Science, Nor is Basketball Statistical Analysis, Part II

Star-divide

"Stats gurus" plainly do not want to discuss or consider the fact that some of their most precious numbers--the raw data that they plug into their formulas, stats like assists, steals, blocked shots and turnovers--are subjectively recorded. During last season's playoffs, I did a detailed post demonstrating that Chris Paul's supposedly record setting playoff assist totals were in fact inflated by generous scorekeeping. Shouldn't that be of interest to the "stats gurus"? Isn't that claim something that they seriously need to investigate on their own to either confirm or reject? I provided very specific information so that anyone could watch a tape of the game and find the exact plays that I described and thus judge for themselves whether or not each of those assists should have been awarded. Yet I see no indication that the "stats gurus" are the slightest bit concerned about the fact that a lot of their basic data is seriously flawed. A lot of these guys spent a good portion of the season pumping up Chris Paul as the MVP and it is highly likely that they did so on the basis of bogus assist numbers. Based on a skill-set evaluation of Paul's game, I consider him to be the best point guard in the NBA and a top five MVP candidate but that is not the point; the point is that if you are basing your whole analysis of the NBA purely on numbers and some of the basic numbers you are using are not right then your whole analysis is bogus. If a real scientist finds out that the raw data he has gathered is flawed then he understands that he has to gather new, accurate data. Unfortunately, many of the basketball "stats gurus" are not scientists; they are "mad scientists" at best.

 

In an insightful post titled Using statistics in basketball: the bar is higher, Rosenbaum writes, "Statistical analysis can play a critical role in basketball decision-making, but it can also be misleading if the complexities of the game of basketball (and the statistical issues generated by those complexities) are not well understood. In other words, the bar is higher for statistical analysis in basketball than it is in baseball. Ultimately this will greatly benefit the teams that incorporate skilled statistical analysts in the right way, because the greater complexities in basketball will mean that it will be harder for other teams to ever catch up with the first teams that get this right. It will be fascinating seeing how this all plays out over the next few years."

 

In The Difference Between Measuring Defense in Basketball and Baseball , I made the important point that basketball statistical analysis is pseudoscience because its practitioners do not base their research on the scientific method:

    1.    Ask a Question
    2.    Do Background Research
    3.    Construct a Hypothesis
    4.    Test Your Hypothesis by Doing an Experiment
    5.    Analyze Your Data and Draw a Conclusion
    6.    Communicate Your Results

Comment 14 comments  |  0 recs  | 

Do you like this story?

Comments

Display:

I like it.

The last block doesn’t dive enough into the point he is trying to make.

by bignerd on Mar 3, 2010 6:18 PM PST reply actions  

Friedman is great...

I found this little nugget in his Knicks article:

Jerry Krause once infamously stated that organizations win championships—a senseless swipe at the talents of Michael Jordan, Scottie Pippen and the other Chicago Bulls players who captured six NBA titles—but it is certainly true that organization wins championships—in other words, a franchise must be committed to doing things the right way from top to bottom in order to achieve the highest level of success. Just look at the New England Patriots, L.A. Lakers and San Antonio Spurs for three examples of 21st century sports franchises that have owners, talent evaluators and coaches who are intelligent and dedicated.

Hey 49ers FO, did you get this?

Well, we're waiting....

by drummer on Mar 3, 2010 11:15 PM PST up reply actions  

Even though I am 49ers fan I’m extremely skeptical about the greatness of an organization concept.

I doubt the Patriots knew what they were doing when they drafted Tom Brady in the 6th round.

What was so magnificent about the Spurs losing David Robinson to injury for the entire season which resulted in winning the lottery and coinciding with the year Tim Duncan declared pro?

Even the Lakers are more blessed than F.O. gifted. They had no post Magic Johnson plan but Shaq was swooned by Hollywood and insisted on playing in LA. The Pau Gasol trade still needs to be investigated by the SEC. The one brilliant move the Lakers have made the past two decades was targeting Kobe in the draft.

The 49ers were lucky in getting a true genius in Walsh and Eddie D. pretty much signed off on all of Walsh’s ideas/plans. The 49ers weren’t so genius in the post-Walsh era. Carmen Policy relied upon the talent Walsh left over and re-enforced the team by spending gobs of money in free agency. The plan worked for most of the decade until Walsh’s left over talent finally thinned out.

Even the Steelers owners are heavily praised without scrutiny. These are same owners using the same methods and philosophy that had the Steelers perennial bottom feeders until Chuck Noll was hired. Same plan for 6 decades: 3 decades the Steelers were rotten, 3 decades the Steelers were good/great.

Now do I believe in stupid franchises? You bet!

by bignerd on Mar 4, 2010 12:13 AM PST up reply actions  

I found these comments very interesting and very illuminating
“Stats gurus” plainly do not want to discuss or consider the fact that some of their most precious numbers—the raw data that they plug into their formulas, stats like assists, steals, blocked shots and turnovers—are subjectively recorded.

Trying to gather data in football is particularly hard because of how hard it is to separate individual effort. If a QB throws the ball and it bounces off his receiver’s hands it still goes down as an incomplete pass in his stats. OTOH if he throws a 2 yard quick out and the receiver makes a couple guys miss and takes it 80 yards to the house he’s credited with an 80 yard TD pass.

I like how FO tackles this with the running stats separating it into the line and the RB effort, but it’s still an issue even there.

Yes Drew K, Tim Tebow will probably get picked in the first round.

by smileyman on Mar 3, 2010 9:50 PM PST reply actions  

It sounds like stats just need to be expanded then for things like WR drops in to a QB stat line and all of things that aren’t measured as cold hard stats need to be considered.

A computer once beat me at chess, but it was no match for me at kick boxing.

by Drew Kerr on Mar 3, 2010 10:39 PM PST up reply actions  

I think so too

but the NFL won’t do it because they’re way behind the times. Sites like Football Outsiders and PFF might take a look at it, or someone else make take a stab at it.

Yes Drew K, Tim Tebow will probably get picked in the first round.

by smileyman on Mar 3, 2010 11:12 PM PST up reply actions  

it's even more difficult than that

When does it count as a drop? Is it a drop if the pass is way behind the WR but he can still get a hand on it in a futile attempt?

Likewise, if a QB leads a receiver perfectly and “throws him open” for a long RAC gain, shouldn’t the passer get some credit?

by microwave donut on Mar 4, 2010 10:08 AM PST up reply actions  

This is like saying...

…that science is subjective because it depends on fallible people to record observations. It’s true, but totally misleading, because it implies that science is therefore incapable of generating reliable predictions.

by Bigmouth on Mar 4, 2010 8:50 AM PST up reply actions  

I really think you should stop writing...

…about a subject you don’t seem to understand. I’m sorry if that’s rude, but give me a break. Quoting some articles by a critic of statistical analysis of basketball is not a substitute for comprehension of statistical analysis generally. And no offense, but nothing you’ve ever said on the subject gives me any confidence that you actually understand what you’re criticizing. Rather, your comments remind me of people who go onto physics forums claiming to have debunked General Relativity.

by Bigmouth on Mar 4, 2010 8:32 AM PST reply actions  

PS

The fact that Friedman would fault economists for failing to predict the behavior of the market reflects a rather basic misunderstanding of economics. You can’t predict the behavior of the market. How do we know this? Because statistical analysis — that’s right, statistical analysis — tells us that the behavior of stocks is random. It’s just betting. To use a famous analogy, monkeys throwing darts at the stock page have as much chance of being right as the most respected market forecasters.

by Bigmouth on Mar 4, 2010 8:40 AM PST reply actions  

But wait...

…it gets better! Friedman actually quotes Taleb’s Black Swan Theory to justify his position that statistical analysis is flawed. Now, in case you haven’t actually read the book, let me clarify that, by Black Swans, Taleb means events that literally change the course of history — e.g., the invention of the personal computer or the start of WWI. Please tell me, what Black Swan events have there been in football or sports generally in the past 100 years?

by Bigmouth on Mar 4, 2010 8:44 AM PST up reply actions  

PPS

I’m sorry to be so hard on you, drummer. I don’t mean to question you knowledge of football, which is deep. I just don’t get why you keep harping on a subject that’s obviously outside your area of expertise. It reminds me of Joe Morgan ripping on Moneyball without even reading the book.

by Bigmouth on Mar 4, 2010 8:57 AM PST up reply actions  

Thanks for your response...

If you will please allow me to clarify a few things…

First, if you look back at my first post on analysis, you would see that I already have addressed my level of understanding of the subject matter. I also addressed that I don’t discount the value of it. I think most of the writers who are “stat guys” here know that I hold them with high regard, and that I understand where they are coming from. I don’t slam the work. In fact, the very reason why this is my favorite fan site, even above the other traditional media outlets is because this site recognizes the cutting edges of modern football. My first post on the subject addressed how I discovered Football Outsiders before way before I became a member here. I didn’t call in to slam Beane on Moneyball. I called in to ask him how it applies to football. The original post was really meant for the people who discount the subject matter with impunity, not even giving them any value at all. The basic premise of the original post was how data has been and is a part of daily life, because in this “Brave New World”, data is being collected every minute of the day. I ended the premise with choice, not slamming the messenger’s who bring us their results, but by choosing on how you value it, and where it applies to your interest.

The second post on this is really for people like yourself who have a deeper involvement with statistical analysis. Again, going back to my original post I wrote “people who endeavor in this science”, whereas Friedman’s premise is that it isn’t a science, so right there we have a fundamental difference of opinion. Of course, he, like yourself and others here, have a greater understanding of the subject, which is why I posted his articles. Basically, I’m just a messenger here. I find his blog fascinating because it’s more pragmatic, and guess what? He still has to use stats as part of his message.

You had a comment in the original post about “statistical entrepreneurs”. Well, part of Friedman’s article may touch on that here:

If I make a skill set-based comparison of two players based on my informed opinion—i.e., an opinion based not only on watching a lot of basketball but also on interacting with professionals who make their living evaluating basketball players—every Tom, Dick and Harry thinks that his opinion is just as valid and informed but if some guy invents a formula, gives it a catchy name and says that player x is worth 30.2 but player y is worth 28.7 then Tom, Dick and Harry are ready to bow down to those numbers as if they are the Golden Calf.

Going back to my original post, I touched lightly on the misuse of stats when it is combined with opinion, and how given to one’s personal belief, stats become subjective. I think everybody can agree to a point that through all this “statistical noise”, statistical analysis with various formulas can lend to it being subjective. Here at Niners Nation, there already is 2 different formula’s posted from Florida Danny and smileyman. Even a “stable” formula like DVOA is constantly being tweaked by others to present their particular results. This stuff is great, and it helps us understand things a bit better. But it’s still not perfected, nor may it never really be. Whether it’s “noise” or “music” to one’s ears is up to that person. I don’t have a particular stake in any of it. I’m just trying to present a different side to it, and not forcing an opinion on anyone.

Again, thanks for the passionate response. I get where you’re coming from. I just want to let you know that I respect statistical analysis, the people who work at it, and I never discount it.

Well, we're waiting....

by drummer on Mar 4, 2010 11:17 AM PST up reply actions  

No, thank YOU...

..for your thoughtful and eloquent reply, drummer. Now I feel like even more of a douche. Thanks as well for the Friedman links — they guy actually makes some interesting points about the limits of statistical analysis in basketball specifically.

by Bigmouth on Mar 5, 2010 9:09 AM PST up reply actions  

Comments For This Post Are Closed


User Tools

Media Requests please email ninersnation@gmail.com

FanPosts

Community blog posts and discussion.

Recommended FanPosts

Small
Site Decorum: Remember, We Are ALL 49er Fans
Steve_young_small
Game Day Food

Recent FanPosts

Small
Concussions...
Small
Is Harbaugh lying or does he mean what he says?
872_small
Where have you seen 49er players?
Download2_small
Can the 49'ers Maintain their Turnover Differential in 2012?
Sfak_small
Why are you a 49er fan?
6a00e5500c77218833011168f234b4970c_small
FOX: "How To Save The Sport"
Small
Old Spice Patrick Willis Football ProCamp
Dave_small
Call For Moderators

+ New FanPost All FanPosts >


Head Ball Coach

Dave_small David Fucillo

Howtheyscoredcat_small howtheyscored

313483_2054510893373_1562580382_31984672_1965025_n_small James Brady

Coordinator

Pirates_small smileyman

Bowman_avi_sm_small Tre9er

Assistant Coach

Pixies_logo_small (Florida) Danny Tuccitto

Memento-lies_small urnext

Me_on_beach_small WesHanson

Dylan_cannes_small Dylan DeSimone

Officiating Crew

Jackalope_card_small wjackalope

These3words_small these3words

Joe_and_bill_small twolfe2

428030_10150598134996875_112852666874_9167376_1157036734_n_small mikeinsp