Don't Miss Your Chance to Grill Football Outsiders

putting words “unvarnished,” and “indisputable” in front of a sentence does not make your argument any more correct. kind of like if i were to say, “it’s indisputable that you are a rambling, underinformed idiot” does not, in fact, all of a sudden mean you are truly a rambling, underinformed idiot. what is so hard about this?

what’s hilarious to me is that just yesterday, i detailed the theory behind DVOA. in that detailing, i made it pretty clear that they follow a methodology developed by a group of authors 20 years ago — via an entire book’s worth of objective statistical analyses, no less. add this to the fact that aaron spent years doing the statistical analyses to see which factors affect play success, points, and winning before he published DVOA for the first time, and continues to re-analyze the impact of those factors every offseason. how on earth can you or anyone charge him with narcissism and a lack of objectivity given these facts?

the freaking guy didn’t even develop the theory that forms the theoretical basis for his stat, the stat is based on objective statistical analyses, and it’s tweaked every year based on a re-analysis of the factors that DVOA accounts for. do you think he pulled “successful plays are more important in late and close games” or “successful plays are more important in the red zone” out of his ass? no! they’re common sense, and they’re backed up by statistical analyses showing the correlation between successful plays and winning are higher when those successful plays occur in late and close games or the red zone. such ignorance of very basic facts surround DVOA tells me either you’re making this stuff up as you go along or you’re a a rambling, underinformed idiot…but i don’t mean that in a bad way. i’m sure you’re a great person.

finally, about name-calling. again, this is semantics. did you say “they have a god complex”? no. but you said they think DVOA is “holier than thou”. um, same thing. did you say “they have contempt for their readers.” no. but you said they correct the mistaken humans who’ve read their couterintuitive predictions “practically spitting as they do.” um, spitting on readers who don’t agree with you = contempt for readers. again, i can call you a rambling, underinformed idiot, or i can just say something like “bkisforsf continuously evinces argumentum verbosum.” in semantics, they’re different. in effect, they’re the same.

If you don’t like the tools, don’t use them. It’s not like Football Outsiders is claiming to be all-knowing.

Are they wrong? Sure, but so is every pundit who decides they’re going to predict how a season goes. Nobody does those right.

Football Outsiders just brings another approach to the table and in my opinion it’s a useful one. You don’t like them and that’s fine but there’s no need to act like they assaulted your grandmother.

how they “hype” their predictions? have i missed their 1-hour “the decision”-esque program on ESPN where they announce their 49ers prediction? and also, i can tell you from a recent conversation i had with aaron @ FO that they are not ecstatic about their projection for the 49ers this year. so, if they’re all “the 49ers are going to suck because we hate the 49ers, bet on it!” i haven’t seen it. their model predicts that the 49ers will most likely win about 6 games. they put that in a book, along with projections — some suprisingly bad, some surprisingly good — for every NFL team. if they’re wrong, which they admit a million times over on their website, then attack their model. once people start saying, “they hate the 49ers” or “they hype their predictions” or whatever personal attributes you’d like to attach to them, it’s really a figment of your imagination.

last year, they called the 49ers “an obscure football team” because, well, they were. they went from one of the gold standard franchises in the league, on MNF 3 times a year, spotlighted in the playoffs every year, with stars up and down a roster to a team that never led off sportscenter, had their highlights recapped 45 minutes into NFL gameday, were never talked about in the pregame shows, etc. indeed, they became a totally obscure football team to the general public. obviously, as niner fans who post and comment on a niners blog, we don’t think they’re obscure. but leave this little cocoon of ours, and ask people outside the cocoon about our beloved team from 2003-2008, and they’d pretty much have nothing to say. from 1981-2002, that was clearly not the case. the team went from high-profile to “obscure.” i don’t see anything at all contemptuous about simply stating the reality of a total lack of popular attention to the 49ers outside of their fan base.

I’ve read several articles going in depth about how Marathe uses a stats-based report towards all drafting decisions. A rather interesting one (which I’ll have to see if I still have) talked about how they had a formula for running backs that compared several factors including 40 time at the combine, plus a bunch of other things.

I think that’s the way the NFL will be trending in the future, especially if the 49ers can show success using those kinds of models.

if you had paid any attention at all, you would have realized by now that i have many of the same concerns about FO’s stuff that you do. for instance, you said this earlier:

regression can take account of interactions amongst variables, regimes and many other dynamics that people associate with complex adaptive systems. There’s more to it than OLS. Though, as you might expect from the above, I take a dim view of it as a tool to attain deified enlightenment of football reality.

and, in a post of mine, just yesterday, i raised this same exact concern with reliance on OLS, and offered up an alternative that improves considerably given what they’re actually trying to model.

my problem is not with legitimate methodological criticisms of FO, as i’ve proven time and again on here (hey, just yesterday even). it’s with the following:

you offering up the factually incorrect this in the guise of a question:

Is there any adjustment for whether ‘successful plays’, big plays, touchdowns, yards, etc. come amidst lopsided blowouts or in close games where the outcome is in doubt?

me correcting you in clear language:

answer: yes. each play is weighted by various types of game situations. one such type is "score gap." so, plays in late and close games are given more weight when calculating DVOA than plays in blowouts, and so on.

and, dissatisfied with my correction, you pivoting unapologetically to this:

Fair enough, however the question is still relevant. If the variable is not missing, it may still be inappropriately measured or, as is my suspicion given what I’ve seen of what they do, given inappropriate weight

oh? so you were asking a question for which you presumably didn’t know the answer. but when presented with the answer, now it’s not that the variable’s missing, it’s that “as is [your] suspicion” it’s measured incorrectly or weighted incorrectly or [insert other throw-stuff-on-the-wall-to-see-if-it-sticks reason for why they suck here].

and then, on top of that, proceeding to just flat out call them snake-oil salesemen:

they have no compunction about presenting their more counter-intuitive assertions as correctives

I’m just no fan of the hubris with which they market themselves, and the overhyped credibility they are afforded amongst the football faithful

i mean, you’re basically saying they can’t accurately predict anything, but are going around proclaiming themselves as the nostradamus of football, just to make money. of course, you offer no proof whatsoever that they actually do exhibit such hubris. in fact, on our own little site here, they’ve said the opposite:

I have a fair amount of confidence in DVOA, but it’s far from foolproof as a predicative measure; if anything, I think you use DVOA to try and find spots in your own analysis from watching games to try and find things that you might have a blind or prejudged eye to.

and on their own website, in the very post that you cite as them being dismissive of their serious suckitude in 2009, they say this:

However, let’s be honest, saying “well, gee, everyone has a bad year, cut us some slack” sounds pretty damn whiny. It’s probably a lot better to go look at how teams changed (and did not change) between 2008 and 2009 to see if that can teach us any lessons as to what went wrong with last year’s projections.

and this

I wrote about it a couple times earlier in 2010, but this chart fully shows how much our 2009 defensive projections sucked. The correlation between our projections and teams’ actual defensive DVOA was pretty much zero — in fact, it was on the negative side of zero. A dartboard would have been just as accurate.

yes, all of these are clearly the words of people suffering from hubris, and of people who, as you say don’t seem to have an

appropriate understanding of the limits of what they’re trying to do

yes, exactly!

so my problem is you faking questions when the reality is you’d like to go on a diatribe about how inaccurate and snake-oil-salesman-y FO is, then after having one of your “questions” answered, pivoting toward basic ad hominem attacks on them that are totally the opposite of what they openly proclaim, both on their own site, and the very site on which you’re sitting here unleashing the attacks; all of this while offering no examples of them hyping themselves (except shilling their book on their own website for obvious reasons). instead, you do the classic debate tactic of asserting a falsehood and then putting the onus on us to prove you wrong:

Seriously, if I’m wrong, please point me to where they appropriately caveat those assertions.

and just to clear up 2 more factually incorrect assertions. this

However, just because they’ve tried to capture all those factors, doesn’t mean they’ve been remotely successful at so doing- much less across a broad swath of circumstances and years.

is directly refuted by the fact that, up until last year, they were more accurate than vegas.

finally, this

The criteria they compile are indeed entirely subjective.

is proven a mischaracterization because, in fact, “they” are not the ones who originally identified these criteria:

i made it pretty clear that they follow a methodology developed by a group of authors 20 years ago — via an entire book’s worth of objective statistical analyses, no less.

again, see how this works? it’s a process whereby, when someone says something you disagree with, you respond with links and quotes providing evidence that they are wrong. i’ve done this several times now, yet you find it totally accurate to say this

put Florida Danny in the position of having to back up his claims

when i’ve been doing just that, backing up my claims (with citations, even!) all along.

everyone, spot the strawman!

There’s more to it than OLS. Though, as you might expect from the above, I take a dim view of it as a tool to attain deified enlightenment of football reality.

is there anyone in their right mind who thinks, is making, or has made the argument that regression is “a tool to attain deified enlightenement of football reality?” anyone? i know neither FO nor i have. in fact, i made the exact opposite case yesterday.

and here’s more evidence of that clear-as-day lack of self-awareness you charged FO with. a journalistic recap of the MIT sloan sports conference reported:

He and Schatz also discussed the inherent inefficiencies in trying to create absolute statistics for player comparison.

who’s “he?” why it was the 49ers’ own paraag marathe, whom schatz participated in a panel discussion with. what was the title of the panel discussion?

I caught a bit of the ‘Emerging Analytics’ session. The session featured Michael Forde – Performance Director, Chelsea Football Club, Simon Wilson – Head of Performance Analysis, Manchester City Football Club, Aaron Schatz – Editor-in-Chief, Football Outsiders, and Paraag Marathe, Executive VP of Football & Business Ops, San Francisco 49ers.

yes, FO definitely has not, as of yet, worked to

develop a more appropriate understanding of the limits of what they’re trying to do, and an attendantly better respect for other ways of dicing the onion.

discussing the limits of what they do, along with ways to address those limitations…in the context of a panel discussion entitled “emerging analytics” no less. have i proven how wrong you are yet?

I mean woof woof.