So, for most NFL teams, we've hit the quarter mark, and that seems like a nice, arbitrary milestone to see how my QB projections based on three-year averages are doing. To catch up on what this thread is about, I highly recommend reading that post and the ensuing conversation. This is especially important if you want to jump into this thread and fail to make the distinction between predictions and projections.
There were a few errors in the original thread, though. For one, I didn't have a projection for Ryan Tannehill, which was stupid of me. I also lightly revised a couple of the projections pre-season, but only by a tenth or two, so nothing that throws what you see there completely out of whack.
To reiterate the original methodology, though:
- Under most circumstances, quarterbacks must regress near their three-year averages (I used the averages for basic rate stats--TD%, INT%, Comp%, Y/C--to get at the counting stats. I also have some hidden sack%/sack yard averages that I used to calculate ANY/A).
- I must assume a 16 game schedule for each QB.
- If a team has two likely enough starters, I must project both of them for the full 16.
- For quarterbacks younger than 28, I can project positive or negative development if a) the trend is already apparent in the three-year sample, or b) if the available sample represents less than three years and there is a coaching or personnel change to suggest it, or c) if the sample represents less than three years and standard development curves suggest it. And even then, I have to be modest.
- For quarterbacks older than 32, I can project decline only if the trend is already apparent in the three-year sample.
- For rookies, I don't really have a system, so I'll just try not to look like an idiot.
- For total attempts, I don't really have a system, so I'll just try not to look like an idiot (insofar as any projection that has John Skelton throwing the ball 550 times does not look idiotic--I'll elaborate on how I know guys like John Skelton will never throw this much, but am going with it anyway, below).
For the quarter report, I get to make a few crucial corrections to parts of the projection system that, at the time were necessary, but were also unfortunately kind of confusing. For one, I am drawing no arbitrary line for attempts. I am pro-rating the projection rates for the actual attempts numbers these quarterbacks have thrown. For another, I am not checking projections against quarterbacks who have not played sufficiently. That makes no sense.
So, the following is a breakdown in four parts of how the projection system has sized up, through four weeks, all of the NFL quarterbacks who have played significantly in at least three games.
I have decided to use Adjusted Net Yards Per Attempt as my closeness meter, because it is my easily calculated stat of choice. I have included Passer Rating because it's a pretty standard thing, not because I especially like it. I think ANY/A does a better job of providing a rough quality count.
The first part shows quarterbacks who, by ANY/A, are within half a yard of their projection. For all of these quarterbacks, the top line is the projection, and the bottom line is their "actual" stats. I put actual in quotes there only because I did something stupid and used the rates to generate the totals. Which is stupid for several reasons, mostly because some of the yards end up being off, and also mostly because it was more work than just typing in the real stuff, but also wasn't worth changing once I realized how stupid it was because any differences between the actual stats and the "actual" stats end up being almost 100% negligible. Seriously, it's like 3 extra yards for Drew Brees or something:
What this tells us:
So far so good! That's roughly a third of the league that, over the short span of only four weeks, my projections are currently doing a pretty good job on. Manning is nailed almost exactly. There are some other interesting things to note here, though. The hidden stats that affect ANY/A are sacks taken and sack yards. This would suggest that, Alex Smith, for instance, is wasting a very real amount of his otherwise fantastic efficiency by taking so many sacks. His sack rate has skyrocketed this year, and generally have skyrocketed under Harbaugh. There may be a chicken/egg argument over whether or not he sacrifices sacks to gain in efficiency, and another argument about whether or not it's worth it. What is clear is that, because his sack numbers are so high, he's landed, production-wise, square on his three-year projection despite throwing the ball as well as ever.
Mark Sanchez is a good example of the opposite. He's generally sucked more than ever from a pure throwing standpoint, but his sack numbers are good enough to bring his overall production back in line with his projection.
The other question, of course, is sustainability. It seems difficult to suggest that Alex Smith could maintain a sack rate near 10%, or even one as high as his 9% from last season. If his sack numbers go down--that is, if they simply regress normally--all that additional efficiency he's displaying will stop being wasted. Essentially, he's a sack rate regression away from being a top-ten QB by ANY/A.
Also, I'm getting hella lucky about Brandon Weeden. Projections based on Zero Data should never be so close.
Onto the next group! This chart shows the guys who were between 0.6 and 1 yards from their ANY/A projection:
What this tells us:
The projection system is still doing pretty well! That's 18 QBs who landed within 1 ANY/A of their projection. The 1 ANY/A mark is sort of my cutoff point for the system doing well, and hitting well on 18/32 with only four weeks of data feels like a pretty solid result to me. This all, of course, could get screwed monumentally by the end of the season.
That said, my luck quotient is still sitting pretty high. I have three guys in this section whose projections were based on one or fewer years of data. So, on the one hand, I can be proud of my ability to make guesses, but on the other hand, well, I'm still doing pretty well.
But, looking at this, I'd say that it tells us some solid stuff about some of these guys. For the most part, though Cutler and Brees have looked much worse than usual (and those guys are held to different standards, obviously), their overall production has not been that far off from what you would expect. We would expect upward regression for both, but probably not as much as a first glance would suggest. I'm not sure if this is useful fantasy information. I like to pretend that it is.
The next group is the QBs who are currently producing between 1.1 and 1.5 ANY/A from their projection. This group is where things start to get weird, and where the regression question starts to become a lot more interesting:
What this tells us:
There are some good quarterbacks over-performing their projections right now. It's hard to say that Matt Ryan is due for regression because he was pegged as a guy from the scouting side who was likely to see a nice, big jump (thank you very much, Mr. Jones). It almost seems safe to trust what you see from him.
Ryan Fitzpatrick, though, is a PRIME suspect for a SHARP regression. Even this early, he's been almost as deeply inconsistent as ever, and it's impossible to believe that he'll sustain anything resembling his current TD rate. Watch out, Fantasy Owners. This guy is not worth picking up for the touchdowns he's already thrown, because he probably won't be throwing them like that going forward... whereas, his INT rate is pretty close to what you'd expect.
Of course, I can't imagine I'm talking to a bunch of people who believe in Ryan Fitzpatrick. So, in a sense, the projection system is telling you what you already know. Which is GREAT. It agrees with reality. Yay!
FINALLY, also, we begin to see the limitations of projecting players who have a small data pool. Andy Dalton is not necessarily a candidate for regression based on the system. Rather, he was always a good candidate for the system to end up with a bad projection for. Nothing to see here. Move along. Things working exactly the way they should.
And, assuming he retains his starting job, I'm going to go out on my first limb here and say that projection system's results for Michael Vick should be heartening to fantasy owners. He's a high-reward prospect right now, underperforming in areas that seem likely to see positive regression. And, as ever, he brings good value on the ground. This might be his low-value point. So if you think he's keeping that job and you need some QB depth... I'm just saying: Think about it.
And now the last group. These are the guys the projection system blew. More than 1.5 ANY/A off of their projections:
What this tells us:
This tells us what we already knew: I hate Jake Locker, and maybe I shouldn't hate him so much. Well, the projection system also hates him, but he's really done a good job so far of justifying the raw-tools argument.
Also important to note that four of the seven guys who the projection system blew are guys who have smaller data sets than the system called for. Nothing to see here! Exactly what should be expected to happen. Yay.
The other guys are Tony Romo, Philip Rivers, and Aaron Rodgers. Rodgers is an obvious choice for positive regression, but that's no news to fantasy owners or anything. Anybody who dropped Aaron Rodgers at this point is a stone cold psycho. But Rivers and Romo are interesting cases. Rivers is now in the midst of his second consecutive worse-than-expected season. It may be time to start thinking about how much positive regression you actually want to expect out of this guy. Maybe this is small sample shenanigans. Maybe he's just a lot closer to the guy we saw in 2011 than the Philip Rivers we know and love.
And Romo. Tony Romo. Here's my other limb. The guy is, like Vick, at his low value point. He probably just played the worst game of his life. He's also probably still a really good quarterback. There seems to be a solid straw hat argument about his mental makeup, but I think he's another guy who could be had right now for less than fair value if you need QB depth and provide real production very soon.
Also: Cam Newton may be better than he looks.
Also: Hahahahahaha Robert Griffin III. I won't step up yet and say that he's this good. But, man. He's been good.
What do you think? I'm pretty happy with the results thusfar. I'll try to check in every four games, but I can't promise I'll have time.