This is a copy (with some things added) of a comment I made on Grant Cohn's blog. I was a bit frustrated at his use of general 49ers offensive data to make judgement calls about Alex Smith in particular, so I did a little game-charting research and came up with a few conclusions. I am a big Alex Smith fan, but I tried to make this as unbiased as possible.
Grant, you have to do some research before you make claims like this. You can't just say "when Smith was on the field, the 49ers were ____ on 3rd downs." It's a useful stat in gaging the Niners offense, but it proves nothing statistically about Smith, as it lumps the Niners woeful run game along with it. Now, as it happens, I agree with you. I went through every game, play-by-play, and it's true: Alex is not great on third downs: he has a roughly 34% third and fourth down conversion rate. This includes scrambles and pass interference penalties, but not designed runs. If you throw out 3rd and 20 or longer, its about 36%. These are accurate numbers if you want to talk about Alex Smith as a player, rather than Alex Smith as the representative of the Niners offense. Still, these are not fantastic numbers.
Smith's main problem has been taking sacks: he took something like 20 sacks on third down last season (I accidentally closed my data without saving after calculating the conversion percentage, so these numbers aren't totally accurate). However, if you compare that to previous seasons, this is far preferable to his penchant for throwing interceptions or fumbling when pressured. He has calmed down in the pocket some, but in obvious passing situations he's not been a consistent passer: in fact, he's much better as a scrambler, scrambling for at least 7 first downs on 3rd down.
One other thing I noticed is a very high number of incomplete passes on third down to Michael Crabtree. Take that as you will: The more I see of Crabtree, the less I think of him. However, to be fair, he had a number of clutch catches, usually medium range (think 10-20 yard catches on third-10 or so). He is a useful WR, but I'm glad we got Manningham and Moss.
One last comment: Only once did Smith exceed a 50% conversion rate last season. Obviously this has got to improve. Even though the conversion rate was only really bad (think <30%), 4 games out of the year, Smith is certainly not doing much to merit a lot of trust in obvious passing situations, like 3rd and long. We'll see what happens with another year in the offense and the addition of Moss and Manningham, as well as possibly another rookie. I'm optimistic, but still skeptical.