FanPost

Rankers & Mockers, Pt.2

In Pt.1 we looked at the predictive ability of our two rankers and two mockers only to discover that they, as a group and individually, were pretty bad at predicting where in the draft individual players would be chosen. We also determined that they were not very good at slotting players in either the round they would be chosen in or even a 64-slot window around any player's draft spot. It was pretty disappointing to me.

 

But what about the usefulness of ordered lists that rank players within their position? Can they be more useful in predicting which players go where? In order to find out, I reordered the 2009 draft according to position, and you will find it at the end of this post. I was forced to make some judgement calls about which position category certain players belonged in, especially on defense where players can easily switch positions when they enter the pros - CBs become FS, OLBs become SS,  and DEs can become either DTs or OLBs. I did the best I could to resolve any differences.

The reordered draft list includes, beginning at the far left, a) the player's order in the draft within his position, b) the player's draft position number, and inside the brackets, c) the player's ranking within his position by our four prognosticators. They are:

MD = Maddog49er, our amateur bigboard creator

DS = NFLDraftScout.com, Bill Walsh's old ranking firm

WF = WalterFootball, a 7-round, all year mocker

FS = Fox Sports, which does a 7-round mock

We are looking for two metrics here; 1) the number of "hits" where our forecaster correctly predicted the player's positional ranking, and, 2) the number of players who made the list regardless of the order in which they were chosen. As an example, if a certain RB was the third one off the board and a forecaster had that player ranked #3 in his positional ranking, that is an "order hit". If a certain WR was chosen 5th in the draft but our forecaster had him as the tenth-ranked WR, that's still a "list hit" because he was on the ranking list even though the order was wrong. At the same time, if there were 30 WRs chosen in the draft and our draft guru has one of those 30 players ranked as the 32nd WR, that's a miss - anything after the last player taken at that position is a miss.

 

So let's see how our gurus did at predicting the entire draft. The first number inside the brackets is the number of "order hits" and the second number inside the bracket set is the number of "list hits". The number inside the brackets directly after the positional designation represents the number of players chosen at that position in the part of the draft we are testing.

POS(p)    MD      DS      WF     FS
QB(11)   (6,7)    (4,6)    (5,9)   (4,8)
RB(20)   (3,15)  (1,14)  (1,14) (2,14)
FB(3)     (2,2)     (2,2)    (0,2)   (0,1)
WR(34)  (4,22)  (3,27)  (2,26) (2,25)
TE(22)    (3,14)  (1,13)  (1,15) (3,13)
OT(18)    (3,16)   (4,15)  (3,13) (4,12)
C(7)        (5,7)      (5,6)    (1,6)   (1,5)
OG(13)   (0,8)     (0,8)     (0,8)   (0,7)
DT(23)    (2,19)  (8,19)   (4,22) (4,16)
DE(20)    (2,16)  (1,15)   (2,16) (2,15)
ILB(8)      (1,5)    (1,5)      (2,5)   (1,5)
OLB(15)  (3,8)    (5,8)     (0,9)    (2,7)
SS(10)    (2,9)    (2,7)      (1,8)   (0,5)
FS(7)      (3,5)    (1,5)      (1,5)    (2,4)
CB(39)   (6,31)  (2,29)    (1,29) (1,27)
------- ------ ------ ------- ------
p=250 (45,184)(39,181)(24,187)(28,164)

Avg. Order Hits = 34 or 13.6%
Avg. List Hits = 179 or 71.6%

As we can see, our gurus are not very good at ordering the players within their position when looking at the entire draft. But they are pretty good at getting the right players onto the list, with an average "list hit" rate of 71.6%. Maybe they can do even better if we just look at the top half of the draft. Let's see:

POS(p)    MD     DS      WF     FS
QB(6)      (6,6)   (4,6)    (4,5)   (4,5)
RB(10)    (1,6)   (1,7)    (1,6)   (2,7)
FB(0)      (0,0)   (0,0)    (0,0)    (0,0)
WR(17)   (4,14) (2,13)  (1,13) (3,14)
TE(11)    (3,9)    (1,9)    (1,10) (3,8)
OT(9)      (3,7)    (4,7)    (3,7)   (3,7)
C(4)        (4,4)    (4,4)    (1,3)   (0,3)
OG(6)     (0,4)    (0,4)    (0,4)   (0,4)
DT(12)   (1,10)  (8,10)  (3,8)  (4,9)
DE(10)   (2,9)    (1,8)    (2,8)  (2,8)
ILB(4)     (1,3)    (1,3)    (1,3)  (1,3)
OLB(7)   (3,4)    (4,5)    (0,3)  (2,4)
SS(5)     (1,3)    (1,3)    (1,4)   (0,3)
FS(4)     (3,3)    (1,5)    (1,3)   (2,4)
CB(20)  (3,15) (2,16)  (1,14) (1,14)
------- ------ ------ ------- ------
p=125  (35,97)(34,101)(20,91)(27,93)

Avg. Order Hits = 29.0 or 23.2%
Avg. List Hits = 95.5 or 76.4%

Well, our gurus managed to improve the number of players they correctly ordered within their position to almost 25% in the top half of the draft. The number of players correctly placed onto the list improved marginally to about 76%. But let's see how they did just with the top two rounds only:

POS(p)    MD     DS     WF    FS
QB(4)      (4,4)  (4,4)   (4,4)  (4,4)
RB(4)      (0,4)  (1,4)   (1,4)  (1,4)
FB(0)      (0,0)  (0,0)   (0,0)  (0,0)
WR(8)     (2,8)  (2,7)   (1,6)  (2,7)
TE(2)      (1,2)   (1,2)   (1,2)  (1,1)
OT(8)      (3,7)   (4,7)   (3,7)  (3,7)
C(3)       (3,3)    (3,3)   (1,3)  (0,3)
OG(1)    (0,1)    (0,1)   (0,1)  (0,1)
DT(7)     (0,6)    (5,6)   (3,6)  (3,6)
DE(10)  (2,8)    (1,8)   (2,8)  (2,8)
ILB(3)    (1,3)    (1,3)   (1,3)  (1,3)
OLB(2)  (2,2)    (2,2)   (0,2)  (2,2)
SS(3)    (1,2)    (1,2)   (0,2)  (0,2)
FS(2)     (2,2)   (1,1)   (1,1)  (1,1)
CB(7)    (1,5)   (1,6)   (1,5)  (1,5)
------- ------ ------ ------- ------
p=64   (22,57)(27,56) (19,54)(21,54)

Avg. Order Hits = 22.3 or 34.8%
Avg. List Hits = 55.3 or 86.3%

Within the top two rounds, our prognosticators did fairly well, corectly ordering over a third of the players within their positions and correctly placing, on average, over 86% of the players chosen in the first two rounds on their lists.

 

So what conclusions can we reach based on this analysis? I think it would be fair to say:

1. None of our gurus is very good at correctly predicting the order in which players will be taken within their position, although they did correctly order over a third of the players in the first two rounds.

2. Our gurus actually are fairly good at getting the right players onto the list, ranging from about 70% for the entire draft to better than 85% for the first two rounds.

 

I'll let the reader make his own judgements about which of our gurus is better at what predictions, but I suspect the differences are statistically insignificant. In addition, this is only one draft and perhaps our gurus had a bad year. Or maybe it was a good year. We don't know because it's a sample of one. But this exercise has made me a little more cautious about accepting mocks as any kind of accurate draft prediction and has tilted me towards favoring the use of positional ordered lists when looking at the draft. They may not get the order right after the first two rounds, but they are pretty good at getting the players who will actually be drafted onto the positional lists. But make your own judgements.

 

2009 NFL DRAFT (by position)

Note1: An "*" asterisk indicates a "miss".

Note2: The order within the brackets is MD,DS,WF,FS

(QB-11)
1-1(1,1,1,1) - Matt Stafford
2-5(2,2,2,2) - Mark Sanchez
3-17(3,3,3,3) - Josh Freeman
4-44(4,4,4,4) - Pat White
5-101(5,6,10,7) - Stephen McGee
6-151(6,5,5,5) - Rhett Bowmar
7-171(*,8,6,9) - Nate Davis
8-174(7,10,11,*) - Tom Brandstater
9-178(*,*,9,10) - Mike Teel
10-196(*,*,*,*) - Keith Null
11-201(*,*,*,*) - Curtis Painter

(RB-20)
1-12(2,2,2,2) - Knowshon Moreno
2-27(3,3,3,3) - Donald Brown
3-31(1,1,1,1) - Chris Wells
4-53(8,4,4,4) - LeSean McCoy
5-65(5,6,7,5) - Shonn Greene
6-74(13,10,15,10) - Glen Coffee
7-111(12,12,13,14) - Mike Goodson
8-129(4,5,6,6) - Andre Brown
9-134(*,*,*,*) - Gartrell Johnson
10-158(*,*,*,*) - Cody Glen
11-173(7,9,9,7) - Javon Ringer
12-185(10,8,8,11) - Cedric Peerman
13-192(16,16,17,18) - Aaron Brown
14-195(14,13,10,13) - James Davis
15-209(15,*,*,*) - Scott Bernard
16-211(*,*,*,19) - Chris Ogbonnaya
17-212(18,14,12,*) - Javarris Williams
18-215(*,*,*,*) - Fui Vakapuna
19-240(*,*,*,*) - LaRod Stephens-Howling
20-250(6,7,5,8) - Rashad Jennings

(FB-3)
1-128(1,1,*,2) - Tony Fiammetta
2-145(2,2,1,*) - Quinn Johnson
3-169(*,*,3,*) - Frank Summers


(WR-34)
1-7(4,3,3,3) - Darius Heyward-Bey
2-10(2,1,1,1) - Michael Crabtree
3-19(1,2,2,2) - Jeremy Maclin
4-22(3,4,9,4) - Percy Harvin
5-29(5,7,5,5) - Hakeem Nicks
6-30(7,6,4,7) - Kenny Britt
7-36(6,5,6,6) - Brian Robiske
8-50(8,9,17,9) - Mohamed Massaquoi
9-82(14,8,8,10) - Derrick Williams
10-83(*,26,33,26) - Brandon Tate
11-84(11,12,12,15) - Mike Wallace
12-85(17,16,11,13) - Ramses Barden
13-87(20,19,25,14) - Patrick Turner
14-91(15,21,21,*) - Deon Butler
15-99(9,10,7,8) - Juaquin Iglesias
16-107(10,13,14,16) - Mike Thomas
17-108(*,*,*,28) - Brian Hartline
18-124(12,11,10,11) - Louis Murphy
19-127(21,15,16,18) - Austin Collie
20-140(18,20,19,23) - Johnny Knox
21-141(19,18,*,20) - Kenny McKinley
22-144(16,14,18,12) - Jarrett Dillard
23-160(29,22,22,*) - Brooks Foster
24-175(*,33,*,*) - Quinten Lawrence
25-194(*,24,13,17) - Brandon Gibson
26-206(22,27,26,31) - Dominique Edison
27-224(*,30,34,*) - Demetrius Byrd
28-229(*,*,*,*) - Manuel Johnson
29-232(*,*,*,*) - Julian Edelman
30-233(*,*,*,32) - Sammie Stroughter
31-243(*,*,29,*) - Marko Mitchell
32-251(*,*,*,*) - Derek Kinder
33-252(*,*,*,*) - Freddie Brown
34-253(*,31,27,25) - Tiquan Underwood

(TE-22)
1-20(1,1,1,1) - Brandon Pettigrew
2-64(7,6,9,*) - Richard Quinn
3-71(*,*,*,*) - Matt Shaughnessy
4-89(2,2,2,2) - Jared Cook
5-98(5,4,3,5) - Chase Coffman
6-100(6,8,5,6) - Travis Beckum
7-121(3,5,6,4) - Shawn Nelson
8-122(9,9,10,9) - Anthony Hill
9-149(10,*,8,*) - Davon Drew
10-152(4,3,4,3) - James Casey
11-153(*,7,7,7) - Cornelius Ingram
12-161(*,*,20,14) - John Nalbone
13-170(*,*,*,*) - George Bussey
14-180(12,*,*,*) - Zach Miller
15-184(13,13,12,10) - Bear Pascoe
16-202(*,*,*,*) - Brandon Myers
17-208(14,12,11,12) - John Phillips
18-221(*,*,*,*) - Eddie Williams
19-237(*,*,*,*) - Jake O'Connell
20-241(*,*,*,*) - David Johnson
21-248(8,10,13,13) - Cameron Morrah
22-255(11,11,16,8) - Dan Gronkowski

(OT-18)
1-2(1,1,1,1) - Jason Smith
2-6(3,3,3,3) - Andre Smith
3-8(2,2,2,2) - Eugene Monroe
4-23(4,4,4,4) - Michael Oher
5-39(5,5,5,5) - Eben Britton
6-54(7,6,7,8) - Phil Loadholt
7-58(11,11,14,11) - Sebastian Vollmer
8-60(6,7,5,6) - William Beatty
9-75(19,16,*,*) - Robert Brewster
10-109(9,9,13,10) - TJ Lang
11-135(12,12,10,13) - Troy Kropog
12-139(*,*,*,*) - Colin Brown
13-155(15,15,12,*) - Xavier Fulton
14-156(*,*,*,*) - Garrett Reynolds
15-159(13,13,11,12) - Fenuki Tupou
16-162(8,8,8,7) - Jamon Meredith
17-181(17,14,*,*) - Andrew Gardner
18-228(14,*,*,*) - Lydon Murtha

(Center-7)
1-21(1,1,1,2) - Alex Mack
2-28(2,2,3,3) - Eric Wood
3-49(3,3,2,1) - Max Unger
4-77(4,4,5,*) - Antoine Caldwell C/OG
5-106(5,5,4,5) - Jonathan Luigs
6-225(7,*,*,*) - Blake Schlueter C
7-226(6,6,6,4) - A.Q. Shipley C

(OG-13)
1-51(3,2,2,3) - Andy Levitre OG
2-78(7,6,7,8) - Louis Vasquez OG
3-79(4,4,5,4) - Kraig Urbik OG
4-123(*,*,13,*) - Rich Ohrnberger OG
5-132(*,*,*,*) - Seth Olsen OG
6-163(1,1,1,1) - Duke Robinson OG
7-167(2,3,3,2) - Herman Johnson OG
8-193(*,*,*,*) - Matt Slauson OG
9-213(*,*,*,*) - Paul Fanaika OG
10-236(8,7,11,6) - Jaimie Thomas OG/OT
11-239(*,*,*,*) - Ryan Durand OG
12-246(10,10,*,*) - Lance Louis OG
13-254(5,5,4,7) - Trevor Canfield OG


(DT-23)
1-3(2,1,2,1) - Tyson Jackson
2-9(1,2,1,2) - B.J. Raji
3-24(4,3,3,3) - Peria Jerry
4-32(5,4,4,5) - Evander Hood
5-40(6,5,5,6) - Ron Brace NT
6-56(7,9,7,7) - Fili Moala
7-62(9,7,9,11) - Sen'Derrick Marks
8-67(8,8,10,8) - Alex Magee
9-68(3,6,6,4) - Jarron Gilbert
10-72(20,18,15,*) - Terrance Knighton
11-81(12,11,18,*) - Roy Miller
12-93(16,17,17,16) - Corvey Irvin
13-103(11,10,11,10) - Dorell Scott
14-113(23,*,24,*) - Vaughn Martin
15-115(15,13,12,14) - Sammie Lee Hill
16-133(*,*,8,*) - Tyronne Green
17-136(18,19,14,15) - Terrance Taylor
18-205(*,21,*,20) - Ra'Shon Harris
19-207(*,*,23,*) - Myron Pryor
20-210(17,15,13,12) - Vance Walker
21-234(*,*,21,*) - Darryl Richard
22-244(19,12,16,9) - Ricky Jean-Francois
23-249(13,16,19,*) - Clinton McDonald

(DE,DE/OLB-20)
1-11(2,3,2,2) - Aaron Maybin
2-13(1,1,1,1) - Brian Orakpo
3-16(7,7,7,6) - Larry English
4-18(3,4,4,3) - Robert Ayers
5-43(6,2,3,5) - Everette Brown
6-45(11,8,9,8) - Clint Sintim
7-46(5,5,6,7) - Connor Barwin
8-52(12,12,20,11) - David Veikune
9-57(9,10,11,10) - Paul Kruger
10-63(10,11,10,13) - Cody Brown
11-70(4,6,5,4) - Michael Johnson
12-105(17,19,19,16) - Henry Melton
13-110(*,*,23,*) - Victor Butler
14-117(13,13,16,12) - Kyle Moore
15-120(14,14,14,14) - Brandon Williams
16-125(8,9,8,9) - Lawrence Sidbury
17-182(*,*,*,*) - Jarius Wynn
18-199(*,*,*,*) - Stryker Sulak
19-204(*,*,*,*) - Will Davis
20-247(18,*,*,*) - Nick Reed

(ILB-8)
1-4(1,1,1,1) - Aaron Curry
2-35(3,3,3,3) - James Laurinaitis
3-38(2,2,2,2) - Rey Maualuga
4-118(*,*,*,*) - Stanley Arnoux
5-137(*,7,8,6) - Jason Phillips
6-146(5,*,6,*) - Scott McKillop
7-150(4,5,*,4) - Jasper Brinkley
8-176(*,*,*,*) - Spencer Adkins

(OLB-15)
1-15(1,1,2,1) - Brian Cushing
2-26(2,2,1,2) - Clay Matthews
3-69(3,3,8,7) - Jason Williams
4-76(11,13,10,*) - Levy DeAndre
5-97(*,5,3,4) - Tyrone McKenzie
6-104(7,6,9,9) - Kaluka Maiava
7-126(*,*,*,*) - Slade Norris LB
8-130(5,*,5,*) - Gerald McRath
9-154(3,4,4,3) - Marcus Freeman
10-186(*,*,*,*) - Robert Henson
11-197(14,14,*,*) - Stephen Hodge
12-214(*,*,*,*) - J.D. Folsom
13-218(*,*,*,*) - Brad Jones
14-230(*,*,*,*) - Moise Fokou
15-235(*,*,6,6) - Zack Follett

(SS-10)
1-34(1,1,2,2) - Patrick Chung
2-47(7,*,*,*) - Michael Mitchell
3-55(2,2,1,1) - William Moore
4-116(3,3,4,3) - Chip Vaughn
5-147(8,7,3,*) - Nic Harris
6-166(4,4,5,4) - Michael Hamlin
7-189(*,*,*,*) - Kevin Ellison SS
8-190(7,*,*,5) - Al Afalava
9-231(6,8,6,*) - Jamarca Sanford
10-245(5,5,9,*) - Courtney Greene

(FS-7)
1-33(1,1,1,1) - Louis Delmas
2-48(2,4,5,3) - Darcel McBath
3-95(3,2,2,2) - Rashad Johnson
4-114(5,5,*,4) - David Bruton
5-165(4,3,6,*) - Chris Clemons
6-219(*,*,*,*) - Curtis Taylor
7-223(*,*,*,*) - Troy Nolan

(CB-39)
1-14(1,1,1,1) - Malcolm Jenkins
2-25(3,3,3,4) - Vontae Davis
3-37(5,4,4,6) - Alphonso Smith
4-41(2,2,2,2) - Darius Butler
5-42(13,8,10,8) - Jarius Byrd
6-59(10,10,12,12) - Sherrod Martin
7-61(4,5,5,3) - Sean Smith
8-66(8,11,18,14) - Bradley Fletcher
9-73(32,*,*,31) - Derek Cox DB
10-80(6,7,7,9) - Kevin Barnes
11-86(11,18,14,10) - Asher Allen
12-88(17,19,20,20) - Lardarius Webb
13-90(16,12,28,15) - Christopher Owens
14-92(23,26,29,27) - Jerraud Powers
15-94(28,*,*,22) - Ryan Mouton
16-96(14,16,9,21) - Keenan Lewis
17-102(19,13,15,13) - Donald Washington
18-112(26,*,27,*) - Glover Quin
19-119(9,6,6,7) - D.J. Moore
20-131(18,15,25,23) - Greg Toler
21-138(*,*,*,*) - William Middleton
22-143(22,14,31,16) - DeAngelo Smith
23-148(20,21,24,18) - Brandon Hughes
24-157(12,20,8,11) - Victor Harris
25-168(30,29,22,*) - Joe Burnett
26-177(24,23,*,*) - Don Carey
27-179(27,25,17,26) - Morgan Trent
28-183(*,22,*,19) - Cary Harris
29-187(21,*,*,*) - Brandon Underwood
30-188(25,24,26,*) - Brice McCain
31-191(15,17,23,17) - Coye Francies
32-200(*,*,*,*) - DeAndre Wright
33-203(33,*,19,*) - Jason McCourty
34-216(*,28,13,29) - Captain Munnerlyn
35-217(*,*,*,*) - E.J. Biggers
36-220(*,27,32,*) - Ellis Lankster
37-238(*,*,*,*) - Stoney Woodson
38-242(*,*,*,*) -  Nick Schommer
39-227(7,9,11,5) - Mike Mickens

(Misc)
142() - Kevin Huber P
164() - Thomas Morstead P
172() - David Buehler K
198() - Jake Ingram LS
222() - Pat McAfee K
256() - Ryan Succop K

This is a FanPost and does not necessarily reflect the views of Niners Nation's writers or editors. It does reflect the views of this particular fan though, which is as important as the views of Niners Nation's writers or editors.