Wednesday, April 27, 2011

Risk and Drafting a QB

What kind of football related blog would this be without some kind of post related to the upcoming NFL draft? It was not difficult to know that a post was necessary, finding an angle that has not been done to death by the football media that, outside of some truly exciting court rulings, has had nothing else to talk about recently.

After surveying the landscape, and seeing yet another discussion of how many first round busts there have been (particularly at the QB position) but no QB in current crop of potential first rounders being labeled as the potential bust, I thought I would take up the mantle of uncertainty in the NFL draft.

As a believer in on field performance above all else (pro day work outs, combine performances etc) I dug into the on field performances of all the QBs that received a draft grade of 2 or higher from Sports Illustrated in order to see which player's performance might have the highest probability of being a mirage. The most direct numerical way to look at this is to build the 95% confidence interval around a QB's completion percentage. Using a player's number of attempts with their completion percentage, I calculated a range of values for each QB's "true" completion percentage. The wider the range, the riskier the pick is.

Based on this admittedly crude measure we can see that Cam Newton has the highest level of uncertainty around his college completion percentage. In his career, Cam completed 65.4% of his passes, but due to the small sample of passes attempted, his "true" completion percentage is somewhere between 59.9% and 70.9%. This is a fairly wide range of potential values, which suggest that we likely do not have enough information to properly evaluate Cam at this point. To be clear, I do not mean that statisticians do not have enough information, but that talent evaluators of all types, do not have enough information to determine Cam's true level of performance.

Cam has the highest standard error in this group at +/- 5.5% while the standard error of the next highest QB to receive a draft grade higher than 3 is Ryan Mallet with a standard error of +/- 3.1%. This suggests that the evaluations of players like Mallett and Gabbert are more likely to be accurate that the evaluation of Newton. This is not to suggest that Cam will not be a successful NFL AB, but that NFL teams should be careful to factor in the high level of uncertainty when drafting him relative to the other highly regarded QBs.


Player  School  Grade Compl% High Low Std Err.
Cam Newton  Auburn  3.12 65.4% 70.9% 59.9% 5.5%
Jeff Van Camp  Florida Atlantic  2.14 57.2% 61.4% 53.0% 4.2%
Scott Tolzien  Wisconsin  2.26 68.1% 71.8% 64.4% 3.7%
Ryan Colburn  Fresno State  2.11 62.1% 65.8% 58.4% 3.7%
Greg McElroy  Alabama  2.34 66.3% 69.9% 62.7% 3.6%
Jordan La Secla  San Jose State  2.1 59.1% 62.7% 55.5% 3.6%
Tyrod Taylor  Virginia Tech  2.36 57.2% 60.5% 53.9% 3.3%
Ricky Stanzi  Iowa  2.5 59.8% 63.0% 56.6% 3.2%
Ryan Mallett  Arkansas  3.02 57.8% 60.9% 54.7% 3.1%
Blaine Gabbert  Missouri  3.14 60.9% 64.0% 57.8% 3.1%
Christian Ponder  Florida State  2.83 61.8% 64.9% 58.7% 3.1%
Ben Chappell  Indiana  2.27 61.1% 64.0% 58.2% 2.9%
Jerrod Johnson  Texas A&M  2.33 58.6% 61.5% 55.7% 2.9%
Jake Locker  Washington  2.92 53.9% 56.8% 51.0% 2.9%
Taylor Potts  Texas Tech  2.22 66.3% 69.1% 63.5% 2.8%
Colin Kaepernick  Nevada  2.9 58.2% 60.9% 55.5% 2.7%
TJ Yates  North Carolina  2.38 62.3% 65.0% 59.6% 2.7%
Andy Dalton  TCU  2.88 61.7% 64.3% 59.1% 2.6%
Nathan Enderle  Idaho  2.41 54.6% 57.2% 52.0% 2.6%

Tuesday, April 19, 2011

Accepting an 18 Game Schedule

The NFL owners have made it clear that they want an 18 game regular season and will push the players hard to get that concession. The primary objection to the 18 game schedule from the players point of view is of course the increased risk of debilitating injury which will likely shorten careers and lower the total earning potential of the individual players. While the incidence of injury within a season will most certainly rise with more games, the focus of much of the discussion undoubtedly also turns on the issue of concussions which is not well understood. After much public pressure and increasing evidence on the effect of repeated brain injury on football players, the NFL is taking steps to at least better handle concussions on the field. The question still remains though about how damaging concussions and other more minor repeated brain injuries really are which allows the NFL to be less than fully aggressive in minimizing not only the impact of specific injuries, but on monitoring and understanding the long term effects of these injuries.

If the players are serious about understanding the impact of repeated brain injury, as I suspect as least some of them are, then they should take a lesson from State Attorney General's from across the country when they sued the tobacco companies and insist on disclosure. Disclosure has been a part of many settlements with the tobacco industry in which the tobacco companies have agreed to make their internal documents available to anyone, including researchers in a broad array of fields such as chemistry, biology, marketing, economics, and political science. These documents have allowed these scientists to learn not only about the behavior of the companies, but gain a greater understanding  of the broader health and policy issues related to tobacco.

The deal for the players should actually require a lot less from the NFL than these settlements required of the tobacco companies. Instead of forcing the disclosure of previously secret memos and financial information, the players should request the collection and disclosure of new information. In exchange for the 18 game regular season, players should request the following three items:

  1. Brain scans at the begining and end of the season for every NFL player.
  2. Insertion of a chip into every player's helmet that tracks the impact that the player's brain is exposed to both on a play by play basis and on a cumulative basis.
  3. Disclosure at the end of the season of all the data (anonymized) to any and all interested parties.
This data would very quickly help researchers understand the full impact that repeated brain injury has on the brain throughout the course of an NFL season, as well as allow all interested parties to see the impact of various policies and equipment on reducing this impact. This data would go a long way from moving the debate from its current stage of arguing about how serious an issue it is, to figuring out the best way to protect players from these types of injuries.

Wednesday, April 13, 2011

Blind Side Project Help

Last season I ran a pilot study on valuing offensive linemen. Some of the results are described here. The results are promising, but are clearly lacking in usefulness as only a sample of games of a few teams are covered. In order to move the process forward, I am looking for a few reliable volunteers who would be willing to assist in collecting the data. If you are willing to spend hours watching and re-watching plays to collect the needed data, I want to hear from you. All consistently contributing volunteers will have access to the complete data set.

The data collection requires some basic understanding of offensive line play as well as some training on a specific game to insure high quality and consistent data. If you are interested, please send an email to: quantsports at gmail dot com. In the email, please rate you knowledge of offensive line play and explain your rating. Also, please indicate which team(s) you would want be able to cover (you must have your own video access that can rewind and pause the games you chart) and how many games you would be able to do in a week (games typically take 2 to 3 hours to complete for experienced trackers).

Thank you to all willing volunteers.