Lies, damn lies, and statistics

Mark Twain said it best.  Statistics has become the weapon of choice these days in everything from science to stimulus (plans, that is).  Maybe because it is one of the most versatile of weapons.  You choose the data, you choose the methods of analysis, and you make claims about how doing things one way is "significantly" different than doing things another way. 

Significance here being the cutting edge of the weapon, by way of its double meaning. For statisticians, a (statistically) significant difference can be (relatively) insignificant because no matter how small the difference, the (statistical) significance is an internal measure related to differences within the data and unrelated to its importance on the outside.  Except, of course, that it allows one to claim they have found a significant difference.     

Two unrelated articles came my way this morning. The first is a recent study by MIT neuroscientist Edward Vul, who:

…analyzed 54 prominent studies that used functional magnetic resonance imaging, or fMRI, to identify the brain areas associated with specific emotions — and found that 28 of them used the statistical methods and filters that were most likely to yield data that would confirm the researchers’ hypotheses. According to Vul, this cherrypicking may have been inadvertent; nevertheless, it produced correlations that “exceed what is statistically possible” and are almost certainly false. (from VSL)

Just when your faith in science was lost, along comes a second article, in the NYT sports section, of all places, that restores it with one of the most interesting and (statistically and personally) significant non-findings I've seen in a long time.

It turns out that the average free-throw percentage in college and professional basketball has remained essentially unchanged for the last 50 years (Free Throws).  While every other sports statistic has fallen, this measure of performance has shown itself resistant to shoes, shoe contracts, performance enhancers, the hem length of shorts, ESPN, tattoos, and television contracts:

Basketball in the United States has changed in myriad ways over the decades, from flat-footed set shots to dunks, from crotch-hugging uniforms to baggy knee-length shorts, from the dominance of American players to the recent infusion of international stars.

But one thing has remained remarkably constant: the rate at which players make free throws.

It's nice to know the more things (significantly) change, the more they stay (significantly) the same.