Abstract:
For decades we have been teaching students Null Hypothesis Significance Tests (NHST) statistics to analyze their data. These tools have yet been criticized since their inception (Meehl 1967, Philosophy of Science 34), because they are only appropriate for taking decisions related to a Null hypothesis (like in manufacturing), not for making inferences about behavioral and neuronal processes (Kline, APA Books 2004). But the times seem ripe for a radical change. In neuroscience, Ioannidis and colleagues revealed that most published studies were severely underpowered (Button et al. 2013, Nature Reviews Neuroscience 14) and, therefore, basically useless. In Psychology, a major endeavor to replicate published studies revealed than most of the effects were severely inflated and not reproducible (Open Science Collaboration 2015, Science). The leading journal Psychological Science now embraces the "New Statistics" defended by Cumming (2013, Psychological Science 25), and encourages our community to publish Confidence Intervals (CI) of effect sizes instead of p-values. Such a small change of practice may foster revolutionary changes in the sociology of our science. At p = 0.05 the 95% CI of the effect size includes zero. The mathematics is the same but the story based upon one or the other presentation differs a lot. In many studies, CIs are often "embarrassingly large" (Cohen 1994, American Psychologist 49) and, if shown, would have prevented publication. One can easily find such major, "historical" (and now textbook) ill-founded publications. Precision is what we should strive for, not ill-named statistical "significance". Precise inference or estimation may often require much more data, and therefore, fewer publications. On the other hand, insisting on precision erases the difference between formerly called "null" and "significant" results, and should help fighting against publication biases, file-drawers problems and even fraud. In 1998, Robert Matthews had written in the Sunday Telegraph: <> and put an end to "our generations-long obsession with p values and the statistical buffoonery" (Lambdin, 2012). Shifting from p-values to CIs is however challenging for complex data like EEG and MRI. I shall present suggestions for MRI results.


Venue:
School of Cognitive Sciences, Institute for Research in Fundamental Sciences (IPM),
Opposite the ARAJ, Artesh Highway, Tehran, Iran