I’m a fan of John Ioannidis and his work. He’s done a lot to raise attention to the use and abuse of frequentist statistics in, well, lots of the sciences.

In a recent article, he made the argument that “Teams which publish scientific literature need a ‘licence to analyse’ and this licence should be kept active through continuing methodological education.” I threw the question out to my PhD students, and there was a mixed reaction. One argument was the pace at which statistical theory advances and the difficulty for applied researchers to keep up. I’m sympathetic with that perspective, because I still feel like I’m playing catch up with my own methods knowledge (still have a long way to go!).

John also made this observation though “Journals also lack the expertise required to understand and properly review statistics. Until a couple of years ago, even Nature and Science did not have trained statisticians routinely reviewing papers, which continues to be true of most biomedical and biology journals.” That got me thinking about my other role as a Field Editor for JBV about the importance of reviewer education of statistical inference.

I’m thinking that flipping the requirement around might be a better way to go. What about a required online training class for members of a journal editorial board? For ad hoc reviewers, this course might be optional, but highly encouraged. The course wouldn’t take long, but would highlight the correct interpretation of the p-value, the importance of standard errors in consistency of inference, etc. It would also dispel some popular myths that still occur in the management literature, with the intention of improving the data science capabilities of the field; to publish, you need to ensure you are meeting the standard that reviewers and editors are trained to look for.

I’m thinking this would be a net value add, and not difficult to implement…