Ask an Expert: Estimation Statistics and Statistical Power

Join experts Robert Calin-Jageman and Katherine Button as they discuss their work in estimation statistics and statistical power. Watch the short videos introducing this work below, then watch the webinar as they, along with moderator, eNeuro Editor-in-Chief, Christophe Bernard, answer questions about how these statistical practices apply to your own work.

This Ask an Expert event is part of SfN's Foundations of Rigorous Neuroscience Research (FRN) program, which aims to increase awareness of the sociological and cultural factors that influence rigorous research practices. Share topic ideas or your own experience with cultural barriers to rigor in an anonymous survey.

Estimation for Neuroscience

The estimation approach to inference emphasizes reporting effect sizes with expressions of uncertainty (interval estimates). This video explains the estimation approach and how it can help you avoid common analysis errors when interpreting neuroscience data.

Why Statistical Power Matters

Statistical power is fundamental to the design and conduct of neuroscience research. This video explains the importance of considering power to avoid falling into the trap of making false inferences.

Ask an Expert: Estimation Statistics and Statistical Power

For more answers from the panelists and to continue the discussion go to the webinar's Neuronline Community thread.

Related Resources

Journal Articles:

• Estimation for Better Inference in Neuroscience — Why estimation?

Robert J. Calin-Jageman and Geoff Cumming. eNeuro 1 August 2019, 6 (4) ENEURO.0205-19.2019; DOI: 10.1523/ENEURO.0205-19.2019

• Equivalence Testing for Psychological Research: A Tutorial — Explains the important approach of equivalence testing and checking a confidence interval (CI) to see if all values are negligible.

Daniël Lakens, Anne M. Scheel, and Peder M. Isager. Advances in Methods and Practices in Psychological Science 1 June 2018, 1 (2), 259-269; DOI: https://doi.org/10.1177%2F2515245918770963

• Erroneous Analyses of Interactions in Neuroscience: A Problem of Significance — Addresses interaction errors in neuroscience and documents the pervasiveness of invalid interaction analyses in neuroscience.

Nieuwenhuis, S., Forstmann, B. & Wagenmakers, E. Nat Neurosci 14, 1105–1107 (2011). https://doi.org/10.1038/nn.2886

• The Burden of the "False-Negatives" in Clinical Development: Analyses of Current and Alternative Scenarios and Corrective Measures

Burt, T., Thomas, H., Button, K. S., Noveck, R. J., Munafò M. R. (2017). Clinical Translational Science. 10 (6): 470-479.

• Statistical Rigor and the Perils of Chance

Katherine S. Button. eNeuro 14 July 2016, 3 (4) ENEURO.0030-16.2016; DOI: 10.1523/ENEURO.0030-16.2016.

• Power failure: Why small sample size undermines the reliability of neuroscience

Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B., Flint, J., Robinson, E. S. J., Munafò, M. R. (2013). Nature Reviews Neuroscience. 14 (5): 365-76.

Textbooks:

• Introduction to the New Statistics: Estimation, Open Science, and Beyond by Geoff Cumming and Calin-Jageman
• Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan by John Kruschke

Software:

• ESCI (Exploratory Software for Confidence Intervals) — Free Excel tools from the Introduction to the New Statistics online community and blog.
• jamovi and JASP — Open-source projects providing software that supports Bayesian and frequentist estimation.
• ESCI for R package on GitHub — Effect sizes and confidence intervals for R and jamovi, by Robert J. Calin-Jageman.

Additional Resources:

• Sample Size Planning for Neuroscientists — Guidance and approaches to sample size planning, on the Open Science Framework (OSF)
• @TheNewStats — A Twitter feed on resources and applications of estimation
• Experimental Design for Laboratory Biologists — A practical guide showing biologists how to design reproducible experiments that have low bias, high precision, and widely applicable results.
• COS Reproducible Research Training — Slide decks, code, examples, and other materials on confidence intervals and power, on the Open Science Framework (OSF).

Speakers

Bob Calin-Jageman, PhD

Bob Calin-Jageman is a professor of psychology and the neuroscience program director at Dominican University just outside of Chicago, IL. Calin-Jageman studies the neural mechanisms of forgetting in an undergraduate-focused lab. He is also is active in promoting Open Science and Estimation Statistics. He received his undergraduate degree in philosophy from Albion College, his PhD in physiological psychology from Wayne State University, and completed postdoctoral training in neurobiology with Paul Katz at Georgia State University.

Katherine Button, PhD

Katherine Button is an associate professor at the University of Bath, UK. Her main research interests lie in mechanisms underlying anxiety and depression and their treatment, but she has a side interest in research rigour and reproducibility. She received her undergraduate degree in neuroscience from Cambridge University and her PhD in Psychiatry from the University of Bristol, where she also held two post-doctoral fellowships before moving to her current role at the University of Bath.

Christophe Bernard, PhD

Christophe Bernard, PhD, is the PhysioNet team leader in at the Institute of Neuroscience Systems at the French Institute of Health and Medical Research (INSERM). His research focuses on understanding how physiological and pathological behaviors emerge from the organization and reorganization of the underlying neuronal architecture, with an emphasis on epilepsy. He currently serves as editor-in-chief of eNeuro.