Light PhD Seminar: Extraordinarily corrupt or statistically commonplace? Reproducibility crises may stem from a lack of understanding of outcome probabilities (A hierarchical bayesian view of experimental science).

Location: Maryam Mirzakhani Seminar Room at BCAM

Title
Extraordinarily corrupt or statistically commonplace? Reproducibility crises may stem from a lack of understanding of outcome probabilities (A hierarchical bayesian view of experimental science).

Abstract
Reports of crisis in reproducibility have abounded in the scientific and popular press, the main culprits allegedly being questionable research practices, lack of incentives for (or simply absence of) rigorous protocols, or outright fraud. Nevertheless, between-experiment variation is just as expected as that within an experiment, but while the latter is built into standard statistical analyses the former is usually completely neglected. In this talk I argue that some instances of failure to reproduce an experimental result attributed to sloppy experimental practices are in fact failures to account for the proper structure of variance, and formalize a hierarchy of distributions that correctly account for the expected phenomena. I also argue that a Bayesian approach allows more general inference that generate posterior distributions for the probabilities of experimental outcomes and identify extreme results. Conversely, this can also be used to integrate data from different sources, allowing complex, nonlinear model-based inference and reduce underdetermination of inverse problems.