This three-day course discusses the evaluation of theory-based hypotheses using p-values, the Bayes factor, and information criteria. Contemporary phenomena will be covered, like publication bias; questionable research practices; the replication crisis; the statistical evaluation of replication studies; and studies in which multiple data sets are used to evaluate the same research question. The course will be non-technical in nature, that is, it is targeted at students and researchers who want to use the approaches presented for the evaluation of their own data.
The evaluation of hypotheses is a core feature of research in the behavioural, social, and biomedical sciences. Over the last decade, the replication crisis has drawn a lot of attention to high rates of false-positive results of hypothesis tests in the literature, caused by problems such as publication bias (selective publication of positive results), ‘questionable research practices’, and statistical misconceptions. This course will give participants a non-technical introduction to the theoretical basis of hypothesis evaluation from different perspectives (frequentist, Bayesian, information theoretic) and teach how to apply hypothesis evaluation appropriately to avoid fooling oneself and others.
The course is targeted at students and researchers who want to improve their understanding of how to evaluate theory-based hypotheses.
The first day of the course will cover classic null-hypothesis significance testing (NHST), common problems with NHST (misconceptions, questionable research practices, publication bias), open-science practices to avoid these problems, and how to draw more informative inferences with equivalence testing.
The second day will focus on hypothesis evaluation using model selection. Model selection provides an alternative to dichotomous decisions that are the default in NHST and allows more nuanced inferences. Two types will be discussed: Bayesian model selection (BMS) and information theoretic model selection, that is, model selection using information criteria. For both types, the focus will be on informative, theory-based hypotheses (as opposed to null hypothesis testing). The methods that will be covered include Bayes factors, posterior model probabilities, the AIC-type criteria called the GORIC and GORICA, and GORIC(A) weights.
The third day of the course will address informative hypothesis evaluation for the evaluation of replication studies, including both direct and conceptual replications (i.e., of multiple studies). Attention will be paid to (among others) hypothesis updating and combining evidence from multiple studies addressing the same research question (using both BMS and the GORICA).
Participants are requested to bring their own laptop computer. Software will be available online.
Among our Methodology and Statistics postgraduate courses, there is another course that deals with Bayes: A gentle introduction to Bayesian Estimation
This course focuses on Bayesian statistics in terms of prediction (focusing on SEM) but does not cover hypothesis testing nor Bayes factors.
The course takes place in the week prior to our course and may be a nice prequel.
Lecturers
Dr. Rebecca Kuiper and Anne Scheel
Target audience
The course will be non-technical in nature, that is, it is targeted at students and researchers who want to use the approaches presented for the evaluation of their own data. The participants can come from a variety of fields, like, for example, sociology, psychology, education, human development, marketing, business, biology, medicine, political science, and communication sciences.
For an overview of all our summer school courses offered by the Department of Methodology and Statistics, please click here.
Aim of the course
After attending the course, you will understand the frequentist, Bayesian, and information-theoretic approaches to hypothesis evaluation. You will obtain the practical skills necessary to evaluate hypotheses using these approaches, using open-science practices to report your research transparently and reduce the risk of error and bias. Moreover, you will learn to evaluate replication studies and to combine the evidence from multiple diverse studies.
Study load
Each day consists of lectures including small hands-on-your-own-laptop exercises and ends with a lab meeting.
Please note that there are no graded activities included in this course. Therefore, we are not able to provide students with a transcript of grades. You will obtain a certificate upon completion of this course.
Costs
The tuition fee for PhD students from the Faculty of Social and Behavioural Sciences from Utrecht University will be funded by the Graduate School of Social and Behavioural Sciences.
The are no scholarships available for this course.
Additional information
We also offer tailormade M&S courses and in-house M&S training. If you want to look the possibilities, please contact us at ms.summerschool@uu.nl
Contact details
Team M&S Summer School | E: ms.summerschool@uu.nl