Replicability, Generalizability, and the ManyClasses Approach to Open Science – June 17, 2025


Event Date & Time

  • ONLINE
    June 17, 2025
    10:00 am - 11:00 am

Event Description

Online

As in many fields, applied research in education is facing serious questions about the replicability of its findings — and further, some ponder whether findings should be expected to replicate in the first place. In education, a particularly vexing challenge is the variability of learners and learning settings, and it is a legitimate question whether any one finding might be expected to generalize across classes. To answer this question, the obvious solution is to conduct research that spans and incorporates this variability. Rather than embedding an experiment in one class, one discipline, or one domain, researchers should distribute an experiment across many such situations, customizing treatments and outcome measures to authentic local contexts, so that they are representative of a range of materials and implementations.

This talk will present ManyClasses, a research model that achieves this need, and it will also describe how the principles of open science are fundamental to the ManyClasses approach. The presenter’s work, which aims to test the generalizability of practical interventions, relies heavily on transparency both in our methods and in our findings.

Speaker Bio:

Dr. Ben Motz is Assistant Professor at Indiana University’s Department of Psychological and Brain Sciences. He received his BS from Indiana University, MS from University of California San Diego, and his PhD at Indiana University, all in Cognitive Science. His research is at the intersection of cognitive psychology, technology, and education, characterized by large-scale experiments and analyses on students from real education settings that test theoretical predictions from the psychological science of learning. He also develops research infrastructure for this work: he currently runs Terracotta (https://terracotta.education), a platform that plugs into the Canvas LMS, and that makes it possible to run controlled experiments in Canvas course sites, and that aims to facilitate open practices by exporting deidentified data in a standard format.

Facilitator:

  • Ben Motz, Assistant Professor, Department of Psychological and Brain Sciences, Indiana University

About POSE

This session is being offered as part of the Program for Open Scholarship and Education (POSE) but it is open and everyone is welcome. POSE is an open, three-month flexible program aimed at helping educators gain foundational knowledge of open scholarship, open education, and open research.  Register for POSE here.

Browse the upcoming 2025 Program for Open Scholarship and Education (POSE) events:

Venue: