A number of AIR projects demonstrate our flexibility and creativity in designing tailored studies that answer critical questions related to program development, efficacy, and improvement.
In addition, we are conducting deep investigations of the types of measures that might be most useful for measuring incremental change in rapid-cycle studies. See examples highlighted below.
AIR is examining this question by conducting a systematic review and meta-analysis of randomized experiments of interventions directed at instructional practice. The goal of this project is to understand how much teachers’ instructional practice changes in randomized intervention studies. A second goal is to understand how changes in teacher practice relate to characteristics of the interventions, characteristics of instructional practice observation rubrics, and characteristics of the teacher and student samples. This work will help us better understand the ways to target and study teacher practice in rapid-cycle studies of teaching interventions, such as coaching.
AIR is using nine years of longitudinal data from a large urban school district to study this question and to provide guidance to researchers who are designing intervention studies that target school climate outcomes. Our focus is on four domains of school climate: safety, support, challenge, and the social and emotional learning environment of the school. This study will help us better understand the benchmarks for reasonable and remarkable changes in domains of school climate and the conditions under which those practices are most likely to change. The study will also help to inform the design of experiments targeting school climate outcomes.
Unobtrusive measures of learning and “noncognitive” factors are of great interest to developers and researchers in need of routine information about how programs are operating. Are the programs engaging? Are they helping students learn? Particularly when students are working in online learning environments, data “exhaust” can be mined to generate useful measures of engagement, motivation, persistence, and learning. The fields of educational data mining and learning analytics are growing, and data scientists and content experts are collaborating to apply these methods for use in rapid-cycle studies.
For more information, check out an AIR working document on measuring student “noncognitive” factors in rapid-cycle studies: