Innovative Designs
AIR specializes in innovative research and evaluation designs that can drive program improvement. Our approach emphasizes the importance of iterative design and development, using rapid-cycle evaluations and other efficient experimental designs, such as A-B testing, factorial experiments, and sequential multiple assignment randomized trials (SMARTs). SMART designs in particular hold great potential for supporting the development of more responsive, more effective interventions in the fields of education, health, workforce development, and more.
Sequential Multiple Assignment Randomized Trials (SMARTs)
SMARTs provide information for development of evidence-based adaptive interventionsWhat is an Adaptive Intervention?
Adaptive interventions are sequences of tailored decision rules that specify whether, how, or when to change the intervention based on how participants respond along the way. Adaptive interventions are personalized in the sense that the intervention for any given person will change (or not) based on their progression, determined at specified time points, by the person’s response to the intervention so far.. In a SMART design, individuals are randomly assigned to one or more treatment options at critical decision points. At these critical points, individuals’ response is assessed. Depending on whether they are considered “responsive” or not, individuals are then further randomly assigned to altered interventions (e.g. an intensified version of the original treatment, or an entirely different treatment).
A SMART design can identify which interventions should be considered “front-line” treatment, and which might be an effective “back-up” strategy for those who are not initially responsive. SMART can also help identify which sequence of interventions works best for a different types of individuals.
SMART designs are powerful tools for evaluating adaptive interventions. Many interventions in education, prevention, and health are adaptive in nature already, but have yet to be studied using SMARTs.
AIR is currently conducting a SMART design evaluation for the U.S. Department of Education’s Institute of Education Sciences. The study will evaluate the impact of an adaptive text messaging intervention designed to reduce chronic absenteeism among elementary school students.
AIR is excited to be a national leader in using SMARTs to help create more robust, responsive interventions in education and other fields.
To learn more about SMART designs, visit The Methodology Center at Penn State https://www.methodology.psu.edu/research-and-rigor/.
Factorial Experiments
Most interventions we develop or study have multiple features and components. Factorial designs can be very useful in R&D because they allow us to isolate effects of different components of an intervention, as well as the effects of the components in combination.
While factorial designs are well known in some fields of research (for example, psychology and cognitive science), they are underutilized in other fields, including education.