Build, Understand, & Tune Interventions that Cumulate to Real Impact

Project No.
PI Name
Timothy Nokes-Malach
University of Pittsburgh

Abstract 1

Build, Understand, & Tune Interventions that Cumulate to Real Impact

Presentation Type
Kevin Binning, Pitt Joe Grabowski. Pitt Nancy Kaufmann, Pitt Chandralekha Singh, Pitt Chris Schunn, Pitt Ben Rottman, Pitt Elizabeth Votruba-Drzal, Pitt


(a) Learning Sciences have produced instructional, cognitive-skill, and social/motivational interventions that can produce large improvements in learning within critical STEM courses, but much remains to be uncovered regarding:
1) What interventions produce meaningful change on important longer-term outcomes?
2) Whether different instructional, cognitive skill, and social/motivational interventions combine negatively or synergistically within and across courses.
3) For which kinds of situations/students are different interventions most effective?
This critical knowledge has remained elusive because of complex technical, statistical, and policy reasons.
(b) The current work aims to benefit undergraduate students in STEM disciplines especially for at-risk students with a focus to improve student learning and long-term performance outcomes.


Our goal is to understand the when, why, and what types of interventions are effective and for whom, how long, and under what circumstances are those most effective.


We bring together a highly interdisciplinary team of 1) discipline based research scientists; 2) learning scientists with expertise in social/motivational, cognitive skills, and active STEM learning techniques; and 3) a learning scientist with expertise in large-scale longitudinal data analysis with cutting edge statistical techniques.
We focus on large introductory courses in
physics, chemistry, biology, and psychology. We have selected a range of promising interventions including instructional, cognitive skills, and social/motivational. Instructional interventions are alternative methods for delivering instruction in the sciences, and are designed to produce more robust core STEM knowledge (e.g., flipped instruction, peer assessment). Cognitive skills interventions focus on providing students with new general cognitive skills that guide how they reason and learn in any STEM course (e.g., metacognitive skills and causal diagraming). Social-motivational interventions are designed to improve academic performance by addressing social psychological factors that inhibit academic engagement and contribute to underperformance in STEM fields (e.g., values affirmation and growth mindset).
All of the courses have at least one current section that is experimenting with one of the proposed interventions, either within the lecture or the lab/recitation. Because many first year students are enrolled in two or more of these classes in one semester, some students will be enrolled in various combinations of interventions. In addition, we will collect data on malleable factors as well as proximal and ultimate success factors. We are working with the University administration to obtain longitudinal data collected at the University on student outcomes (later course grades and performance, graduation).


The project is just getting started so we do not have results yet. But should have some initial assessments in the Spring on the current educational interventions.

Broader Impacts

This project will produce interventions that reliably effect important long-term outcomes; these interventions will be distributed to other departments within the University of Pittsburgh. Further, the network of institutions connected via the dB-SERC and its sister organization at the University of Pittsburgh, the Engineering Education Research Center, will play an important role in distributing findings to other universities (e.g., via the NSF-funded CIRTL network,, and the Reinvention Center,, of which the University of Pittsburgh is a core partner). The findings from this project will be based in very strong science, providing a particularly powerful message to support distribution.

Unexpected Challenges

The grant has just started so we have not yet encountered unexpected challenges.


Not yet.