Rigorously Assessing the Anecdotal Evidence of Increased Student Persistence in an Active, Blended, and Collaborative Mechanical Engineering Environment
Retention of engineering undergraduates, in particular during the sophomore and junior years, is a major national concern. Active learning classroom techniques, Blended online and in-class materials, and Collaborative learning environments have each shown some promise of improving engineering student outcomes. However, these promising tools are seldom deployed together and rarely utilized in core disciplinary classes. Our proposal is rigorously studying the Purdue Mechanics Freeform Classroom (Freeform), which has piloted the use of an Active-Blended-Collaborative (ABC) structure and experienced anecdotal success for the last six years. On one key metric of course success, the rate of students who drop, fail, or withdraw from (DFW), the course has experienced near-constant improvements since the ABC structures were introduced. However, this evidence is anecdotal. In this study, one of a number beginning to come out of this recently funded project, the authors utilize rigorous longitudinal methods to determine whether this drop in DFW rates can be directly attributed to increased implementation of ABC features. This rigorous model will help investigators determine the efficacy of the Active, Blended, and Collaborative course and identify productive characteristics of the environment to support instructional decisions.
The projectﾒs broad goals are to better understand (a) the student experience in using ABC
tools and classroom structures, (b) mediating individual characteristics of the faculty, discipline, and institution, and (c)
the way Freeform operates in the context of studentsﾒ and instructorsﾒ other courses.
Using an iterative process of research and implementation, the project team is testing Freeform in its most mature implementation (Dynamics), in other mechanical engineering areas,
in courses at Trine University, a small private post-secondary institution, at Purdue University-Calumet, a regional campus with larger populations of non-traditional and part-time
students, and in other disciplines across Purdue West Lafayette with the Purdue Center for Instructional Excellence. This course has built on the growing body of literature citing active learning (Freeman et al., 2014), blended structures (Bowen & Ithaka, 2012), and collaborative engagement (Jeong & Chi, 2007) as positive influences on college and university science, technology, engineering, and math (STEM) outcomes. For the last six years, ﾓDynamicsﾔ, a core mechanical engineering course at a large public university, has utilized in-class activities, highly-watched problem-solving videos, and a collaborative blog space to realize an ABC environment.
The authors hypothesize that as instructors become accustomed to the ABC environment and increase the level of in-class activity, use of blended resources, and collaboration, the likelihood of DFW in each subsequent year would drop. However, in the same time period, each subsequent class came in with higher levels of performance on proxy measures for prior knowledge. We therefore build a logistic regression model to predict individual-level DFW and determine whether the anecdotal drops in DFW that we observe can be attributed to the expansion of the ABC environment. More specifically, we predict likelihood of DFW based on studentsﾒ prior knowledge (grade in the preceding course, SAT math score), key demographics (gender, race/ethnicity), the semester and year they took Dynamics, their instructor, their year in school, and their major. We test for year fixed effects to determine whether odds ratios for DFW consistently and significantly decrease over time. We also test for instructor effects, in particular for differences between the instructors who were involved in the design and development of the ABC environment and more independent instructors who only partially implemented the ABC course components.
We anticipate results that will provide more rigorous, less biased, and efficient estimates for the individual- and class-level components that explain variance in DFW rates. This would be followed by the estimation of a multinomial logistic model predicting grade and a multilevel linear regression model predicting final exam score and percentile in the course.
The initial results from this first step in our long-term project would provide immediate implications for the next phase of our work, as we assess the next on-term implementation of the course in 2016. Our findings would also have long-term significance for other classes in mechanical engineering and related disciplines and for classes at other institutions that are considering implementing a comprehensive ABC learning environment. We are exploring Freeform at Purdue West Lafayette and two other diverse campuses in Indiana�Purdue Calumet and Trine University, which serve underrepresented populations that are disproportionately affected by stopout. Our study is generating useful outcomes for other interested program adopters, including online course templates and video creation strategies, student study guides, and faculty orientation workshops, and we are disseminating these to interested practitioners working in mechanical engineering and other STEM disciplines.
We have started gathering off-term data on our main campus and collaborator campus, so we are ahead of schedule, but data coordination and combining datasets is a unique challenge. To deal with this, we have brought on an experienced team member from our Office of Institutional Research.
None yet (began Sept 2015)