Assessing Faculty Professional Development in STEM Higher Education: Sustainability of Outcomes
Faculty professional development (PD) programs are critical components of efforts to improve teaching and learning in the STEM disciplines, but reliable evidence of sustained impacts of these programs is lacking. By definition, if a professional development program is effective then desired outcomes are sustained by participants after program completion. Assessment of the impact of PD programs is necessary to identify the types of PD activities that are most closely associated with change in participantsﾒ teaching. Knowledge of effective activities and approaches will allow us to improve the efficiency and impact of PD programs.
Expected outcomes of the FIRST project were that participants would adopt learner-centered beliefs about teaching, implement learner-centered courses, and use assessments that incorporated questions requiring higher-order thinking.
We tested the effectiveness of Faculty Institutes for Reforming Science Teaching (FIRST IV) a professional development program for postdoctoral scholars, by conducting a study of program alumni upon completion of the program. We determined whether the perceptions of teaching and teaching practice of faculty who participated in FIRST differed from those of faculty in the same department who did not participate. We designed our research to test the null hypotheses that there were no differences in the perceptions about teaching, teaching practices used, and levels of cognition targeted by course assessments, between FIRST participants and non-FIRST faculty.
We used a paired design in which we matched a FIRST alumnus employed in a tenure-track position with a non-FIRST faculty member at the same institution (N=21 pairs). The members of a pair taught courses that were of similar size and level. To determine whether teaching practices of FIRST participants were more learner-centered compared with those of non-FIRST faculty, we compared faculty perceptions of their teaching strategies, perceptions of environmental factors that influence teaching, and actual teaching practice.
Non-FIRST and FIRST faculty reported similar perceptions about their teaching strategies and teaching environment. FIRST faculty reported using active learning and interactive engagement in lecture courses more frequently compared with non-FIRST faculty. Ratings from external reviewers also documented that FIRST faculty taught class sessions that were learner-centered, contrasting with the teacher-centered class sessions of most non-FIRST faculty. Despite marked differences in teaching practice, FIRST and non-FIRST participants used assessments that targeted lower-level cognitive skills.
Our study demonstrated the effectiveness of the FIRST program and the empirical utility of comparison groups, where groups are well-matched and controlled for contextual variables (e.g. departments), for evaluating the effectiveness of professional development on subsequent beliefs and teaching practices. Key questions investigators must address about PD programs are how to scale projects nationally and how to sustain them over time. The success of a program is not measured along a single axis, and as our results demonstrate, some aspects of teaching are more difficult to change than others. Results from FIRST can be used formatively to inform the design of future professional development in STEM disciplines, particularly with regards to addressing instructorsﾒ beliefs about how students learn and perceptions about instructional practices and designing assessments.
Accruing funding for a longitudinal study of the participants re: sustaining teaching practice in different departments. 2013 proposal not funded. We will resubmit an Institutional/Transformation proposal in Jan. FIRST IV includes a large cohort of early career faculty and presents the basis for a singular, even obligatory, opportunity to study the sustained use of intensive training in learner-centered teaching methods.
Ebert-May D, Derting TL, Henkel TP, Middlemis Maher J, Momsen JL, Arnold B, Passmore HA. 2015. Breaking the cycle: Future faculty begin teaching with learner-centered strategies after professional development. CBE Life Sci Educ. 14:1-12; doi: 10.1187/cbe.14-12-0222.
Ebert-May D, Derting T, Hodder J. 2014. Professional development of faculty: how do we know it is effective? In M. A. McDaniel, R. F. Frey, S. M. Fitzpatrick, & H. L. Roediger (Eds.), Integrating cognitive science with innovative teaching in STEM disciplines. St. Louis: Washington University in St. Louis Libraries. doi: 10.7936/K7G44N61.
Ebert-May D, Derting TL, Hodder J, Momsen JL, Long TM, Jardeleza SE. 2011. What we say is not what we do: Effective evaluation of faculty development programs. BioScience 6(17): 550-558.
Derting, TL, D Ebert-May, TP Henkel, J Middlemis Maher, B Arnold, HA Passmore. Assessing faculty professional development in STEM higher education: Sustainability of outcomes. Submitted: Science Advances.
Momsen, JL, Wyse SA, Long TM and Ebert-May D. 2010. Just the facts? Undergraduate biology courses focus on low-level cognitive skills. CBE Life Sci Educ. 9(4): 435-440.