Measuring Instructional Practices and Climate for Improvement: Two New Instruments

Project No.
1256505
PI Name
Andrea Beach
Institution
Western Michigan University


WIDER

Abstract 1

Measuring Instructional Practices and Climate for Improvement: Two New Instruments

Presentation Type
Paper
Team
Andrea L. Beach Emily M. Walter Charles Henderson


Need

Valid and reliable measures of baseline organizational climate and postsecondary instructional practices are essential to plan better change initiatives, but few such instruments exist that can be used by STEM researchers and change agents.

This project offers two newly developed and validated instruments that are built on strong conceptual work, easy to administer and score, and freely available.

Goals

The WIDER project team developed and validated two new instruments to measure instructional practices in higher education settings and organizational climate for instructional improvement.

Research Tool 1 - Postsecondary Instructional Practices Survey (PIPS)

Research Tool 2 ヨ Survey of Climate for Instructional Improvement (SCII)

Approach

For the PIPS, there is no single or definitive conceptual model of instructional practice despite excellent literature reviews describing research on instructional practices (e.g., Pascarella & Terenezini, 1991; 2005). Without a model from which to develop instructional practice items, we shaped the dimensions of our instrument by finding themes among (a) developed instruments, (b) teaching observation protocols and (c) patterns in research on instructional practice.

An initial set of 153 items was revised by removing redundant items, items that did not refer to actual teaching practices, and lists of generalized practices
New item development. All items are designed for respondents to describe teaching the largest enrollment, lowest level course they have taught in the last two years. The resulting instrument has 24 items measuring four components: Instructor-student interactions, student-content interactions, student-student interactions, and assessment of learning.

The SCII was developed using Gappa, Austin and Trice's (2007) elements of faculty work as well as literature on workplace ムclimate for changeメ (Bouckenooghe, Devos, & Van den Broeck, 2009), academic work and workplaces (Massy, Wilger, & Colbeck, 1994); departmental teaching climate (Beach, 2002; Knorek, 2012), and leadership for teaching (Ramsden, Prosser, Trigwell, & Martin, 2007).

Items refer to group rather than individual perceptions
e.g., モthe instructors in my department thinkヤ rather than モI thinkヤ. The 52 item instrument addresses resources, rewards, leadership, academic freedom and autonomy, professional growth, collegiality, and collective attitudes about students and teaching.

The instruments were pilot tested at three large universities in multiple departments and analyzed for construct and instrument validity and reliability.

Outcomes

Testing and analysis of the two instruments indicates that the PIPs is valid and reliable and that the SCII can be shortened to address key constructs without non-performing items. The SCII has been revised and both instruments are being further tested. The next steps are to test the validity of using climate for instructional improvement to predict instructional practices.

Broader Impacts

The PIPs is non-evaluative, designed for respondents to score practices as descriptive of their teaching without judging the quality of these practices. Furthermore, PIPS is concise (20 items), non-proprietary, and designed with clear and consistent item scales. That makes it unique among self-report instruments that address teaching.

The SCII is unlike any other instrument available. SCII is built in alignment with the essential elements of faculty work described by Gappa et al. (2007). Our results not only provide empirical support for the factors described by Gappa et al. (2007), but it also elicit constructs that could serve as levers for change in planned initiatives.

Unexpected Challenges

None.

Citations

Walter, E. M., Beach, A. L., Henderson, C., & Williams, C. T. (2015). Measuring
postsecondary teaching practices and departmental climate: The
development of two new surveys. In G. C. Weaver, W. D. Burgess, A. L.
Childress, & L. Slakey (Eds.), Transforming institutions: Undergraduate
STEM in the 21st Century. Purdue, IN: Purdue University Press.

Williams, C. T., Walter, E. M., Henderson, C., & Beach, A. (In Press). Describing undergraduate STEM teaching practices: A comparison of instructor self-report instruments. International Journal of STEM Education.

Walter, E. M., Henderson, C., Beach, A., & Williams, C. T. (In Review). The
development and validation of the Postsecondary Instructional Practices
Survey (PIPS). CBE - Life Sciences Education.



Project Page