Making students and teachers smarter by analyzing relationships between student behavior and outcomes in a data-rich online course
Neither students nor teachers are perfect, and even the most well-intentioned in both groups may not know how to address their own shortcomings. In order to help students succeed in a challenging course, they must receive timely, relevant feedback. In order for teachers to refine an existing course, they must have access to detailed information about which parts of the course are most challenging and why those aspects are difficult.
An inquiry-based, active-learning online course, such as Habitable Worlds, designed using a learning platform that collects data about every student interaction and adapts accordingly, provides a great opportunity for students to receive tailored feedback about their content mistakes and diligence, even at scale. It also provides insight for teachers into which exercises are causing problems for even the best students. Therefore, our goals are to leverage interaction data to make targeted course improvements and to mine multiple semesters of student data to identify patterns of student behavior that may predict outcomes.
We have looked for relationships between specific wrong answers, time spent on lessons, individual lesson score, number of attempts, or the date of first attempt and a studentﾒs overall course success. We have also examined how these characteristics change from the beginning to the end of the course as well as whether there are composite factors among these characteristics that better predict course success. In addition, we administered the Classroom Undergraduate Research Experience (CURE) survey to assess attitudinal changes.
Among the observable behaviors, the most reliable predictor of student success or failure is how quickly a student begins each week's work. A substantial portion of the course grade is earned based on completion, so students who fail to complete lessons will not earn those points, nor will they learn the concepts covered in those lessons. We observe highly significant correlations (p < .001) between a student's first lesson start delay and the number of lessons they went on to complete as well as their lesson grades. We are still working to implement an effective intervention to help these students. From the teaching perspective, student responses have led to a number of targeted improvements. The CURE results indicate a shift towards a more accurate understanding of the scientific method (p < .001). Students also left the course with greater belief in their ability to succeed in science courses (p = .005).
The lessons we have learned developing Habitable Worlds will be applicable to other educators and instructional designers seeking to use digital tools to improve science education at scale. In addition, Habitable Worlds is itself being offered at scale, via the Inspark teaching network which is facilitating deployments at a number of two-year colleges
Novel methods of instruction and new courses are typically compared against a 'business as usual' course/pedagogy in order to evaluate their effectiveness. For two reasons, this was challenging for evaluating Habitable Worlds. First, because the subject matter of the course - the scientific basis underlying the search for exoplanets - cuts across astronomy, physics, and geology, there is no true comparison course to evaluate content knowledge gains. Second, we put a large emphasis on data collection and reasoning skills in the course, and improvement on those skills was a key learning outcome. However, we have been unable to find a suitable instrument to measure those skills for hundreds or thousands of students each semester. Ultimately, designing our own instrument for this task has become an aspect of our research.
Interactive technology enabling a virtual exploration of our evolving planet
The emerging landscape of digital Virtual Field Trip (VFT) technology enables geoscience teachers to expose whole classrooms to geologically significant but remote or inaccessible regions, overcoming obstacles of distance, hazards, cost, time, and logistics. Integrating VFTs into cyberlearning environments affords students an infrastructure that enables them to do authentic field science in the classroom.
Our team and collaborators are producing a diverse suite of freely accessible, Immersive VFTs (iVFTs) to teach key concepts in geology, astrobiology, ecology, and anthropology. In addition to the iVFTs themselves, we are also developing a unified series of lesson plans and learning objectives that utilize the iVFTs as a means to teach Historical Geology. Topics and locations are shown at https://vft.asu.edu.
We are applying a technology platform we devised, supported by NASA and the NSF, and now HHMI, that integrates a variety of digital media. This technology utilizes hardware and software that gives developers the tools needed to capture high-resolution spherical content, 360ﾰ panoramic video, gigapixel imagery, and unique viewpoints via unmanned aerial vehicles as they explore remote and physically challenging regions of our planet. In addition, advanced software enables integration of these data into dynamic, immersive, interactive, adaptive, learner-driven virtual field explorations, experienced online via HTML5.
We have already produced 16 iVFTs and we plan to continue to develop new locations. We expect that students who complete lessons using the iVFTs will have an equivalent learning experience to students attending in-person field trips. Moving forward, we plan to investigate what differences do exist between the virtual and the in-person field trip experiences. This research will inform future iVFT creation as well as iVFT-based instructional strategies.
Our iVFT platform will bring dozens of unique geological, biological, and anthropological settings to students and lifelong-learners around the world who would not have had access to these sites through other means. By teaching science through these evocative locations, we have the potential to excite learners who might otherwise not be interested in science at all. These resources are freely available at https://vft.asu.edu. Because the iVFTs are designed to accommodate both guided inquiry and open inquiry, they are broadly applicable. We expect that they will be useful to other undergraduate or Kﾖ12 instructors and to informal educators and STEM interpreters.