The ChemCollective: Learning about learning from student interactions with online resources

Project No.
PI Name
David Yaron
Carnegie Mellon University

Abstract 1

The ChemCollective: Learning about learning from student interactions with online resources

Presentation Type
Dave Yaron, Carnegie Mellon University Michael Karabinos, Carnegie Mellon University Colin Ashe, Indiana University of Pennsylvania Gaea Leinhardt, University of Pittsburgh James Greeno, University of Pittsburgh Jodi Davenport, Wested Anna Rafferty, Carleton College Kobi Gal, Ben-Gurion University


Introductory chemistry is gateway course for many STEM careers and resources are needed to both help students succeed in this course and to ensure that students leave the course with the conceptual understanding required for success in follow-on courses.


The ChemCollective is a digital library hosting virtual labs, tutorials and particulate level simulations for introductory chemistry courses at the college and high school level. Our goal is to provide resources that both enhance student learning and gather data that may be used to iteratively improve the resources.


The resources couple domain content with science practices by engaging students in the design and interpretation of chemical experiments, connecting molecular-level interactions to observable properties, and relating chemical phenomena to real-world goals. Design of the materials is done through collaborations of chemists, cognitive psychologists, and human computer interaction researchers. We have developed a technique we call モexpert learnersヤ to help in the early design phases. This involves teaching the material to cognitive scientists with little domain knowledge. These experts are able to simultaneously learn the material and discover tacit knowledge that is essential to understanding, but held so innately by experts that they fail to make it explicit to students. Once identified, think aloud data from student problem solving is used to refine the nature of this tacit knowledge. Instruction on chemical equilibrium developed through this approach more than doubled student performance on difficult problems in chemical equilibrium. Once resources are released, data is gathered from both classroom observations and log files of interactions with online content. This data helps reveal how students approach the tasks and provides insights into how we can best support student learning through engagement in those tasks.


Examples of what we have learned about learning include: activities in which students must design experiments to generate data breaks the shallow problem solving strategy of means-end analysis and so challenges them to better understand chemical concepts; how to develop algorithms that can detect, with greater than 90% accuracy compared to human coders, the problem solving strategies a student is using when solving an ill-defined problem; and the ways in which a computer agent can, by overseeing and intervening in a chat room, guide student discussions in a way that most enhances learning outcomes.

Broader Impacts

The ChemCollective provides free resources that are both incorporated into introductory chemistry courses as required homework activities, and provide students with anytime access to tutorials on challenging topics. Each year, we have more than 550,000 unique visitors, over 300,000 online experiments are performed, and over 50,000 use chemistry concepts to solve the Mixed Reception murder mystery.

Unexpected Challenges

Shifts in web technology prompted us to divert some effort away from development of new materials and towards porting of activities written in Java and Flash to HTML5.


Yaメakov (Kobi) Gal, Oriel Uzan, Robert Belford, Michael Karabinos and David Yaron, モMaking Sense of Studentsメ Actions in an Open-Ended Virtual Laboratory Environmentヤ, Journal of Chemical Education, Vol. 92, No. 4, pp. 610ヨ616. 2015.

Jodi L. Davenport, Gaea Leinhardt, James Greeno, Kennith Koedinger, David Klahr, Michael Karabinos, and D. Yaron, モEvidence-Based Approaches to Improving Chemical Equilibrium Instructionヤ, Journal of Chemical Education, Vol. 91, No. 10, pp. 1517-1525. 2014.

Ofra Amir, Kobi Gal, David Yaron, Michael Karabinos and Robert Belford, モPlan Recognition and Visualization in Exploratory Learning Environmentsヤ. In Educational Data Mining. Springer International Publishing. pp. 289-327, 2014.

Bruce M. McLaren, Tamara van Gog, Craig Ganoe, Michael Karabinos and David Yaron. モExploring the Assistance Dilemma: Comparing Instructional Support in Examples and Problemsヤ. In Intelligent Tutoring Systems, Springer International Publishing. pp 354-361, 2014.

David Adamson, Colin Ashe, Hyeju Jang, David Yaron, and Carolyn Ros�, �Intensification of group knowledge exchange with academically productive talk agents�. In Proceedings of the 10th International Conference on Computer Supported Collaborative Learning, Madison, Wisconsin. July, 2013.

Colin Ashe, and David Yaron, モDesigning analogy-based simulations to teach abstractionsヤ, In J.P. Suits and M. J. Sanger (Eds.), ACS Symposium Series: Pedagogic Roles of Animations and Simulations in Chemistry Courses, American Chemical Society, pp. 367-388, 2013.

Ryan S.J.d. Baker, Gregory R. Moore, Angela Z. Wagner, Jessica Kalka, Aatish Salvi, Michael Karabinos, Colin A. Ashe and David Yaron, 'The Dynamics Between Student Affect and Behavior Occuring Outside of Educational Software.', in S. D'Mello, A. Graesser, B. Schuller, J Martin (Eds.), Proceedings of the 4th bi-annual International Conference on Affective Computing and Intelligent Interaction. Springer Berlin Heidelberg: Berlin, Heidelberg, Vol. 6974, pp. 14-24. 2011.

David Yaron, Michael Karabinos, Gaea Leinhardt, James Greeno, Jodi Davenport, Bruce McLaren, Laura Bartolo, John Portman, Colin Ashe, W. Craig Carter and Donald Sadoway., 'Instruction Online: Core Components for Reuse', in R. Belford, J. W. Moore, and H. E. Pence (Eds.), Enhancing Learning with Online Resources, Social Networking, and Digital Libraries Washington, DC, pp. 235-262. 2010.

David Yaron, Michael Karabinos, Donovan Lange, James G. Greeno, and Gaea Leinhardt. 'The ChemCollective: Virtual labs and online activities for introductory chemistry courses', Science, Vol. 328, No. 5978. April, 2010.

Project Page