Engagement in Action - Multimodal Learning in Collaborative Classrooms
OER 19/19 TCL - Uncovering the Process and Outcome of Computer-Supported-Collaborative-Learning (CSCL) using Multimodal Learning Analytics
“Our work shows that students not only experience a wide range of epistemic emotions during collaborative discourse, but such emotional behavior appears more heightened when they engage in sensemaking and negotiation of their ideas”
Excerpt from Teo, C. L., Ong, A., & Lee, V. Y. A. (2022, June). Exploring students’ epistemic emotions in knowledge building using multimodal data. In Weinberger, A., Chen, W., Hernandez-Leo, D., and Chen, B. (Eds.), Proceedings of the 15th Computer-Supported Collaborative Learning (CSCL) 2022 (pp. 266-273). Hiroshima, Japan: International Society of the Learning Sciences. https://2022.isls.org/proceedings/
Project Team
PI: Dr Teo Chew Lee, CRPP, NIE
Co-PI: Dr Wong Lung Hsiang, CRPP, NIE
Co-PI: Dr Elizabeth Koh Ruilin, CRPP, NIE
Co-PI: Professor Looi Chee Kit, LSA, NIE
Collaborators:
Associate Professor Tan Seng Chee, LSA, NIE
Dr Esther Tan, Technical University of Delft
Dr Shien Chue, NTU
Professor Marcus Specht, Technical University of Delft
Professor Marlene Scardamalia, University of Toronto
Project Description
Multimodality involves utilising various communication channels to acquire and convey information. When students participate in collaborative discussions, they exhibit multimodal behaviors to demonstrate their knowledge and engage in discourse. In this study, we aimed to develop a learning analytics system that analyses multiple data streams from different sources, providing deeper insights into student engagement in collaborative discourse within the context of Knowledge Building (KB). In KB lessons, teachers design activities that encourage students to explore significant problems, fostering discussions centered on generative and creative ideas rather than adhering to traditional approaches that focus solely on finding a single correct answer. We collected data on student interactions in KB lessons from a primary school. We completed a total of 4 data collection cycles from 2020 to 2023 and captured a total of 511 online student notes, about 10 hours of group collaboration video and audio data as well as student self-reports on their emotional level during group collaboration. Based on analyses of students’ talk and online contributions and their body movement and reported emotions, we found consistent patterns of productive collaborative discourse characterized by more sophisticated student responses (such as questions and explanations with elaborations) along with active physical movement and a wide range of emotional engagement. We produced a prototype of a Multimodal Learning Analytics (MMLA) for KB that offered teachers a way to holistically understand student engagement during various KB processes with feedback to enable teachers to provide necessary support for student learning.
Project Implications
This Multimodal Learning Analytics (MMLA) project has developed a comprehensive approach to collecting, analysing, and interpreting data from various modalities, enhancing our understanding of complex collaborative learning activities. Key implications include:
- Teachers can use Multimodal Learning Analytics (MMLA) to gain insights into less observable students’ communication and collaboration behaviors such as speech patterns, physical movement and emotions.
- Teachers can foster a more positive classroom culture for collaborative learning by using MMLA feedback to help students reflect on their interactions and engagement.
Resources
NIE Research Brief:
Selected Articles:
- Teo, C. L., Ong, A., & Lee, V. Y. A. (2022, June). Exploring students’ epistemic emotions in knowledge building using multimodal data. In Weinberger, A., Chen, W., Hernandez-Leo, D., and Chen, B. (Eds.), Proceedings of the 15th Computer-Supported Collaborative Learning (CSCL) 2022 (pp. 266-273). Hiroshima, Japan: International Society of the Learning Sciences. https://2022.isls.org/proceedings/
- Ng, A. D. X., Ong, A., Lee, A. V. Y., & Teo, C. L. (2024). Implementing learning analytics interventions to support student agency in knowledge building. Pedagogies: An International Journal, 19(3), 372–402. https://doi.org/10.1080/1554480X.2024.2379786
- Lee, A. V. Y., Teo, C. L., & Ong, A. (2023, December). A Step toward Characterizing Student Collaboration in Online Knowledge Building Environments with Machine Learning. In J.-L., Shih, A. Kashihara, W. Chen, & H. Ogata (Eds.), Proceedings of the 31st International Conference on Computers in Education (ICCE), Volume 1 (pp. 815-825). Matsue, Japan: Asia-Pacific Society for Computers in Education.
.tmb-listing.jpg?Culture=en&sfvrsn=b0306905_1)