AI for 21CC: Collaborative Problem Solving Diagnosis among Secondary School Students using Multimodal Audio Data
| Beyond using AI for increased work productivity and mimicking teaching strategies, can AI be used to provide insights into complex behaviours, such as when learners work collaboratively to solve problems? With the current capabilities of transformer-based models, how should we think about leveraging them for diagnosing important 21st Century Competencies (21CC) in education? This study points out 3 key considerations: 🪙 Value of multimodal audio data: transcription with acoustic-prosodic data of students processed by a transformer-based model can improve the diagnosis of CPS classification compared to using transcription data alone; multimodality has its value! 📈 Improvements by multimodality: model architectures for different classes and data structures can yield varied performance; complex models may not always be better! 🛑 Limits to automated CPS diagnosis: some CPS classifications remain too complex even for the best models to detect; Human-AI complementarity is needed! For more details, please see here for the conference paper. https://doi.org/10.1007/978-3-031-98417-4_2 If you are unable to access or would like to see an extended version of the paper, https://doi.org/10.48550/arXiv.2504.15093 Related to this paper are two presented AIED workshop papers that discusses further on Explainable AI (XAI) and Human-AI Complementarity. 1) “Explainable Collaborative Problem Solving Diagnosis with BERT using SHAP and its Implications for Teacher Adoption” http://arxiv.org/abs/2507.14584 2) “Exploring Human-AI Complementarity in CPS Diagnosis Using Unimodal and Multimodal BERT Models” http://arxiv.org/abs/2507.14579 If you would like to understand more, please feel free to reach out to Kester Wong at [email protected] |