Published on 17 Feb 2025

Evaluating STEM Success - Practical Tools for Engaged Learning

OER 05/21 OYS - Actualising Integrated STEM Lessons

“The iSTEM protocol posits students’ PIE in deliberating decisions towards a solution as a worthy integrated STEM education outcome, which was not articulated in the previous protocols. Additionally, the iSTEM protocol items corresponding to problematising, resources, authority, and accountability suggest ways through which these four design principles could be achieved and balanced to foster the 3-dimensional PIE outcomes. By articulating details of our protocol development, including our design considerations, rationale, and challenges, we hope to engage other researchers in the conversation of how to characterise and design “good” integrated STEM learning experiences.”

Excerpt from Developing an integrated STEM classroom observation protocol using the Productive Disciplinary Engagement framework, published in the journal of Research in Science Education. 

 

Project Team

PI: Assistant Professor Ong Yann ShiouNSSE, NIE
Co-PI: Associate Professor Tan Aik LingNSSE, NIE
Co-PI: Mr Ng Yong Sim, NSSE, NIE
Research Associate: Ms Jaime Koh Li-ChingNSSE, NIE
Collaborators:
Dr Chin Tan Ying, CPDD1, MOE
Ms Tay Wee Beng , CPDD1, MOE

 

Project Description

While STEM education has gained popularity internationally, specifications of how features of integrated STEM lessons lead to desired outcomes, and how to measure these outcomes are lacking. Classroom observation protocols could provide such guidance, but existing ones seem inadequate. This motivated us to develop the integrated STEM classroom observation protocol (iSTEM protocol). 

An initial version of the iSTEM protocol informed by the productive interdisciplinary engagement (PIE) framework was constructed. The PIE framework proposes that a worthy outcome of integrated STEM is students’ productive interdisciplinary engagement i.e., collaborative, systematic, and justified decision-making to achieve improved solutions to STEM problems. This is achieved by balancing four design principles: problematising (having meaningful STEM problems), resources, student authority/agency, and accountability. 

The initial protocol was iteratively improved through analysing video-recorded and live integrated STEM lessons enacted by 10 primary and 17 secondary teachers between 2022 to 2023. One student group served as the focus group to analyse productive interdisciplinary engagement in each lesson. This resulted in the validated iSTEM protocol version 5.3 for researchers. A version suitable for educators, the STEM Learning Experience (LE) Reflection Tool (RT) 2.0 (revised from the original STEM LE RT by MOE CPDD), was also developed. 

Findings suggest while most teachers could balance problematising and resources, they encountered challenges giving students authority/agency to propose and drive ideas while holding students’ ideas accountable to standards/criteria. Thus, PIE outcomes were not always achieved. 

Project Implications

Implications for practice: STEM LE RT 2.0 can inform lesson planning and enactment of integrated STEM lessons to achieve PIE student outcomes.

Implications for policy and research: iSTEM protocol/STEM LE RT 2.0 serves as research/educative tool for analysing and communicating PIE design principles that foster groups’ decision-making during integrated STEM problem-solving, which involve 21CCs (collaboration, communication, critical thinking, creativity). 

Recommendations: Educators interested in actualising integrated STEM lessons with students’ productive interdisciplinary engagement as a worthy outcome of integrated STEM education are encouraged to review the relevant resources and hold professional conversations around the PIE design principles and outcomes. 

 

Resources

Research Outputs: 

  • Integrated STEM Classroom Observation Protocol (iSTEM protocol) version 5.3
  • STEM Learning Experience Reflection Tool 2.0 (STEM LE RT 2.0) access via OPAL 2.0 or write to the PI for a copy. A microlearning unit on using this tool will be available on OPAL 2.0.
  • PIE Design Principles Best Practices Summary
    This is a summary of the best practices for enacting PIE design principles informed by exemplars from Singapore STEM classrooms involved in the project.

Selected Talks:

Selected Articles:

  • Ong, Y.S., Koh, J., Tan, AL. & Ng, Y.S. (2023). Developing an integrated STEM classroom observation protocol using the Productive Disciplinary Engagement framework. Research in Science Education, 54, 101-118. https://doi.org/10.1007/s11165-023-10110-z