Assessment Design
Let’s explore the implications of generative AI on assessment design and consider when and how generative AI can be leveraged to make assessments more meaningful, efficient, and effective.
To support course chairs and assessment designers in cascading the NTU guidelines, refer to the Assessment Decision Flowchart above, which serves to guide pedagogical and implementation decisions on why, what and how to re-design the assessments.
The Two-lane Approach
Consider the University of Sydney’s ‘two-lane approach’, available on their comprehensive AI for Educators guide:
The two-lane approach below emphasises balance between assurance, and human-AI collaboration. The reality in any one unit will likely be a situation where some assessments lie in lane 1 in order to assure attainment of all learning outcomes, but most other assessments lie in lane 2. Fundamentally, we want to develop students who are well-rounded and can contribute and lead effectively in authentic, contemporary environments (which will include AI), and also be assured of their learning. Therefore in this context, it is important to privilege lane 2 assessments with a higher weighting than lane 1 assessments.
Lane 1 – Examples of assured ‘assessment of learning’
- In-class contemporaneous assessment e.g. skills-based assessments run during tutorials or workshops
- Viva voces or other interactive oral assessments
- Live simulation-based assessments
- Supervised on-campus exams and tests, used sparingly, designed to be authentic, and for assuring program rather than unit-level outcomes
Lane 2 – Examples of human-AI collaboration in ‘assessment as learning’
- Students use AI to suggest ideas, summarise resources, and generate outlines/structures for assessments. They provide the AI completions as an appendix to their submission.
- Students use AI-generated responses as part of their research and discovery process. They critically analyse the AI response against their other research. The AI completion and critique provided as part of the submission.
- Students initiate the process of writing and use AI to help them iterate ideas, expression, opinions, analysis, etc. They document the process and reasoning behind their human-AI collaboration. The documented process demonstrates how the collaborative writing process has helped students think, find their voice, and learn. The documented process is graded and more heavily weighted than the artefact.
- Students design prompts to have AI draft an authentic artefact (e.g. policy briefing, draft advice, pitch deck, marketing copy, impact analysis, etc) and improve upon it. They document the process and reasoning: initial prompt, improvements, sources, critiques. The documented process demonstrates learning, is graded, and is more heavily weighted than the artefact. More information.
They advise integrating higher-order thinking, real-world data sets, creative artefacts, oral assessments and multi-stage assignments to develop AI-resistant evaluations, embracing AI as a learning tool while upholding academic integrity through updated pedagogical models. Check out their full guide for examples of uses, wording and rubric criteria.
The AI Assessment Scale
The AI Assessment Scale (AIAS) (Perkins et al., 2024) is a useful scale to help you decide. Find out more about it here.
You can find even more examples of uses in the appendices of Cornell University’s Committee Report: Generative Artificial Intelligence for Education and Pedagogy.
Curated Examples
- The University of Sydney (How Sydney academics are using generative AI this semester in assessments)
- MIT (Rethinking Your Problem Sets in the World of Generative AI)
- University of Massachusetts Amherst (How Do I (Re)design Assignments and Assessments in an AI-Impacted World?)
- Assignments curated by The AI Pedagogy Project
- Untangling Critical Interaction with AI in Students’ Written Assessment by Antonette Shibani, Simon Knight, Kirsty Kitto, Ajanie Karunanayake, Simon Buckingham Shum (2024)