AI.X Seminar 1 July 2026
Title: Creative Physical Intelligence
Speaker: Professor Heng Ji, UIUC
Time: July 1st, 2026 (Wed), 4:30pm – 5:30pm
Venue: Academic Building North (ABN) Conference Room 1-1
Abstract: No scissors to open a taped box? Use a key. A bottle can hold water -- but it can also knead dough or shatter a window in an emergency. Humans effortlessly repurpose tools based on the situation. For AI, this kind of physical-world creativity is surprisingly hard. On a challenging tool repurposing benchmark, today's best large vision-language models get it right only ~20% of the time. While large language model (LLM)-based agents have greatly advanced analytical intelligence through problem solving, and simulation environments emphasize practical intelligence by testing the application of knowledge, creative intelligence (i.e., the ability to generate novel and useful ideas and solutions, such as novel functional tool design and creative ways to use tools) remains a moonshot goal. I will introduce a “decompose and reassemble” framework that enables agentic AI models to automatically break down real-world objects into distinctive, function-specific components and then use these components to perform compositional reasoning, generation and action. We propose affordance-grounded alignment to encourage models to prefer attribute-affordance reasoning grounded in visual evidence over hallucinated alternatives. In addition, we incorporate supervision derived from an affordance knowledge base to guide broader entity exploration and multi-turn planning. Our results suggest that creative intelligence is not a peripheral capability, but a foundation for the next stage of multimodal AI, enabling systems that learn over time, adapt to unfamiliar environments, and solve problems beyond their training, moving closer to human-like intelligence.
Bio: Heng Ji is a Professor of Computer Science at Siebel School of Computing and Data Science of University of Illinois Urbana-Champaign. She is the Founding Director of Amazon-Illinois Center on AI for Interactive Conversational Experiences (AICE), and the Founding Director of CapitalOne-Illinois Center on AI Safety and Knowledge Systems (ASKS). She is a faculty member affiliated with Coordinated Science Laboratory, and Carl R. Woese Institute for Genomic Biology of University of Illinois Urbana-Champaign. She is named as an Association for Computational Linguistics (ACL) Fellow in 2025 for her "significant contributions to information extraction, multimodal and multilingual knowledge extraction and AI for science." She received her Ph.D. in Computer Science from New York University, and B.A. and M.A. in Computational Linguistics from Tsinghua University. The awards she received include Outstanding Paper Award at ACL2024, two Outstanding Paper Awards at NAACL2024, "Young Scientist" by the World Laureates Association in 2023 and 2024, "Young Scientist" and a member of the Global Future Council on the Future of Computing by the World Economic Forum in 2016 and 2017, “Outstanding Advisor of the Year” by UIUC Engineering Council in 2026, Amazon Scholar 2020-2025, "Women Leaders of Conversational AI" (Class of 2023) by Project Voice, "AI's 10 to Watch" Award by IEEE Intelligent Systems in 2013, NSF CAREER award in 2009, PACLIC2012 Best paper runner-up, "Best of ICDM2013" paper award, "Best of SDM2013" paper award, ACL2018 Best Demo paper nomination, ACL2020 Best Demo Paper Award, NAACL2021 Best Demo Paper Award, Google Research Award in 2009 and 2014, IBM Watson Faculty Award in 2012 and 2014 and Bosch Research Award in 2014-2018. She has given many keynote speeches at top conferences including IJCAI2025 and EMNLP2025. She served as the Program Committee Co-Chair of many conferences including NAACL-HLT2018 and AACL-IJCNLP2022. She was elected as the North American Chapter of the Association for Computational Linguistics (NAACL) secretary 2020-2023.