Seminar on New Advances in Multimodal Reasoning

Assistant Professor Paul Liang

Massachusetts Institute of Technology

This seminar will be chaired by Asst Prof Marvin Carl May.

Today's language models are increasingly capable of reasoning over multiple steps with verification and backtracking to solve challenging problems. However, multimodal reasoning models that can reason over an integrated set of modalities such as text, images, audio, video, sensors, and external knowledge are sorely lacking, and can pave the way for a next frontier of AI. I will describe our group's work on advancing the frontiers of multimodal reasoning, from new multimodal reasoning benchmarks to training multimodal foundation models capable of interactive and long-range reasoning, with real-world applications to sensing, health, and wellbeing.