Seminar: Specification-Guided Reinforcement Learning
Abstract
Reinforcement Learning (RL) is being touted to revolutionize the way we design systems, but a key challenge to reaching that holy grail comes from the lack of guarantees that the synthesized systems offer. Logic and formal reasoning can address some of these issues... or can they?
In this talk, I will cover recent progress in using logical specifications in RL as an alternative to traditional reward-based approaches, where designing effective reward functions can be tedious, error-prone, and inadequate for complex objectives. We will introduce formal logical specifications as a more intuitive and precise way to define agent
behavior and discuss the triumphs and pitfalls of theoretical guarantees and the algorithmic aspects of learning from specifications.
Biography
Dr Suguman Bansal is an assistant professor in the School of Computer Science at Georgia Institute of Technology, USA. Her research interests lie at the intersection of Artificial Intelligence and Programming Languages. Specifically, she works on developing tools and techniques to improve the quality of automated verification and synthesis of computational systems. Her recent work concerns providing formal guarantees in learning-enabled systems with a focus on Reinforcement Learning.
She received her Ph.D. (2020) and M.S. (2016) in Computer Science from Rice University, and B.S. (with Honors) degree (2014) in Mathematics and Computer Science from Chennai Mathematical Institute. She is the recipient of the Amazon Research Award 2024, ATVA Best Paper Award 2023, MIT EECS Rising Stars 2021 and 2018, Future Faculty Fellowship 2019, Andrew Ladd Fellowship 2016, and a Gold Medal at the ACM Student Research Competition at POPL 2016. She was the keynote speaker at the 29th Symposium on Static Analysis (SAS) 2022 and 44th Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS) 2024, and presented an invited tutorial at the 28th Joint Conference on Theory and Practice of Software (ETAPS) 2025.