A Decision-Theoretic Foundation for Trustworthy AI by Dr Yifan Wu

Abstract:

Artificial intelligence (AI) is increasingly important to decision-making across various domains. However, recent research has observed that AI systems lack key elements of trustworthiness, such as calibration and complementarity with human decision-makers. This talk presents a decision-theoretic foundation for AI trustworthiness.  

The main part of the talk focuses on how decision theory can be used to evaluate and improve the trustworthiness of predictive models. A central concept is calibration, which ensures that predictions can be reliably interpreted as probabilities. My paper proposes a decision-theoretic calibration error, the Calibration Decision Loss (CDL), defined as the worst-case payoff loss from miscalibration suffered by any downstream decision maker. We show separations between the decision-theoretic CDL and existing calibration error metrics, including the canonical metric Expected Calibration Error (ECE). Our main technical contribution is an efficient algorithm for online CDL minimization. By minimizing CDL, the algorithm achieves near-optimal loss minimization simultaneously for downstream decision makers, bypassing the lower bound from minimizing the canonical ECE. The technical result highlights that a decision-theoretic error metric helps identify better predictive algorithms for decision making. This will be based on the paper “Predict to Minimize Swap Regret for All Payoff-Bounded Tasks” in FOCS 24 (https://ieeexplore.ieee.org/document/10756171).  

In the final part of the talk, I will mention the applications of this decision-theoretic perspective to a theoretically justified understanding of reliance in human–AI systems.

Biography:

Yifan is a postdoctoral researcher with the EconCS group at Microsoft Research New England. She earned her Ph.D. from Northwestern University, advised by Prof. Jason Hartline, and her B.S. in Computer Science from the Turing Class at Peking University. She was also a visiting student researcher at Stanford University during the 2023–2024 academic year.

Her research sits at the intersection of theoretical computer science, AI, and economics, with a recent focus on the theory of AI trustworthiness. She received an Honorable Mention for the 2025 ACM SIGecom Doctoral Dissertation Award and a Best Paper Award at FORC 2025.