Why AI can't solve turbulence: a lesson on complexity

Abstract

One of the recent trends is training AI to understand the physical world. Some advocate for a foundational model for physical data, while others pursue alternatives to traditional numerical methods, like neural operators. In this talk, I will outline a framework of what physical AI entails based on information theory. Starting with a brief contemplation of the nature of physical laws as a ML representation of physical data, I will present the idea of ML models as "compressed representation of data", and investigate how the entropy of the ML model scales with the domain size and parameters of a physical PDE solution. Borrowing ideas from Kolmogorov-Sinai entropy, we will show that the Kolmogorov complexity of temporal data scale linearly with temporal domain size. This scaling is the root cause of the exploding gradient problem, which historically plagued the development of vanilla RNNs and NeuralODEs and give rise to stiffness in the inverse problems like data assimilation in long time-series data. I will also discuss existing methodologies that can mitigate this challenge, noting that they all share the common characteristic of "forgetfulness."

Extending the concept of Kolmogorov complexity scaling, we will then demonstrate how Kolmogorov complexity sometimes scales with key defining parameters of physical systems, such as the Reynolds number in turbulence. I propose that this scaling, which serves as a lower bound on the degrees of freedom necessary to accurately represent a PDE solution, should be employed as an evaluation metric for any new ML architecture or physical AI. Finally, drawing on historical development of numerical methods, I will offer my perspective on why current hype surrounding physical AI will not ultimately "solve" turbulence.

About the Speaker

Lloyd Fung is an Imperial Research Fellow in the Department of Aeronautics at Imperial College London, working with Prof. Luca Magri on scientific machine learning and its interface with traditional data assimilation and numerical methods. He is also a Fellow in the Schmidt Sciences-funded AI in Science programme at IX (Digital Foundry), an Imperial AI initiative.

His research spans machine learning and fluid dynamics, with particular interests in data assimilation for complex fluid problems, data-driven discovery of symbolic models, and complex fluid phenomena.

Fung graduated from Imperial’s Department of Aeronautics in 2017 under the supervision of Dr Yongyun Hwang. He was subsequently elected a Research Fellow at Peterhouse, Cambridge, where he worked with Prof. Raymond Goldstein in the Department of Applied Mathematics and Theoretical Physics.