Mathematical and Computational Understanding of Neural Networks: From Representation to Learning and From Shallow to Deep, and Beyond

Abstract

In this talk I will present some understanding of a few basic mathematical and computational questions for neural networks, as a particular form of nonlinear representation, and show how the network structure, activation function, and parameter initialization can affect its approximation properties and the learning process. In particular, we propose structured and balanced multi-component and multi-layer neural networks (MMNN) using sine as the activation function with an initialization scaling strategy.  At the end, I will discuss a few issues and challenges when using neural networks to solve partial differential equations. 

Biography:

Hongkai Zhao is the Ruth F. DeVarney Distinguished Professor and Chair of the Mathematics Department at Duke University. He obtained his B. Sc from Peking University and his Ph.D in Mathematics from UCLA in 1996.  His research interest includes scientific computing, numerical analysis, inverse problems and imaging, and scientific machine learning.

He is a SIAM Fellow and has received Sloan Fellowship and Feng Kang Prize in Scientific Computing.