An Introduction to the Gradient Descent Method

 

Biography

Thai Doan Chuong is currently a Lecturer in Mathematics and Statistics at Brunel University of London, United Kingdom. He has published over 85 research articles in the field of Optimization and Operations Research, including in leading journals such as SIAM Journal on Optimization and Mathematics of Operations Research. He serves as an Editor of Journal of Nonlinear and Variational Analysis and is a member of the Editorial Board of Applied Analysis and Optimization.

Lecture Abstract

Gradient descent is one of the fundamental algorithms in optimization and machine learning. This lecture introduces the core idea of using iterative updates based on gradients to locate minima or stationary points of differentiable functions. Through simple examples, we derive the update rule, explain the role of the step size (or learning rate), and visualize how the algorithm progresses toward an optimum. The lecture emphasizes conceptual understanding and intuition, supported by graphical illustrations and examples that highlight both the strengths and limitations of the method.