A Functional Approach to Local Geometric Convergence of Coordinate Ascent Variational Inference
Mean-field variational inference approximates a target distribution by a product of factors chosen to minimize Kullback–Leibler (KL) divergence. Coordinate ascent variational inference (CAVI) computes each factor update by optimizing over that distribution while holding the others fixed. Applying these updates in parallel or sequentially leads to different convergence behaviour.
For Gaussian targets, the factor variances can be optimized explicitly, leaving a quadratic problem in the means. The parallel and sequential schedules then coincide exactly with Jacobi and Gauss–Seidel iterations. I will explain their convergence in terms of the precision matrix: positive definiteness suffices for Gauss–Seidel, whereas Jacobi requires an additional spectral condition. Beyond Gaussian targets, I will show that differentiable factor updates have the same block Jacobi and Gauss–Seidel structure to first order near a variational fixed point.
Joint work with Rocco Caprio and Sam Power.
Short bio: