Repository: Freie Universität Berlin, Math Department

Stein Variational Gradient Descent:many-particle and long-time asymptotics

Nüsken, Nikolas and Renger, D.R. Michiel (2021) Stein Variational Gradient Descent:many-particle and long-time asymptotics. arxiv preprint . pp. 1-25. (Submitted)

[img] PDF

Official URL:


Stein variational gradient descent (SVGD) refers to a class of methods for Bayesian inference based on in-teracting particle systems. In this paper, we consider the originally proposed deterministic dynamics as well asa stochastic variant, each of which represent one of the two main paradigms in Bayesian computational statis-tics:variational inferenceandMarkov chain Monte Carlo. As it turns out, these are tightly linked througha correspondence between gradient flow structures and large-deviation principles rooted in statistical physics.To expose this relationship, we develop the cotangent space construction for the Stein geometry, prove its ba-sic properties, and determine the large-deviation functional governing the many-particle limit for the empiricalmeasure. Moreover, we identify theStein-Fisher information(orkernelised Stein discrepancy) as its leadingorder contribution in the long-time and many-particle regime in the sense ofΓ-convergence, shedding some lighton the finite-particle properties of SVGD. Finally, we establish a comparison principle between the Stein-Fisherinformation and RKHS-norms that might be of independent interest

Item Type:Article
Subjects:Mathematical and Computer Sciences > Mathematics > Applied Mathematics
Divisions:Department of Mathematics and Computer Science > Institute of Mathematics
ID Code:2549
Deposited By: Monika Drueck
Deposited On:06 Apr 2021 10:07
Last Modified:06 Apr 2021 10:07

Repository Staff Only: item control page