Duncan, A. and Nüsken, N. and Szpruch, L. (2019) On the geometry of Stein variational gradient descent. SFB 1114 Preprint in arXiv:1912.00894 . (Unpublished)

PDF
851kB 
Official URL: https://arxiv.org/abs/1912.00894
Abstract
Bayesian inference problems require sampling or approximating highdimensional probability distributions. The focus of this paper is on the recently introduced Stein variational gradient descent methodology, a class of algorithms that rely on iterated steepest descent steps with respect to a reproducing kernel Hilbert space norm. This construction leads to interacting particle systems, the meanfield limit of which is a gradient flow on the space of probability distributions equipped with a certain geometrical structure. We leverage this viewpoint to shed some light on the convergence properties of the algorithm, in particular addressing the problem of choosing a suitable positive definite kernel function. Our analysis leads us to considering certain nondifferentiable kernels with adjusted tails. We demonstrate significant performs gains of these in various numerical experiments.
Item Type:  Article 

Subjects:  Mathematical and Computer Sciences > Mathematics > Applied Mathematics 
ID Code:  2387 
Deposited By:  Silvia Hoemke 
Deposited On:  04 Dec 2019 14:18 
Last Modified:  04 Dec 2019 14:18 
Repository Staff Only: item control page