Repository: Freie Universität Berlin, Math Department

On the geometry of Stein variational gradient descent

Duncan, A. and Nüsken, N. and Szpruch, L. (2019) On the geometry of Stein variational gradient descent. SFB 1114 Preprint in arXiv:1912.00894 . (Unpublished)

[img]
Preview
PDF
851kB

Official URL: https://arxiv.org/abs/1912.00894

Abstract

Bayesian inference problems require sampling or approximating high-dimensional probability distributions. The focus of this paper is on the recently introduced Stein variational gradient descent methodology, a class of algorithms that rely on iterated steepest descent steps with respect to a reproducing kernel Hilbert space norm. This construction leads to interacting particle systems, the mean-field limit of which is a gradient flow on the space of probability distributions equipped with a certain geometrical structure. We leverage this viewpoint to shed some light on the convergence properties of the algorithm, in particular addressing the problem of choosing a suitable positive definite kernel function. Our analysis leads us to considering certain nondifferentiable kernels with adjusted tails. We demonstrate significant performs gains of these in various numerical experiments.

Item Type:Article
Subjects:Mathematical and Computer Sciences > Mathematics > Applied Mathematics
ID Code:2387
Deposited By: Silvia Hoemke
Deposited On:04 Dec 2019 14:18
Last Modified:04 Dec 2019 14:18

Repository Staff Only: item control page