VarGrad: A Low-Variance Gradient Estimator for Variational Inference

Abstract

We analyse the properties of an unbiased gradient estimator of the evidence lowerbound (ELBO) for variational inference, based on the score function method withleave-one-out control variates. We show that this gradient estimator can be obtainedusing a new loss, defined as the variance of the log-ratio between the exact posteriorand the variational approximation, which we call thelog-variance loss. Undercertain conditions, the gradient of the log-variance loss equals the gradient of the(negative)ELBO. We show theoretically that this gradient estimator, which we callVarGraddue to its connection to the log-variance loss, exhibits lower variance thanthe score function method in certain settings, and that the leave-one-out controlvariate coefficients are close to the optimal ones. We empirically demonstrate thatVarGrad offers a favourable variance versus computation trade-off compared toother state-of-the-art estimators on a discrete variational autoencoder (VAE)