Repository: Freie Universit├Ąt Berlin, Math Department

Happy People--Image Synthesis as Black-Box Optimization Problem in the Discrete Latent Space of Deep Generative Models

Jung, S. and Schwedhelm, J.C. and Schillings, Claudia and Keuper, M. (2023) Happy People--Image Synthesis as Black-Box Optimization Problem in the Discrete Latent Space of Deep Generative Models. arXiv . (Submitted)

Full text not available from this repository.

Official URL: https://doi.org/10.48550/arXiv.2306.06684

Abstract

In recent years, optimization in the learned latent space of deep generative models has been successfully applied to black-box optimization problems such as drug design, image generation or neural architecture search. Existing models thereby leverage the ability of neural models to learn the data distribution from a limited amount of samples such that new samples from the distribution can be drawn. In this work, we propose a novel image generative approach that optimizes the generated sample with respect to a continuously quantifiable property. While we anticipate absolutely no practically meaningful application for the proposed framework, it is theoretically principled and allows to quickly propose samples at the mere boundary of the training data distribution. Specifically, we propose to use tree-based ensemble models as mathematical programs over the discrete latent space of vector quantized VAEs, which can be globally solved. Subsequent weighted retraining on these queries allows to induce a distribution shift. In lack of a practically relevant problem, we consider a visually appealing application: the generation of happily smiling faces (where the training distribution only contains less happy people) - and show the principled behavior of our approach in terms of improved FID and higher smile degree over baseline approaches.

Item Type:Article
Subjects:Mathematical and Computer Sciences > Mathematics > Applied Mathematics
Divisions:Department of Mathematics and Computer Science > Institute of Mathematics > Deterministic and Stochastic PDEs Group
ID Code:3115
Deposited By: Ulrike Eickers
Deposited On:21 Feb 2024 11:27
Last Modified:21 Feb 2024 11:27

Repository Staff Only: item control page