Soft Quality-Diversity Optimization

Discrete archive

Soft illumination

Soft QD replaces discrete behavior space partitioning with a smooth illumination of behavior space.

Abstract

Quality-Diversity (QD) optimization seeks collections of solutions that are both high-performing and behaviorally diverse. Existing approaches rely on discretizing the behavior space into archives, limiting scalability and preventing end-to-end differentiable optimization. We introduce Soft Quality-Diversity, a formulation that removes discretization by defining QD as a smooth integral over behavior space. Building on this objective, we derive SQUAD, a fully differentiable algorithm that balances quality maximization with repulsive forces between behaviorally similar solutions. Across multiple benchmarks, Soft QD scales effectively to high-dimensional behavior spaces and achieves competitive or superior performance compared to state-of-the-art methods.

Soft Quality-Diversity

Given a population of $n$ solutions, Soft QD defines the behavior value of a point $b$ in the behavior space as:

$$v_{\theta}(b) = \max_n f_n \exp\left(-\frac{\|b - b_n\|^2}{2\sigma^2}\right)$$

The overall quality-diversity of a population is measured by the total behavior value that they induce over the entire behavior space, called the Soft QD Score:

$$S(\theta) = \int_{\mathcal{B}} v_{\theta}(b)\, db$$

Key intuition. Each solution acts as a light source whose brightness is proportional to its quality and whose influence decays smoothly in behavior space.

Soft QD Score quantifies the overall quality and diversity of a population. It does not require partitioning of the behavior space, and can compare populations of different sizes.

SQUAD

Soft QD Score admits a tractable lower bound that leads to a simple and effective optimization objective:

$$\tilde{S}(\theta) = \sum_n f_n - \sum_{i < j} \sqrt{f_i f_j} \exp\left(-\frac{\|b_i - b_j\|^2}{\gamma^2}\right)$$

Solutions are attracted toward higher quality, while high-quality solutions that are behaviorally similar repel each other. SQUAD optimizes this objective end-to-end using standard gradient-based optimizers.

Scalability

By avoiding discretization, Soft QD remains effective as the dimensionality of the behavior space increases. SQUAD consistently outperforms archive-based methods in higher-dimensional settings.

Qualitative Results

Image Composition

SQUAD discovers diverse reconstructions of a target image, spanning different stylistic and compositional attributes. Here, each image is composed of a few colored circles, put together so that they best resemble the target image.

Latent Space Illumination

When used to explore the latent space of a StyleGAN2 model, SQUAD can identify latent codes that generate a diverse set of images closely aligned with a given text description. The examples below show images produced by SQUAD in response to the prompt "a detective from a noir film", exhibiting diversity across attributes such as age, hair color and style, and facial expression.

BibTeX

@inproceedings{
hedayatian2026softqd,
title={Soft Quality-Diversity Optimization},
author={Saeed Hedayatian and Stefanos Nikolaidis},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026}
}