Sculpting Art from Chaos: Diffusion Models — SMLDs

Author(s): Son Cain

Originally published on Towards AI.

Imagination creates reality…
Generated image from Stable Diffusion

In the previous article, we covered the general structure of diffusion models as well as a very popular category, Denoising Diffusion Probabilistic Models (DDPMs). Now, we will take a look at a completely different but equivalent approach to formulating the diffusion process. The models in this class are called SMLDs, Score Matching via Langevin Dynamics, a name that will become clearer later on.

Let’s begin our journey!

At the core of these models lies the (Stein) score function of a probability density p(x) which is given by ∇ₓlogp(x).

This quantity provides the directions according to which we move from a random sample x₀ towards a sample xₙ in a region with high density. The algorithm that is used for this process is called Langevin sampling algorithm.

Just like DDPMs took inspiration from thermodynamics, the fundamental idea behind SMLDs can be traced back to physics. In particular, Langevin sampling, known in physics as Langevin Dynamics, is an approach to the mathematical modeling of the dynamics of molecular systems originally developed by French physicist Paul Langevin.

But enough talk. Let’s now take a look at the actual math behind this beautiful idea!

The theoretical foundation for this category of diffusion models wasestablished in the… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI