Reversing Time: Diffusion Models and Stochastic Differential Equations

Author(s): Son Cain

Originally published on Towards AI.

In the beginning, there was chaos…
AI-generated image with Freepik

Diffusion models allow us to reverse time. Yes, time. But I am getting ahead of myself … In the previous two articles, we discussed two different formulations of the diffusion process, Denoising Probabilistic Diffusion Models (DDPMs) and Score Matching via Langevin Dynamics (SMLDs). In this article, we will unite these two formulations and discover how we can describe the diffusion process and its reversal using Stochastic Differential Equations (SDEs).

Let’s begin our journey through time!

Before we jump into diffusion models, let’s take a minute to make ourselves comfortable with the main concepts of stochastic differential equations.

As you probably already know, a differential equation is an equation that relates one or more functions to their derivatives. In physics and mathematics, we use differential equations to model the dynamic behavior of various systems.

In general, a differential equation has the following form:

We can also express this equation in terms of the infinitesimal differential:

Intuitively, this states that a very small — infinitesimal — change to the value of the function x = x(t) is equal to a very small change in time scaled by a factor of magnitude f = f(t, x(t)).

It is noted that the above formulation describes a deterministic differential… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI