The Broken Mirror: What Generative Models Still Don’t Understand About Symmetry

Author(s): Riccardo Di Sipio

Originally published on Towards AI.

At TMLS 2025, I attended a talk titled “Generative AI for Retail: Personalization with Diffusion Models.” The speaker opened with a compelling analogy: diffusion models simulate the reversal of a thermodynamic process — like ink dissolving back into a pen, undoing entropy one pixel at a time.

That caught my attention. In physics, we call that time-reversal symmetry — when a system looks the same whether time flows forward or backward. It’s a powerful idea, but something didn’t looked quite right to me.

I asked the speaker a question: If diffusion models are inspired by physical symmetries, why do they so often fail at generating things that are mirror-symmetric such as faces, shoes, logos? The output frequently includes distortions that a human eye catches instantly.

The answer may lie in the kind of symmetry these models forget.

The Missing Mirror

In physics, parity (or mirror symmetry) describes what happens when you flip a system left-to-right. Some properties and laws remain unchanged. Others — like the weak nuclear force — do not. Mirror symmetry is fragile, and in generative models, it’s often completely absent.

Diffusion models are trained to denoise images step-by-step to recover structure from randomness. But they don’t conserve symmetry unless symmetry is present in the data and reinforced in the model. And it usually isn’t.

The result? Models that can paint beautifully, but can’t quite replicate the reflection.

Emmy Noether’s Insight

In 1915, mathematician Emmy Noether made a discovery that rewired modern physics:

Every continuous symmetry corresponds to a conservation law.

Time symmetry → conservation of energy.
Rotational symmetry → conservation of angular momentum.
And mirror symmetry — more than a visual transformation — is deeply tied to how fundamental forces behave.

Just last week, the Quanta Magazine podcast released an episode titled “How Emmy Noether’s Theorem Revolutionized Physics.” It reflects on how her work showed that what we took as basic truths — like conservation laws — actually emerged from symmetry itself. “On paper, symmetries seem to have no impact on the physics of the system,” the narrator says.

“The impact of symmetry had been hiding beneath the equations all along — just out of view.”

In many ways, that’s the situation with generative models today. In machine learning, symmetry is often treated as a cosmetic detail — something that can be approximated or left to emerge from training. But maybe, like in physics, its absence is more consequential than we realize.

A Familiar Struggle

This isn’t just a philosophical worry. It’s something I’ve run into before.

A few years ago, I co-authored a paper on using Generative-Adversarial Networks (GANs) to simulate particle collisions at CERN— a project called DiJetGAN. We aimed to reproduce realistic particle jets events with correct angular distributions, but we hit a wall: mirror symmetry wouldn’t emerge naturally.

Our workaround? Restrict the angular observable to a limited range and inject randomness to simulate symmetry. It worked — sort of. But it always felt wrong. We weren’t capturing parity. We were faking it.

That experience has stayed with me. Because if our models can’t respect symmetries that even nature holds dear, what are they really learning?

Symmetry in ML — But Not in Generation

To be fair, symmetry isn’t foreign to machine learning — just foreign to generative image models.

• In reinforcement learning, researchers have designed policies that explicitly respect rotational and mirror symmetry.
  👉 SO(2)-equivariant agents have shown improved sample efficiency in robotic control (Wang et al., ICLR 2022).
• In partially observable environments, symmetry-aware actor–critic models outperform vanilla networks by encoding invariances (Nguyen et al., CoRL 2023).
• In computer vision, Group-equivariant CNNs (Cohen & Welling, 2016) successfully model translation, rotation, and reflection — showing clear gains in tasks where symmetry is structural.

But in diffusion-based generative models?
I think the mirror is still broken.

Toward Symmetric Generators?

We don’t need to force physics into AI, but maybe we should listen to it more closely. Symmetry isn’t just aesthetic. It’s a constraint that Nature uses to preserve structure. And when our models ignore it, the consequences are visible: distorted reflections, broken anatomy, uncanny details.

Diffusion models in AI simulate the reversal of entropy — a noise-to-image process that runs against the arrow of time. But this reversal is purely mathematical. Unlike physical systems, where entropy increases and symmetry breaks irreversibly, AI models are trained to mimic structure without experiencing dissipation. They invert diffusion without the thermodynamic cost — and without the real-world asymmetries that give rise to meaning, memory, and direction.

Systems evolve from low entropy (order) to high entropy (disorder). This gives time a direction — because while the physics equations might not care which way time runs, entropy does. This is the essence of the Second Law of Thermodynamics, from which originates the concept of the Arrow of Time.

In that sense, the asymmetry in diffusion models isn’t physical. It’s conceptual — and maybe that’s why symmetry can be lost so easily in their outputs.

The asymmetry of time lies at the heart of non-equilibrium thermodynamics, and few understood this better than Ilya Prigogine. He showed how dissipative structures — ordered states emerging in open systems far from equilibrium — form through entropy production, not in spite of it. This formalized the arrow of time as a creative force.

You can see this, too, in geology – the most visible record of time’s asymmetry. Where physics writes in equations, geology writes in strata. Erosion, heat, mineral diffusion – they leave traces that don’t run backward. To those who know how to read it, the Earth is an open book, written in irreversible processes.

Prigogine was, incidentally, an honorary citizen of my hometown. A quiet coincidence — but a fitting one. His work reminds us that broken symmetry is not failure. It’s where new structure begins.

Symmetry Breaking

In physics, some of the most beautiful structures emerge not from perfect symmetry, but from its breaking. The Standard Model itself relies on this: the symmetry of its gauge group is broken in a way that gives particles mass, defines interactions, and shapes the complexity of the universe. Symmetry breaking isn’t a flaw — it’s a mechanism for emergence.

I believe the same can be said in the human world: a perfectly symmetrical building often feels sterile; the intricate irregularities of Gaudí’s architecture, by contrast, feel alive. We recognize beauty not in perfection, but in patterns interrupted by intent.

Even faces — the most intimate symmetry we know — are never perfectly balanced. And maybe that’s what makes them unforgettable.

In physics, broken symmetry gives us structure and dynamics .
In humans, it gives us unique character.
In AI, it’s still just a glitch.

References

• Cohen, T., & Welling, M. (2016). Group Equivariant Convolutional Networks. ICML. arXiv:1602.07576
• Wang et al. (2022). SO(2)-Equivariant Reinforcement Learning. ICLR. arXiv:2203.04439
• Nguyen et al. (2023). Equivariant Reinforcement Learning under Partial Observability. CoRL.
• DiJetGAN paper — arXiv:1903.02433
• Quanta Magazine podcast (2025). “How Emmy Noether’s Theorem Revolutionized Physics”. [link]

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI