Deep Learning May Solve The Tokamak Magnetic Control Problem

Author(s): Ömer Özgür

Originally published on Towards AI the World’s Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses.

Deep Learning

At the beginning of everything, there was energy. Energy is one of the most important things in the universe . Without usable energy, you could not read this article. Fusion power gives life to earth. Sun is a very big fusion reactor, without it earth would be frozen. Our food’s chemical bonds, petroleum, solar, wind, hydropower originated in the heart of the sun.

Nowadays energy is a very big problem for the earth. The more we produce energy more we produce pollution and CO2 which is not sustainable. Many scientists believe that solution for energy is nuclear power. There are 2 types of nuclear power.

First of all, we invented fission power. Fission power relies on splitting heavy nuclei into smaller ones and generates heat energy. Fusion combines smaller nuclei with heavy ones which generate heat energy. Fusion is the ultimate energy generation process because it creates way more energy than fission and is very very clean.

Humanity tries to create sun in the lab and we succeed but there is a big problem. It generates less energy than we put in and is not safe to use. Recently DeepMind team published an article named Magnetic control of tokamak plasmas through deep reinforcement learning [1].

In this article, I will explain Deepmind’s paper and we will see how Deep Learning can help us to achieve unlimited clean energy.

Table Of Contents:

• How do Tokamak works?
• Tokamak Magnetic Control Problem
• Deepmind’s solution

How does Tokamak Works?

Nuclear fusion can be used to generate electricity. Generated heat collected from the walls of the Tokamak generates steam power and runs the turbines. For setting up the sun in your lab, you need fuel such as deuterium and tritium. They are easy to find and store. The next step is to bake your ingredients.

But we have a problem called Coulomb Forces. As we know that nuclei of an atom consist of protons and neutrons. Protons are (+) charged and neutrons are not charged. It is a known fact that the same charges will repel each other. Fusion occurs when the nucleus of 2 atoms gets so close that a strong nuclear force act upon them and fuse. When this occurs reaction gives off energy to the environment.

Let’s look at how to beat Coulomb Forces:

Coulomb Force Formula = (k*q1*q2)/r^2

The closer 2 protons more they will feel forced. How can we penetrate the Coulomb barrier? The answer is easy, more speed! As we know that temperature is related to the velocity of the atoms. If we can heat the fuel mixture to 100 million Kelvin, fusion is inevitable.

Sun does this process with its massive body. Gravity compresses the gas and when the conditions right ignition starts. But we need to use another force of nature for heating the plasma. It is a magnetic force. Changing magnetic fields are both used for heating and maintaining the shape of the plasma. A combination of magnets makes a toroidal field and a poloidal field which is a donut shape.

Okay, we started the fusion, how to control it? Millions of Kelvin will melt everything apart. To solve this problem, a vacuum is created inside of the Tokamak, and keeping the plasma away from the walls is provided by the magnetic field. As we know that plasma is electrically charged, thus it will affect by the magnetic field.

Tokamak has a very special donut shape. And the surface of this device is covered with field coils. This formula shows how the magnetic field is created from the current passing wire.

B=(μ*I)/(2πr)

μ=(4π×10−7T⋅ m/A) is the permeability of free space
I = Current passing the wire(A)
π= (3.14...)pi number
r = radius of the wire [2]

If we look at the formula of the generated magnetic field from a current passing wire. The direction and magnitude of the magnetic field are directly related to the current. We can control the magnetic field, if we can control the magnetic field we can also control the plasma. By changing current we can give wanted form to the plasma, it is the key concept in Tokamak and Deepmind’s approach.

Sun creates an equilibrium between nuclear pressure and Gravitational force. In Tokamak, equilibrium is created between nuclear pressure and Magnetic force.

Tokamak Magnetic Control Problem

How to control chaotic nuclear fireball? When the plasma touches the walls of the reactor it cools down and the reactor gets damaged. The main challenge is the shape and maintains hot plasma within the reactor. Hot plasma is a very chaotic place, small instabilities accumulate and cause fusion impossible.

This is Tokamak's Magnetic Control Problem.

Deepmind’s Solution

DeepMind tackled very hard scientific problems such as playing go better than any human, predicting folding of proteins in very low error. But this time they are facing a really hard problem. If DeepMind can solve this problem, future nuclear plants will be controlled by AI.

When we have a look at DeepMind’s paper. They have collaborated with the Swiss Plasma Center and experimentations made here. Tokamak à configuration variable(TCV) is Swiss research, medium-sized fusion reactor.

DeepMind proposed Reinforcement Learning for solving Tokamak Magnetic Control problems. The tokamak is the most complex real-world system for Reinforcement Learning. First of all, they created a simulated environment, for the mechanics of environment Grad–Shafranov equations used [3]. These equations are ideal for magnetohydrodynamics.

DeepMind created a single neural network to control all 19 coils, the agent automatically learns which voltages needed to be supplied to them to best contain the plasma. The model has three hidden layers and finishes with a 19 Dense Layer. This model is only trained in simulation than applied to the real world which is very interesting. All control policy learned from interaction with simulator then deployed to TCV’s hardware.

The method is divided into three stages. First, a researcher defines the experiment’s goals, which may include time-varying control targets. Second, a deep RL algorithm interacts with a tokamak simulator to determine a near-optimal control strategy to achieve the defined objectives. Third, the control strategy, represented as a neural network, is executed in real-time on tokamak hardware. Reward functions for the agent were simple position and plasma current stabilization to complex combinations of multiple time-varying targets.

As we looked at the results the experiment was quite successful. DeepMind stated that the model can control a variety of shapes of plasma. The fusion problem has not been solved yet but this experiment created a paradigm shift. Conventionally used in Tokamak was expensive and complex. Using one model for control makes the process more efficient.

Conclusion

Energy is a very big problem on the earth, it caused wars and pollution. Fusion seems to solve all these problems. Human intelligence does not evolve to create control systems to create stable sun earth. Maybe we can do it but we do not have enough time. It is time to use machine learning to tackle the most complex problems.

At the same time, if humans are to be an interstellar species, we must overcome this problem. There is plenty of hydrogen and helium in the universe…

References

[1]https://www.nature.com/articles/s41586-021-04301-9

[2]https://courses.lumenlearning.com/physics/chapter/22-9-magnetic-fields-produced-by-currents-amperes-law/

[3]https://en.wikipedia.org/wiki/Grad%E2%80%93Shafranov_equation

---

Deep Learning May Solve The Tokamak Magnetic Control Problem was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Join thousands of data leaders on the AI newsletter. It’s free, we don’t spam, and we never share your email address. Keep up to date with the latest work in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI