Researchers From Deepmind and Swiss Plasma Center at EPFL Developed a Deep Reinforcement Learning (RL) System to Control a Nuclear Fusion Reactor

Researchers have been looking for a source of clean, inexhaustible energy to alleviate the global energy dilemma for a long time. One option is nuclear fusion, the process that fuels the universe’s stars. The intense process generates enormous energy by crushing and fusing hydrogen, a common ingredient in saltwater.

Nuclear fusion can provide an unlimited, sustainable supply of clean energy, but only if we can grasp the intricate physics within the reactor can we fulfill this grand ambition.

Scientists have been making little steps toward this aim for decades, but there are still numerous obstacles to overcome. One of the biggest challenges is regulating the reactor’s unstable and superheated plasma, but a new technique shows how we can achieve it. The plasmas in these devices are intrinsically unstable, making it difficult to maintain the nuclear fusion process.

Researchers from Deepmind in partnership with the Swiss Plasma Center at EPFL developed a deep reinforcement learning system to investigate the plasma behavior and control inside a fusion tokamak. This donut-shaped device uses a series of magnetic coils placed around the reactor to control and manipulate the plasma inside it. The coils must make millions of minor voltage adjustments every second to properly keep the plasma confined within magnetic fields, so it’s not an easy balancing act.

While dozens of tokamaks operate worldwide, they are pricey and in great demand. TCV, for example, can only sustain plasma for three seconds in a single experiment before needing 15 minutes to cool down and try again. Furthermore, the tokamak is frequently shared by numerous research groups, significantly reducing the amount of time available for experimentation.

This “plasma sculpting” demonstrates that the RL system has successfully controlled superheated matter and, more crucially, lets scientists explore how plasma behaves under various conditions, allowing them to better comprehend fusion reactors.

Complex, multi-layered systems are required to regulate the coils to sustain nuclear fusion processes, which require keeping the plasma stable at a temperature hotter than even the Sun’s core. However, researchers in a new study indicate that a single AI system can handle the work entirely independently.

Controllers were created to maintain the plasma constant and correctly mold it into diverse shapes using a learning architecture that blends deep RL with a simulated environment. The researchers achieved this by training their AI system in a tokamak simulator. The machine learning system learned how to negotiate the complexity of the magnetic confinement of plasma through trial and error.

The controller first molds the plasma to the desired shape, then moves it lower and detaches it from the vessel’s sides, suspending it in the middle on two legs. The plasma is kept stationary to allow for measurements of plasma characteristics. Finally, the plasma is directed back to the vessel’s top and destroyed safely.

In addition to traditional forms, the AI could sculpt the plasma into complex configurations such as ‘negative triangularity’ and ‘snowflake configurations.’

The ‘ITER-like shape’ (as seen above) is one of the configurations controlled by the system, and it may hold particular promise for future research by the International Thermonuclear Experimental Reactor (ITER) – the world’s largest nuclear fusion experiment, which is currently being built in France.

The magnetic control of these plasma forms, according to the researchers, is “one of the most demanding real-world systems to which reinforcement learning has been applied” and might pave the way for a drastic shift in how real-world tokamaks are created. Indeed, some believe that it will significantly impact the development of improved plasma control systems in fusion reactors.

Paper: https://www.nature.com/articles/s41586-021-04301-9.pdf

Reference: https://www.deepmind.com/blog/article/Accelerating-fusion-science-through-learned-plasma-control

Suggested