Deep learning despite sparse data
The German Research Foundation (DFG) is funding the new research group “Deep Learning on sparse chemical process data” for four years with around 3.5 million euros. It is coordinated at the Technical University of Kaiserslautern (TUK). The group works to make Deep Learning methods, a subfield of artificial intelligence, usable for the chemical industry. So far, this has not been possible, in particular due to the limited data available. The interdisciplinary computer science and process engineering team develops new methods for this purpose. These are intended to help detect errors in chemical processes at an early stage in order to avoid accidents and stoppages.
Search millions of images in fractions of a second for a keyword like “beach” or use voice recognition to play a song for you on your streaming service. Deep learning – a branch of artificial intelligence – makes such things possible. With the help of huge amounts of data, algorithms learn; they classify, sort and filter data. The technology is used in many fields, such as medicine, agriculture and robotics.
However, this has not yet been the case for processes in the chemical industry. “There’s a lot less data here than, for example, in a web image search, and in some cases there’s no data at all or the companies don’t make it available,” explains Professor Dr. Marius Kloft, who heads the Intelligent Systems Department and the Machine Learning Working Group at TU Kaiserslautern and is the spokesperson for the new DFG research group. “Furthermore, often all the data also looks the same, which makes learning here much more difficult.” For example, in a chemical plant, the same process is always performed under the same conditions over a long period of time, such as converting raw materials into products. The sensors then always measure the same temperature, pressure, etc.
It is precisely in these areas that the new research group nevertheless wishes to apply deep learning methods. The group benefits from its interdisciplinary approach. Otherwise, the development of these new methods would not be possible. In addition to computer scientist Kloft and computer science professor Dr. Heike Leitte, the group includes TUK process engineering professor Dr. Hans Hasse, as well as junior professor Dr. Fabian Jirasek, who conducts research on the machine learning in process engineering, and junior professor Dr Sophie Fellenz, who also works on machine learning. The peculiarity of Fellenz and Jirasek is that the Carl Zeiss Foundation finances their positions as junior professors in tandem with 1.5 million euros; their common goal is to bring machine learning closer to physical modeling. The new research group DFG now also benefits from this. Under a Mercator Fellowship, the project will also involve computer science professor Dr. Stephan Mandt from the University of California, Irvine, whose research includes applications of machine learning in the natural sciences.
“For us, the focus is less on the number of data and more on the quality of the data,” continues Kloft. The team takes two approaches. First, it performs chemical processes in the laboratory itself to collect experimental data; second, it also generates synthetic data. “Every experiment is complex and expensive, so our main concern is to collect the right data, that is, the data from which deep learning methods can learn the most,” adds Jirasek. For this purpose, a so-called batch distillation facility is used on the Kaiserslautern campus to collect data from dynamic processes. In addition, at the project partner, the Technical University of Munich, a plant for the continuous production of synthetic fuels is operated on a pilot scale at the Straubing site.
“These plants are equipped with various sensors to record a wide range of data, such as pressure and temperature, but also videos of the internal workings or the composition of the mixtures used”, continues the junior professor.
In addition to laboratory data, other data is generated using physical simulations and artificial intelligence methods. A particular challenge here is to make them as realistic as possible. “We take two approaches to this,” says Assistant Professor Fellenz. “First, we develop methods to learn the style from experimental data, and then apply it to simulation data.” Such methods already exist to change the tone of texts. In the research group, they are now applied to chronological data of chemical processes, which can be understood as sequential data, just like words in a text. “On the other hand, we integrate physical laws, for example from thermodynamics, directly into our models, so that we can also generate realistic data from areas where we have no measurements”, continues the computer scientist. .
The goal for the next four years is to use new deep learning methods to detect anomalies or faults in chemical plants at an early stage, but also to identify appropriate countermeasures. This is of great practical importance, since any failure of a power plant is at least costly and, in the worst case, represents a danger to people and the environment.
Examples abound, such as in 2013 when an explosion occurred at a chemical plant in Geismar, Louisiana, USA, killing one person and injuring several. The cause was a fault in a heat exchanger. “Particularly in such factories, anomalies in the process can have disastrous consequences,” continues Kloft. Deep learning methods could help here to automatically detect such faults in order to issue a warning message in time. “Our anomaly detectors should be much more sensitive than conventional techniques thanks to Deep Learning.”
The object of the work is also to explain the information, to visualize it and to present it clearly. They must be simple and quick to understand so that professionals in the chemical industry can use these techniques and meet the appropriate technology recommendations. The team will also work on verifying its methods. This will involve using mathematical methods to verify that the algorithms work correctly. Kloft continues, “Through our work, we are generating a whole new set of data that we will make available to others.”
Professor Dr. Werner R. Thiel, Vice President for Research and Technology at TUK, emphasizes the importance of the new DFG research group: “We were able to lay the foundations for this success with our two junior tandem chairs by the Carl Zeiss Foundation. We bring our expertise in computer science and process engineering to the project. Without this combination, such a project would not be possible. This is an advantage of our research location, which is interdisciplinary in many fields. This now pays off again. I warmly congratulate everyone involved.
Rhineland-Palatinate Minister of Science Clemens Hoch also congratulates: “I am very satisfied with the success of the researchers from Kaiserslautern and their cooperation partners from Oldenburg and Straubing. The DFG-funded research group is proof of TU’s research strength in this highly topical area of research and fits perfectly into the state’s AI strategy. This project once again clearly demonstrates that the key technology of artificial intelligence can make positive and performance-enhancing contributions across a wide range of scientific disciplines and applications. Therefore, I am very happy that we in Rhineland-Palatinate are fortunate to have leading AI researchers, such as Professor Kloft and his colleagues, in our institutions whose innovative projects are developing new possibilities for the use of artificial intelligence in Rhineland-Palatinate and beyond.”
The long-term goal of the research group is to develop methods for the autonomous operation of plants in the chemical industry. With its work, the group also aims to advance process simulation in process engineering by developing new types of tools and integrating data types that are not even currently considered.
In addition to the working groups at TU Kaiserslautern, the teams of Dr. Michael Bortz from the Fraunhofer Institute for Industrial Mathematics ITWM in Kaiserslautern, Prof. Dr. Jakob Burger from TU Munich and Prof. Dr. Daniel Neider from the University of Oldenburg are involved in the project. Additionally, Professor Dr. Stephan Mandt from the University of California at Irvine is involved as a Mercator Fellow.