Journal Article

Sensor-Driven Surrogate Modeling and Control of Nonlinear Dynamical Systems Using FAE-CAE-LSTM and Deep Reinforcement Learning

Kherad, Mahdi, Fotouhi-Ghazvini, Faranak, Moayyedi, Mohammad Kazem, Vahabi, Maryam, Fotouhi, Hossein

2025

Overview

In cyber-physical systems governed by nonlinear partial differential equations (PDEs), real-time control is often limited by sparse sensor data and high-dimensional system dynamics. Deep reinforcement learning (DRL) has shown promise for controlling such systems, but training DRL agents directly on full-order simulations is computationally intensive. This paper presents a sensor-driven, non-intrusive reduced-order modeling (NIROM) framework called FAE-CAE-LSTM, which combines convolutional and fully connected autoencoders with a long short-term memory (LSTM) network. The model compresses high-dimensional states into a latent space and captures their temporal evolution. A DRL agent is trained entirely in this reduced space, interacting with the surrogate built from sensor-like spatiotemporal measurements, such as pressure and velocity fields. A CNN-MLP reward estimator provides data-driven feedback without requiring access to governing equations. The method is tested on benchmark systems including Burgers’ equation, the Kuramoto–Sivashinsky equation, and flow past a circular cylinder; accuracy is further validated on flow past a square cylinder. Experimental results show that the proposed approach achieves accurate reconstruction, robust control, and significant computational speedup over traditional simulation-based training. These findings confirm the effectiveness of the FAE-CAE-LSTM surrogate in enabling real-time, sensor-informed, scalable DRL-based control of nonlinear dynamical systems.

• Publisher: MDPI AG,MDPI
• Subject: Auto Encoders / Autoencoders / Circular Cylinders / Comparative analysis / Computer Control Systems / Control Of Dynamical System / Control Of Dynamical Systems / Control systems / Convolution / Cybe-physical Systems / Cyber-physical Systems / Deep learning / Deep Reinforcement Learning / Differential equations / Dynamical Systems / Dynamics / Embedded Systems / Explicit knowledge / Flow Control / Fluid dynamics / Fuel Additives / High-dimensional / Higher-dimensional / Long Short-term Memory / Machine learning / Measuring instruments / Methods / Neural networks / Nonlinear Dynamical Systems / Optimization / Partial differential equations / Real Time Control / Reinforcement Learning Agent / Reinforcement Learnings / Robust Control / Sensor-driven Modeling / Sensors / Short Term Memory / Simulation / Simulation methods