Journal Article

Adaptive Control of Nonlinear Non-Minimum Phase Systems Using Actor–Critic Reinforcement Learning

Jouili, Khalil, Charfeddine, Monia, Ben Moussa, Mongi

2025

Overview

This study introduces a novel control strategy tailored to nonlinear systems with non-minimum phase (NMP) characteristics. The framework leverages reinforcement learning within a cascade control architecture that integrates an Actor–Critic structure. Controlling NMP systems poses significant challenges due to the inherent instability of their internal dynamics, which hinders effective output tracking. To address this, the system is reformulated using the Byrnes–Isidori normal form, allowing the decoupling of the input–output pathway from the internal system behavior. The proposed control architecture consists of two nested loops: an inner loop that applies input–output feedback linearization to ensure accurate tracking performance, and an outer loop that constructs reference signals to stabilize the internal dynamics. A key innovation in this design lies in the incorporation of symmetry principles observed in both system behavior and control objectives. By identifying and utilizing these symmetrical structures, the learning algorithm can be guided toward more efficient and generalized policy solutions, enhancing robustness. Rather than relying on classical static optimization techniques, the method employs a learning-based strategy inspired by previous gradient-based approaches. In this setup, the Actor—modeled as a multilayer perceptron (MLP)—learns a time-varying control policy for generating intermediate reference signals, while the Critic evaluates the policy’s performance using Temporal Difference (TD) learning. The proposed methodology is validated through simulations on the well-known Inverted Pendulum system. The results demonstrate significant improvements in tracking accuracy, smoother control signals, and enhanced internal stability compared to conventional methods. These findings highlight the potential of Actor–Critic reinforcement learning, especially when symmetry is exploited, to enable intelligent and adaptive control of complex nonlinear systems.

• Publisher: MDPI AG
• Subject: Accuracy / Adaptive control / Analysis / Canonical forms / Cascade control / Control engineering / Control systems / Controllers / Coordinate transformations / Data mining / Decoupling / Feedback linearization / Learning / Machine learning / Mathematical optimization / Multilayer perceptrons / Nested loops / Nonlinear control / Nonlinear systems / Output feedback / Reference signals / Symmetry / Time varying control / Tracking