Journal Article

Autonomous Trajectory Planning Method for Stratospheric Airship Regional Station-Keeping Based on Deep Reinforcement Learning

Liu, Sitong, Cui, Yuxuan, Lu, Ying, Shang, Hai, Zhou, Shuyu, Miao, Jinggang

2024

Overview

The stratospheric airship, as a near-space vehicle, is increasingly utilized in scientific exploration and Earth observation due to its long endurance and regional observation capabilities. However, due to the complex characteristics of the stratospheric wind field environment, trajectory planning for stratospheric airships is a significant challenge. Unlike lower atmospheric levels, the stratosphere presents a wind field characterized by significant variability in wind speed and direction, which can drastically affect the stability of the airship’s trajectory. Recent advances in deep reinforcement learning (DRL) have presented promising avenues for trajectory planning. DRL algorithms have demonstrated the ability to learn complex control strategies autonomously by interacting with the environment. In particular, the proximal policy optimization (PPO) algorithm has shown effectiveness in continuous control tasks and is well suited to the non-linear, high-dimensional problem of trajectory planning in dynamic environments. This paper proposes a trajectory planning method for stratospheric airships based on the PPO algorithm. The primary contributions of this paper include establishing a continuous action space model for stratospheric airship motion; enabling more precise control and adjustments across a broader range of actions; integrating time-varying wind field data into the reinforcement learning environment; enhancing the policy network’s adaptability and generalization to various environmental conditions; and enabling the algorithm to automatically adjust and optimize flight paths in real time using wind speed information, reducing the need for human intervention. Experimental results show that, within its wind resistance capability, the airship can achieve long-duration regional station-keeping, with a maximum station-keeping time ratio (STR) of up to 0.997.

• Publisher: MDPI AG
• Subject: Adaptability / Aircraft / Airships / Algorithms / Altitude / Control tasks / Decision making / Deep learning / deep reinforcement learning / Doppler effect / Environmental conditions / Kinematics / Machine learning / Methods / Nonlinear control / Optimization algorithms / proximal policy optimization (PPO) / Regional planning / regional station-keeping / Robotics / Space ships / Space vehicles / Stationkeeping / Stratosphere / stratospheric airship / Stratospheric winds / Task complexity / Trajectory planning / Wind resistance / Wind speed