Journal Article

Policy-Gradient and Actor-Critic Based State Representation Learning for Safe Driving of Autonomous Vehicles

Khwaja, Ahmed Shaharyar, Anpalagan, Alagan, Gupta, Abhishek, Venkatesh, Bala, Guan, Ling

2020

Overview

In this paper, we propose an environment perception framework for autonomous driving using state representation learning (SRL). Unlike existing Q-learning based methods for efficient environment perception and object detection, our proposed method takes the learning loss into account under deterministic as well as stochastic policy gradient. Through a combination of variational autoencoder (VAE), deep deterministic policy gradient (DDPG), and soft actor-critic (SAC), we focus on uninterrupted and reasonably safe autonomous driving without steering off the track for a considerable driving distance. Our proposed technique exhibits learning in autonomous vehicles under complex interactions with the environment, without being explicitly trained on driving datasets. To ensure the effectiveness of the scheme over a sustained period of time, we employ a reward-penalty based system where a negative reward is associated with an unfavourable action and a positive reward is awarded for favourable actions. The results obtained through simulations on DonKey simulator show the effectiveness of our proposed method by examining the variations in policy loss, value loss, reward function, and cumulative reward for ‘VAE+DDPG’ and ‘VAE+SAC’ over the learning process.

• Publisher: MDPI AG,MDPI
• Subject: Artificial intelligence / autonomous driving / Autonomous vehicles / deep deterministic policy gradient / Deep learning / Literature reviews / Markov analysis / Markov decision process / Neural networks / soft actor-critic / state representation learning / variational auto encoder / Wireless networks