This on-going project aims to develop autonomous car-following strategies based on deep Reinforcement Learning (deep RL).
Deep Reinforcement Learning
Deep RL is a field that seeks to combine the advances in deep neural networks with reinforcement learning algorithms to create agents capable of acting intelligently in complex environments, and exciting breakthroughs have been witnessed, like deep Q-network and AlphaGo.
Deep reinforcement learning can achieve better generalization capability because it learns decision-making mechanisms from training data rather than parameter estimation through fitting the data.
Autonomous Car Following
The critical element in developing autonomous car-following strategies by reinforcement learning is the design of the reward function. Currently, we have developed two types of reward functions: 1) reward function reflecting deviations from human driving performance, leading to human-like car following; and 2) reward function reflecting driving efficiency, safety, and comfort, leading to multi-objective autonomous car-following.
The following demo animation shows the car-following learning process of RL agents. In early training episodes, cars become red frequently, indicating penalties caused by bad car-following performance, e.g., rear-end crashes. As training converges, the RL agents maintain steady car-following headways, and receive fewer penalties.
Demonstration of the car-following learning process of RL agents