Lane Changing of Autonomous Vehicles in Mixed Traffic Environments: A Reinforcement Learning Approach
Overview
According to an accident report from Volvo, lane changing maneuver is the main cause of various severe highway accidents, due to human drivers’ inaccurate estimation and prediction of the surrounding traffic, illegal maneuver, and inefficient driving skill. Autonomous driving is regarded as a solution to reduce these human errors. At present, there are many obstacles to developing automated lane-changing technology, including dynamically changing interactions between vehicles, complex routing choices, and the interactions between vehicles and the environment.
In this proposal, in order to guarantee the safety of autonomous vehicles (AV), improve passenger comfort, and increase traffic efficiency, we aim to develop innovative learning-based control methods for lane changing of connected and autonomous vehicles (CAVs) in mixed traffic by a combined use of reinforcement learning and optimal control techniques.
Conventionally, automated lane changing consists of three phases: surrounding environment detection and prediction, trajectory planning, and vehicle control. With the predicted states of the surrounding vehicles and the environment, the aim of the trajectory planning module is to plan a safe trajectory from the CAV’s original position in the current lane to the desired position in the target lane.
For the low-level vehicle control, the aim is to design a trajectory tracking controller to track the aforementioned trajectory. Most of the existing controllers are developed by model-based approaches, such as sliding mode control and model predictive control, where the accurate dynamic model of the CAV is indispensable. However, system modeling is time-consuming, costly, and often involves modeling errors. A controller designed based on an inaccurate model may destabilize the vehicle system, which threatens the safety of the CAV and its surrounding traffic. Robust control can stabilize the vehicle system if there are errors or uncertainties in the model, but this compromises the system’s
transient response.
Research Objectives & Deliverables
In this proposal, we will extend our recent work from lane-keeping to lane changing, by developing data-driven adaptive learning algorithms to achieve the optimal lane-changing maneuver in the mixed traffic. Novel learning-based vehicle lane changing techniques will be proposed based on reinforcement learning and optimal control techniques. Once the lane changing task is completed, the CAV automatically switches to our previously developed learning-based controls for lane keeping.
Simulation of Urban MObility (SUMO) simulation platform will be used while the trajectories of HDVs in the mixed traffic will be obtained from the next generation simulation (NGSIM) or similar realistic datasets with actual HDV trajectories to demonstrate that our results are not a simple artifact of the simulation models that are being used.
Deliverables:
- Learning-based control algorithms for lane changing and lane-keeping for CAVs in mixed traffic environments
- Implementation of learning-based lane-changing control algorithms in SUMO simulations
- Experimental testing and validation in small-scale vehicles
- Lesson plans for NYC DOT staff, students, and junior faculty
- Final report
Personnel
| Principal Investigator | Jhong-Ping Jiang |
| Funding Source | Request from C2SMART: $63,990 Cost share from NYU: $35,000 |
| Total Project Cost | $98,990 |
| USDOT Award # | 69A3551747124 |
| Implementation of Research Outcomes | To validate the performance of the proposed learning-based adaptive optimal controllers, the Simulation of Urban MObility (SUMO) simulation platform will be used in a setting where the trajectories of human-driven vehicles (HDV) are obtained from the next generation simulation (NGSIM) and other realistic datasets. Additional experiments to further incorporate hardware and communication-related disturbances that cannot be easily simulated in a microscopic traffic simulation will be carried out in the PI’s Control and Networks (CAN) Lab using three small-scale autonomous vehicles equipped with on-board sensors. |
| Impacts/Benefits of Implementation | This research will significantly extend our previous work in data-driven control from lane-keeping to lane changing under constraints. Practical implementation in CAVs either in SUMO or real-world experiments is a necessary intermediate step toward the transition of advanced vehicle control algorithms to practice for the wide deployment of self-driving cars in real traffic networks. This interdisciplinary project provides an opportunity to train undergraduates and graduate students in both theory and applications of automatic control and reinforcement learning through microscopic simulations and experimental validations in real-world test scenarios and small-scale vehicular platoon. |