Center for Safety Equity in Transportation

rural • isolated • tribal • indigenous

Safe Reinforcement Learning for Intersection Management in RITI Communities Under Rare Extreme Events

  • Active

    CSET Project #: 2009

    Project Funding: University of Hawaii

  • Start Date: August 2020

    End Date: July 2021

    Budget: $60265

Principal Investigator(s)

Yuanzhang Xiao

Guohui Zhang

The PI has emphasized their research on crash severity formulation, analysis, and mitigation in the transportation program at the University of Hawaii. Zhang has conducted several relevance projects: 1) Alcohol Influenced Driver Injury Severity Mitigation in Intersection-Related Crashes in New Mexico; 2) Exploratory Multinomial Logit Regression Model-based Teenage and Adult Driver Injury Severity Analyses in Rear-End Crashes; and 3) Mixed Logit Model-based Driver Injury Severity Investigations in Single- Vehicle and Multi-Vehicle Crashes on Two-lane Rural Highways. The first project provides valuable insights in driver behavior analysis under the influence of alcohol in intersection-related crashes. Findings of the second project can be beneficial to better understand the difference between teenage and adult drivers, and their specific attributes in rear-end crashes. The third project enhances our understanding of single-vehicle and multi-vehicle involved crashes and advances crash severity research methodology. All these projects would provide solid contributions to the proposed project. The modeling approaches used in those three projects can be helpful to this study. Our experiences and findings from these three projects will make the proposed project start at a higher level. Guided by the USDOT’s priorities to promote the safe, efficient and environmentally sound movement of goods and people, this project will develop crash record database and research findings are helpful for transportation agencies to develop cost-effective solutions to reduce crash severities and improve traffic safety performance in RITI communities.

Project Summary

The advances of artificial intelligence (AI) is transforming the transportation sector, one of the most critical infrastructures in modern society. AI-based technologies have been used in many facets of the transportation system, such as autonomous vehicles, driver injury prediction and prevention, and traffic management. In particular, efficient traffic management can greatly reduce traffic congestion, a problem that we all face on a daily basis. Therefore, it is of paramount importance to develop better traffic management systems, which will in turn boost the efficiency of the overall transportation system. One effective measure of traffic management is intersection management, where we
optimize the phasing of traffic signals at each intersection of the transportation network. Recently, reinforcement learning has been applied to adaptive traffic signal control, and demonstrated superior performance. In a nutshell, reinforcement learning is a branch of machine learning, and aims at learning to optimally interact with dynamic environments. In the context of adaptive traffic signal control, reinforcement learning algorithms can learn to optimally set the phasing of traffic signals under time-varying environments of traffic conditions, given enough training data.

In this project, we propose safe reinforcement learning for intelligent traffic signal control for RITI communities under rare extreme events. The key innovation behind safe reinforcement learning under significant rare events is the adjustment of rare event probabilities in the training process. More specifically, we can artificially increase the probabilities of the significant rare events (e.g., extreme weather conditions that paralyze the transportation system) in a simulator, such as the Simulation of Urban MObility (SUMO) platform. Then, we need to make proper adjustment (e.g., through importance sampling) in the learning process to account for the altered rare event probabilities. Under correct adjustment, the learned policy can be optimal under the true rare event probabilities. In addition, we will propose techniques, such as hierarchical reinforcement learning and transfer learning, to further improve the convergence speed of the proposed reinforcement learning algorithm.