Work Zone Safety III: Calibration of Safety Notifications through Reinforcement Learning and Eye Tracking
Overview
According to the Federal Highway Administration (FHWA), work zone fatalities at road construction projects account for up to 3% of all workplace fatalities in a given year [1], and the primary causes are runovers/backovers, collisions, and caught in-between mobile equipment. Hence, drivers and the way they perceive the work zone and related notifications are primary factors required to reduce fatalities. A study of work zone crash data in five states showed that around half of the crashes occur within or adjacent to work activities, putting workers in danger together with drivers [2]. To reduce work zone injuries and fatalities, regulations such as mandated Personal Protective Equipment (PPE), traffic control plans, advance warning signs, the share of traveler information, and signal timing adjustments (ANSI, OSHA) were introduced by the regulatory bodies. However, these mainly aim for changing the behavior of drivers instead of workers. Although there is a large body of analysis and modeling literature related to work zone accidents as documented in [3], the actual safety treatments applicable to real-world work zones are limited at best and there is still a need for proactive approaches to be deployed at highway work zones, capable of warning construction workers of approaching hazards in advance.
To improve work zone safety, in the previous two phases of this project, we proposed a virtual reality (VR)-based platform that integrates with SUMO and hardware in the loop sensors to realistically simulate dangerous situations in work zones (i.e., enabling worker-initiated changes in the work zone to be accounted in SUMO and updated simulation to be displayed real-time in VR). In this phase, we propose to add two main components to the existing VR work zone safety testing platform. The first component focuses on monitoring construction workers’ attention. To that end, we propose adding new functionality to the current VR platform to track the subjects’ attention through his/her head-movement and eye-movement to infer his/her gaze pattern. With the introduction of this method to measure the subject’s attention, we plan to capture additional critical information about the decision a worker makes.
Research Objectives & Deliverables
In the third phase of this work zone safety project, we aim to calibrate the frequency, timing, and modality of alarms through (1) the introduction of an eye-tracking component to the existing VR platform, and (2) the introduction of a reinforcement learning-based model to optimize the calibration of alarms.
Specifically, we have the following research objectives:
- Understand the construction workers’ attention to work zone conditions (e.g., vehicle head direction/speed toward work zone, worker vicinity to work zone perimeters) when an alarm is delivered by tracking the head-movement and eye-movement of subjects in VR.
- Evaluate the effectiveness of alarms sent in different modalities, frequencies and timings to the workers and determine the key factors that influence the reactions of workers to alarms.
- Develop an alarm delivery system that is tailored to maximize workers’ attention to the alarms sent regarding dangerous situations using reinforcement learning.
Deliverables for this phase of the project will include:
- An implementation of an attendance monitoring system embedded in the current VR platform to measure subjects’ gaze patterns when a dangerous situation is simulated in the VR.
- The evaluation of whether data arriving with different delays/frequency from the sensors and wearables are meant to correlate with human responses to notifications.
- The development of an alarm delivery system that is trained to maximize the attention of the workers towards alarms sent regarding potentially dangerous situations in the work zone.
- A report describing the RL-based model for calibration of notifications, and worker attention level to dangerous situations in highway work zones.
Personnel
| Principal Investigator | Semiha Ergan |
| Funding Source | Request from C2SMART: $60,000 50% cost share from NYU Department of Civil and Urban Engineering: $60,000 |
| Total Project Cost | $120,000 |
| USDOT Award # | 69A3551747124 |
| Implementation of Research Outcomes | The outcome of this research will help to calibrate when, at what frequency, and how to (with what modalities) share warnings from simulated virtual environments as well as real vehicles and sensors with workers involved in active work zones for effective responses towards the reduction of incidents. The reinforcement learning model trained from the user studies data can be deployed into physical work zones for alarm delivery to notify construction workers on-site for potentially dangerous situations. |
| Impacts/Benefits of Implementation | The proposed research can potentially help mitigate highway work zone incidents (i.e., injuries, fatalities, and near misses), in particular, leveraging the eye-tracking component, we can continuously monitor works’ attention (e.g., where exactly a worker was paying attention when s/he receives a notification and resulting decision on how to acknowledge that notification) and determine when, how and at what frequency to push alarms/warnings/notifications to workers at work zones. The proposed VR-based hazard simulation can serve as a safe way to capture worker response under hazardous conditions without putting them in danger while capturing a realistic representation of real-world work zones from the integrated hardware (i.e., sensors and vehicles). The reinforcement learning model trained from the user study data can be deployed to real work zones for alarm delivery to mitigate incidents. |