Pedestrian Tracking & Collision Prediction to Enhance Mobility Safety

Business challenge

A team of high-profile AI/ML engineers at Intellias has developed a pedestrian tracking and collision prediction module as an R&D project. Harnessing advanced technologies for road and pedestrian safety has long been one of our focuses, and our new solution addresses the rising need for safe mobility in big cities.

Urban populations are growing alongside the number of personal, public transportation, and last-mile delivery vehicles. These vehicles share increasingly crowded streets with pedestrians and cyclists, who are the most vulnerable road users. Globally, pedestrians accounted for 25% of all road traffic fatalities in 2018 according to the WHO Global status report on road safety 2018.

Besides street congestion, blind spots of large buses and heavy vehicles are another cause of incidents. Drivers must maneuver quickly and accurately while staying alert to any nearby movement. Workers on foot can also be exposed to potential harm on industrial storage sites from forklifts. With a full load completely blocking the view, forklift drivers may have more blind spots than areas of clear vision.

All these pedestrian safety issues encouraged us to work on a pedestrian collision prediction module. Our skills in machine learning and AI along with experience in object detection solutions led to the success of our R&D project for predicting pedestrian collisions.

Solution delivered

Our approach to developing a pedestrian collision prediction module involved analyzing data about the position of pedestrians, their predicted locations, and road coordinates. The module we developed comprises the following components:

• Pedestrian detection
• Pedestrian trajectory prediction
• Road segmentation
• The pedestrian collision prediction module itself

Pedestrian detection is conducted with the YOLOv3 algorithm. The module for pedestrian trajectory prediction with Kalman Filter obtains the speed and velocity of a pedestrian from the detection module to predict their motion. Then the pedestrian detector searches for the best-match appearance to update measurements. The output of the previous calculation is an input for the next one. The result of the Kalman filter is an adjusted pedestrian trajectory.

The road segmentation module distinguishes driving lanes from the sidewalk and outputs images with labeled road pixels. The module is usually handled by a Convolutional Neural Network (CNN), so for our R&D project, we used Mask R-CNN trained with the CityScape dataset extended with images of Lviv.

The pedestrian collision prediction module calculates the probability of a collision using the coordinates of a pedestrian bounding box, which are predicted and obtained from the object detector paired with the Kalman filter, and the road coordinates from the road segmentation module. If these coordinates of a pedestrian bounding box do not intersect with road coordinates, the vehicle and pedestrian collision is of zero-probability. In case of an intersection, the collision probability equals the ratio of the distance to the predicted pedestrian location and the distance of detected road available for the car.

ML and AI in the automotive industry make instant road analysis possible, which is demonstrated in the video above showing pedestrian tracking software in action. The results of automatic road segmentation are highlighted in green, blue bounding boxes stand for pedestrian detection confidence, and green/red bounding boxes stand for predicted collisions with a low/high probability, respectively.

Business outcome

Active safety technology for preventing pedestrian collisions is important to protect vulnerable road users and move toward zero-traffic-accident society. Leveraging expertise in AI and ML solutions, Intellias provides software engineering services to help OEMs build intelligent vehicles that can perceive and react to road conditions up to 99.8% better than human drivers.

Being an extra pair of eyes, a pedestrian collision prediction module addresses the challenge of monitoring hazardous blind spots for drivers of both private cars and large vehicles maneuvering in narrow lanes and around people on foot. Once the possibility of hitting a pedestrian is detected, a driver will receive a visual and audio alert, thus getting more time to react. If the situation is defined as critical, the brakes will be applied automatically.

The advanced pedestrian trajectory prediction module we’ve developed will support automakers in enhancing the safety parameters of vehicles in urban, rural, and industrial environments. By equipping fleets with a pedestrian collision prediction module, companies will reduce incidents and associated costs and, most importantly, save lives.