Cost-Effective 3D Object Detection for Autonomous Vehicles

Business challenge

The rapid evolution of technology is prompting all industries, and especially the automotive industry, to adopt innovative products. Although self-driving cars have great potential, they still face a lot of challenges. The biggest are related to road perception and cost-effectiveness. Intellias has addressed market demand and invested in autonomous driving technology by enhancing object recognition using computer vision.

Modern solutions for self-driving cars use Lidar hardware to obtain three-dimensional information about the position of objects. This hardware is extremely costly but is required to feed data to robust object recognition software. Our team of artificial intelligence and machine learning developers discovered a more cost-effective approach to acquiring the same data with stereo camera equipment and a mathematical algorithm that simulates Lidar data outputs.

Solution delivered

We set up an R&D project with the aim of validating the use of stereovision cameras as an alternative to Lidar/radar sensors as the primary data source for object recognition algorithms.

Our approach comprises two stages. The first involves an algorithm applied on images taken with a pair of calibrated cameras to get a disparity map. This disparity map is then processed by another algorithm that transforms it into a depth map. Based on depth and the vertical focal length, we acquire the 3D location of each pixel. After backprojecting every pixel in 3D coordinates, a 3D point cloud is acquired that imitates the Lidar signal.

The second stage involves training existing Lidar-based 3D object detection pipelines directly on Lidar-simulation datasets. For that purpose, we combine initial visual information with Lidar-like representations transformed to a bird’s-eye view (BEV) image. The algorithm relates visual features and features of the BEV representation to 3D box proposals and then merges both to perform box classification and regression.

A significant increase in the accuracy of image-based object detection with deep learning algorithms is achieved by changing the 3D depth representation to a Lidar simulation. Hence, this solution can be integrated into ADAS systems in autonomous vehicles.

This project proved that stereoscopic vision combined with object recognition AI-based algorithm can output data of comparable quality to that of actual Lidar/radar sensors. Moreover, simulated Lidar data has an advantage of higher signal density than actual Lidar data, so the strengths of these data modalities can be complementary.

Business outcome

Object recognition based on stereoscopic vision and AI/ML algorithms contributes to the global change in the domain of maps compilation. By developing such R&D projects, Intellias fulfills its aim of being a technological enabler. Lidar simulation technology promises to increase the cost-effectiveness of object detection, which is the principle holdup to developing self-driving vehicle capabilities. With this approach, it is highly possible that image-based 3D object recognition for automated and autonomous driving will become a default feature even for non-premium cars.

The affordability of self-driving vehicles could also make a difference for people with disabilities, enabling them to get a driver’s license and no longer have trouble commuting to work, going to the store, and getting to doctor’s appointments. It would also be a real game changer for shared mobility by reducing fixed costs and, in turn, increasing the number of car-sharing services.

Business benefits of stereoscopic vision go beyond the automotive industry. Robotics and sensor systems, for instance, can benefit from it too. Most robots are still operated with prebuilt commands and programs. With 3D object recognition software, any service robot can assist humans in performing ordinary tasks more effectively by being able to perceive, locate, and interact with different objects, therefore planning its movements and executing sustainable operations.

It is no longer a question of if autonomous cars will be in common use but rather when. At Intellias, we’re contributing to bringing the autonomous future closer.