3D Point Cloud Classification and Object Labeling

Business challenge

Our client is an advisor and a provider of artificial intelligence (AI) technologies for competitive and customized data science and machine learning solutions. While providing geospatial artificial intelligence (GeoAI) services, the client needed to build intelligent point cloud processing software for managing and classifying big data sets in a 3D point cloud. The most challenging part of this project (with which our client sought assistance) was classifying and labeling nearly 150 million 3D points to distinguish between different types of objects: cars, roads, trees, and so on. Labeling these objects would allow users to see the entire picture of a scanned area. Finally, users should be able to render, view, and choose classified objects in a web browser.

Initially, Intellias worked with this client on machine learning projects. Our big data analytics expertise qualified us to develop the client’s new 3D point cloud processing software from scratch.

3D point cloud solution delivered

We started by developing an MVP of the point cloud processing software and continued to full-cycle development with continuous delivery of custom features for end customers. In the scope of this 3D cloud point project, we also took responsibility for designing a unique user experience (UX). Our specialists were involved at all stages of implementing the 3D point cloud processing software, from designing the architecture and selecting core components to creating UI/UX prototypes, developing customer-side functionality, testing, deploying the cloud infrastructure, and finally launching the product.

The point cloud processing product we’ve been building processes and classifies 3D point cloud input data collected from Lidar sensors installed on vehicles, aerial photos from helicopters and drones, and portable stations set up throughout a scanned area. After data has been collected, it’s gathered into big data sets for further Lidar point cloud processing. Our client receives 3D point clouds from customers for processing, segmentation, and classification; in the future, they plan to render all this data in web maps applying custom mapping services.

Our client’s point cloud processing software can recognize objects in previously selected segments and classify them by criteria such as shape and location and whether they’re mobile or static. In such a way, our client’s 3D point cloud software can classify objects like road signs, trees, roads, vehicles, buildings, bridges, and traffic lights. This 3D point cloud technology can cope with billions of data points, classifying them as real-world objects and assigning characteristics like color, geographic coordinates, angle, and speed. After objects are labeled, users can build and view a 3D model of an entire city or any landscape that needs to be digitally represented and render it right in a web browser. After successful classification of cloud points, these 3D models can be used for smart city planning, construction, architecture, and infrastructure development.

How point cloud processing works in practice

Upload and segment the point cloud

First, customers upload files with data points to the point cloud processing service, where an AI engine automatically generates segments of different objects. At this stage, the system cannot sort objects by categories, but it prepares large segments for further labeling and annotation.

Classification and labeling

A segmented 3D point cloud serves as the input for manual annotation and labeling. Manually marking large data sets that include nearly a billion points could take forever. So the data points are labeled using certain object determiners. Users can click on one object (for example, a tree) and the classifier will automatically highlight all nearby trees in green. The same algorithm works for other objects: ground, roads, infrastructure, cars. Click on one object and the classifier marks all nearby objects with determinators similar to those of the chosen object. In this way, the classifier groups similar points into object categories for distinguishing objects one from another.

Render, view, and export

After all objects in the 3D point cloud are recognized and the whole 3D terrain model is available for the user’s viewing, the system offers the possibility to view a particular object (for example, a road) separately from other objects. This helps municipalities or construction firms plan and optimize road infrastructure without being distracted by other objects in the terrain model. Each view can be rendered right in a web browser. Customers can export and use these models with third-party software for city planning, construction, and landscape design.

Key features of point cloud processing software

• Simplifies the classification of objects and creation of 3D models with visual editing tools
• Guarantees near millimeter accuracy of 3D point cloud classification and object recognition
• Increases classifier computing power thanks to the Microsoft Azure technology stack
• Processes big data sets quickly thanks to an elastic and scalable architecture
• Offers a friendly user interface with customizable views of GIS analytics and spatial data

Results

Throughout the project, our client’s 3D point cloud technology has been evolving into a complete solution that relies on comprehensive machine learning algorithms for automated classification of 3D point cloud data. Its computing power allows end customers to process big data sets that consist of 150 million points and weigh up to 5 gigabytes with remarkable speed, enabling users to render 3D models in a browser and choose classes in seconds.

Our client’s point cloud processing product has received positive feedback after being presented at several tech conferences. Its UX was recognized as the most user-friendly among competitors in the field of 3D cloud point classification.

Currently, our client provides 3D point cloud processing services that allow end customers to upload custom and specific point clouds for applying 3D point cloud data in a wide range of industry use cases.

Use case 1: Landscape design and city planning

The customer may need to reshape a large landscape for a park zone with bicycle paths on hillsides and green spaces. For this, they may need to raise, level, and lower the landscape in different areas to adjust the terrain to the necessary angles. The legacy approach would suggest digging soil for valleys, transporting it, and then looking for more soil to pile up hills. Using 3D point cloud processing software, the customer can get a precise 3D model of terrain in advance. With this digital model, they can plan to dig the exact amount of soil for valleys that will be used for other needs like piling up hills. This approach saves time and money on transporting soil from other places.

Use case 2: Road construction and monitoring

The customer may need to rebuild a stretch of road and crossways. They may have found a vendor who has offered them a specification with the total amount of building materials required. But halfway through this construction project, the vendor might suggest adding more materials. Materials used can be difficult to monitor, resulting in cost overhead. Using a 3D point cloud model would help the customer accurately calculate the required number of sidewalk tiles and other materials to plan a precise budget and schedule.

Use case 3: Updating road signs

After road reconstruction and installation of new road signs, the customer from Use case 2 might have to update signs and new traffic intersections on public maps. Doing so could require months to collect images and scans of the newly built area. Instead, by applying Lidar point cloud processing during construction, the customer could already have a 3D model with all objects recognized to accurately place new signs on any public map.