It wouldn’t be surprising today if you came across a comparison of a car with an electronic device. This is the result of a desire for higher levels of autonomous driving and the development of advanced driver-assistance and infotainment systems. Vehicles require a dramatic amount of power to process data flows. Automation, connectivity, and electrification are setting the tone for vehicle software development. Software engineers in the automotive industry are asked to balance the need for building a high car performance software with other automotive objectives. Is this mission possible? And if yes, how can it be achieved? Let’s find the answers.
Quality challenges in car performance software development: balancing vehicle functionality and software performance
The complexity of the software architecture under the hoods of modern vehicles has a direct impact on the car performance software. A sizable number of functional components, both within the vehicle and in the cloud, are responsible for the functioning of a car’s every capability. In a single car, automotive software that includes telematics, mapping, infotainment, various third-party apps and, of course, ADAS can be managed by up to 100 different electronic control units (ECUs) from different software providers.
As vehicles become more software-dependent, software engineers scratch their heads. A single team of software engineers can no longer build all the software for a vehicle. First, OEMs’ internal development teams need to decide which components they’re building in-house and which they’ll acquire or delegate to technology partners. It’s a common approach for part or all of a car’s application software to be supplied by an OEM or contractors while deployment of ECUs falls on the shoulders of another contractor, a subcontractor, or even the same OEM.
Next, the development team should make independent automotive software modules work together. Finally, this bulky system should move to production and show decent results. That’s why car performance software efficiency and the baseline for measuring it take center stage.
Role of software in automotive innovations
Source: McKinsey & Co
How performance testing of automotive software helps automakers measure vehicle functionality and software performance
Even though the word nightmare is in the title of this article, based on our automotive experience, we can tell what engineering bliss looks like. It’s a core platform that powers high-performance connected car services at scale while addressing critical concerns around data privacy, security, and reliability. Such a platform provides best-in-class latency and uptime, no less.
Vehicles are complex, embedded, distributed, and safety-critical systems, and maintaining performance is challenging. Engineers used to compromise vehicle functionality and software performance for infotainment systems, for example.
A realistic baseline for performance testing of automotive software that makes a difference
Performance testing of automotive software should evaluate how a system is expected to perform in production. The key performance indicators (KPIs) will depend on the perspective from which a vehicle’s performance is being tested and the goals the engineering team sets. There are countless ways of measuring car performance software efficiency, but the key is to set a realistic baseline.
While remaining vehicle- and hardware-agnostic, engineering departments need to think over baseline hardware and software requirements for the performance testing of automotive software in general or one system or service in particular. Bear in mind that today, reusability is the central factor in the success of vehicle software.
Configure your car performance software parameters so they’re the minimum required for the effective functioning of the indicator under test. Set your baseline to obtain the best possible performance with the fewest resources. Unless you’re developing software systems for premium-class cars, the mass market is where your application or system will start its journey. But thanks to reusability, it won’t be a problem to roll out your software on higher-end hardware. However, things don’t work the other way round. If you’re building premium-class software and testing it on a CPU running at maximum capacity, it will be impossible to implement that software on entry-level models with less memory, for example.
How to build a test pipeline to meet software performance KPIs
This way or another, customers with their experience are at the heart of automotive programming initiatives manufacturers and OEMs endorse. Broadly speaking, initiatives in the vehicle engineering field should prevent quality problems that affect customer safety. On the other hand, car functionality and infotainment components should keep users entertained and prevent them from looking for memorable driving experiences elsewhere.
This is one of the reasons we suggest sorting out KPIs in the following metrics that are truly pivotal to the car performance software efficiency.
Client metrics. These are focused on measuring the end user’s perception of performance, including how long it takes for in-vehicle applications to perform local processing and render results. Client metrics cover performance KPIs such as:
- Throughput, indicating how many operations the system can perform during a certain time
- Concurrency, indicating the number of operations that can be performed simultaneously
- Responsiveness and latency, measuring how long the system takes to perform an operation
Business metrics. A high rate of user satisfaction depends on the ability of manufacturers and OEMs to define users’ needs. If they’re able to think out business cases with logical operations that resonate with end users’ activities, they’ll win. In practice, software performance testing investigates whether business transactions the system performs satisfy customer demands. Think of rear-seat entertainment. Does it differ between a premium-class sedan and a mass-market Golf-class car? Sure it does. Back-seat infotainment modules are placed in premium models for the convenience of vehicle owners while someone else drives them to their destination. So testing efforts, in this case, will be focused not only on the components of the center console, like in many other vehicles, but on the rear-seat consoles as well.
System metrics. These metrics mainly cover KPIs like network, memory, and CPU use. Performance testing captures information about low-level performance of the vehicle’s infrastructure. High CPU, network, and memory use will reduce infotainment system responsiveness, harming the entire system’s performance.
Performance testing of automotive software in the CI/CD paradigm
The effectiveness of automotive software performance testing directly depends on the ability of a team or even the entire company to apply agile and DevOps methodologies. If your company has already adopted DevOps, it’s natural to do software performance testing as part of continuous integration/continuous delivery (CI/CD) releases. The increasing complexity and number of features to be integrated into automotive systems, fueled by client demands and fierce competition, force companies to shorten development cycles. Decision-makers in automotive programming have finally woken up to the fact that CI/CD is of particular value.
Being a safety-critical highly distributed system, a vehicle is a challenge for continuous delivery. The fail-fast principle may be more dramatic when applied to cars compared to a system featuring some users and a web browser. Running performance tests on embedded devices and car-specific hardware is not a cakewalk.
To avoid hardware manipulations, complex automotive systems are often tested in virtual environments. Let’s take ADAS, for example. Under the method called hardware-in-the-loop, sensors can be replaced with computers that output the same signals the sensors would produce in real-world conditions. Another computer can then broadcast signals that represent physical actions in response to those simulated sensor signals.
Hardware-in-the-loop testing in automotive means that target software can be installed into a virtual vehicle with no need for specific hardware. This way, the software can be tested live in a hypothetical driving situation on the developer’s workstation.
What can the engineering team do to avoid issues in the test pipeline?
Companies adopting DevOps should count on automated, simplified, and properly documented processes. The hardest part of testing automotive performance, though, is choosing an appropriate baseline.
Also called the 80/20 rule, the Pareto principle says that 80% of what matters for a successful outcome can be achieved by 20% of what is being done. When testing automotive systems, it would be rational to focus on 20% of the performance indicators to make them work really smoothly. Also, as we’ve suggested earlier in the article, start from the minimum required KPIs for 80% of vehicles. You’ll have a chance to be more sophisticated while configuring systems for the privileged 20% of vehicles. So use the Pareto principle while defining and executing the CI/CD roadmap for automotive products.
Source: Interaction Design Foundation
Striking a balance between KPIs on car performance software efficiency: execution speed vs memory use
Execution speed. Real-time software systems in cars must immediately react to signals. The cost of a delay in response to road signals or events can be tragic. Car systems must react within exact time constraints according to system and environmental events. Execution speed has an impact on the correct behavior of embedded real-time vehicle systems.
Memory use. This type of KPI defines what in-vehicle software uses the most memory. On top of that, it shows where resource conflicts typically take place. Finding underlying conflicts in your system can help engineers increase system performance and enhance the user experience.
Both speed and memory use are critical KPIs to ensure the performance of automotive software. There’s no winner in the execution speed vs memory usage battle, mostly because improvements in one indicator will immediately impact the other. For instance, let’s assume we’re using a caching approach to improve system performance. We store instructions and frequently used data in a pool of fast memory. This way, we get better speed, since the CPU has faster access to the instructions or data it uses regularly. But on the flip side, there’s higher memory usage. The more information you cache, the more memory you’ll need.
The balance between execution speed and memory use should be crucial in automotive programming when system performance is the goal.
The bottom line
From both a technical and organizational perspective, increasing the performance of automotive software systems is a tall task. OEMs and Tier 1 suppliers need to think over best practices for developing, testing, and deploying new features. All the efforts OEMs and Tier 1 companies put into increasing the performance of automotive software should first be measured against such vital criterion as user satisfaction.
Intellias helps clients achieve a smarter, safer, and more convenient in-vehicle experience. For more than 15 years, Intellias has been a trusted partner to OEMs and Tier 1 suppliers. Our areas of expertise cover automotive programming services for all major automotive systems including ADAS, ECUs, infotainment systems, telematics, and navigation.
Get in touch with our experts to discover how Intellias can help you increase the performance of your automotive software.
