Back in the day, dashboard elements such as a speedometer, tachometer, and fuel gauge felt like an absolute novelty. Today’s car buyers are far harder to impress. Touchscreen displays, real-time in-car navigation, a variety of infotainment options, and even business tools are what drivers have come to expect from premium vehicles and more budget-friendly alternatives alike.
In the nearest future, the automotive industry will add more than 44 million new display systems, enabling all-new content flexibility for automakers to attract new consumers.
Vehicle connectivity is now a given. Over 60% of US drivers and 24% of EU drivers used connected vehicles in 2021. By 2025, the numbers are expected to increase to 70% and 49%, respectively.
Built-in sensors, local wireless networks, and SIM network connectivity, coupled with software-defined architectures, have enabled an unprecedented level of in-car innovation. Nearly every OEM now has a vision for a fully computerized car.
Mercedes-Benz plans to offer “supercomputer-like performance” in every car with automated driving sensors and advanced navigation technology. GM plans to unveil an end-to-end software platform, Ultifi, in 2023, which will enable new in-car subscription services and over-the-air (OTA) software updates.
Stellantis has an ambitious Dare Forward 2030 strategic plan, calling for major digitization across the brand portfolio and the launch of a unified software platform, which has a target of €20 billion in incremental software-enabled revenue.
Less than a decade ago, only premium vehicles contained 100 electronic control units (ECUs) and 100 million lines of code. Today, every low-end vehicle hosts an equivalent number of technologies, as Advanced Driver Assistance Systems (ADAS) have become standard.
To further capitalize on car computerization, however, OEMs will need to get two components right — backend system implementation and human machine interface (HMI) design.
HMI design is key to selling the connected car experience
A car HMI is made up of functional elements (such as the vehicle dashboard, head-up display (HUD), and instrument cluster), as well as different interaction modalities (touch, voice, haptic, eye movement) that enable effective driver experiences.
In the best-case scenario, poor interactivity upsets the user; in the worst-case scenario, it can create dangerous on-road situations.
That’s why a third of an ECU’s software is now dedicated to ensuring quality vehicle performance and powers safety-critical systems such as emergency braking, airbag deployment, traction control, and collision warning.
Effective HMI design helps users drive safer by progressively discovering new in-car features. Users’ feedback must be effectively captured and communicated back to the machine, which, in turn, must adapt its performance.
As one of the ten usability heuristics for UI design suggests: “The design should always keep users informed about what is going on, through appropriate feedback within a reasonable amount of time.”
HMI shouldn’t add extra complexity. On the contrary, it should make the user experience more intuitive.
Because here’s the deal: Users don’t always understand the perks of car connectivity.
An FIA survey found that in France, 70% of respondents claimed to have knowledge about connected cars, but only 59% could accurately define what they are.
Likewise, not all consumers are eager to share data about their driving behavior or preferences in return for access to services. This indicates a gap in understanding how in-vehicle connectivity generates extra value and how automakers will guarantee the privacy of users data.
Level of consumers interest in connected vehicle features
| Connected vehicle features | China | Germany | India | Japan | Rep. of Korea | Southeast Asia | US |
|---|---|---|---|---|---|---|---|
| Maintenance updates and vehicle health reporting/alerts | 80% | 56% | 84% | 62% | 66% | 81% | 60% |
| Updates regarding traffic congestion and suggested alternate routes | 78% | 58% | 83% | 66% | 75% | 81% | 58% |
| Updates to improve road safety and prevent potential collisions | 80% | 53% | 84% | 66% | 72% | 81% | 57% |
| Suggestions regarding safer routes (i.e., avoid unpaved roads) | 82% | 43% | 85% | 63% | 68% | 78% | 54% |
| Maintenance cost forecasts based on your driving habits | 81% | 47% | 81% | 54% | 61% | 78% | 53% |
| Customized suggestions regarding ways to minimize service expenses | 80% | 46% | 81% | 62% | 74% | 77% | 50% |
| Over-the-air vehicle software updates that correct or improve your driving experience |
76% | 37% | 80% | 50% | 62% | 74% | 49% |
| Access to nearby parking (i.e., availability, booking, and payment) | 78% | 51% | 82% | 60% | 68% | 76% | 48% |
| Receiving a discount for access to a Wi-Fi connection in your vehicle | 77% | 38% | 78% | 57% | 60% | 71% | 47% |
| Customized/optimized vehicle insurance plan (e.g., “pay how you drive” plans) | 76% | 46% | 79% | 47% | 65% | 72% | 46% |
| Special offers regarding non-automotive products and services related to your journey or destination | 76% | 32% | 79% | 49% | 55% | 69% | 40% |
Source: Deloitte — 2023 Global Automotive Customer Survey
The challenge of great HMI design is to progressively introduce consumers to the value of new connected services and features until they have an aha! moment — a breakthrough realization of the beauty, efficiency, and attractiveness of the HMI solution.
Drivers aren’t always ready to pay more for OTA maintenance updates, but they sure want to shell out extra for greater vehicle safety and reliability. Likewise, HD maps may sound less attractive than the promise of higher fuel/energy efficiency.
HMI design helps connect the dots between new in-car features and real-world benefits customers are seeking.
HMI design: hardware and software solutions for a delightful driver experience
HMI design includes two subjects: the user and the connected vehicle. The goal is to establish an effective process input request and precise response.
Source: MDPI — Developing a Multimodal HMI Design Framework for Automotive Wellness in Autonomous Vehicles
Great HMI experiences are:
- Multimodal: Drivers can interact with the vehicle via visual, auditory, haptic, olfactory, and multi-channel cues.
- Connected: Vehicle sensors, telematics, and internal/external connectivity enable drivers to better interact with the surrounding environment.
- Safe: The experiences prioritize driver and passenger safety, minimizing distractions and unsafe behaviors.
To achieve the above synergy, OEMs need to plan HMI design on three levels:
- Hardware
- Middleware
- Software (user-facing elements)
Hardware platform
Since 2007, the cost of semiconductor-based electronic systems in vehicles has doubled and will approach 50% of the total car value by 2030.
Modern automotive systems consist of over 100 sensors and up to 3,000 chips of different types. But it’s not always the more hardware, the merrier.
The growing number of ECUs, sensors, and lines of code increases vehicle manufacturing complexity (and, consequently, costs). According to Deloitte, systems integration, testing, verification, and validation now comprise 40% or more of the total vehicle development budget.
Whereas McKinsey estimated that automotive software development complexity increased by a factor of 4 over the past ten years, software development productivity only grew by a factor of 1 to 1.5. New in-car systems such as ADAS require meticulous work on both the hardware and software levels and take more time to develop than traditional embedded software.
The panoply of sensors and ECUs also complicates development. An Aurora Labs survey found that automotive software developers are facing mounting difficulties in establishing how code changes in one ECU will affect others.
How difficult is it to know when a change in the code of one ECU affects another ECU?
Source: Aurora Labs — 2020 Automotive Software Survey
To deal with the complexities of automotive software development, OEMs establish new approaches to electrical/electronic (E/E) architecture design.
Instead of using multiple ECUs, E/E architectures promote the creation of specific zone ECUs responsible for certain features (such as infotainment or navigation).
A zone-oriented E/E architecture facilitates the independence of sensors and actuators from the central vehicle computing unit. In simpler terms, the hardware and software levels are decoupled, meaning the update cycles for each can differ. Moreover, zone architectures drastically reduce the total number of ECUs required, as well as the in-car cable length. Bosch, for example, is developing a new E/E architecture that contains 20% fewer embedded control units but doesn’t have any loss in performance.
To further improve the performance of E/E, Intellias recommends embedded virtualization at the hardware level. In this case, you assign different hardware resources to multiple in-car operating systems, which in turn power various in-car applications (navigation, entertainment, ADAS, etc.).
Overview of virtual machine and containerization approaches
Your goal is to establish the key control points for each in-car system. These will serve as the starting points for determining where to focus design and development and which unique hardware and software assets to exploit.
This way, OEMs can maximize the value of available hardware by augmenting its powers with best-fit digital technologies.
Middleware
Middleware — the connective tissue between applications, data, and vehicle hardware — is also gaining traction in automotive HMI development.
Middleware can dramatically reduce the duration and complexity of developing new digital car systems with:
- Streamlined integrations
- Added standardization
- Easier testing
- Higher maintainability
- Improved security
Let’s take infotainment systems. On average, OEMs spend up to three years developing a new in-vehicle infotainment (IVI) system, with several hundred people working on each interaction. Between 30% and 50% of the development efforts are dedicated to integration.
Middleware acts as the link between software applications and the vehicle’s hardware components. A growing number of production-ready middleware solutions provide premade components for:
- Communication
- Memory management
- Job scheduling
- Input/output processing
Many middleware solutions also include off-the-shelf UI components, application libraries, and APIs, which further reduce the complexity and increase the speed of automotive software development. Instead of fiddling with standardization and system testing, software engineers can focus on higher-value tasks such as developing new features.
Middleware also helps to prevent costly quality issues. In 2020, a record-setting 15 million vehicles were recalled due to electronic component defects according to Stout Risius Ross. Half of these recalls involved software defects, of which 30% were integration-related.
For instance, Mitsubishi Motors had to recall over 68,000 SUV models due to a software bug. The hydraulic unit ECU interfered with multiple safety systems, making vehicles unsafe to use. BMW had to recall over 120,000 electric vehicles due to software issues within its high-voltage battery ECU, which caused an interruption in electrical power.
A middleware layer can ease system development, testing, and integration processing, resulting in more standardized, maintainable, and quality hardware and software systems.
Software solutions
The final layer of HMI design is software — user-facing applications placed around the cabin on the dashboard, head-up displays, and rear-seat screens.
When it comes to creating HMIs for user applications, two elements matter most: the operating system (OS) and user experience (UX).
Car OS
Since cars have become computers, they now require an operating system. OEMs have different strategies for car OS development.
Stellantis is building a custom OS, whereas automakers like Ford, Volvo, GM, and Renault-Nissan-Mitsubishi have gone with the open-source Android Automotive OS, according to S&P.
Other popular car operating systems include Automotive Grade Linux (AGL), AUTOSAR Classic and Adaptive OS, and Green Hills INTEGRITY®.
The perk is that OEMs don’t necessarily need to choose just one car OS. With a zoned E/E architecture and a strong middleware layer, you can use different operating systems for independent subsystems (navigation, commerce, entertainment). In other words, you can have a best-of-breed collection of operating systems to maximize memory, network, and application management.
From a user perspective, a car OS has to feel familiar: Offer meaningful icons, support familiar gestures, and provide straightforward navigation between individual apps and screens.
Multi-touch gesture support is now a given, as well as air gestures and eye tracking. Mid-air haptics and gesture control in an automotive HMI can make the driving experience safer and more delightful. A recent UX study in the UK found that adding haptic feedback to gesture control increased preference scores among drivers by 83%.
BMW has one of the most magically delightful air gesture control systems, allowing users to accept or reject calls with point and swipe gestures. To change the rearview camera angle, the driver simply needs to make a circle with a thumb and forefinger, without lifting the hands from the steering wheel. BMW also holds a patent for a gesture control system for motorcycles.
Voice control is another helpful feature for keeping eyes on the road. Among drivers who have tried a voice assistant, 60% now indicate that its presence is a driving factor in their purchasing decision. In-car voice assistants help drivers complete lots of things on the go — check directions, book appointments, and place eCommerce or restaurant orders.
For OEMs, voice assistants also represent new opportunities for revenue diversification. By blending voice-driven commands and in-car payments, OEMs can capture new profits from users’ commercial transactions, ranging from fuel and parking payments to on-the-go commerce.
In-dashboard navigation
Navigation is the most frequently used feature in cars. As such, it has to be done right. On the hardware level, advanced E/E architectures provide OEMs with extensive data on:
- Car positioning
- Driving direction
- Battery/fuel usage
- Average driving speed
On the front end, this information has to be translated into real-time route-building and driving experiences.
To create a convenient navigation UI/UX, you have to:
- Provide drivers with an interface that emphasizes action plus overview: the driver’s present position and immediate surroundings, plus a full picture of the complete route.
- Ensure that all content is adaptive and automatically scales based on different screen sizes (dashboard, HUD, RSD, etc.).
- Adapt the application view to the driving scenario based on the usage scenario and display only essential information to avoid overloading the driver.
- Deliver a consistent UX by making sure that all in-car screens are treated as part of the same interface ecosystem. Strive to provide continuity in information.
Intellias applied the above principles when designing an HMI for a British luxury automotive company. We helped the client develop the necessary hardware base and establish a seamless cloud-based navigation system, complemented by a distraction-free HMI.
Our team has helped deploy an embedded navigation system to head-up displays, rear-seat displays, plus iOS and Android companion apps. Drivers can now build routes in real time and optimize their journeys based on traffic conditions, points of interest, and e-vehicle charge rates.
Advanced driver assistance systems (ADAS)
ADAS is largely designed and implemented on the hardware level but activated and controlled by software. To serve drivers well, an ADAS HMI must give instant feedback to users and support various interaction modalities.
In connected cars, you can provide multiple outputs from ADAS using visual, acoustic, and haptic modalities (or a combination thereof). Mercedes was among the first to implement a touch-sensitive steering wheel for its E-class models.
Mercedes E-class model’s steering wheel has two-zone sensors that detect if a driver’s hands are on the wheel, meaning no extra movement is required to send a signal to the ADAS system that the driver is paying attention. On-wheel buttons have surfaces similar to a smartphone, making them responsive to typical finger swipes. This helps drivers engage with other in-car systems without switching off the ADAS.
The challenge is ensuring that critical information is emphasized when the driver is tired or navigating heavy traffic.
Great HMI solutions are context-aware when it comes to driver and vehicle states. Based on data from the vehicle telematics unit and in-cabin driver monitoring system, an HMI should personalize data outputs and prompt extra ADAS controls.
For example, Honda’s Adaptive Cruise Control (ACC) prompts the driver to select short, medium, or long intervals behind the vehicle detected ahead. Afterward, the ACC automatically controls throttle and moderates braking to hold the selected following interval, which can be helpful when driving in heavy traffic.
Entertainment & comfort features
As driving progressively becomes more autonomous, customers come to expect more lifestyle features for comfort, entertainment, commerce, and business.
And drivers are growing more connected to their connected vehicles. A BearingPoint study of European consumers found that 59% of connected car owners selected their vehicle due to connectivity features, while 32% indicated that it was their primary reason for purchase.
Premium-brand vehicle owners are also more interested in hearing about new connected features: 62% of Audi, 62% of BMW, and 55% of Mercedes owners say they’d like to get notified about new releases.
Moreover, 74% of automotive consumers now say that the level of personalized CX influences their future considerations of an OEM brand. By providing impeccable in-salon user experiences, OEMs can improve customers’ brand perceptions, intention to purchase, and loyalty.
Take it from Smart, which made personalization the focus of their driver experience. Its newest Smart #1 premium edition comes with a continuous UX experience, achieved via the Hello smartphone app and an in-car digital cockpit. The driver can program individual preferences (e.g., AC controls or seat position) before sitting down for a drive, or share a digital key with another person in several taps. In-cabin, drivers can engage with the system using hand gestures or an AI voice assistant, which helps with navigation and activates entertainment.
Overall, the digital cockpit presents ample new monetization opportunities, especially in subscription segment such as:
- Sponsored POI listing and stop suggestions
- Embedded, usage-based insurance deals
- Predictive maintenance services
- Access to office/business tools
- Premium entertainment subscriptions
- And more!
Using new data generated by connected cars and new in-car interface elements, OEMs can meet a wider range of drivers’ functional and emotional needs.
Creating a continuously delightful driver experience
HMI design is complex to get right because you need to create an effective symbiosis between the vehicle’s hardware and software components (oftentimes via means of middleware).
A harmony between physical car elements and virtual gestures; a responsive relationship between navigation instructions and real-world conditions; salon ambiance personalization based on your preferences — these are the core elements of delightful HMI design.
You can watch them in action with our brand-new automotive kit, demonstrating the in-vehicle experience under various driving conditions.
Our automotive team is always at your disposal — online and in person — for any questions you have about HMI design. Contact us.
