Intelligent cockpit HMI automated testing - vehicle computer performance test

In recent years, with the popularization of intelligent cockpits, users have higher and higher requirements for the experience of car systems. Users hope to enjoy the same convenience and smooth experience as smartphones in the car. However, many car systems have shortcomings in application response time, operation sensitivity and sliding smoothness, resulting in a decline in user experience. These problems may be caused by hardware performance limitations, unreasonable software design or insufficient system optimization. For users, the intuitive reaction is slow response and jamming, but for testers, how to quantify these indicators is a headache.

Test scenario glossary

Application response time

It refers to the time it takes from when a user triggers an application to when the application completes the response. This includes application loading time, page switching time, etc. Users want to see feedback from the application immediately, rather than waiting for a long loading time.

For example, when you click the "Music" icon with your finger, the music app starts loading and waits until the app is fully loaded. The time from when you lift your finger to when the app is fully loaded is the response time of the app startup.

In addition to the common application startup response time, there are also exit response time, screen flip response time, etc. In addition, it also involves cold start, hot start and other response time tests.

Operation sensitivity

It refers to the speed at which the vehicle system responds to user operations. Users expect to get immediate feedback when operating the vehicle system, rather than experiencing noticeable delays.

For example, when a user slides the screen on the home page, the time difference between when the finger touches the screen and when the screen starts to slide is quantified. Common list sliding and map sliding are also such scenarios.

Sliding smoothness

Sliding smoothness refers to whether there is obvious lag or delay when sliding a list or zooming a page.

For example, when sliding a song list, users want the list to scroll smoothly without any lag or delay. It is necessary to quantify the number of stuck frames and the average frame rate.

Necessity of testing

In order to provide an excellent in-vehicle user experience, in-vehicle manufacturers and developers need to test application response time, operation sensitivity, and sliding smoothness. The purpose of these tests is to discover and solve potential problems, ensure that users can operate the in-vehicle system smoothly, and improve user satisfaction and user stickiness. Through testing, we can fully understand the performance of the in-vehicle system in different scenarios, find performance bottlenecks and problems, and make corresponding optimizations and improvements.

Test plan introduction

Dongzhou's vehicle computer performance test solution adopts a non-intrusive method. The following figure is the test solution architecture diagram:

The system integrates high-frame industrial cameras and high-sensitivity touch sensors to test screen response performance. When the user clicks the screen, the touch sensor can accurately capture the moment of finger pressing and lifting. The high-frame industrial camera collects the entire process from application change to loading completion at high speed. Combined with the built-in image analysis algorithm, the application response time, operation sensitivity and list fluency in different scenarios can be tested and compared to find performance bottlenecks and room for improvement.

Test results display

Response Time & Sensitivity

Application response time and sensitivity are both quantitative time indicators, and both belong to the response time test. In our system, they belong to the same type of test scenario. The following figure shows the results of the actual list sliding test:

▲ Touch the starting point

▲ Response end point

[Touch start point] is the moment when the click starts. The figure shows the start point marks for three situations. In actual testing, you only need to set one of them. Some working conditions may require "touching the screen" and some may require "leaving the screen". Users can choose according to specific use case requirements.

*Note: The specific situation depends on the use case scenario

[Response End Point] is the moment when the application loading ends or starts to react (i.e. sensitivity). The figure also shows the end point marks for the three situations. Similar to [Touch Start Point], users can choose according to the test conditions.

Sliding smoothness

Fluency is quantified by the number of stuck frames and the average refresh frame rate. The user's intuitive feeling is whether there is any stuck. The location where the stuck occurs indicates that the list is not refreshing and is in a static state. In actual testing, a high-frame camera is used to capture images of the entire process of list sliding, and then the algorithm dynamically obtains feature points for analysis. If the feature points do not change during the sliding process, a stuck occurs.

▲ Fluency results

The above picture shows the actual result of the smoothness test, which includes the number of freezes, the maximum number of consecutive freeze frames, the total number of freeze frames, and the average frame rate. Users can quickly locate the freeze position through the [Stuttering Interval] in the interface, and can also view the image through [Frame-by-Frame Playback].

Application response time, operation sensitivity, and sliding smoothness are important factors affecting the user experience of the car machine. By testing and optimizing these aspects, we can improve user satisfaction and provide a better car machine user experience.

During the design and development process, we need to focus on the optimization of hardware and software, and adopt reasonable technologies and methods to improve the performance and response speed of the system.

We regularly update and optimize the car system to keep pace with the times and meet users' needs for convenience, smoothness and efficiency. Through continuous improvement and optimization, we can provide users with a more pleasant and seamless car system experience.