Intelligent cockpit domain controller function automation test solution

1. Background

Smart cockpits are the focus of development, design and differentiated competition in the current automotive industry. Currently, most smart cockpit controllers are complex systems with multiple screens in one machine, integrating traditional IPK, HMI, HUD, DMS and other controllers. In terms of software architecture, multiple operating systems are also a major feature. For example, integrating Android and QNX systems is the most common solution, while hardware interfaces are usually automotive Ethernet, CAN/CAN FD and LVDS.

Due to its own characteristics, the cockpit domain controller has tens of thousands or even hundreds of thousands of functional test cases. If it relies entirely on traditional manual testing, it will require a lot of human resources and it will be difficult to meet the needs of shorter and shorter project development cycles and rapid software iteration. In order to improve testing efficiency, it is necessary to use automated/semi-automated methods to complete the functional testing of the cockpit domain controller.

2. Test content analysis

From the perspective of component-level functional testing in the smart cockpit domain, the focus is on the following aspects:

a) Vehicle and environment information display function: central control, instrument, HUD, streaming media rearview mirror, electronic rearview mirror, A-pillar and other display interactions

b) Entertainment and browsing functions: local and online media playback, games, document and web browsing and reading, etc.

c) Configuration, setting and control functions: personal account, vehicle/driving information settings, APP installation/uninstallation, etc.

d) AR navigation, surround view, rear view, night vision and other functions

e) In-vehicle monitoring: driver/passenger head, face, eye monitoring and health monitoring

f) Interactive functions: voice interaction, gesture control

g) User APP online and offline service functions

h) Connectivity: Bluetooth, WiFi, USB, 4G/5G, etc.

3. Test system framework

From the perspective of the cockpit automation test system, the intelligent cockpit test system can be divided into the following main functional modules:

a) Conventional vehicle network and IO simulation acquisition and fault injection

b) UE simulation and monitoring

c) Image simulation and UI monitoring

d) Speech simulation and recognition

e) Wireless signal related simulation

The schematic diagram of the complete test system is shown below. This article will focus on the process and basic methods of implementing UE simulation and UI monitoring based on Eggplant and the comparison of various mainstream solutions.

Figure 1: Test system block diagram

4. Introduction to automated testing based on Eggplant

(1) Software and solution framework

Eggplant is a black box automated testing tool developed by TestPlant. It connects PC and DUT via VNC/RDP transmission protocol and can be used across platforms (Linux, Mac, ISO, QNX, Android and Windows). It monitors DUT images based on images and OCR algorithms, facilitates access to various states of images (color and brightness), and supports running in a dynamic environment using Flash. It can automatically generate test scripts by recording user operation processes. The integrated SenseTalk language is aimed at non-developers, and testers do not need to understand the underlying code and architecture, making it simple and easy to use.

Supports integration with Vector CANoe. Operation instructions are sent through CANoe, and the execution results are automatically generated in HTML format for easy viewing.

Figure 2: CANoe+vTESTstudio+Eggplant overall solution framework

(2) Implementation process overview

This time, we take the DUT of Android system as an example to carry out joint debugging of CANoe, vTESTstudio and Eggplant. The debugging process is mainly divided into the following steps:

a) Test system and DUT connection settings

Enable DUT debugging permission, connect PC, set up the connection for Android system, and generate the IP address and port number of the server for connecting to DUT in Eggplant software.

Figure 3: Connection effect diagram

b) Eggplant test project creation

Use Eggplant to write, run and execute low-level scripts for UE simulation and UI monitoring. Use the Log function in Eggplant to record the test results.

Figure 4: Eggplant operation interface display

c) CANoe calls the Eggplant test project

According to the API provided by Eggplant software, use CANoe software to call the Eggplant test project. In the CANoe project, you need to fill in the relevant information of the Eggplant test project, such as: software installation address, test project storage address, port number and password.

Figure 5: Configuring the Eggplant test project information in the CANoe project

d) Simulation implementation of “opponent software” (optional)

For the cockpit domain controller based on SOA, it is necessary to simulate the "opponent" that interacts with it, such as the vehicle domain controller, autonomous driving domain controller, and networked controller. To meet this requirement, the new version of CANoe can support protocols such as SOME/IP and MQTT, and efficiently complete the simulation of SOA applications by importing databases (such as .arxml). In our experience, it is more efficient and professional to implement this simulation based on CANoe.

e) Test case implementation

The test script is written based on the graphical programming method of vTESTstudio software, compiled to generate executable files, and imported into CANoe for test execution.

Figure 6: Test script writing based on vTESTstudio

f) Test execution and analysis

The Eggplant test results are read and judged through CANoe, and the original images are printed in the test report to facilitate the analysis of test problems.

Figure 7: Test report format

(3) DEMO example

Since the real DUT modeling and display interface are confidential, the following video uses debugging based on an Android phone as an example.

Video 1: CANoe and Eggplant UE simulation and UI monitoring automated test results

5. Solution comparison

(1) Comparison of UE simulation test solutions

UE simulation is to simulate user operations, such as single/double click of soft keys and screen sliding. Its difficulties are:

a. Adapt to multi-level window operation

b. Re-adaptation after the operation interface design is changed and the operation object is changed

c. Implement special control methods, such as multi-finger click/slide, two-finger zoom

Table 1: Comparison of UE simulation test solutions

(2) Brief introduction to comparison of UI and UE monitoring test solutions

UI and UE monitoring is used to determine the correctness and performance of functional logic. The difficulties are:

a.Multi-screen interaction

b. Display style switching, APP dynamic loading recognition adaptation

c. Dynamic flashing icon monitoring and screen response time monitoring

Table 2: Comparison of UI and UE monitoring test solutions

6. Summary

From early instrumentation/central control to today's smart cockpit, attempts to automate testing have been going on for more than a decade. Today's test objects are more complex, test plans are constantly being upgraded, and test requirements are constantly increasing. At this time, we need to conduct rational analysis or review based on the origin of test requirements and the characteristics of cockpit domain controllers. We need automation but not blindly pursue full automation. We need to subdivide different test functions, the resource consumption of the test work, the feasibility and adaptability of automation, and the input-output ratio.

Beihui Information has accumulated nearly ten years of experience in the development of test specifications/cases, test implementation and test system construction for components, systems and real vehicles in the field of smart cockpits. It can provide "combined" technical solutions and cooperation models to achieve a balance between test results, efficiency and cost performance. This time, TestPlant's Eggplant software was selected as a new attempt, and a brief comparison of related simulation and monitoring solutions was made, hoping to provide some reference and inspiration for everyone. Finally, I would like to thank the colleagues and partners of TestPlant (Keysight) and Vector for their strong support.

References

[1] Vector - Eggplant integration.pdf

[2] Automatic software testing on Mac_OS_X platform based on Eggplant_Deng Yingqiu.pdf

[3] http://docs.eggplantsoftware.com

[4] GUI - Testing_HanserAutomotive_201706_PressArticle_EN.pdf