Solutions for CNC Equipment Software Testing

Since the 1980s, when 32-bit CPUs were widely used in CNC devices to form multi-microprocessor systems, the status of computer software in CNC equipment has gradually become important. After the 1990s, with the rapid development of computer technology, the software in the CNC system with an open architecture developed using the rich software and hardware resources of PCs has become more powerful in supporting intelligence and networking, and the scale and function of the software have been further enhanced. Contemporary CNC equipment has become a comprehensive system with highly integrated hardware and software.

 Features of software in CNC equipment

Most of the software in CNC equipment is embedded software, that is, software that has a close relationship with hardware and runs in a specific hardware environment. The biggest feature of embedded software is that it is closely related to the hardware environment. The performance, intelligence level and reliability of the entire CNC system are all determined by the hardware environment and software. However, it is well known that under current technical conditions, the reliability of software is one order of magnitude lower than that of hardware. According to statistics, 75% of the failures of embedded systems are caused by software failures. In fact, system failures caused by software failures have become a topic that cannot be ignored in CNC equipment fault diagnosis.

Testing and Diagnosis Methods of Software in CNC System

Compared with hardware failures, software failures have the following two main characteristics: (1) Software failures do not change over time. A large part of hardware failures is caused by equipment wear and aging of materials, but this problem does not exist in software. Once the software runs correctly, it will not degrade over time. (2) Most software failures are caused by inherent errors in the program code, and for embedded software, interface errors between software and hardware are also an important factor leading to failure.

Due to the difference in failure mechanisms, the diagnostic methods for software are different from the fault diagnostic methods commonly used for hardware. From the perspective of ensuring equipment reliability, hardware equipment must undergo a series of reliability tests before leaving the factory, with the aim of exposing problems that arise during the design and processing process in advance. When it comes to the user use stage, the focus of the work is to monitor the operating status of the equipment and diagnose and repair any faults that occur. The focus of software work is on testing the software before it leaves the factory and in the early stages of use, especially the system testing performed after the software is integrated with the hardware. It can be said that system testing is one of the most effective ways to discover software problems in embedded systems.

In the theory of software testing, system testing is a kind of dynamic black box testing, that is, the tester does not need to go into the details of the software code, but only needs to drive the actual operation of the tested software by controlling the input conditions of the software. Simply put, dynamic black box testing is to simulate the actual use of the tested system as much as possible, and to achieve the purpose of discovering faults through actual operation of the tested system. According to the principle of system testing, our laboratory independently designed and developed an embedded software system testing environment with certain versatility, and summarized a set of effective system testing methods on this basis. This article introduces the system testing environment and testing methods in combination with a specific test example.

System testing of engine numerical control system

System test environment

The first step in system testing of embedded software in a CNC device is to build a system testing environment. The role of the system testing environment is to enable the software to run in a real hardware environment, and to allow testers to apply test cases to the software under test and collect test result data. The system testing environment is a test platform composed of hardware and software.

Figure 1 shows the system test environment we designed and developed when testing the CNC system software for a certain model of engine.

The entire test environment consists of three parts: test case and test script generation environment, system detection software and system detection box. The test case and test script generation environment and system detection software belong to the software part of the test platform, while the system detection box and the I/O devices connected to it belong to the hardware part. The test case and test script generation environment is a graphical editing environment, in which testers can convert the designed test cases into test scripts by drawing or writing text. It is an executable form of test cases. The test script contains a series of manipulation actions on the system under test. Its actual function is to realize the test intention of the tester by controlling the test process.

The system detection software consists of four parts: script interpreter, model, test result collection and display module. The script interpreter is used to interpret the test script into executable code form, which is used to control the test process during the test. The model calculates and generates test data according to the requirements of the test script and sends it to the system under test. The test result collection is used to collect the status information and output data of the system under test during operation. The display module is used to display the status information and output data of the system under test during the test process according to the requirements of the test staff, so that the tester can observe and control the progress of the test.

The ground detection device is a tool for detecting the integrity of the numerical control system and the reading and writing of the flash memory in the device before the normal operation of the equipment, as well as for data downloading and data analysis after the equipment is running. The functions of the ground detection device include simulation of electronic controllers and engine models, detection of electronic controllers, detection of relay boxes, detection of sensors, and detection of electro-hydraulic servo valve coils. In the test environment, the ground detection device obtains the test data generated by the model according to the test script from the system detection software, converts all these test data into data forms that can be input into the system under test, and sends them to the system under test through the I/O interface, driving the operation of the system under test and completing the test. From the above introduction, it can be seen that this system test environment has the following main features:

It has a certain degree of versatility. This environment can not only test engine numerical control software, but also can test a similar type of embedded software.

Easy to operate. The test environment only requires the tester to write the test cases into test scripts to implement automated testing in the test environment without directly operating the system under test.

High test efficiency. With the support of system detection software and system detection box, the execution efficiency of this automated test has been greatly improved compared with manual testing.

Supports repetitive testing. Since test cases exist in the form of test scripts, the same test can be executed infinitely without any difference. Repetitive testing is widely used in the backtest after fault correction.

System testing process

After setting up the test environment, you can use it to perform system testing. The specific process can be divided into the following steps:

1. Design test cases. System test cases consist of two parts: test input sequence and expected test results. The test input sequence describes the specific operation process of the device under test in a test, including how to start the device, when to accelerate, when to stop, and when to shut down the device. The expected test results describe what kind of action or state the device under test should produce under the above operation.

The quality of the test case is the key to whether the equipment software fault can be found. When designing the test case, it can be considered from several aspects such as normal function, boundary conditions and abnormal operation according to the different test focuses. Normal function test is to simulate some representative operations according to the normal use of the engine under test. The test of boundary conditions refers to the test of the boundary value of the analog quantity input into the system under test. Experience shows that it is most likely to make mistakes when the input value is on the boundary or near the boundary. For example, the engine is prone to failure when running around the boundary point of the normal speed range. Abnormal operation mainly examines the ability of CNC equipment software in handling some abnormal situations. For example, when the user makes an erroneous operation or a hardware failure suddenly occurs during use, whether the CNC software has a certain protection function to avoid major accidents. This test mainly examines the reliability and robustness of the CNC system.

2. Run the test. The designed test cases can be converted into test scripts that can be recognized by the test environment by drawing or text in the graphical editing environment provided by the test environment. Each test case is converted into a test script, which can be directly loaded into the test environment for a test. During the test run, the operations in the operation sequence of the test case are executed in sequence, and the tested device is also run. During the run, the status of the device operation can also be observed through the display module provided by the test environment.

3. Analyze the test results and draw test conclusions. After a test is completed, the test environment will automatically collect various operating data of the tested device. Compare these data with the expected results in the test case to determine whether the tested device is operating normally or whether a fault has occurred.

We generated 100 test cases from three aspects: normal functions, boundary conditions, and abnormal operations, and successfully ran them in the test environment. Finally, through the analysis of the test result data, we found several key problems. For example, when the flight altitude of the aircraft exceeds a certain value, the interpolation calculation performed by the software will be wrong. In addition, there are problems with the design logic of the software, which causes the engine to stop when it should not stop. After analysis, it was determined that these faults were caused by defects in the CNC software. After the software designers made correct modifications to these problems, the reliability and robustness of the system have been greatly improved.

Conclusion

Using the system test environment and test methods introduced in this article, we have successfully conducted tests on multiple embedded software and discovered many important defects hidden in the software, which played an important role in improving the quality and reliability of the equipment.