Embedded software technology software testing strategy and solution design

    Embedded systems that combine software and hardware are increasingly being used in our common instruments and equipment. The application systems of target systems in the embedded field are becoming increasingly complex, and development technologies are changing with each passing day. At the same time, as hardware technology develops more and more steadily, software failures are becoming increasingly prominent, and the importance of software has gradually attracted people's attention. More and more researchers have realized that it is imperative to optimize the testing technology of embedded systems. The significance of this topic is to develop a suitable embedded software system testing method.

    Embedded system introduction and software features

    Introduction to Embedded Systems

    Embedded systems are application-centric, based on computer technology, and have customizable software and hardware. They are dedicated computer systems tailored for application systems and have strict requirements on functionality, reliability, cost, size, and power consumption.

    Embedded systems generally refer to non-PC standard systems, which also include hardware and software. The hardware includes processors/microprocessors, memory and peripheral devices and I/O ports, graphics controllers, etc. The software part includes operating system software (OS) (requires real-time and multi-tasking operations) and applications. Sometimes designers combine these two types of software. The application controls the operation and behavior of the system, while the operating system controls the interaction between application programming and hardware.

    Analysis of the characteristics of embedded system software

    Embedded system development has its own characteristics. Generally, the hardware part is developed first, mainly including forming a bare metal platform, transplanting the real-time operating system as needed, and developing the underlying hardware driver. After the hardware platform test is passed, the development and debugging of the application software is based on the hardware platform, which is also a test of the hardware platform.

    The development process of embedded systems is a process of mutual coordination, mutual feedback and mutual testing between software and hardware. Generally speaking, in embedded system software, the interfaces of the underlying driver, operating system and application are unclear, and are even mixed together as needed. This is mainly due to the dependence of software on hardware in embedded systems. Based on the dependence of embedded software on hardware, it is required that the actual operating environment of the software under test must be simulated to the maximum extent during software testing to ensure the reliability of the test, while the unclear boundaries between the underlying program and the application program increase the difficulty of testing. During testing, the application program can only be tested if the embedded system platform and the underlying program are confirmed to be correct, and it is more difficult to locate errors during system testing.

    The specificity of software is also an important feature of embedded software. Since embedded software design is based on a certain target hardware platform and is oriented to fixed tasks, once loaded onto the target system, its functions must be completely determined. This feature determines that the inheritance of embedded application software is poor, and also prolongs the system testing time and increases the testing cost.

    Another important feature of embedded software is real-time performance. This is based on the execution perspective of the software, that is, the execution of embedded software must meet certain time constraints. In embedded systems, the complexity of the application software's own algorithm and the operating system's task scheduling determine the allocation and consumption of system resources. Therefore, when testing the real-time performance of the system, it is necessary to use certain testing tools to analyze and test the complexity of the application algorithm and the operating system's task scheduling. It can be seen that embedded software has its own characteristics compared to traditional object-oriented and process-oriented software. Therefore, the development and testing of embedded software is very different from the development and testing strategies of general commercial software. It can be said that embedded software is the most difficult type of software to test. It is necessary and meaningful to study the testing of embedded software based on these characteristics.

    Embedded Software Testing

    Software testing is a key factor in software quality assurance, representing the final check of specifications, design, and coding. It is a process of ensuring the quality and correctness of software code from the perspective of economy and efficiency. Software testing is an important part of software development and a key link in the process from software development to application, and embedded software is no exception.

    Embedded software testing strategy and solution design

    The first question we encounter when discussing embedded software testing is: Why not put all tests on the target? Because if all tests are placed on the target platform, there are many disadvantages:

    It may cause a bottleneck of competing with the target environment developers for time, avoiding the provision of more target environments;

    The target environment may not yet be feasible;

    Target environments are often less sophisticated and less convenient than host platform environments;

    The target environments and development environments provided to developers are usually expensive;

    Development and testing efforts may prevent the target environment from having ongoing applications already in place.

    After determining the target host (host-target) test environment, development testers will encounter the following problems:

    1) How many developers will do testing?

    2) How much software should be tested and how long will the testing take?

    3) What software tools are available for the host and target environments, and how much do they cost?

    4) How many target environments can be provided to developers?

    5) What is the connection between the host and the target host?

    6) How fast is the software under test downloaded to the target host?

    7) What are the restrictions between using host and target environments?

    When testing embedded software, you should consider the above issues in depth, combine your actual situation, and select reasonable testing strategies and plans.

    Embedded software testing process and methods

    According to different indicators, software testing has different division methods.

    From the different stages of testing in the software development process, it can be divided into module testing, integration testing and system testing; according to whether it is necessary to run the target code, it can be divided into dynamic testing and static testing; according to the visibility of the target code, it can also be divided into white box testing (structural testing) and black box testing (functional testing).

    In software testing, each test method is not isolated. In order to achieve the purpose of testing in the most economical and effective way, various test methods are often nested with each other. For example, in the unit testing phase of software, dynamic testing can be performed using black box testing and white box testing methods respectively.

    In recent years, in software testing, test code coverage has gradually become a unified standard for software testing. Therefore, no matter what test method is used, it is necessary to improve the code coverage in software testing as much as possible. Software testing code coverage is based on the white box testing method. Therefore, in order to improve the code coverage of software testing, testers must understand the structure of the source code and have program design documents in order to design test cases so that the test can cover every statement of the program's internal structure as much as possible and improve the code coverage.

    Embedded Software Testing Process

    According to the development process of embedded systems, in order to realize the functions of the system in the most economical way, a top-down, step-by-step approach is adopted to test the embedded system, using the test process shown in the following figure.

    Platform testing: This part includes hardware circuit testing, operating system and underlying driver testing, etc.

Hardware circuit testing requires special testing tools, which will not be described here. The testing of the operating system and the underlying driver mainly includes testing the task scheduling, real-time performance, and data transmission rate of the communication port of the operating system. After the testing of this stage is completed, the system should be a complete embedded system platform, and the user only needs to add applications to complete specific tasks.

    Module testing: Divide a large embedded software system into several relatively small task modules, which are coded by different programmers at the same time. After coding, before integrating the modules, the individual modules must be tested. Since there is no support for data transmission from other data modules, the test at this stage is generally performed on the host machine (the host machine has rich resources and a convenient debugging environment). This stage mainly conducts white box testing, testing every function, every conditional branch, and every program statement as much as possible to improve the coverage of code testing. Since only when a single module is correct is there a need for overall integration, the test of a single module must be sufficient and complete. In the module testing stage, the construction of test cases should not only test the normal operation of the system, but also perform boundary testing. Boundary testing is to test the maximum and minimum values ​​of a certain data variable, and at the same time perform out-of-bounds testing, that is, input data variables that should not be input to test the operation of the system. An ideal embedded system should not crash due to user information interaction, which is also a basic requirement of embedded design. Therefore, no matter what kind of test is performed, system crashes should be treated as test errors.

    In the module testing phase, large modules can also be divided into small modules. Within the program, the entry interface function for data transmission between small modules should be designed to quickly locate errors. Software testing with the idea of ​​module nesting requires a clear internal structure of the module and a simple data link.

    Integration testing: After a single software module has been tested correctly, all modules are integrated for testing. This stage is mainly to find errors in data transmission between modules and the logical structure of the system. A combination of black box and white box methods is used for testing on the host machine. To maximize the simulation of the actual operating environment, some functions that do not affect system execution and data transmission that are difficult to simulate can be shielded.

    Integration testing is the stage where the advantages of embedded software testing are fully reflected. Before integration testing, programmers should write module interface functions based on the input and output of data between modules. This requires the coordination of programmers responsible for different software modules, and then integrate the module interface functions into the entrance of the module receiving data. It is easy to locate the software module where the error is during integration testing of software modules with single-link data transmission. The data of a software module is not necessarily provided by only another module, that is, the data link of the software module is not necessarily a single link. During testing, the data transmission of complex link structures can be divided into single-link structure data transmission for error location.

    Integration testing is the testing of errors that occur during the integration of software modules when program design documents, program structure and data structure are available.

    System testing: After the integration test is completed, exit the host test environment, transplant the system to the target machine, complete the transplantation of the application to the field environment, and perform black box testing on the system from the user's perspective to verify each specific function. Since the tester knows nothing about the program content and program execution, it is difficult to locate errors in this testing phase. The system testing phase should conduct unexpected tests and destructive tests, that is, activities that should not occur under normal execution of the test system and man-made destructive tests to further verify system performance. The system testing phase should not modify the code immediately after the error is determined. The test cycle should be determined based on a certain frequency of error occurrence, and the code should be modified at the end of each test cycle for repeated testing; otherwise, it will not only increase the workload of the full test, but also reduce the credibility of the test.

    Test result analysis: Test result analysis can locate errors, guide programmers to modify the code, and point out the direction of further testing of the program under test. Test result analysis is a process of analyzing, comparing and locating errors based on test results and predicted test results. Test result analysis is the last step of a test. When analyzing, the differences between the software's operating environment and the actual operating environment, as well as the influence of various external factors, should be considered.

    White Box Testing Methods

    White box testing is also called structural testing or logic-driven testing. It is to know the internal working process of the product. It can be used to test whether the internal actions of the product are carried out normally according to the specifications. According to the internal structural test program of the program, it is checked whether each path in the program can work correctly according to the predetermined requirements. The main methods of white box testing are logic drive and base path testing, which are mainly used for software verification. The "white box" method requires a comprehensive understanding of the internal logical structure of the program and testing of all logical paths. The "white box" method is an exhaustive path test. When using this solution, the tester must check the internal structure of the program, start from checking the logic of the program, and obtain test data.

    Black Box Testing Methods

    Black box testing is also called functional testing or data-driven testing. It is to test whether each function can be used normally when the functions that a product should have are known. During testing, the program is regarded as a black basin that cannot be opened. Without considering the internal structure and internal characteristics of the program, the tester tests the program interface. It only checks whether the program functions are used normally according to the requirements specification, whether the program can properly receive input data and generate correct output information, and maintain the integrity of external information (such as databases or files). The main black box testing methods include equivalence class partitioning, boundary value analysis, causal diagrams, error speculation, etc., which are mainly used for software confirmation testing. The "black box" method focuses on the external structure of the program, does not consider the internal logical structure, and tests the software interface and software functions. The "black box" method is an exhaustive input test. Only by using all possible inputs as test cases can all errors in the program be found in this way.

    Test case construction

    Test cases are designed to test the target program, including input items and expected results. Depending on the test environment and the test target program, it can be divided into a document of a certain format or a certain input behavior (such as a key press). The construction of test cases should cover all possible value ranges as much as possible, so that the test covers all program codes as much as possible, improve the test coverage of the code, and at the same time avoid redundant, repeated and meaningless tests. Different test cases should be constructed at different stages of embedded software testing; in the system platform testing stage, test cases for system task scheduling, real-time performance and underlying drivers should be constructed; in the module testing stage, test cases for a certain module should be constructed; in the integration testing stage, corresponding test cases should be constructed for data transmission and structural connection problems during system integration; in the system testing stage, test cases for a certain function or a combination of multiple functions should be constructed, or test cases that have been verified to be correct on similar products should be used. Test cases are reusable.

    Conclusion

    Embedded design has become the only way for industrial modernization and intelligence, and embedded products have penetrated into all walks of life. The degree of specialization of embedded systems is high, and the overall inheritance of the system is relatively small. In order to ensure the stability of the system, software testing has become an important part of embedded development. Due to the characteristics of embedded software itself, traditional software testing theory cannot be directly used for embedded software testing. This article analyzes the characteristics of embedded software, summarizes the strategy and solution design of embedded software testing, and proposes embedded software testing process and testing methods. The research content of this article is of great significance to the testing of embedded software.