Detailed explanation of application test steps for Bluetooth personal area network

　　The testing of communication protocols and applications is a very critical and difficult problem in the field of communication. It has become one of the most active areas in communication protocol engineering. The two most basic tests include consistency testing and interoperability testing. Conformance testing is a necessary condition for interoperability testing. The testing of Bluetooth hardware products is mainly interoperability testing and performance testing, which are completed by the provider; while the protocol software has both consistency testing, interconnection testing and performance testing; for applications, consistency testing is only in the theoretical stage, but interconnection testing and performance testing are indispensable. Therefore, completing consistency testing can ensure the correctness of the product on the one hand, and on the other hand, it can help implementers know where the error is and how to correct it when an error occurs.

　　This paper takes the interoperability test model of Bluetooth PAN as an example, starting from the actual engineering needs, using the automation technology of software testing, combining the general theory and methods of protocol consistency testing to illustrate the characteristics of interoperability testing of Bluetooth applications, and summarizes an automated testing method for application specifications suitable for Bluetooth software models.

　　Test Theory Overview

　　Software quality is the sum of features and characteristics related to the ability of software products to meet explicit or implicit requirements (ISO 9126). Software quality assurance has always been a major issue in the software industry. With the improvement of the degree of social informatization, software applications are becoming more and more extensive, software products are becoming more and more complex, and the quality of software products is increasingly valued by people. Software quality assurance has become the focus of attention of developers and users. The strength of quality assurance capabilities directly affects the development and survival of the software industry.

　　Software testing is a program execution process, the purpose of which is to discover and correct errors in the tested software as much as possible and improve the quality of the software. It is a very important and complex task in the software life cycle, and is extremely important for software reliability assurance. At present, when formal methods and program correctness proof technology have not yet become practical methods, software testing will still be an effective method for software reliability assurance for quite some time in the future. Therefore, studying software product testing technology and developing software automatic testing tools have become an important task for software quality assurance.

　　Communication protocol testing methods

　　Currently, protocol testing mainly includes four tests:

　　Conformance test

　　The meaning of consistency is: if the communication process between an actual system and other actual systems conforms to the international standard of the protocol used, the actual system is said to demonstrate consistency. Conformance testing is used to detect the degree of compliance of the implemented system with the protocol specification. The main purpose of consistency testing is to increase the probability of interoperability between different systems. Although consistency is a necessary condition to ensure interoperability, it is not a sufficient condition. Even if two implementations are consistent with the same protocol specification, they may not be able to interoperate at all. If two implementations can be consistent with a subset of a protocol standard, it is easier to achieve interoperability between the two systems in this case than in other cases. Therefore, ensuring protocol consistency is the basis for achieving protocol and application interoperability.

　　Interoperability testing

　　Interoperability testing is used to test the interoperability and interconnection capabilities between different implementation versions of the same protocol, or between different implementation versions of the same type of protocol (such as email protocols X.400 and sMTP). Interoperability testing of application models on protocols is an important indicator to ensure that applications can be implemented correctly.

　　·Performance Testing

　　It is used to detect the performance indicators of protocol entities or systems (data transmission rate, connection time, execution speed, throughput, concurrency, etc.);

　　Robustness testing

　　Detect the ability of a protocol entity or system to operate in various harsh environments (channel being cut off midway, communication technology being powered off, interference messages being injected, etc.).

　　Automation Technology for Software Testing

　　Software testing is the process of verifying whether the software produces the correct output. It is achieved by completing one or more comparisons between the actual output of the test and the expected output (the output when the software is executed correctly). Automated execution of test cases is the starting point of automated testing. However, comparison work is a repetitive and complex task. It is the task with the highest degree of automation in software testing. It is often the task that benefits the most from automation. Automated execution of test cases will produce a large amount of output, which usually needs to be verified by some method, but not all tests require detailed comparison of outputs. Therefore, automated execution of test cases and automated comparison of execution results are the two main issues that automated testing focuses on.

　　Automated comparison can be divided into dynamic comparison and post-execution comparison according to the timing of comparison. Dynamic comparison is the comparison performed when the test case is executed. Using dynamic comparison helps to program some intelligent functions into the test case so that the test case takes different actions according to the output that appears. For example, if unexpected output appears, it means that the test script is inconsistent with the tested software, so it is best to terminate the test case in an abnormal way instead of continuing to execute it. Post-execution comparison is the comparison performed after the test case has finished running. Both comparison methods play a decisive role in the test results, so the performance of the automated comparison module is crucial to the impact of the automated test program, and it is also a key issue for PAN automated testing.

　　Design of Interoperability Test Model of PAN

　　Generic Interoperability Testing Model

　　Interoperability testing is the next step after consistency testing. Conformance testing verifies that both system A and system B comply with specification X, while interoperability testing verifies to what extent system A and system B can work with each other. Its basic structural model is shown in Figure 1:

Figure 1 Interoperability testing model

　　In the above structure, in order to monitor the data transmitted between the two systems under test, monitors can be placed at monitoring points A, B and C.

　　Interoperability testing generally starts with defining the test purpose and specifying the abstract test suite (ATs). Then, based on the ATS, the ATS is implemented on a specific hardware platform (such as a protocol analyzer) to turn it into an executable test suite (Executive Test Suite ETS). The ETS is then executed on the analyzer to test two or more systems under test (SUT). Finally, a test report is obtained from the test process (the data obtained by the monitor at each monitoring point can be used to discover errors in the SUT).

　　PAN interoperability testing model

　　The PAN test structure consists of five parts, as shown in Figure 3: automatic test program (Auto-test Program), system under test (SUT), test system (TestSystem), PAN application (Applicatiffa), record log (Trace&log) and analysis equipment (Analyzer).

　　The automatic test program replaces the role of the Test Operator in the general test model. It is responsible for executing commands on the system under test and returning results to the test system. It is also responsible for interacting with the analysis equipment to confirm each test result.

　　The system under test is the system being tested, including the file transfer application on the client and server sides, the Bluetooth protocol stack, and the man-machine interface unit (MMI) that provides a graphical user interface for automatic testing. Due to the complexity of PAN applications, corresponding system support is also required. PAN Block is a module that specifically handles system-related transactions.

　　The test system is similar to the general model. Because it is an interoperability test, the test system and the system under test need to have the same hardware equipment and configuration.

　　PAN application is a common application based on PAN service running on the client or server. It can be telnet, FTP and other applications. During the running of the automatic test program, PAN application is scheduled for execution, and the test results are recorded in the log and reported to the analysis device by the log.

　　The analysis device acts as a monitoring tool. It is responsible for detecting test results during the automatic test execution process. It provides detailed data in the interoperability test to verify the credibility of our PAN-based application execution.

　　According to the Bluetooth interoperability test model and the relevant definitions in the Bluetooth specification, the test model of a specific PAN is shown in Figure 2:

Figure 2 Bluetooth PAN interoperability test model

　　The characteristic of the PAN test model is that the automatic test program monitors the test results during the test. If the test results obtained by the analysis device are wrong, the analysis device will feed back the relevant information to the automatic test program, and the automatic test program will schedule the test to continue, terminate the operation, troubleshoot errors, or directly require the tester to intervene. This can make the automatic test more efficient and prevent small errors from affecting the entire test result.

　　Implementation of PAN interoperability automated testing

　　Functions of PAN automated testing

　　According to the above interoperability automatic test model, combined with the requirements for Bluetooth PAN stability in engineering practice, and referring to the test cases of IVVT, an automatic test program based on PAN application specifications was implemented. This automatic test program is an automatic test for application layer programs, so it indirectly reflects the consistency and interoperability performance of the Bluetooth protocol through the performance of the application program. The specific functions of the PAN automatic test in this project are described as follows:

　　1. Establish a connection.

　　Simulate the user clicking the shortcut to automatically establish a PAN from the local machine to the target machine, requiring the local machine to be PANU, the target machine to be GN (NAP), and both machines to correctly allocate port addresses. Monitor the connection process, and once the analysis device finds that the connection is incorrect or the port address is incorrectly allocated, it will immediately feedback to the automatic test program, which will analyze the error level and reschedule.

　　2. Wine test accessibility and connectivity.

　　When the connection is established, the ping command is simulated, requiring the sender to send the command, the receiver to receive and give feedback, and the feedback information can reach the sender correctly. If an error occurs, the log is saved and recorded, and after the test is completed, the tester will analyze and give the final result.

　　3. Transfer files.

　　When a connection is established, a file is sent from the sender to the receiver via the Bluetooth network card, requiring the receiver to receive the file and the size and content of the file to be correct. During the entire transmission process, the automatic test program monitors the sending and receiving ports, which can ensure the most accurate tracking of the source of the error.

　　4. Disconnect.

　　Simulate a user's request to disconnect the PAN connection and perform corresponding operations.

　　5. Scheduling control.

　　Integrate modules 1, 2, 3, and 4 into one module, and automatically complete the above operations in sequence by inputting parameters from the external loop number control through the automatic test program, and repeat this cycle. At the same time, analyze and record the test results in the log file.

　　Test program input information

　　After the test program is started, it will read some input information from the initialization file. This information is set by the tester before the test starts for different test tasks, and cannot be intervened after the test starts. During the entire test process, the devices participating in the test must have the Bluetooth application software installed and configured accordingly, otherwise it will affect the accuracy of the test.

　　Data structures for automated testing

　　Mainly used to store the class of the port address information

　　class CIPAddress

　　{

　　public：

　　charm_sHostName［MAX__PATH］; , used to identify the host name

　　charm

　　sIPAddress［MAX PATH］; ,/ used to identify the port address

　　public：

　　CIPAddress0；

　　int startupo；

　　int CleanUP0；

　　int GetLocalHostName（ch卸r‘sHostName）；

　　int

　　GetIPAddress（char。slPAddress）；

　　void ShowIP0；

　　}；

　　2. Defined data structure for obtaining PAN setting information

　　typedef

　　stnm PanInfo

　　{

　　SHORT

　　Role; //Sender or receiver

　　SHORT

　　PingTnnes; /, the number of times the ping command is executed

　　CHAR

　　SendFilePath［MAX_PATH］； //The file path to be sent

　　CHAR

　　ReceivedFilePath［MAX_PATH］； // Path where the received file is stored

　　}PANINFO，*PPANINFO；

　　The overall process of the PAN automatic test program is shown in Figure 3. Sending and receiving files are completed by different roles. Therefore, it is very difficult to record, analyze, and debug the entire process synchronously, and sometimes manual intervention is necessary. The requirement for the automatic test program is that it cannot be disturbed by abnormal situations. If the robustness of the automatic test program itself cannot be guaranteed, the correctness of the test results is difficult to guarantee.

　　Based on the above considerations, when an error occurs, the severity of the error and whether manual intervention is required are first determined based on the error type. This not only takes into account the impact of serious errors on the system but also reduces the burden on testers. Of course, if the errors accumulate to a certain level, further testing will be meaningless. At this time, the test must be interrupted and manual analysis and processing must be requested. In this system, this is handled through an accumulator.

　　Figure 3 Bluetooth PAN interoperability test flow chart

 

　　The process of the sender in the PAN automatic test is shown in Figure 4. This flowchart is a detailed process of the step "Schedule and execute the sender program" in Figure 3. The sender is divided into multiple roles such as PANU, G|N, and NAP, so the device needs to be initialized at the beginning of the program. The purpose of registering the callback function is to display the test process status information on the interface and trigger the log event. The sender can only judge whether the file is sent successfully based on the monitoring of the sending process. Therefore, this judgment cannot be guaranteed to be completely correct to a certain extent. It needs to be analyzed again after being summarized with the results generated by the receiver. This analysis process is controlled by the automatic test program.

　　Figure 4 Automatic test sender flow chart

 

　　The process of the receiver in the PAN automatic test is shown in Figure 5, which is a detailed process of scheduling and executing the receiver program steps in Figure 3. The process of the receiver is relatively simple, and the receiver must cooperate with the sender to complete the entire automatic test process.

　　Figure 5 Automatic test acceptor flow chart

 

　　Test results and summary

　　The purpose of PAN automatic test program is to test interoperability. The whole test process is to replace some manual tests and save time and manpower for version testing. By making basic functions into automatic test tools to simulate user operations to complete the verification test of most functional points, the workload of testers can be greatly reduced. In practice, the automatic test program is generally run at night after get off work. The next morning, the tester will evaluate the test results by checking the log files generated after the automatic test is run. PAN automatic test program has achieved good results in this regard. By analyzing the results of automatic tests with a scale of 10,000 test cases running continuously, it is very helpful to improve the stability of the software. The field data saved during the operation of the automatic test program is also very helpful for developers to fix bugs in the source program. Especially for long-term projects and incremental development models, automatic testing is a good choice.