Softswitch Performance Testing Technology

As the core equipment in NGN/VoIP network, soft switch has rich interfaces, complex protocols and high performance requirements. Therefore, it has become the focus of equipment manufacturers and operators for laboratory testing or network acceptance testing. Soft switch is generally located in the center of the network. It needs SIGTRAN interface to communicate with PSTN, H.248 control interface to communicate with relay media gateway, sip/H.323/mgcp/H.248 interface to communicate with IP terminal, and sip/sip-I/sip-T/BICC interface to communicate with other soft switches. If any of these interfaces have problems, it will become a bottleneck restricting system performance, and even cause NGN/VoIP network paralysis in serious cases. Therefore, the performance of soft switch has become a key indicator for operators to evaluate the quality of NGN system.

For the above reasons, how to effectively verify the performance and stability of each interface function of soft switch equipment in a complex network environment has become an important research topic in the field of NGN/VoIP testing. Spirent Communications has long focused on testing technology in the field of communications. With a strong R&D team and years of experience in live network testing, it provides customers with a complete soft switch system testing solution, becoming the inevitable choice for operators and equipment manufacturers in NGN field testing.

2 Common softswitch test methods

As a complex core control device, softswitch has a variety of test methods to choose from. Common test methods are classified into the following categories:

●By networking mode: single node full surround test, multiple network element intercommunication test, end-to-end system test
●By test function: C4 tandem office function test, C5 local office function test
●By test index: peak stress test, maximum simultaneous conversation capacity test, stability test, long call test, overload protection test, CDR accuracy test.

Below we introduce these different testing methods in detail.

2.1 Single-node full enclosure test

Single-node full-surround test is generally used to verify whether a single device has a performance bottleneck. The test instrument simulates all interfaces around the node and simulates all network environments around the device, thereby eliminating the impact of other devices on the test environment and accurately verifying the performance of the device under test.

Figure 1 shows an example of a full-surround test of a soft switch. The instrument simulates the signaling gateway through the Sigtran (M2UA or M3UA) protocol, simulates the trunk gateway through the H.248 protocol, simulates the IP Phone through protocols such as sip/H3.23/H.248/MGCP, and then simulates a large number of voice calls from the PSTN to the IP network side to achieve performance testing of the intermediate soft switch equipment.

Figure 1 Schematic diagram of soft switch full surround

2.2 Multiple NE interoperability testing

This type of test is generally used to perform performance tests on a limited system consisting of multiple nodes in a certain, controllable environment. It is used to discover interoperability problems between network elements and determine whether new performance problems are introduced during the interoperation process.

For example, when deploying a VoIP network, an operator may use a soft switch from manufacturer A and an access gateway from manufacturer B. How can they confirm that there are no performance issues when the two devices are interoperating? In this case, the instrument can be used to simulate the analog line terminal under the access gateway, initiate a call to the gateway and soft switch, and then the soft switch will route the call back to the gateway and then to the terminal. In this way, thousands of calls initiated by the instrument through the access gateway and soft switch can effectively verify whether the two devices have performance and stability issues when interoperating.

2.3 End-to-end system testing

End-to-end system testing is generally used to verify the function and performance of a complete system, or for distributed testing of an operating network. As shown in Figure 2, instruments are generally deployed at different branch nodes in the system or network to simulate various different terminals involved in the system, and then a large number of calls are initiated between different terminals for testing. In this way, not only can system performance be verified, but even the end-to-end network delay and network transmission quality can be tested.

Figure 2 Schematic diagram of end-to-end system testing

2.4 C4 Tandem Office Function Test

As a control device in the NGN network, the soft switch has both the function of a tandem office and the function of a local end office. When the soft switch is used as a tandem office, it is generally used to tandem PSTN or long-distance voice traffic, or to tandem traffic from other offices. Therefore, when testing the tandem office function, there are several networking methods:

● Through the E1/STM-1 interface of the instrument, the PSTN end office is simulated, and a voice call from the PSTN to the VoIP side is initiated through traditional No.7 signaling (see Figure 3).

Figure 3 Simulating PSTN end office to test soft switch C4 function [page]

● Through the IP interface of the instrument, the signaling gateway + trunk media gateway is simulated, and a voice call from the PSTN to the VoIP network is initiated through the Sigtran protocol (see Figure 4).

Figure 4 Simulating SG+TG to test softswitch C4 function

●Use the instrument to simulate another tandem office implemented by soft switching, and initiate a voice call from one tandem office to another tandem office through the sip-T/BICC protocol.

Among these three solutions, the first solution can verify the performance of SG and TG while testing the soft switch. The second and third solutions can achieve very high performance because they simulate traffic directly through the IP interface, and can also save a lot of trouble in connecting E1 or STM-1 lines.

2.5 C5 local office function test

When the soft switch is used as a local terminal in the NGN network, it is generally used to control the traffic connection of terminal users and provide various supplementary services. Therefore, when testing the terminal function, the instrument needs to simulate the user under the access gateway or the IP terminal user to realize the simulation of local traffic. As shown in Figure 5, in actual testing, terminal equipment based on various protocols such as SIP, H.323, Megaco or MGCP can be configured according to the situation to realize call testing of large traffic volume between terminals of the same protocol or between different protocols.

Figure 5: Simulating IP terminal to test softswitch C5 function

2.6 Peak Pressure Test

Peak stress test is generally used to verify the ability of the tested equipment to handle calls per second when the system is at its highest load. The tested equipment is often examined through two indicators, BHCA or CPS, that is, the number of calls processed by the system per hour or per second. In actual testing, the instrument is generally used to generate a certain amount of calls per second to accurately verify the system capacity, and the pressure is adjusted by changing the call holding time and call interval time. The test time is generally more than 1 hour, and even up to 24 hours or 48 hours; the call loss cannot exceed one ten-thousandth.

2.7 Maximum simultaneous session capability test

The maximum simultaneous conversation capability test is generally used to test the number of conversations that the device under test can maintain at the same time at a certain time. Compared with BHCA and CPS, the maximum number of maintained conversations is also an important indicator. This test generally simulates a certain number of calls through the instrument, and then sets the call holding time to be long enough to observe whether these simultaneously maintained calls have dropped calls, abnormal interruptions, etc. within a certain period of time. Reflected in the call loss value, it cannot exceed one ten-thousandth.

2.8 Stability Test

Stability testing is generally used to verify the system's ability to operate stably for a long time and handle traffic without failure. Generally, the instrument simulates a certain amount of traffic that accounts for a certain load of the system (such as the system can handle 80% of the CPS at most), and then sets the holding time of each call to the average call holding time of the existing network (such as 2~3 minutes). The system is required to operate stably for at least 24 hours, sometimes even more than 48 hours, and the call loss rate cannot exceed one in ten thousand.

2.9 Long-term call test

Long call test is generally used to verify the system's ability to handle long calls. This type of test generally does not require very high performance. It often uses instruments to simulate dozens of voice calls, sets the call holding time of each call to at least 24 hours, and observes whether there are dropped calls, abnormal interruptions, abnormal voice, etc. during the test. In addition, the accuracy of the soft switch for such long call records, whether the call records are lost, and whether the duration billing is accurate will also be observed.

2.10 Overload protection test

Soft switches generally have an overload protection function. When the call volume exceeds the maximum load or the system CPU load has reached 100%, subsequent calls are actively rejected to prevent excessive calls from occupying system resources and causing system paralysis. The overload protection test is used to verify the soft switch system's ability to effectively control the call volume and avoid system paralysis when encountering sudden call traffic that exceeds the system's maximum load. This type of test generally simulates the system's call load step by step and in quantity through instruments. For example, the system's call volume of 80%, 100%, and 120% is simulated respectively. It is required that the call loss rate should not exceed one in ten thousand when the system is at 80% and 100% load. At 120% call volume, a large number of call losses can be allowed, but the number of successfully connected calls per second should be maintained at around the maximum CPS that the system can handle. In addition, there should be no system paralysis, board restart, or message no response.

2.11 CDR Accuracy Test

The CDR accuracy test is used to verify the accuracy of the call records generated by the softswitch system while processing a large number of calls. Generally, a large number of calls are simulated by the instrument for a period of time (for example, more than a few hours). Then, after the test, the total number of call records given by the instrument statistics is compared with the total number of call records given by the softswitch system to check whether the softswitch has dropped call records. Generally, the call record error rate is required to be no more than one in ten thousand.

3. Key parameter statistics and performance test results analysis

Softswitch equipment has become the most complex device under test in the NGN test field because of its multiple interfaces, rich protocols, and high performance. Therefore, the statistical indicators involved in the test are also diverse. How to filter and observe key test parameters from complex test data, effectively analyze the quality of performance test results, and check the performance bottlenecks of the system will be what we will discuss in this section.

[page]

(1) Statistics and analysis of key indicators such as BHCA/CPS/call loss rate

In the soft switch performance test, BHCA/CPS and call loss rate are two very important interrelated indicators. We conduct a large number of tests to find out the maximum BHCA/CPS that the system can support when there is no call loss or the call loss rate is less than the specified value (such as one in ten thousand). Generally, the two-point method is used to accurately find the fault-free balance point of the system. At this time, if the CPU and memory occupancy of the soft switch system are observed, it can help testers more easily determine whether the system load has reached a bottleneck. Generally, as the BHCA/CPS generated by the instrument gradually increases, when the system approaches the performance bottleneck, it is often accompanied by a sudden sharp increase in the CPU occupancy or the memory occupancy, and approaches the maximum value; if the call loss increases greatly at this time, it can be determined with certainty that the large number of call losses are caused by the system load reaching the bottleneck and not due to other reasons such as the network.

(2) The importance of signaling delay statistics for performance testing

Signaling delay is an indicator to examine the speed of the soft switch system in processing signaling messages. When looking for the system's fault-free balance point, the statistics of this indicator are also a very important inspection content. For VoIP signaling, signaling delay mainly includes several specific indicators such as Response Time, Post Dial Delay, Call Setup Time, and Tear Down Time.

Response Time

The time interval between the terminal initiating a call establishment request message and receiving a provisional response message. Taking SIP as an example, it generally refers to the delay between the Invite message and the 100 Trying message.

Post Dial Delay

The interval between the terminal initiating a call establishment request message and receiving a ringing response message. If it is Sip signaling, it generally refers to the delay between the Invite message and the 180 ringing message; if it is analog line signaling, it generally refers to the delay between picking up the phone and dialing the number to receiving the ringback tone.

●Call Setup Time

The interval between the terminal sending a call setup request message and receiving a call setup confirmation message. In Sip signaling, it generally refers to the delay between the Invite message and the 200 OK message.

●Tear Down Time

The interval between the terminal sending a session release request message and receiving a disconnect confirmation message from the other end. In SIP signaling, it generally refers to the delay between the Bye message and the 200 OK message.

Signaling delay statistics are an important auxiliary means for judging and analyzing softswitch performance. This is because these delay statistics often represent the actual user experience during a call. For example, Post Dial Delay represents how long a user has to wait to hear the ringback tone after dialing a number. If these indicators exceed the standard value, it means that the service quality of the softswitch has deteriorated. At this time, even if the system can establish a call normally and the call loss rate is zero, the softswitch performance under the current load can no longer meet the requirements of actual operations. [page]

Operators and standard organizations often have corresponding specifications for signaling delay. For example, the Chinese specification requires that the Post Dial Delay should be less than 1s under light load and less than 2s under heavy load. Therefore, it is very important to observe the signaling delay statistics while determining the fault-free balance point of the soft switch system.

Figure 6 is a statistical diagram of the CPS and PDD indicators during the soft switch performance test. We can see that at the beginning, when the CPS is relatively small, the Post Dial Delay indicator is relatively stable. When the CPS value climbs to a certain stage, the Post Dial Delay begins to rise significantly. When CPS=200 reaches the peak, PDD=3S. If we do not comprehensively examine various indicators during the test, we often simply think that the maximum CPS indicator of the system is 200. However, the actual situation is that the PDD indicator at 200 CPS can no longer meet the operational requirements. Therefore, the actual and reasonable maximum CPS of the system should be the CPS value when PDD<2s, that is, about 180 CPS.

Figure 6 Schematic diagram of the relationship between CPS and Post Dial Delay

From this example we can also see the importance of comprehensively examining various important indicators for performance testing analysis.

(3) Fault location analysis

When the soft switch test encounters a fault (such as a large number of abnormal call losses, no response to messages, etc.), we need effective means to locate and analyze the cause. Is it the abnormal operation of the soft switch equipment, or is the message loss caused by the instability of the bearer network? In this case, we can take several auxiliary positioning methods for further analysis.

●Analyze the cause of the failure by modifying the call model

Take the example of CPS VS PDD mentioned in the previous section as an example. When the system load increases and Post Dial Delay increases significantly, we can appropriately reduce the traffic volume (for example, reduce it to half of the original volume) and then observe whether the statistical value of Post Dial Delay decreases significantly. If it decreases significantly, it means that the slow message response is indeed related to the traffic load of the soft switch device; if the value does not decrease significantly, it means that the slow message response may be related to the network, such as the network transmission delay is too large, causing the slow message response.

●Observe the network status through the statistics of network layer parameters provided by the instrument

While simulating the call signaling and voice transmission, Abacus can test and count the following parameters: packet loss, delay, jitter, and disorder, and can output the statistical values ​​corresponding to different time periods. When the test encounters a problem, the statistics of the above parameters can be used to determine whether there is a problem with the network. For example, the statistical chart of One Way Delay shown in Figure 7 shows that when the instrument counts to 14:10 and 14:30, the One Way Delay increases significantly. If the system call loss occurs at this time, we have reason to believe that the system call loss is likely to be caused by network reasons. It is also possible to combine other factors such as packet loss, jitter, disorder, etc. to comprehensively determine whether the problem is caused by poor performance of the bearer network.

Figure 7 One Way Delay VS time statistics provided by Abacus instrument

●Use the packet capture monitoring function of the instrument to analyze the signaling process and determine the cause of the fault

While simulating calls, the Abacus instrument can record all the signaling during the problematic call process. In addition, Spirent also provides a monitoring instrument called Clearsight, which can monitor and record all VoIP calls flowing through the IP network. In this way, combining Abacus, Clearsight and the signaling log function provided by the soft switch itself, it is very convenient to check and compare the message log output by the instrument with the message log captured on the network and the message log of the soft switch, so as to find out which node the error message or message loss and other faults are caused by.