This article details methods for testing Thread mesh network performance. With the increasing number of mesh networks available in today’s wireless market, designers must understand the use cases for these networks and their expected performance. When selecting a network or device, designers need to understand the performance and behavioral characteristics of the network, such as battery life, network throughput and latency, and the impact of network size on scalability and reliability.
Silicon Labs conducted testing using the Thread Mesh software stack and Wireless Gecko SoC platforms capable of running Thread Mesh and proprietary protocols to describe how Thread mesh networks compare in performance and behavior to other standards. The test environment was a commercial office building with Wi-Fi and Zigbee networks in range. Wireless test clusters were deployed in hallways, conference rooms, offices, and open areas. Methods for performing benchmark tests were defined so that others can run the same tests. These results are primarily intended to provide guidance on design practices and principles as well as expected field performance results.
The test items for Thread mesh network include:
For detailed performance data, please visit the Silicon Labs website and download
the
“
Thread Mesh Network Performance”
application note: https://www.silabs.com/documents/login/application-notes/an1141-thread-mesh-network-performance-cn.pdf
For additional performance benchmark information on other technologies, see http://www.silabs.com/mesh-performance
Basic physical layer and packet structure
Network performance depends on the size of the payload, as the packet overhead does not include application usage. Thread uses IEEE802.15.4 2006, 127-byte packets and a base data rate of 250 kbps. The Thread packet format is shown below, resulting in a 63-byte payload. For payloads above 63 bytes, Thread stack fragmentation uses 6lowpan. Our performance numbers depend on the payload size, as this is a design parameter that needs to be considered when building an application.
Thread Packet Format
Network routing differences
Thread supports next-hop routing and flooding. As part of normal network maintenance, Thread maintains next-hop routes to all routers, rather than devices performing flooding based on route discovery. Thread also minimizes the number of active routers, which can address scaling issues for large networks. Previously, multicast flooding was considered a limitation of embedded 802.15.4 networks because flooding in the presence of a large number of routers limited the frequency and reliability of multicast traffic. Note that Thread networks manage the number and spacing of active routers, so no user intervention or management is required.
The network breaks up larger messages into smaller messages to fit within specific PHY constraints. For Thread, segmentation is done at the 6lowpan layer, in the source-to-destination direction (not a single hop).
For unicast forwarding in these networks, the message is forwarded whenever the device is ready to send. For multicast forwarding, there are usually network requirements on how the message is forwarded. Thread devices use RFC 7731 MPL forwarding. For this method, the trickle timer is set to 64 milliseconds, so the device backs off a random number up to this value before retransmitting.
Note: This performance data applies to the Silicon Labs implementation of these mesh networking stacks. Testing was not performed with other stacks or systems, as shown with the test network and infrastructure provided for this testing.
Objectives and methods
This application note defines a series of tests for evaluating mesh network performance, scalability, and reliability. Test conditions and infrastructure are described, as well as message latency and reliability. The testing is performed with actual wireless devices in the test network, not simulations.
This test is primarily intended to provide a comparison between different mesh technologies to better understand and recommend their use. Different network and system designs have different requirements for devices and networks. Therefore, no one network can meet all network requirements. However, the three mesh network technologies we will compare are all targeted at low-power and battery-powered mesh networks for security monitoring in homes and commercial buildings.
Typically, when analyzing network performance data, we consider what improvements could be made to the network to improve performance. Because there is currently limited public data on mesh network performance for large networks, it is difficult to have an industry discussion about possible improvements or changes. For example, in commercial buildings, there are concerns about:
-
Other network traffic
, because there may be many subnets interfering with each other.
-
Wi-Fi
interference
with
normal building
Wi-Fi
infrastructure
, as these technologies typically operate in the 2.4 GHz ISM band.
-
Network throughput and latency, as well as large network multicast latency and reliability
, because multicast is often used for lighting control in dense office environments and system users expect lighting controls to be responsive.
Note: The test results presented here are limited to comparing system performance under normal operating conditions or when stressed as indicated in specific tests. This application note does not provide solutions to system interference or other such effects, which can be addressed by other published results. However, testing was performed at our SiliconLabs R&D facility with over 100 Wi-Fi access points within RF range. The facility also has a 300-node Zigbee lighting network that was not part of this testing but was used for general lighting control.
Test network and conditions
To minimize variance, device testing can also be performed in a fixed topology where the RF paths are connected together through splitters and attenuators to ensure that the topology does not change over time and between tests. This approach is used in 7-hop testing to ensure network topology. MAC filtering can also be used to implement network topology.
Large network testing is best done in an open-air environment where device behavior is subject to existing and changing RF conditions. Silicon Labs R&D facilities are used for this open-air testing.
The Silicon Labs R&D facility consists of a central core with an elevator shaft, other services at the west end of the building with an open floor plan, and offices and meeting rooms at the east end. The entire facility occupies approximately 120 feet by 200 feet.
Test equipment is installed at various locations around the facility. These devices all have Ethernet backchannel connections that allow:
-
Firmware Update
-
Command Line Interface
-
Script processing
-
Timing Analysis
-
Packet Collection
-
Energy Measurement
The test cluster is spread throughout the facility, including high and low locations, open areas, and enclosed conference rooms and offices. Devices are regularly added or removed from this test network, but at the time of this testing it consisted of the following devices:
This network represents the devices used by network and software quality assurance teams for open-air testing. All devices are controlled by a central test server and infrastructure, allowing scripted regression testing or manual testing by engineers.
in conclusion
Thread exhibits excellent reliability and latency below the 200 milliseconds required for typical human-device interactions. Even in multicast, large network conditions, Thread networks can handle traffic every 0.5 seconds and maintain latency and reliability. Because the network layer has flexible router configurations, Thread's network behavior remains almost unchanged even as the network scales.
Thread running on the EFR32 shows better multi-hop latency performance compared to the EM35x platform. This is expected as the device is a newer architecture, runs at a higher clock speed, and has more RAM for packet processing. As packet payload increases, latency in the network also increases, but this was less of an issue when testing 5, 25, and 50 byte payloads.
You can also scan the following QR code to follow Silicon Labs on social media platforms