Where to start testing LTE?

　　When working with any new technology, the biggest challenge facing production engineers is what to test and why to test it. This is especially tricky with complex devices like modern smartphones or tablets. With LTE, the complexity is unprecedented, and full testing requires leaving the device on the tester all day.

　　The fundamental assumption in production must be that the design delivered by engineering meets all customer requirements and functions consistently when assembled correctly. While supporting this assumption places a burden on the design team and its processes, without this assurance, the test coverage for today's extremely complex devices will be too large to check all possibilities. The manufacturing floor is not the place to verify millions of firmware production lines or the hardware functionality associated with multi-million gate digital signal processing (DSP)/application-specific integrated circuit (ASIC) designs.

　　The main goal of production test is to test as many mobile devices as possible to find manufacturing defects while minimizing test time. Software and digital designs have been verified in engineering and compliance testing. Digital integrated circuits have been extensively tested during their production process. Once a digital failure occurs, it usually results in catastrophic consequences such as the phone not turning on, not generating output, or not receiving signals. These failures are usually best found through simple techniques such as internal power-up testing and the use of checksums without any tester intervention at all. Therefore, optimal production testing focuses on physical layer measurements, which is the area that exhibits the greatest degree of variability associated with the manufacturing process.

　　The following sections discuss LTE testing and how to optimize testing with a physical layer tester such as LitePoint's Iqxstream (see figure).

　　Physical layer measurements

　　Physical layer testing focuses on the lowest level of the air interface. Its goal is to determine the consistency of important parameters that are essential for the successful transmission of wireless signals. Transmit power, transmit waveform quality, and transmit frequency accuracy are all critical to the performance of mobile stations. On the receiving end, the ability of a mobile device to successfully decode the received signal at both the lowest and highest signal levels is key to its successful operation in the network.

　　The 3GPP test specifications for LTE include a number of different tests that are used to determine compliance with the LTE specifications. Many of these tests overlap to some extent. Given the level of implementation in the digital domain, many measurements will not differ from one mobile device to another. User Equipment (UE) transmitter (Table 1) tests are generally considered sufficient to detect problems in a production environment.

Table 1: UE transmitter measurement indicators

　　To a large extent, adjacent channel leakage power ratio (ACLR), occupied bandwidth, and spectrum emission mask (SEM) are all addressing the same problem. There is usually some kind of degradation in the final part of the analog output chain, or there are noise sources within the DUT that are generating spurious signals. Therefore, only one of these measurements will be specified as part of the test plan.

　　Unlike the transmit chain where the final output is located at the antenna connector for evaluation, the receive signal remains internal to the DUT until it is fully decoded. Fortunately, while there are many components in the receive chain that can be degraded, nearly all degradations will show up in a receive bit error rate measurement at or near the receive threshold. Physical layer testers typically rely on the ability of the DUT to report receive test results. Since receive quality monitoring is an essential part of modern air interface operation, routing this data to an external terminal interface is a straightforward approach. Most, if not all, integrated circuit manufacturers support some form of bit error rate testing. [page]

　　Table 2: UE receiver measurement indicators

　　The two tests in Table 2 are used to verify reception performance. With the above combination of measurements, the challenge now becomes how to apply them to the almost infinite number of possible mobile device configurations.

　　LTE Test Plan Development

　　There are many approaches to test plan development, including: looking for possible failure modes in the design; using the recommendations of standards bodies; recommendations from integrated circuit manufacturers; and past history of similar devices in production.

　　Unfortunately, with new technologies such as LTE, there may be very limited experience on which to base a test plan. The various device manufacturers may not disclose the details of what goes on inside the design, and the manufacturers themselves may have limited experience with relatively new designs.

　　As a result, manufacturers often develop their own test plans and may fall back on standards bodies’ test specifications as a benchmark.

　　Table 3 represents a test plan developed for an LTE user equipment transmitter. Although there are several additional tests we would like to perform before declaring the device under test (DUT) “qualified” for production testing, this subset is useful for this discussion.

　　Each column in Table 3, from left to right, represents a test configuration, with each configuration specified by the parameters at the top of each column. Generally, when discussing test configurations, we are talking about the steady state that the DUT is in, such as constant modulation rate and constant power level. The bottom half of each column represents the measurements to be performed for each configuration.

　　
　　

　　Table 3: Test specifications from 3GPP

　　We will walk through the development of this test plan section by section in Table 4. The test developer in this case has extensive LTE knowledge and a good understanding of the 3GPP test specifications for LTE - he is considered an expert in testing mobile devices of other technologies.

　　Table 4: Discussion of test specifications from 3GPP

　　Note that the author used an RX power level of -57dBm for all tests. Since the RX power is not directly related to the TX measurement, it does not matter what level it is set to.

　　Note: It is generally assumed that the test plan will be applied consistently across the various bands/channels depending on the capabilities of the DUT. 3GPP recommends testing the device using the low, mid and high channels of each band. With certain channel allocations, this may mean that only a single channel is tested.

　　From a test coverage perspective, the author of this test plan did a very good job: He tested the performance boundaries of the DUT. He tested the maximum and minimum RB (resource block) allocations, the maximum and minimum modulation rates, and the maximum and minimum power levels. He looked at variations across the channel in terms of RB allocations. This testing was taken from and fully complies with the recommendations of the 3GPP test specification. It is unlikely that a test plan like this in production would miss many, if any, issues.

　　Let's examine this test plan in terms of test throughput, since our goal after all is to efficiently test the DUT as quickly as possible. Two things stand out when you look at this plan. The tables are very sparse, but the number of configurations is very high.

　　Given that Iqxstream supports a separation of data capture and analysis, test time is largely determined by the configuration capture cycle rather than the number of measurements calculated per capture. This suggests that a test plan optimized for throughput minimizes the number of configurations while increasing the number of measurements. This favors narrower tables with greater measurement density.

　　Let's also examine how the test engineer selected different configurations. In the example above, the tests fully investigated one set of parameters and then orthogonally moved on to the next set. Tests 1-3 fully investigated changes in RB offsets, then varied the RB block size and again investigated the effects of different offsets. In a lab environment, this control is critical to tracking down the source of unacceptable variations in a design, but in a manufacturing test environment, this orthogonality is less important.

　　Simple defect example

　　Let's take a look at a simple example of how imperfections in analog performance can occur. Assume that the post-modulation analog filter is frequency offset and the cutoff frequency intrudes into the upper edge of the channel. The result will be that the power output will be low at the upper edge of the channel. This failure will show up in both the 1RB test and the 12RB test on the upper side of the band, which is the power measurement in test configurations 3 and 8. The failure will also show up in the EVM flatness measurement of the 50RB block. [page]

　　Remember, in production we simply want to determine if the DUT is "good" or "bad." Once it is identified as "bad," the DUT can be set aside for further inspection and repair. If isolating the problem significantly increases test time, then the testing required to isolate the problem is not necessary and should not be performed on the production line.

　　It would then be logical to remove configurations 3 or 8 from the test table since they provide similar test coverage. These types of duplications often appear throughout a test plan. While some duplication may be desirable or necessary, it should not be wasteful.
　

　　Compression test plan

　　When looking at the original test plan, compression is an appropriate term to use to improve it for execution on Iqxstream. The total test time will be largely determined by the number of test configurations. And by separating the analysis portion from the data capture, we can complete many more measurements for a given capture with minimal cost. Therefore, our goal should be to complete more measurements per capture while reducing the number of test configurations. Let's walk through this compression operation (Table 5).

　　

　　Table 5: LTE test plan reduction

　　The various modulation schemes in a cell phone generally use different data paths in the circuit. Therefore, although we would like to check all modulation schemes in a cell phone, it is likely that we do not need to verify all variations. This is because they are generally generated in the digital domain and are not affected by analog variations.

　　Let's start by picking the configurations that we absolutely want to keep. Configurations 1, 12, and 20 examine extreme cases of modulation and RB allocation, and configuration 4 provides a moderate and perhaps typical test of RB allocation. These four configurations are logical candidates to keep.

　　For TX quality measurements, the focus should generally be on the maximum power measurements, as these measurements are generally the most challenging for high power circuits. If there are output power variations from one edge of the band/channel to the other, they will show up in the single RB quota measurements. Therefore, Configuration 3 with the maximum RB offset should also be retained as a complement to the minimum RB offset of Configuration 1.

　　Configuration 2 is a candidate for being cut because it only tests the middle RB offset of the channel. This intermediate measurement is of little value because simulation problems will usually manifest themselves across the band or at the edge of the band.

　　The simulation was tested at the band edge at different RB offsets using configurations 1 and 3. Therefore, we can safely eliminate configurations 8 to 11, as they differ from configurations 4 to 7 only in RB offset.

　　Configuration 20 represents a form of stress testing on the PA and other high power circuits, while Configuration 21 actually represents the most realistic case of maximum rate operation. Maximum rate operation is only possible when close to the base station, so the PA is generally set to a low power setting. We should complete at least one TX quality measurement at low power because the PA will operate in a very different mode 63dB below its maximum power setting. Therefore, Configuration 21 is still a valuable test configuration.

　　Configurations 14 and 15 can be used to test a specific absolute power setting capability, but the premise is that this capability should be consistently applicable over the entire operating range where any intermediate power measurements are likely to be made. Therefore, we will retain Configuration 5 as a measurement of the ability to set intermediate power levels and remove Configurations 14 and 15.

　　Let's go through the rest of the configuration and see what's left.

　　Configurations 6 and 7 reduce the power to -30dBm and -40dBm respectively, but there is no reason to believe that the simpler modulation and lower RB allowances of these tests will reveal any issues that the more complex waveforms of Configuration 21 do not reveal, and there will not be a huge difference between -30dBm and -40dBm, so these tests can be removed.

　　The same reasoning applies to test configuration 13. This is also a simpler configuration than test 21.

　　The same is true for test configurations 16, 17, 18, and 19. These tests validate operation under RB offset and power variations for a simpler 12RB allocation operating at 16QAM. Different RB offsets have been validated in configurations 1 and 3, while the modulation schemes were confirmed in configurations 20 and 21. So these four tests are candidates for removal.

　　In the process we removed a lot of test configurations, but as mentioned earlier we are looking at a sparse test matrix. Since there is little or no cost associated with adding measurements to a particular test configuration, let's fill in some of the gaps.

　　More tests were added to the test plan (Table 6) and some further adjustments were made.

　　

　　Table 6: Compression test plan adjustments

　　Conclusion

　　The main goal of production test is to test as many mobile devices as possible to find manufacturing defects while minimizing test time. To achieve this, using IQxstream's "capture once, measure many" capabilities can gain significant advantages in test speed and overall test coverage.

　　Compared to a plan centered around 3GPP test specifications, a compressed plan with similar test coverage can run in just one-third the time.

　　For complex air interfaces such as LTE, it is important to narrow down each parameter in the test plan development, confirm what parameters will stress the DUT, and identify parameters that IC testing and lab testing have proven locked into the digital design.

　　IQxstream represents a new value proposition when discussing production test versus the more common lab test environment. Its multi-DUT functionality and “capture once, measure many” capabilities combined with an architecture that separates data capture from analysis, deliver unprecedented levels of throughput and flexibility to manufacturing environments.