Confidence and accuracy estimation methods to speed up bit error rate testing

In the bit error rate test of wireless systems, the reliability of test results can be improved and the test time can be shortened by estimating the confidence and accuracy before the test. This article introduces the confidence and accuracy estimation method to speed up the bit error rate test.

Most engineers are familiar with the term "bit error rate" (BER) and related tests. BER testing is essentially outputting a known data bit stream to the device under test, then capturing and analyzing the data stream returned from the device under test. In order to make different instruments have the same test results, a special pseudo-random sequence is often applied, which is a standard defined by the communications industry.

In today's communications world, the telephone is of general interest along with other industries such as wireless data. Wireless communications are themselves very dynamic and include many sub-technologies such as point-to-point microwave communications, satellite communications, radio frequency broadcasting and "two-way" communications with cell phones.

Several instruments other than bit error rate testers have entered various technical fields to measure the performance of related systems or components. For example, protocol analyzers are a special test method that is not universal, but can test bit error rates concisely and effectively.

Furthermore, engineers designing wireless systems need a fast, accurate way to simulate basic wireless communication channel interference (such as noise and attenuation) and verify the performance of the designed product in this environment. Evaluating a unique wireless system is a very complex task, especially if all the digital modulation schemes used today are taken into account.

For wireless systems, it is necessary not only to test the bit error length, but also to pay attention to what kind of background noise causes so many bit errors. Therefore, wireless testing usually requires evaluating the corresponding relationship between the bit error rate and different levels of simulated background noise, which is represented by Eb/No (energy per bit to noise density) or bit signal-to-noise ratio.

Applying AWGN (Additive White Gaussian Noise) to wireless devices can generate a "BER - Eb/No" curve: commonly called a waterfall curve. The waterfall curve shows the relationship between the bit error rate BER and the decreasing background noise. The waterfall curve is the final test result and is an important factor in comparing the performance of wireless systems.

The most immediate goal of a responsible engineer is to produce test results efficiently and make credible comparisons. Therefore, understanding the possible distribution of noise interference can effectively perform the test work. In simple terms, he can make a compromise between acceptable test time and the desired confidence level and accuracy level. When the test is completed, the confidence and accuracy of the test can be applied to the waterfall curve, so that this curve can be used for credible quantitative comparisons.

This article analyzes the problems faced by such tests, describes in detail the requirements of today's tests, and explains the important performance of Fastbit FB100A bit error rate tester and FB200A noise generation and channel interference simulator as test and measurement instruments.

A communication channel model

In all practical cases, wireless communication systems can be divided into some simple modules. We give a block diagram of this model and explain how the FB100A Bit Error Tester performs the test. The source of the information is a person speaking or a string of characters. The sensor converts this information into an electrical signal that can be processed by a computer or processor. This basic form is a bit or binary number.

Obviously, an undefined data stream does not represent the transmission of information. A set of rules is needed to convert basic recognizable speech words or strings into recognizable data streams. This is the basic function of the source encoder and is usually the starting point for the work of wireless communication engineers.

Once the engineer knows the type and form of information (i.e., the information source and the method of source encoding), he needs to estimate the possible damage and degree of damage to the information transmitted by the transmission channel. This is where error testing is superior to other testing methods. In short, the engineer's job is to measure the bit error or bit error rate for the information source of a given communication channel and design the appropriate encoder/decoder and modulator/demodulator for the circuit, or find ways to reduce these errors.

Bit Error Rate Test and FB100A Meter

The Fastbit family of products for wireless data communications testing supports serial and parallel communications, programmable 4M-byte data word streams, and an additional 4M-byte programmable memory for packetized or framed data. Designed for R&D engineers, the FB100A is not suitable for low-end error testing and field protocol analysis verification. The features included in the FB100A, such as the ability to test packet and frame errors, and the large memory space for defining and analyzing long sequences, make it impossible to compete with low-end BER test equipment on price. The programmability of the FB100A makes it very valuable to R&D engineers because it is not limited to a specific protocol. The FB100A is not designed to replace a dedicated laboratory protocol analyzer, nor can it replace expensive, high-speed (Gbs) error testing (such as required in SONET system development laboratories). The FB100A provides 100Mbs serial data streams and up to 160Mbs parallel data streams. Therefore, the FB100A is an ideal choice for laboratory engineers who need flexibility and high performance but have a tight budget.

Most low-end error testers only use the data stream generator as a slave function, and even if they have a small memory, they can only be used to generate and analyze PRBS (pseudo-random code) serial data streams. Most protocol analyzers are just as their names suggest - they can only generate and analyze a specific sequence. FB100A provides outstanding performance in this regard, whether it is a user-defined sequence or a variety of standard PRBS sequences provided by the instrument, its INSTALOK synchronization technology can provide fast synchronization of data bit streams and error analysis.

The FB100A's unique INSTALOK technology includes a fast two-step process that provides special frame structure header synchronization and fast PRBS data stream synchronization regardless of whether the PRBS data stream is embedded in the frame structure payload. In addition, the PRBS data stream will start at the end of the first frame structure header (i.e., the starting point of the first frame payload) and terminate at the end of the frame structure payload, so that the next frame structure header is transmitted seamlessly and continuously, and then the PRBS sequence is continued from the last termination point in the payload part of this frame. In this way, the complete PRBS sequence can be transmitted in the frame structure without any limitations in the early BER test. Therefore, simple setup and testing are achieved, avoiding the complex problems encountered by other instruments, such as designing and installing external logic circuits or subsystems.

Another unique feature of FB100A's INSTALOK technology is that it provides word (byte-width information) and word block bit error rate testing. When a word represents a frame structure header, all bits in the header are ignored, thereby simulating the frame structure header masking function in the protocol. In parallel data transmission, each bit of the word can be programmed and regarded as a separate channel, and the bit error test can be targeted at the word or any of the 8 channels. Therefore, whether the data being analyzed is a data bit stream, a byte-width word, a word block, or a frame and frame structure header, a comprehensive and accurate bit error rate test can be performed.

The FB100A has two more ports or channels (in addition to the data and clock ports and their complements). These extra ports can be used as one or two additional control lines, the frame structure synchronization channel and its complement; or the single-ended frame structure synchronization channel and an additional single-ended frame structure data valid indicator synchronization control line. These additional ports can be used to test simple data receivers that include these control lines in the design, avoiding additional upfront hardware investment.

Another important feature of the FB100A is that the instrument's internal data generator and data analyzer can be used as separate instruments. The data generated by the generator can be different from the data analyzed by the analyzer. This is an important tool when you need to analyze signals from the device under test (rather than the FB100A). This feature is due to the use of separate 4M bytes of memory for the generator and analyzer in the FB100A. For the versatile control line function, the FB100A data generator has an additional 4M bytes of programmable memory for a unique frame structure synchronization sequence.

Finally, the FB100A's versatility in the physical layer provides users with a variety of different choices for logical interface interfaces. In serial communication mode, standard outputs provide TTL and ECL, or TTL and PECL logic levels. In parallel communication mode, users can purchase a variety of POD options, including byte-wide data and control lines. These parallel PODs provide user equipment connections through the FB100A standard physical logic interface. Interfaces include: MPEG using SPI (same frequency serial interface), ASI (asynchronous serial interface), or RS422, as well as more common serial data transmissions such as RS232, RS449 and HSSI, including logic levels such as TTL and LVDS.

When the functions of the general data generator and protocol analyzer mentioned at the beginning are added, the above performance of FB100A makes a simple bit error rate test device become a powerful communication analysis tool.

AWGN and Gaussian Probability Analysis

In theory, you can measure for an infinite amount of time to get accurate BER test results. However, in practice you need to test for a relatively short time. Therefore, the true BER statistics may be significantly higher or lower than your test results.

By adding weights of known statistical probabilities to the system, you can describe the occurrence of bit errors with a known probability distribution. Using the probability distribution, you can simply describe a single test with a confidence level and an estimate of the actual accuracy. This approach can produce repeatable test results in a short time with acceptable uncertainty.

Before giving the formula, it is helpful to review confidence levels and precision estimates. The confidence level defines the probability that the actual bit error rate is contained within the precision range of your test. Precision is related to the difference between the test bit error rate and the true bit error rate. You usually express it as a percentage.

As an example, suppose you run a test and record 100 bit errors. If you set the confidence level to one standard deviation (or 68.27%), the test accuracy is 10%. That is, there is a 68.27% chance that the true bit error rate falls within 10% of the bit error rate test result. If you run the test 10,000 times, 6,827 times are within 10% of the test result error, and the remaining 3,172 times are outside the 10% range of the test result error.

Calculate bit error rate test parameters using Gaussian probability distribution

The Gaussian probability distribution provides a formula that includes a confidence level and an accuracy estimate for bit error rate measurements. The basic formula relates the confidence level and the accuracy estimate to the number of errors actually measured. You can use this formula to solve any three-parameter problem using two of the parameters. Most often, you use it to solve accuracy estimates, but it also solves the problem of how many errors you need to measure for a given level of accuracy and confidence. The second formula presented in this article describes the relationship between the minimum error-free test time and the ideal upper limit of bit errors that can be trusted.

The first formula provides a calculation for an estimate of the expected accuracy assuming that the bit error is not zero. You must also set a confidence level when performing this calculation. This equation solves for the measurement accuracy and can be expressed as a ± error for bit error rate testing. This error factor is independent of the test time and bit error rate; it depends only on the number of bit errors tested.

Accuracy = s/vn

s: Standard deviation validity period n: Number of bit errors

The standard deviation is directly related to the confidence level. Tables 1 and 2 show examples of the relationship between confidence level and standard deviation. The value of the standard deviation in the equation above represents the confidence level; the confidence level represents the probability that the true bit error rate falls within the accuracy range you calculated.

For example, suppose a test result has 4,331 bit errors. The desired confidence level is 99.9%, and you use a standard deviation of 3.29053. You can calculate that the accuracy is about 5%:

precision = 3.29053/v4331

=3.29053/65.8027

=0.050006

You can use the same formula to determine the number of bit errors required to achieve the desired accuracy and confidence. This equation effectively determines when to end the test, and you can terminate the test when the number of bit errors reaches this value. Solving the problem of calculating the number of bit errors, the formula becomes:

Number of bit errors = (s/a)/v

s: standard deviation; a: expected accuracy

Continuing with the previous example, assume the target accuracy is ±5% and the confidence level is 99.9%. The number of bit errors to be tested is 4,332:

Bit error rate = (3.29053/0.05)/v

= 65.81062/v

= 4331.035

Notice the relationship between confidence, precision, and the number of bit errors in the equation. Confidence and precision are inversely proportional, while they are both directly proportional to the number of bit errors. Basically, as confidence increases, precision deteriorates or the number of bit errors increases. The same relationship applies as precision increases, confidence decreases, or the number of bit errors increases. This concept is very useful when you use formulas to determine confidence, precision, or test time before testing.

Calculation of bit error rate parameters when there is no bit error

The formula introduced above determines the accuracy but requires the number of bit errors. Note that this assumes that the number of bit errors will accumulate during the test. But what happens when the test is not error-free? Since the bit error rate is zero, it makes no sense to set the accuracy, because any accuracy multiplied by zero will result in zero. You cannot assume that the true bit error rate is zero because you only tested one sample. Therefore, you need to find a reliable upper limit on the bit error rate for measuring the zero bit error case.

To perform this test, you need to consider a different probability distribution, called the Poisson distribution. To analyze the Poisson distribution, you use a formula to calculate the probability of an event that is extremely unlikely to occur again. If the true bit error rate is known, you can use the formula to calculate the probability of zero bit errors in a given period of time:

P(0)=e - rT

R: frequency of bit errors T: test time

This formula gives the probability of testing zero bit errors. This probability can be converted to confidence. The probability of zero bit errors is equal to the chance that the true bit error rate is equal to or higher than the bit error rate set in the formula. Therefore, if you transform the probability to "1-P(0)", it becomes the probability that the true bit error rate is equal to or lower than the bit error rate in the formula.

Before using the formula, take a look at the following example. First, assume an error-free test lasts 4.6 seconds. Substituting 1 bit error per second into the formula gives a 1% probability. Therefore, at this bit error rate, you are 99% likely to find a bit error during the test time. If you increase the bit error rate to 2 bit errors per second, the probability rises to 99.99%. Therefore, you are 99% sure that the true bit error rate is less than 1 bit/second, and 99.99% sure that the bit error rate is less than 2 bits/second.

Now you can flip the equation around. Instead of calculating the probability, you can calculate the upper limit of the bit error rate and the required test time. Substitute the confidence level in place of the zero bit error probability, and subtract the confidence level from 1 to get the zero bit error probability. Likewise, replace the frequency of errors and the test time with the bit error rate and the number of bits tested. The number of errors per second times the time equals the bit error rate times the number of bits tested:

P(0)= e-rT

Using the previous equation, substitute confidence level, bit error rate, and number of bits.

1-C =e-Rb

R: bit error rate upper limit b: number of bits tested

ln(1-C)=-Rb.

Calculate the upper limit of the bit error rate:

R =-ln(1-C)/b

Calculate the number of bits needed to test:

b =-ln(1-C)/R.

The first two equations above determine the upper limit of the credible bit error rate based on the number of error-free bits. When an error-free test is completed, you choose the desired confidence level, and the equation provides a credible upper limit of the bit error rate. For example, if you choose 1,000,000 error-free test bits and a confidence level of 99.99%, the upper limit of the bit error rate is 9.21x10-06:

BER=-ln(1-0.9999)/1,000,000

=9.21x10-06

The second equation is for pre-test estimation. It determines the number of error-free bits that need to be tested to meet the bit error rate standard. You need to define a bit error rate level, if the true bit error rate is confidently lower than this level, the test does not need to be performed. This way, you can determine when to end the test and still get a reliable test. For example, if the bit error rate level is 1.00x10-06, the confidence level is 99%, and the number of bits required to be tested is 4,605,171:

Bits=-ln(1-0.99)/1.00x10-06

=4,605,170.186

You can use these equations and the previous section to analyze test results. You can also use it to estimate test times before testing. For error-free testing, these estimates only require the actual bit rate. But to provide a useful estimate, you must predict the range of bit error rates. You can do this for wireless systems using waterfall curve theory calculations, which can be found in many textbooks.

Waterfall curve and test time estimation

Typically, test results are presented as waterfall curves showing the bit error rate versus the decreasing background noise. Some waterfall curves are the lowest theoretical limit lines that can be achieved for a given modulation method. These theoretical waterfall curves provide a starting point for estimating test time. The bit error rate you test will be at or above this level. Using real signals, the bit error rate tested will be close to or significantly higher than the theoretical line. Through some initial tests, you can test this bit error rate and, if necessary, make a new curve representing the expected bit error rate.

For each point in the curve you wish to test, you can accurately estimate the test time by substituting the theoretical or expected bit error rate into the equation as the actual bit error rate. You can examine these estimates to determine which points take the most time. Similarly, the confidence level can be set to a fixed value, and by adjusting the target accuracy and the upper limit of the reliable bit error rate, you can control the estimated time so that the test time can be set to the optimal state.

This estimation makes the test results reliable and meets the test requirements without wasting test time. When the test is finished, you can analyze the test work. You can apply the calculated accuracy to each bit error rate measurement and find the bit error rate upper control line to apply to each error-free test. You can apply these tests to the waterfall curve of the test results for more reliable and quantitative comparison.

The unique test interface of the instrument FB2000A can adjust the accuracy, confidence and test time before a single test or a group of tests. Figure 2 shows this interface.

Building Confidence in Wireless Data Communications Testing

For wireless systems, confidence and accuracy in testing are somewhat difficult to understand. However, by taking the time to make these estimates before testing and analyzing them after testing, you will reduce overall test time and make test results more reliable and quantitative. Before any testing, determine all parameters regarding confidence and accuracy. Determine the bit error rate limits. At this time, you do not need to measure the lowest bit error rate for any Eb/No value, and use these Eb/No test points to determine the test time estimate.

Next, fill in the test plan by adjusting the confidence, accuracy, and bit error rate limits based on the acceptable test results. Then, use these parameters to calculate the criteria for ending the test. This criterion is expressed as the number of bit errors and the number of error-free bits required for each Eb/No test point.

When the test is complete, confidence and precision can be applied to the measurement to perform reliability analysis of the waterfall test results.

The FB100A and FB2000A instruments provide Windows NT graphical interface control and graphics windows, automatically displaying the bit error rate-time curve graph, including indicator functions and additional traditional table data results. In addition, when the FB100A and FB2000A instruments are combined, the bit error rate-Eb/No test is simplified. The unique drawing program provides integrated testing and test result display. Figure 3 shows a typical waterfall curve drawing diagram from the Fastbit 100 and 2000A instruments.

Author: Yan Yong

Technical Support Engineer

Aeroflex Asia Pacific Ltd.

Email: harry.yan@ifrsys.com.cn