Three potential problems of embedded software and their testing technologies

This article will show you how to avoid those subtle yet common errors and introduce several techniques to help engineers find hidden errors in software. Most software development projects rely on a combination of code inspection, structural testing, and functional testing to identify software defects. Although these traditional techniques are very important and can find most software problems, they fail to detect many common errors in today's complex systems.

Structural testing, or white box testing, is effective in finding logic, control flow, calculation, and data errors in code. This testing requires a clear view of the inner workings of the software (hence the term "white box" or "glass box") to understand the details of the software structure. It checks every conditional expression, mathematical operation, input, and output. Because of the many details that need to be tested, structural testing checks one unit of software at a time, usually a function or class.
Code review also uses the same complex techniques to find implementation defects and potential problems. Like white box testing, reviews are usually conducted on individual units of software because an effective review process requires focused and detailed inspection.

Unlike reviews and white box testing, functional testing, or black box testing, assumes no knowledge of the software's implementation and tests outputs driven by controlled inputs. Functional testing consists of test procedures written by testers or developers that specify the expected program outputs for a specific set of program inputs. After the tests are run, testers compare the actual outputs to the expected outputs to look for problems. Black-box testing is effective in finding unimplemented requirements, interface problems, performance problems, and bugs in the most commonly used functions of a program.

While these techniques combined can find most of the bugs hidden in a particular software program, they have limitations. Code review and white-box testing focus on only a small portion of the code at a time, ignoring the rest of the system. Black-box testing typically treats the system as a whole, ignoring implementation details. Some important problems can only be discovered when the details of their interaction within the entire system are focused; traditional methods cannot reliably find these problems. The software system must be examined as a whole to find the specific cause of a specific problem. Because it is usually impossible to thoroughly analyze every detail in the program and its interaction with all other parts of the code, the analysis should focus on specific aspects of the program that are known to cause problems. This article will explore three of these potential problem areas:

* Stack overflows
* Race conditions
* Deadlocks

Readers can read the second part of this article online, which will explore the following issues:

* Timing problems
* Reentrancy conditions

All of the above problems are quite common in systems that use multitasking real-time design techniques.

Stack Overflows

Processors use stacks to store temporary variables, pass parameters to called functions, save thread "state", and so on. If the system does not use virtual memory (in other words, it cannot transfer memory pages to disk to free up memory space for other uses), the stack is fixed to the size it was shipped with. If for some reason the stack grows beyond the amount allocated by the programmer, the program becomes undefined. This instability can cause serious system failure. Therefore, it is critical to ensure that the system can allocate enough stack in the worst case.

The only way to ensure that stack overflows will never occur is to analyze the code, determine the maximum stack usage of the program under various possible conditions, and then check whether enough stack is allocated. It is unlikely that the test will trigger a specific combination of transient inputs that will cause the system to behave in the worst case.

The concept of stack depth analysis is simple:

1. Build a call tree for each independent thread.
2. Determine the stack usage of each function in the call tree.
3. Examine each call tree and determine which call path from the root to the outer "leaves" requires the most stack usage.
4. Add the maximum stack usage of each individual thread call tree.
5. Determine the maximum stack usage of each interrupt service routine (ISR) within each interrupt priority level and calculate the sum. However, if the ISR does not have its own stack and uses the stack of the interrupted thread, then add the maximum stack used by the ISR to the stack of each thread.
6. For each priority level, add the amount of stack used to save the processor state when an interrupt occurs.
7. If an RTOS is used, add the maximum amount of stack required for the RTOS's own internal purposes (as opposed to system calls initiated by the application code, which are included in step 2).

There are two more important things to consider. First, the call tree built from high-level language source code alone is likely to be incomplete. Most compilers use a run-time library to optimize common computational tasks, such as multiplication and division of large integers, floating-point operations, etc. These calls are only visible in the assembly language generated by the compiler. Runtime library functions themselves may use a lot of stack space, so they must be included in the analysis. If using C++, all of the following types of functions (methods) must also be included in the call tree: constructors, destructors, overloaded operators, copy constructors, and conversion functions. All function pointers must also be resolved and the functions they call included in the analysis.