Various techniques for finding defects in embedded C programs/software

Software verification techniques such as pattern-based static code analysis, runtime memory monitoring, unit testing, and data flow analysis are effective ways to find defects in embedded C language programs/software. Each of the above techniques can find a specific type of error. Even so, if users only use one or a few of the above techniques for verification, such a verification method is likely to miss some defects in the program. A safe and effective strategy to solve this problem is to use all the complementary techniques in the above software verification at the same time. In this way, a solid framework can be established to help users check for defects that may escape a specific technique. At the same time, users naturally establish an environment that can detect critical and difficult-to-find functional errors.

This article will detail how automated techniques such as pattern-based static code analysis, runtime memory error detection, unit testing, and data flow analysis are used together to find defects in embedded C language programs/software. This article will use Parasoft C++test as an example to demonstrate each of the above techniques. C++test is an automated integrated solution that has been proven to improve the development efficiency and software quality of software development teams through a wide range of best practices.

It is important to pay attention to the screenshots in the article when reading this article and whenever you think about the defects you find. Automating the detection of bugs such as memory corruption and deadlocks is undoubtedly an essential task for any development team. However, the most lethal bugs are functional errors, which are often difficult to find automatically. We will briefly discuss techniques for finding these bugs in the conclusion of this article.

Scenario Introduction

To give a concrete example, we will introduce and demonstrate the recommended bug finding strategy for a case we recently encountered: a simple sensor application running on an ARM board.

Assume that we have built the application system, but when we upload the program to the system target board and try to run it, we do not see the expected output on the LCD screen. We

are not sure why the system does not work properly, so we try to debug the system, but debugging on the target board is time-consuming and annoying. Because we have to manually analyze the debugger results and try to manually determine the real cause of the problem. Or we can use some tools or techniques that are proven to automatically locate the errors to help us relieve the burden.


At this point, we can either hope for good luck with debugging the program using a debugger, or we can try to use an automated testing strategy to find the errors in the code. If automated techniques still don't help us find the bug, then we have to fall back on using the debugger as a last resort.

Pattern-based static code analysis

Here, we assume that debugging with a debugger is only done when absolutely necessary, so we start by running pattern-based static code analysis. It will find issues like the following:

This is a violation of MISRA rule, which indicates that there is something fishy about the assignment operator here. Indeed, the programmer intended to use comparison operators instead of assignment operators. So we modify the detected conflict here and rerun the program.

We find some improvement: some output is displayed on the LCD screen. However, the program crashes due to an access violation. So we need to make a choice again. Should we use the debugger or continue to use automatic error detection techniques. Since experience has shown that automatic error detection techniques can effectively detect problems such as memory corruption encountered in our current program, we decided to use runtime memory monitoring to find the problem.

Runtime memory monitoring of the entire program

For runtime memory monitoring, we use C++test to instrument the application. Such instrumentation is lightweight, so the instrumented program is suitable for running on the target board. After uploading the program to the target board and running the instrumented program, we download the results to the PC and the following errors will be reported:

The result indicates that an array out-of-bounds read error has occurred at line 48. Obviously, the value of the msgIndex variable must be outside the range of the array. If we follow the stack trace to the previous level, we will find that the value indicated by the printed message here is indeed outside the range of the array (because we gave an incorrect condition before calling the printMessage() function). We can delete the unnecessary condition (value <= 20) to fix this error.

void handleSensorValue(int value)
{
initialize();
int index = -1;
if (value >= 0 && value <= 10) {
index = VALUE_LOW;
} else if ((value > 10) && (value <= 20)) {
index = VALUE_HIGH;
}
printMessage(index, value);
}

Then we rerun the program and no memory error will be reported. When we upload the program to the target board, it seems to be working as expected. However, we still have some concerns.

We only found one instance of a memory overwrite in the code path we executed. How can we conclude that there will be no memory overwrite errors in the code we have not yet executed? If we check the coverage analysis, we will find that the reportSensorFailure() function has never been executed. We need to test this function, but how to do it? A good way to do this is to create a unit test case that calls this function.

Use runtime memory monitoring in unit tests: We use C++test's test case wizard to create a test case skeleton and add some test code to it. Then run the test case - to check the untested function mentioned above, and turn on the runtime memory monitoring function. Using C++test, the whole process only takes a few seconds.

The results indicate that the function has been covered, but new errors have also been found:

Our test case found more memory related errors. Apparently, we had a problem with initializing memory (null pointer) when the failure handler was called. Further analysis revealed that we had a function call order error in reportSensorValue().

finalize() was called before printMessage(), but finalize() freed the memory that printMessage() needed.

void finalize()
{
if (messages) {
free(messages[0]);
free(messages[1]);
free(messages[2]);
}
free(messages);
}

After changing the function call order, we rerun the program.

This solved the first error in the report above. Now let's analyze the second error in the report: the AccessViolationException in the printed message. This error is caused by the uninitialized message list. To solve this problem, we call initialize() once before printing the message to initialize it. The modified function looks like this:

void reportSensorFailure()
{
initialize();
printMessage(ERROR, 0);
finalize();
}

When we run the test case again, only one task is reported: an unvalidated unit test case, which is not actually an error. We just need to validate the output to convert this test case into a regression test. C++test will automatically do this for us by creating the appropriate assertions.

Next, we run the entire program again. Coverage analysis tells us that almost the entire program has been covered and no memory errors have been found.

Is this the end? Actually, not really. Although we ran the entire program and created unit test cases for uncovered functions, there are still some paths that are not covered. We can still continue to create unit test cases, but it will take a long time to cover all paths in the program in this way. Or we use another method and use data flow analysis to simulate these paths.

Data flow analysis

We use C++test's BugDetective to perform data flow analysis. BugDetective can simulate different paths in the system and check whether there are potential problems in these paths. After data flow analysis, we get the following results:

After carefully analyzing the results of the report, we found that there is an uncovered potential path in the program that may cause two free operations in the finalize() function. In the program, the reportSensorValue() function calls the finalize() function, and then the finalize() function calls free(). At the same time, the finalize() function is also called by the mainLoop() function. We can modify the finalize() function to make it more intelligent to fix this problem. The modified code is as follows:

void finalize()
{
if (messages) {
free(messages[0]);
free(messages[1]);
free(messages[2]);
free(messages);
messages = 0;
}
}

Now we run the data flow analysis again, and the result will only have two problems:

Here we could have used -1 as index to access the array. This is because the integer variable index is initialized to -1, and there is a possible path through the if statement that calls the printMessage() function before the integer variable is properly initialized. Runtime analysis does not detect such a path, and it is very likely that this path will never be executed in the real world. This is the main disadvantage of static data flow analysis compared to real runtime memory monitoring: data flow analysis can detect potential paths, which may include paths that will never be executed or do not exist during the actual execution of the program. However, to be on the safe side, we remove the unnecessary condition (value>=0) above to fix this potential error.