Helium Technical Lecture | Overcoming the influence of Amdahl's Law

This article is reproduced from: Arm Community

Serial code is common in loop processing, and the overhead caused by serial code can be considerable, especially for small loops. The following memory copy code is a good example:

The processor must record several pieces of data:

Keeping track of all this data while handling an interrupt can be problematic, so some DSPs simply delay the interrupt until the loop is complete. This can take quite a long time if there are a large number of iterations to perform, and is completely inconsistent with the requirements for fast and deterministic interrupt latency that a Cortex-M processor should achieve. This approach is also not suitable for handling precise faults, such as memory management fault exceptions (MemManage) caused by permission violations. Another approach is to add additional registers to handle loop state. But these new registers must be saved and restored on exception entry and return, which in turn increases interrupt latency. To address this problem, Armv8.1-M uses a pair of loop instructions:

Adding a loop instruction at the end of the loop has a nice side effect that if the cached loop information is refreshed, the instruction will be re-executed. Re-executing the LE instruction will then refill the cache. As shown in the figure below, the existing fast interrupt handling function is preserved because there is no need to save the loop start and end addresses.

Apart from the first iteration and some setup when recovering from an interrupt, all the time is actually spent in memory copying rather than loop processing. Furthermore, since the processor knows the order of the instructions in advance, it can always fill the pipeline with the correct instructions. This eliminates pipeline flushes and the resulting branch penalties. Therefore, we can vectorize this loop and no longer worry about the effects of Amdahl's law, and we have (temporarily) overcome these difficulties.

When vectorizing code, a loop often starts and ends with different types of instructions, such as vector load (VLDR) and vector multiply-add (VMLA). When executing such a loop, a long, uninterrupted chain of alternating VLDR/VMLA operations is generated (as shown in the figure below). This uninterrupted chain allows the processor to gain the most benefit from instruction overlapping, as it can even overlap from the end of one loop iteration to the beginning of the next, further improving performance. For more information on instruction overlapping, see: "The origin of Arm Helium technology: why not just use Neon?"

Vectorized code has problems when the amount of data to process is not a multiple of the vector length. The typical solution is to process the full vector first, then use a serial/non-vectorized tail cleanup loop to process the remaining elements. Before you know it, Amdahl's Law is back, and it's annoying! Vectors in Helium can hold 16 8-bit values, so when we vectorize our 31-byte memcpy function, less than half of the copies will be performed serially by the tail loop instead of in parallel by the vector instructions. To address this, we added tail-predicted variants of the loop instructions (e.g., WLSTP, LETP). For these tail-predicted loops, the LR holds the number of vector elements to process instead of the number of loop iterations to execute. The loop start instruction (WLSTP) has a size field (".8" in the memcpy function example below) that specifies the width of the elements to be processed.

If you've seen other optimized memcpy routines, you might be surprised at how simple this example is, but for Helium, this is all you need for the best fully vectorized solution. Here's how it works: the processor uses the size field and the number of remaining elements to calculate the number of remaining iterations. If the last iteration has fewer elements to process than the vector length, the corresponding number of elements at the end of the vector will be disabled. So in the example of copying 31 bytes above, Helium will copy 16 bytes in parallel on the first iteration, and then 15 bytes in parallel on the next iteration. This not only avoids the impact of Amdahl's law and achieves the expected performance, but also completely eliminates the serial tail code, reducing the amount of code and simplifying the development process.

With high performance goals and tight area/interrupt latency constraints, we designed Helium like a multi-dimensional puzzle where half of the shape is fixed. Seemingly unrelated parts of the architecture can interact with each other to produce unexpected effects or help solve interesting puzzles.

The entire Helium research team and I are looking forward to seeing Helium technology bring strong support to new applications. Currently, three Cortex-M products support Helium technology - Cortex-M52, Cortex-M55 and Cortex-M85 . I can't wait to see Helium technology continue to empower our ecosystem partners' AI innovation applications.