Using FPGA to implement medical imaging

Medical imaging technology plays an increasingly important role in the healthcare industry. The development trend of this industry is to achieve early disease prediction and treatment through non-invasive means to reduce patient expenses. The integration of multiple diagnostic imaging methods and the advancement of algorithm development are the main driving forces for designing new devices to meet patient needs.

To achieve the functions required by these industry goals, equipment developers are beginning to use commercial off-the-shelf (COTS) CPU platforms that provide FPGA support and are updateable for data acquisition and co-processing. When developing updateable medical imaging equipment flexibly and efficiently, several factors need to be considered, including the development of imaging algorithms, the integration of multiple diagnostic methods, and updateable platforms.

Developing imaging algorithms requires intuitive, advanced modeling tools to continually improve digital signal processing (DSP) capabilities. Advanced algorithms require an updateable system platform that significantly improves image processing performance and enables devices that are smaller, more convenient to use, and easier to carry.

The performance requirements of real-time analysis require system platforms that can be adjusted with software (CPU) and hardware (configurable logic). These processing platforms must be able to meet various performance price requirements and support the integration of multiple imaging diagnosis and treatment methods. FPGAs are easily integrated into multi-core CPU platforms to provide DSP functions for the most flexible high-performance systems.

System planners and design engineers use advanced development tools and intellectual property (IP) libraries to quickly partition and debug algorithms on these platforms, accelerating design implementation and increasing profits.

This article describes some of the trends in medical imaging algorithms, the integration of multiple diagnostic and treatment approaches, and the updateable platforms to implement these algorithms.

Algorithm development for medical imaging

First, let’s take a look at the development trends of imaging algorithms for each diagnostic and treatment method and how FPGAs and intellectual property are used.

MRI

Magnetic resonance imaging (MRI) reconstruction technology creates cross-sectional images of the human body. With the help of FPGA, three functions are used to reconstruct 3D human images. From the frequency domain data, 2D reconstructed slices are generated by fast Fourier transform (FFT) to produce grayscale slices, generally in the form of a matrix. 3D human image reconstruction uses slice interpolation to make the slice spacing close to the pixel spacing, so that the image can be viewed from any 2D plane. Iterative resolution sharpening uses spatial deblurring technology based on an iterative inverse filtering process to reconstruct the image while reducing noise. In this way, the visual diagnostic resolution of the cross section is greatly improved.

Ultrasound

The small particles that appear in ultrasound images are called speckles. The interaction of various unrelated scatterers produces ultrasound speckle (similar to multipath RF reflections in the wireless field), which is essentially a multiplicative noise. Lossy compression technology can achieve speckle-free ultrasound images. The image is first processed logarithmically, and the speckle noise becomes additive noise relative to the useful signal. Using the JPEG2000 encoder for lossy wavelet compression can reduce speckle noise.

X-ray imaging

Coronary X-ray image motion correction technology is used to reduce the effects of breathing and heart beats during imaging (heart-breathing cycle). The movement of the "3D plus time" coronary model is projected onto the 2D image and used to calculate the correction function (translation and magnification) to correct the movement and obtain a clear image.

Molecular imaging

Molecular imaging is the characterization and measurement of biomedical processes at the cellular and molecular level. The goal is to detect, acquire, and monitor abnormal conditions that lead to disease. For example, X-ray, positron emission tomography (PET), and SPECT technologies combine to map low-resolution functional/cellular/molecular images to corresponding high-resolution anatomical images, down to 0.5 mm. Miniaturization and algorithm development have driven the use of FPGAs in these compact system platforms, further improving performance over multi-core CPUs.

Fusion of diagnostic methods

Early prediction and non-invasive treatment have promoted the integration of diagnostic and treatment methods such as PET/computer-assisted tomography (CT) and X-ray diagnosis/CT equipment. To achieve higher image resolution, it is required to use fine geometric micro-array detectors and combine them with FPGA to pre-process the photoelectric signals. After pre-processing, the CPU and FPGA coprocessor work together to process the collected signals and reconstruct the human body image.

Non-real-time (NRT) image fusion (coincidence) technology is generally used to analyze functional and anatomical images obtained at different times. However, NRT image coincidence processing is difficult due to factors such as patient position, scanning bed shape, and natural movement of internal organs. Using FPGA processing technology to fuse PET and CT in real time can obtain functional and anatomical images simultaneously during a single imaging process, rather than synthesizing the images afterwards. In surgical treatment, the fused images are clearer and more precisely positioned.

Image processing for surgical guidance uses preoperative (CT or MR) images and real-time 3D (ultrasound and X-ray) image registration (correlation) technology to perform surgical treatment of diseases through non-invasive means (ultrasound, MR intervention and X-ray therapy). Various algorithms have been developed to achieve the best image registration results for the fusion of diagnostic methods and treatment types.

In this type of converged system, FPGAs that support high-speed serial interconnects can reduce the interconnection of data acquisition functions in the post-processing part of the system, greatly reducing the total system cost related to circuit boards and cables.

Image Algorithm

Various imaging algorithms are often implemented in FPGAs, including image enhancement, stabilization, wavelet analysis, and distributed vector processing.

Image enhancement is generally achieved using convolution (linear) filtering. The high-pass and low-pass filtered images are linearly combined and weighted by a matrix multiplication template, producing an image with enhanced detail and reduced noise.

Video image stabilization technology normalizes the effects of rotation and scaling in a video data sequence to average out the noise in consecutive frames. It also smooths the jagged edges of still images extracted from the video, correcting image shake of approximately 1/10 pixel.

To obtain information about events in a signal, wavelet analysis uses a variable window technique to analyze a small portion of the signal at a time. Wavelet analysis uses longer time intervals for accurate low-frequency information and shorter intervals for high-frequency information. Wavelet applications include detecting discontinuities and breakpoints, detecting self-similarity, suppressing signals, removing signal noise, removing image noise, compressing images, and fast multiplication of large matrices.

The recently developed S-transform (ST) combines the FFT and the wavelet transform. It reveals how frequencies vary in space and time. Its applications include texture analysis and noise filtering. However, ST is computationally intensive and too slow to implement using a traditional CPU. Distributed vector processing technology solves this problem by using both vector and parallel computing in an FPGA, reducing processing time by 25 times.

A method for early detection of cancer takes advantage of the patient's ability to regenerate blood. Digital sensors detect infrared energy radiated by the human body, thereby "seeing" the tiny differences in increased blood flow caused by cancer. Its typical implementation is based on a programmable cardiac contraction matrix and uses a general-purpose workstation and FPGA dedicated hardware engine to achieve it. Compared with current high-end workstations, this engine speeds up the core algorithm by nearly 1,000 times.

What key FPGA building block functions are needed for these complex imaging algorithms? In CT reconstruction, interpolation, FFT, and convolution functions are required. In ultrasound, processing methods include color stream processing, convolution, beamforming, blending, and elastic estimation. Common imaging algorithms include color space conversion, graphic overlay, 2D/median/time filtering, scaling, frame/domain conversion, contrast enhancement, sharpening, edge detection, clipping, translation, polar/Cartesian conversion, non-uniformity correction, and pixel replacement functions.

Updatable platform

Many imaging systems were previously built using dedicated computing systems. Now, with the introduction of high-performance COTS CPU boards, system engineers can use a more off-the-shelf approach. Although software can perform non-real-time processing of many algorithms, real-time image processing still requires auxiliary hardware. Current FPGAs have built-in DSP modules, broadband memory modules, and a large number of programmable units, making them ideal devices for implementing these auxiliary hardware.

Altera has worked closely with its partners to achieve reliable integration of FPGA coprocessing resources and COTS CPU solutions. For Intel and AMD single-board computers (SBCs), Stratix II GX FPGAs with built-in serializer/deserializers can directly implement PCI Express-compatible coprocessor boards to handle algorithm functions. For dual-slot AMD SBCs, Altera partner XtremeData provides coprocessor daughter cards that plug directly into an Opteron slot to provide a very good CPU+FPGA processing solution (see Figure 1). For high-performance, computationally demanding applications, four-slot AMD SBCs can provide a variety of CPU+FPGA coprocessor combinations (1+3, 2+2 or 3+1). Multiple 1-U blade servers can be used to implement a very flexible platform, with each blade completing the CPU+FPGA coprocessor solution.

Figure 1 XtremeData XD1000 system diagram

NOTE: The XD1000™ plugs directly into the Opteron™ Socket 940 of a multi-Opteron motherboard, using the motherboard's existing CPU infrastructure.

The application acceleration of these platforms depends on the algorithm - the more parallel calculations in the algorithm that the FPGA undertakes, the faster it runs. For example, when accelerating the implementation of a CT imaging algorithm by FPGA-based hardware, each 3-GHz CPU combined with an FPGA coprocessor runs 10 times faster, greatly reducing system-level power consumption, saving space and cost.

Development Methodology

Now, let's consider algorithm development methods and corresponding implementation tools.

Algorithm tools

Imaging designers use advanced software tools to model various algorithms and evaluate the results. The best general-purpose tools for digital signal processing are MathWorks' MATLAB processing engine and Simulink simulator GUI. Most OEMs and medical design agencies use MATLAB to develop fast and accurate algorithms, including digital image processing, image quantitative analysis, pattern recognition, digital image coding/compression, automated microscopy, forensic image processing, and two-dimensional wavelet transforms. In addition to algorithm development, MATLAB can also simulate fixed-point algorithms commonly used in FPGAs and provide optional tools to generate C code that can run on general-purpose CPUs.

Partitioning and debugging

After the algorithm is developed, system designers must determine how to divide the functions between the CPU and FPGA to provide the best overall solution and achieve the best balance in performance, cost, reliability and service life. Equipment designers believe that it is very difficult to divide and debug algorithms on high-performance hardware systems. Many previous designs use an assembly line approach in FPGAs to divide algorithms into multiple functions and execute them sequentially. 90% of debugging functions are spent on integration. Because the execution time of each function must be balanced with each other to achieve maximum throughput, many difficulties arise and local memory and latency cannot be observed.

The solution is a “software-centric approach” based on a distributed co-processor computing model (see Figure 2), where:

Each function in the coprocessor is an execution machine (function subprocessor), and the subprocessors have message-based control pass-through capabilities.
All memories, CPUs, and subprocessors are fully switchable, allowing for comprehensive observation and easy debugging.
Message pass-through capabilities can be adjusted between internal FPGA subprocessors and between other CPUs and coprocessors in the system.

Figure 2 Software-centric design

For example, in the Altera/XtremeData coprocessor solution, these concepts are used to achieve an excellent design methodology. AlteraFPGA's Avalon switch architecture and chip programmable system (SOPC) integration tools automatically build a flexible crossbar switch architecture between all functional units. The interface IP provides pre-tested interfaces from FPGA to host CPU and FPGA to DIMM memory. The communication between the host CPU, FPGA sub-processor and FPGA memory controller is controlled based on the basic structure of pre-tested messages (software defined by Nios CPU). Message passing and full switching capabilities simplify debugging during development and greatly increase flexibility. Data paths can be soft-defined (redefined) during execution, and data can be interpreted and the transmission direction changed during system integration and debugging to improve its observability.

Design Tools and IP

Although tools such as MATLAB are very suitable for software algorithm development, they are not sufficient for implementation in FPGAs. Designers can accelerate the implementation of their designs in FPGAs by using Altera and third-party EDA tools and IP. For example, Altera provides a full set of tools: DSP Builder, SOPC Builder, Nios II CPU Development Kit, Nios II C to Hardware Acceleration (C2H) Compiler, and Quartus Development Kit.

Altera's video and image processing packages and DSP libraries provide key IP building blocks to accelerate the development and implementation of complex imaging algorithms. The video and image processing module library and other Altera/partner IP modules and reference designs (including IQ modem, JPEG2000 compression, FFT/IFFT, edge detection, etc.) provide designers with a large number of IPs to accelerate the FPGA implementation of large computing tasks.

The DSP Builder tool provides an IP library-based design flow that links algorithms developed in MATLAB and implemented in the FPGA using the Altera Quartus toolkit.

SOPC Builder is a system integration tool that automatically generates interconnect code (Verilog or VHDL) between IP blocks, Altera MegaCores functions (including Nios II CPUs), partner IP, and user-defined functions.

The Nios II Development Kit enables C programs to be embedded in and debugged on one or more Nios II CPUs in an FPGA. The Nios II CPU is a 32-bit configurable RISC soft-core processor CPU.

The Nios II C2H compiler is a new tool that analyzes the inner loop of a C code algorithm and generates coprocessor logic in the FPGA, greatly improving the performance of software running on the Nios II CPU. With a moderate increase in logic, the running speed can be increased by 10 to 100 times.

The Quartus development kit is the main tool for implementing programmable hardware and software functions in Altera FPGA. It provides IP import functions, as well as all functions such as simulation and layout and routing, and performs FPGA programming on Altera development boards or user target systems.

Other EDA vendors, such as Celoxica, also offer C-to-HDL conversion tools to accelerate FPGA implementation.

in conclusion

Baby boomers are seeking new and more feasible treatments for common diseases (especially heart disease and cancer), including early detection and non-invasive surgical treatments. Advances in the integration of imaging diagnostic methods and the development of related algorithms have greatly promoted the development of new devices to meet patient needs. Advanced algorithms require an updateable system platform that can significantly improve image processing performance.

Integrated into COTS multi-core CPU platforms, FPGAs provide DSP capabilities for the most flexible, high-performance systems. To help accelerate the implementation of complex imaging algorithms on these platforms, advanced development tools and IP implementation libraries are needed. Altera has developed tools and IP libraries with these requirements in mind. These libraries include the key building block functions required for imaging and are integrated into Altera's complete toolkit, which, combined with MathWorks' algorithm development tools, enables rapid development.