Integrating DSP functions with FPGAs to improve performance in imaging applications

rain

Integrating DSP functions with FPGAs to improve performance in imaging applications [Copy link]

Intevac是商用和军用市场光学产品的前沿开发商。本文介绍该公司NightVista嵌入式电子系统的开发，该产品是高性能超低亮度紧凑型摄像机。该摄像机最初采用了流行的数字信号处理器、几个ASSP和外部存储器件。系统对性能的需求越来越高，工程师团队决定试验一种替代方案——在可编程逻辑中实现可配置软核处理器。这一决定带来了以下好处：

达到了目标所要求的性能
在单个FPGA中集成了分立的元件和数字信号处理(DSP)功能
功耗降低了近80%
将五块元件板缩减到一块，显著降低了成本
缩短了开发时间

Figure 1. Functional block diagram of
the Cyclone series FPGA in the Intevac NightVista camera

DSP-Based Implementation

Some of the features of the NightVista electronics system include:
Camera power-up test and initialization
Video sensor calibration and characterization
Automatic gain control for image enhancement management
Graphics, text, and watermarks On-screen display functions
Real-time adaptive contrast adjustment
Gamma correction, video freeze frame capture, and storage to flash memory
Real-time clock
User-defined programmable preset configuration Communication
with host PC via RS-232
Remote update of camera functions and parameters, camera-to-host PC video data transmission

Intevac's initial approach to developing the NightVista electronics was to use a digital signal processor. In addition to the processor, several other major components were required, including an NTSC video encoder, an RS-232 interface, multiple phase-locked loops (PLLs), a CPLD to implement various logic functions, and several memories (FIFOs, SDRAM, and flash). In addition, these components required four different operating voltages (requiring four power supply regulators and different PCB board layers), four independent clock systems, and their own oscillators and power supply decoupling circuits. In total, these components occupied five PCBs, each about two square inches, stacked in the camera's two-inch square housing.

After several months of hardware development, it became clear that the solution would not meet Intevac’s performance goals for NightVista. The weight and power consumption were unacceptable. In addition, the high power consumption of the densely stacked PCB presented significant thermal management issues. Intevac decided to discontinue the digital signal processor-based design and move to an FPGA-based mixed logic and soft-core microprocessor solution. Intevac had no experience with integrated microprocessor programmable logic, but an evaluation solution using an Altera demonstration board was attractive. Its latest low-cost FPGA integrates a complete 32-bit RISC processor with memory blocks, PLLs, and a large number of logic resources to implement dedicated video signal processing functions. Integrating PLLs in FPGAs can solve a number of problems associated with multi-clock systems at the board level.

The engineering team considered a variety of factors, including:
Performance and features of multiple FPGA families
Availability of intellectual property (IP) cores
Device integration technology and business capabilities of multiple suppliers
Providing mature hardware and software development tools
Vendor support resources Reliability

After analyzing the above factors, the company decided to purchase a solution that implements the Altera Nios processor in an Altera Cyclone FPGA. The functions of the FPGA are shown in Figure 1.

After deciding

to use Altera's solution, Intevac had to determine how well the existing DSP software could be ported to the Nios processor. The company had already invested 18 months of manual effort in developing the previous DSP software, and the team was now faced with the problem of passing image data through the processor to the output without performing any video processing. The Nios processor in the FPGA camera has different characteristics and can only communicate with the host PC and video sensor via RS-232 serial communication protocol. Fortunately, software development for the Nios processor is straightforward, and using a Nios development board, Intevac established communication between the processor and the host PC in a few hours. The

new FPGA board was completed within a month, and during this period, Intevac continued to use the Nios processor development board to write and debug software code. Intevac originally planned to use the real-time operating system (RTOS) of the digital signal processor to manage the timing of the complex video processing algorithms. Because the Nios processor does not include an off-the-shelf RTOS, the software team was unsure whether all timing requirements could be met. After discussing with the hardware team, the software team quickly discovered that the configuration function of the Nios processor can control the signal timing very well, and generally only a slight change in the FPGA design is needed to achieve the target timing requirements. The hardware and firmware processing in the same FPGA environment are highly integrated, which can quickly and easily achieve the best control and video processing tasks.

FPGA solution customized to improve performance

After further research, Intevac began to develop custom functions and peripherals to meet its needs. Once a bottleneck was encountered in the software, the hardware team developed a processor to improve performance, which could usually be done within an hour. The hardware team designed a custom video encoder, a FIFO module for buffering video data, and a dedicated DMA controller to provide a stable video data stream to the encoder, avoiding the use of an external encoder and FIFO buffer. In addition, a custom SDRAM controller was built so that all video, attribute, Nios processor command reading and data storage can use the same memory, thereby improving performance. Some functions require their own clock, so the FPGA's on-board PLL is used to generate three different clocks from a master clock: the first is for the video encoder, the second is for SDRAM timing, and the third is for the external pixel sensor.

Improved Performance Through Integration

After implementing the functions previously implemented by external devices, Intevac also added functions that were not possible with the original DSP processor. A video test pattern generator was added to simulate the operation of the camera, allowing the software team to implement various video processing algorithms to make the system work in harmony. Another function was added, a statistics generator, which analyzes the characteristics of the video data for image enhancement and brightness processing. The image statistics generator requires mathematical operations that would be slow if implemented in software. Intevac used the logic resources in the FPGA to implement the function, set up, and transfer the results to the processor.

After the board was built, the development board software code was transferred, set up, and running on the new board within a few hours. Intevac spent the next few months further debugging and optimizing the design while continuing to develop hardware and software. Although the processor and other parts of the FPGA design were modified several times, they did not affect the board layout. In the end, the use of Cyclone devices and Nios soft-core processors reduced the number of boards from five to one. This integration approach reduced the weight of the camera, reduced the required support voltages from four to two, and reduced power consumption by nearly 80%. Intevac is also able to efficiently produce multiple products using the same PCB setup.

Easily Meet RoHS with Altera Lead-Free Products

Altera offers the broadest range of lead-free products in the industry, with more than 1,200 products available in lead-free packages. As a leading supplier of environmentally friendly programmable logic solutions, Altera has shipped 25 million lead-free products since 2002. Altera's lead-free devices comply with the maximum concentration values specified in EU Directive No. 2002/95 on the restriction of the use of hazardous substances ("RoHS Directive"), including lead (Pb), mercury, cadmium, hexavalent chromium, polybrominated biphenyls (PBB) and polybrominated diphenyl ethers (PBDE). Easily complete your RoHS transition by integrating non-compliant ASSP functions with Altera's PLDs.

Conclusion

After simplifying the design, Intevac achieved its performance goals, greatly reduced component and production costs, and improved the quality and reliability of NightVista. The solution also added more functions based on the original product specifications. Intevac reserved logic resources in the FPGA so that the camera can be further updated when it is applied in the field. This solution helped Intevac research and improve a faster and more efficient design and development process, saving a lot of time and resources for future product development.