Image acquisition and stitching of manual PCB appearance inspection machine

The printed circuit board appearance inspection machine is an important quality inspection equipment on the PCB product production line. It is based on the principles of optical image processing and computer vision recognition technology. Its main function is to detect the appearance defects encountered in the production process of PCB components. China is a major PCB producer in the world, but not a strong country. The weak PCB equipment and instruments, an important link in the PCB production chain, is one of the important reasons for this situation. In order to promote the development and progress of China's PCB industry, a manual PCB appearance inspection machine with simpler operation, more compact system and higher performance-price ratio has been developed compared with the automatic appearance inspection machine. The manual PCB appearance inspection machine automatically transmits the PCB through the conveying device after the manual board is placed. The linear array CCD camera scans the PCB board evenly to obtain an accurate image. Then the scanned circuit board is compared and processed by the computer and judged. Then the device automatically inspects the PCB (OK/NG) according to the control signal fed back by the computer. It can accurately detect appearance defects such as foreign matter, exposed copper, oil replenishment, scratches, poor gold plating, character errors, uneven green oil, uneven pads, residual copper, missing printing, and development. Since the speed and effect of image acquisition and processing will directly affect the accuracy and effectiveness of PCB board inspection, the image acquisition system is studied in detail according to the characteristics of manual appearance inspection machine. ActiveMil is secondary developed in C# under the .net framework and the GDI+ graphic interface library is used to realize the development of image acquisition software system.

1 Overall structure of PCB appearance inspection machine system
The structure of PCB online inspection system is complex. Its transmission control equipment, electrical control system and camera must be under the precise control of the computer to coordinate the processing work and complete the complex inspection and sorting tasks. The structure of PCB online inspection equipment based on machine vision is shown in Figure 1.

The system is mainly divided into motion control, image acquisition and image processing. The image acquisition part is an important part of the whole system. The camera and lens are equivalent to human eyes in machine vision and are responsible for capturing the image of the object. The image acquisition part is an important part of the PCB inspection system and the basis of inspection processing. The PCB inspection system emphasizes the speed and accuracy of inspection, so the image acquisition part is required to provide clear images in a timely and accurate manner.

2 Hardware structure of the image acquisition system
1) Working principle of the image acquisition system
When the system is powered on, the MCU automatically detects whether the PCB stage is reset to the starting point. This process is mainly completed by two fiber optic sensors and servo motors. The two sensors are installed at the starting point of the four-view motor track, that is, the reset point and the track end point. The functions of reset, stop and motor reversal. After power-on, the MCU detects that sesorl (reset point or starting point) is invalid, then calls the motor reversal program to return the PCB stage to the starting position, and the serial port sends a color image invalid signal to the PC. Then the MCU continues to determine whether a key is pressed. If a key is pressed, the motor starts to rotate forward, and the serial port sends a color image valid signal to the PC. This process is also the PCB line scanning process to complete the image acquisition. During this process, the motor goes through three stages: acceleration stage, uniform speed stage and deceleration stop stage. During the forward rotation of the motor, starting from 7 o'clock, the MCU counts the pulses fed back by the servo motor encoder through the unique capture comparison unit (CCU6). When the count value reaches the valid value for image acquisition, the serial port sends an image acquisition start signal to the PC, and the linear CCD starts to acquire images of the PCB. When the motor decelerates and rotates to the sensor sensor2 at the end of the track, the motor stops and immediately reverses back to the starting point. During this process, the serial port sends an invalid image acquisition signal to the PC. At this point, a complete detection process is complete. The MCU continues to detect whether a key is pressed to perform the next detection. The acquisition validity and start signals sent by the serial port can effectively avoid false triggering. The image signal acquired by the CCD is sent to the image acquisition card by the Camrelink interface and then further processed by the PC.

[page]

2) Receiving and detecting sensors and buttons
The use of optocoupler isolation technology can achieve optoelectronic isolation between circuits, which allows the input signal to pass through unimpeded and prevents the output signal from being fed back to the input end, which is beneficial for suppressing the interference of spike pulses and various noises. It has stable operation, no contacts, long service life, and high transmission efficiency. In order to detect the feedback mirror speed signal in real time, this design uses a high-speed optocoupler ACPL-072L, which can reach a transmission rate of up to 25 MBd, and the peripheral circuit is simple, as shown in Figure 3.

This design uses the capture/compare unit CC25 port in the XC164CS peripherals, and uses the optocoupler transmission signal as an external interrupt signal to trigger the XC164, achieving high-speed and accurate sensing positioning and button detection.
3) CCD camera system
The image acquisition system of the manual PCB appearance inspection machine uses the NED Rainbow series 3CCD color line scan camera NUCLi7300. This camera is widely used and can detect color differences that black and white cameras could not do before. The external interface is a high-speed serial interface (Camera Link), which can be easily connected to the acquisition card, and can also easily set the gain and offset, and has the function of correcting the RGB line delay. The number of pixels is 7 300x3 Line, the pixel size is 10x10 μm, the data rate is 60MHz, and the shortest scan rate is 7.6kHz. Figure 4 is the timing diagram of the continuous line scan rate model used in this system.

The image acquisition card in the system is Matrox's SoliosXCL-SU74, which has a built-in processor to handle two independent base modes or one medium mode Camera Link configuration, a 66 MHz acquisition rate, a 64 MB buffer, and can acquire area and line array cameras, and has multiple acquisition modes.
At present, most of the image inspection systems on the market use area scan cameras for image acquisition and analysis. However, since the PCB product size range inspected by the PCB appearance inspection machine is large - 50 mmx50mm~330mmx250 mm, the accuracy requirement is sub-pixel level, and the resolution and image acquisition speed of the area scan camera cannot meet these requirements, so the system selects a line scan CCD. However, the line scan detection system must use the movement speed to obtain the area image, that is, when the PCB under test moves to the camera's field of view, the trigger signal is sent to the camera through the hardware to start acquisition, so the acquisition mode of the acquisition card is set to hardware triggered synchronous acquisition, and the maximum memory is allocated 64 M. The schematic diagram of the connection between CCD and the acquisition card is shown in Figure 5.

3 Software System Composition
The software development language used in this image acquisition system is C#, and the software development kit is AetiveMil9.0 component.
Matrox has a rich package - Image Processing and Pattern Recognition Library (MIL) and MIL's sub-library MIL-LITE (Basic Image Processing). MIL is a hardware-independent 32-bit image processing library. It optimizes the image processing library by using Intel's MMX multimedia graphics acceleration function. It can process binary, grayscale or color images. It is independent of the hardware platform. And for rapid Windows application development, MIL is bundled with ActiveMIL. AetiveMIL is a collection of dynamic controls that manage image acquisition, processing, analysis, display and archiving. AetiveMIL is fully integrated into Microsoft Visual Basic, Visual C++ and .net rapid application development environment. The image acquisition system uses the AetiveMil control library, which can realize all the functions of MIL. And because of the use of ActiveX controls, it greatly reduces the difficulty of program design, shortens the software development cycle, and improves the stability of the application system. [page]

3.1 Image acquisition program based on AclivrMil
The general software structure for secondary development of Matrox image acquisition is shown in Figure 6.

The basic idea of ​​the image acquisition program is:
1) Open the communication channel, that is, determine an application object (Application), and create one or more system objects (System) for each application object.
2) Initialize hardware resources, that is, allocate data buffer (Data Buffer), data digitizer (Digitizer) and data display (Display) objects for each system object.
3) Start the acquisition process, that is, read the image into the data buffer, and assign the buffer to the array. The image is processed by processing the array. After the image data is associated with the data display, the pre-processed results can be displayed through the display control or form.
ActiveMIL allows developers to quickly and easily combine an image application with the Windows user interface. Application development includes drag and scroll tool placement as well as punctuation and single-click configuration, which fully reduces the amount of code. In the .net framework3.0 card development environment, after installing the MIL development kit and configuring it correctly, components such as Application, System, Image, ImageProcess, Display, etc. will be automatically added to the toolbar.
The developer drags the required controls onto the image acquisition form, one Application, corresponding to a single-lens manual appearance machine, one System, a single-window display - one Display, in order to improve CPU usage, double-buffer asynchronous acquisition should be enabled - two ImageBuffers, and one ImageProcess.
3.2 Double-buffer asynchronous acquisition ideas
Matrox MeteorII image acquisition card supports both synchronous and asynchronous acquisition modes.
The synchronous acquisition mode is suitable for the case of continuous acquisition and display of images without calculation processing. This is because in the synchronous acquisition mode, before each image acquisition, the CPU sends an acquisition synchronization signal to the acquisition card and then waits for the acquisition to end. Therefore, during the acquisition process, the CPU can only be in a waiting state because it cannot obtain image data. Since the CCD takes about 40 ms to acquire only one frame of image, which is roughly equivalent to the visual retention of humans, it is impossible to ensure that the acquisition and processing are completed within 40 ms in the synchronous mode, and it is impossible to meet the requirements of real-time processing. In the
asynchronous acquisition mode, the acquisition work and the computer's processing of the image can be carried out simultaneously, that is, when the CPU processes the image of the current frame, the CCD camera can acquire the image of the next frame. Therefore, when programming, a cache mechanism is needed to cache the currently acquired image and the image that the CCD camera is about to acquire or is acquiring, so as to achieve parallel work of image acquisition and processing. Obviously, using this double-buffer acquisition method can not only reduce the requirements for the hardware system, but also greatly improve the processing speed of the system to meet the requirements of real-time processing.
3.3 Image acquisition code implementation

[page]

3.4 Image stitching with GDI+


4 Image acquisition example
Figure 7 is a PCB image acquired by the synchronous acquisition system. The image width is 7300 and the height is 1000. It takes 1.3 seconds and there is a frame drop phenomenon. Figure 8 is a PCB image acquired by the double-buffer asynchronous acquisition system. It takes 1.0 seconds and the acquired image effect is good.

结果表明，双缓存异步采集系统的图像采集速度显著提高，并且由于利用ActiveX组件缩短了开发周期，降低了开发的人力成本。且提高了图像采集的质量。采集一幅7300x10000的图像并实时拼接仅需1．3 s，不存在掉帧现象，无损拼接拼接。计算图像缓存区大小的公式为MemSize=ImageWidthxlmageHeightxFrames，实验还表明，采用同步采集方式采集7 300x1 000，每像素8 bits的1帧图像大致需要6Gb的缓存空间；采用异步双缓存采集方式采集相同分辨率的图像只需要不到32 mb的缓存就可以连续不问断地保存图像。由此可见，使用异步双缓存采集方式不仅可以节约系统资源，而且可以获得更高效更好的采集效果。

5 结论
双缓存技术近年在图像显示领域得到广泛应用，该设计首次将双缓存技术与异步采集结合应用在PCB手动外观检查机的图像采集系统中。利用Matrox公司的SoliosXCL-SU74，使用异步双缓存采集方法可以实现PCB外观机的实时连续采集(本实验设为每秒采集10帧图像，每帧图像宽为7 300，高为1 000)的要求，并利用GDI+库实现图像实时无损拼接，该方案较传统的图像采集效率更高，对硬件的要求更低，更适合PCB外观机等大数据的图像采集系统。