Using TI DLP technology to drive structured light systems for bin picking accuracy

alan000345

Using TI DLP technology to drive structured light systems for bin picking accuracy [Copy link]

 

In industrial environments, parts of different shapes, sizes, materials, and optical properties (such as reflectance, absorption, etc.) need to be processed every day. These parts must be picked and placed in a specific orientation before processing. The activity of automatically picking and placing these parts randomly from the storage environment (container or other) is often called bin picking. But this poses a challenge for the robot end effector (a device attached to the end of the robotic arm), which needs to accurately know the 3D position, size, and desired orientation of the object to be grasped. In order to accurately navigate around the outer walls of the box and other objects inside the box, the robot's machine vision system needs to obtain depth information in addition to the 2D camera information.

For bin picking, the challenge of capturing a 3D image of an object can be solved by structured light technology. 3D scanners/cameras based on structured light technology work by projecting a series of patterns onto the object being scanned, and the pattern distortion is captured by a camera or sensor. A triangulation algorithm then calculates the data and outputs a 3D point cloud. Image processing software (such as Halcon developed by MVTech) calculates the object's position and the optimal approach path for the robot arm (Figure 1).

查看详情

Figure 1 : Example of using Halcon to match pipe fittings to their respective 3D models (Source: Halcon developed by MVTech )

DLP technology provides high-speed pattern projection capabilities through a micro-mirror matrix (also known as a digital micro-mirror device, DMD) mounted on top of a semiconductor chip, as shown in Figure 2. Each pixel on the DMD represents a pixel in the projected image and allows pixel-accurate image projection. The micro-mirrors can be switched at ~ 3us to reflect the incident light through the projection lens onto an object or onto a light block. The former can obtain bright pixels in the projected scene, while the latter can create dark pixels. DLP technology also has the unique advantage of being able to project patterns over a wide wavelength range (420 nm – 2500 nm) using a variety of light sources such as lamps, LEDs, and lasers.

Structured light powered by DLP technology for bin picking offers several advantages:

• Strong resistance to ambient lighting. Lighting conditions in factories, such as low exposure and high contrast between different lighting areas, causing underexposure of sensors or flash that interferes with machine vision systems, are a major challenge for applications that require machine vision, such as box picking. Structured light driven by DLP technology has its own active illumination, which makes it resistant to these conditions.
• No moving parts. Structured light systems can capture the entire scene at once, eliminating the need to sweep the beam across the object or move the object through the beam (as in scanning solutions). Structured light systems protect against mechanical wear and tear by using no moving parts at the macro scale.
• Real-time 3D image acquisition. The micromirrors in the DLP chip are controlled at high speeds, providing custom pattern projection at up to 32kHz. In addition, the DLP controller provides trigger outputs and inputs that can be used to synchronize cameras and other devices with the projected pattern sequence. These features help achieve real-time 3D image acquisition that allows simultaneous scanning and picking.
• High contrast and high resolution of the projected pattern. Since each micromirror reflects light onto a target or absorbing surface, high contrast is achieved, enabling accurate point detection regardless of the object's surface properties. Combined with the use of a high-resolution DLP chip with 2560 x 1600 mirrors, objects can be detected down to the micron level.
• Adaptable to object parameters. Programmable patterns and various point coding schemes (such as phase shifting or Gray coding) make structured light systems more adaptable to object parameters than systems using diffractive optical elements.
• Accelerate development time. Although robots offer high repeatability, bin picking requires precision in unstructured environments, where the position and orientation of the object being picked changes every time an object is removed from a storage bin. Successfully addressing this challenge requires a robust process—from machine vision to computing software to the dexterity of the robot and gripper. Making everything work together can be a challenge that consumes a lot of development time.

查看详情

Figure 2 : DLP chips contain millions of micromirrors that can be individually controlled at high speeds to reflect light to create projected patterns.

TI's DLP technology evaluation module can quickly embed structured light into machine vision workflows. To demonstrate this capability, factory automation and control system engineers mounted the DLP LightCrafter 4500 evaluation board to a monochrome camera at a certain distance and angle. The DLP evaluation board was triggered by the camera through an interconnected trigger cable; as shown in Figure 3.

查看详情

Figure 3 : Structured light setup, including DLP Products LightCrafter 4500 (left), gray-dot FLIR Flea3 camera (right), and calibration (after)

Both the board and the camera are connected to a PC via USB, and the entire setup is used to calibrate the board. The software in the Reference Design for Accurately Generating Point Clouds for 3D Machine Vision Applications Using DLP Technology is then used to calibrate the camera and projector parameters such as focal length, focus, lens distortion, camera translation and rotation relative to the calibration board. The reference design user guide walks through this process step by step.

Recalibration is only required if the camera is moved relative to the DLP product board.

After the setup is complete, point clouds of real-world objects can be created. These clouds are output by the software in an arbitrary file format and then read and displayed by some short code developed on Halcon's HDevelop platform. Figure 4 shows a point cloud that color-codes the depth information of a box full of coffee cups.

查看详情

Figure 4 : A captured cup (left) and a point cloud of multiple cups captured using structured light driven by DLP in a box displayed in Halcon HDevelop (right).

Halcon’s surface matching determines the 3D pose of the cup by comparing the point cloud to the 3D CAD model of the cup. This way, the robotic arm can now “see” the object and the optimal approach path for the robotic arm can be calculated, allowing it to pick the object from the bin while avoiding obstacles in an unstructured and ever-changing environment.