The robot's eyes and brains--intelligent photoelectric sensors

Robots that work in complex environments must meet the following prerequisites: they must be able to use sensors to identify their surroundings and have certain "learning" and "self-regulation" capabilities. On the market, there are various sensors that rely on image monitoring or force detection to control robots, but there is still a lack of "slip" sensors, that is, sensors that can detect and compensate the relative movement between the robot's fingers and the object being grasped online. Most of the solutions recommended for you are based on tangential force or friction force detection. That is, the tangential force at the time of contact is detected using a contact sensor or the relative displacement is indirectly detected and estimated using a friction vibration acceleration sensor. However, common sensors so far cannot directly detect the relative slip speed. The high R&D costs are the main reason why direct relative slip detection technology has not been successfully industrialized. Unlike the solutions currently recommended, the optical sensors developed at the IITB Hollenhofer Institute can directly detect relative slip. In combination with contact force detection sensors, extremely demanding intelligent robot solutions can be met. In this solution, optical sensors can be used economically and reasonably, and their compact structure can be well integrated into mechanical and machine tool equipment. This solution has a wide range of applications, but is currently focused on the following two areas: (1) fragile workpieces that rely on force for reliable clamping and manipulation; (2) surface scrubbing and grinding of non-positioned or movable workpieces. Working principle The working principle of the photoelectric slip sensor is similar to that of an optical mouse. The surface of the object in contact with the robot finger is illuminated by light emitted by a light-emitting diode or a laser diode. The light reflected from the illuminated surface of the object is refracted by a lens and enters the micro camera in the sensor chip (Figure 1). The captured image enters the DSP digital signal processor in the sensor chip as a grayscale image and is first converted into a velocity signal in this microprocessor. Finally, the slip data (△x and △y values) are calculated based on these velocity signals.

The computational language used in the velocity calculation process is optical flow language, which has a small amount of computation. As optical flow is actually a vector field, it can represent each picture, pixel, and the vector field of the image's 2D motion direction and speed.

The detection function of the photoelectric sensor is based on image processing. The sensor must be able to receive images with a sampling accuracy of 10,000 frames/s and a maximum spatial resolution of 3200 fps. Not only the shooting speed, but also its resolution is much higher than most contact force sensors.

This high-definition slip sensor can detect:

(1) the material and surface structure of the object;
(2) the distance to the surface of the object being measured;
(3) the nonlinear relative speed between the sensor and the object being measured (Figure 2).

The determination of this nonlinear relationship is achieved by the introduction of the inverse module. The parameters of the module are precisely set according to the object to be measured. When detecting the surface of an unknown object, the detection of the nonlinear relationship can also be completed through the module's automatic metrology calibration function.

Since it has the function of detecting the relative slip speed between the robot and the object to be measured, this slip sensor is mainly used in automated cleaning and grinding production. Through the nonlinear characteristic curve of the detected material, different surface structures can be automatically identified and the type of material can be finally evaluated. Through the automatic metrology calibration method, the material of the unknown object can also be judged by using the detected nonlinear characteristic curve. After combining the use of force-torque sensors, the dynamic force-slip situation can also be detected, adjusted and controlled, and finally the desired constant grip contact force and cleaning speed are obtained to ensure that the curved surface of the object can be cleaned. Thus, the complex workpiece positioning operation is omitted, so that the workpiece to be cleaned or ground can be held by people, and even the workers can move during the cleaning and grinding process.

Figure 3 Cleaning or grinding without positioning, fixing or even moving the object

The difficulty in modern robot programming technology is to quickly and easily complete the local trajectory programming when the workpiece is not moving in a fixed coordinate system in space. Usually, great efforts are required in robot trajectory programming. However, this technology can automatically identify the contour of the workpiece, thereby realizing the automated processing of the workpiece surface.

At the IITB Hollenhof Institute, a practical application experiment was specially carried out to realize the local trajectory programming of the robot to keep the surface clamping force of the workpiece constant during grinding. When the surface of the workpiece was ground, the workpiece was not clamped in any way and was "held" in the hands of an operator or a robot. The image capture rate of the slip sensor used reached 2300fps, the sampling accuracy reached 400 frames/s, the outer dimensions of the sensor were 35mm×75mm×6mm, and the shell diameter was 100mm. In

the sensor control effect test based on clamping force and slip adjustment, the surface processing accuracy of the workpiece without clamping was studied in detail under different processing speeds and movement speeds. In the detection curve shown in Figure 4, the plate is serpentine motion with a trajectory speed of 300mm/s. During the grinding process, the workpiece moves at a speed of 37.5 mm/s and an acceleration of 30 mm/s2. Although the workpiece moves in a highly interfering serpentine motion, the workpiece processing error caused by it is quite small. The test results show that the trajectory error is equivalent to about 1% of the actual motion.

Reliable gripping of smooth surfaces

In the industrial and household electrical sectors, robots and manipulators often encounter many difficulties when gripping fragile objects with smooth surfaces. On the one hand, the robot's fingers are required to have sufficient gripping force to ensure that the gripped object does not fall off; on the other hand, the gripping force must not be too large to crush the gripped object. Fragile objects (such as glass tubes, test glasses, etc.) can be crushed if too much force is applied when gripping. It is necessary to prevent the object from falling off by itself and to reliably grip objects with unknown friction coefficients. This can only be achieved if the robot can intelligently control the slip and adjust the clamping force.

IITB Hollenhofer Institute successfully completed the first test to meet the above requirements using their photoelectric sensors and the PG070 standard manipulator produced by Schunk (Figure 5). The Array DAS 9205 contact sensor developed and produced by Weiss Roborics is installed in one finger of the manipulator, and a photoelectric sensor for detecting slip is installed in the other finger. The photoelectric sensor has a shooting speed of 1500fps, a resolution of 300 sampling units per inch, and an overall size of 23mm×60mm×6mm. The first practical test of the slip photoelectric sensor has achieved ideal results, ensuring that it can reliably and non-destructively "hold" plastic and ceramic objects. Even when the load and friction coefficient vary, it can reliably complete the task.

Figure 5: A two-finger manipulator with integrated slip sensors. The slip sensor is installed in finger a and
the contact array sensor is installed in finger b.

Summarizing the inventions of IITB Hollenhofer, it can be confirmed that the slip sensor they developed has opened the door to the development and production of intelligent robots, solving the long-standing problems in laboratory automation and servo robots in industrial production and assembly processes. In particular, the application prospects are very broad in the cleaning of objects and surface processing of workpieces as well as the reliable grasping of fragile objects. This slip sensor can be economically mass-produced, and its compact structure allows it to be easily integrated into complex mechanical mechanisms.