3D machine vision and hand-eye calibration technology setup principles

1. Visual system The basic hardware includes the following:

• 1 Industrial Computer

• 2 network cables

• 3 routers

• 4 calibration plates

• 5 Calibration plate fixing flange

• 6 cameras and cables

2. Wiring diagram

Connect one end of the three network cables to the LAN port of the router (be careful not to connect to the WAN port), and the other end to the network port, industrial computer network port, and camera (camera cable includes wire and network cable) network port. The router end connection is shown in the figure below:

The robot network port is the network port on the control cabinet, as shown in the following figure (taking the UR robot as an example):

3. Software environment construction

After confirming that the power supply, network cable and other connections are normal, start setting the IP and use the router to build the industrial computer, camera and robot into the same network segment.

Enter the URL tplogin.cn in the browser to enter the router management interface (the login management interface address can be viewed on the back of the router). If it is a pre-configured router, directly enter the configured network segment to enter the router management interface, for example 192.168.3.1 (set to network segment 3).

When entering the router management interface, you will be asked to enter your username and password, as shown below:

The management interface is shown in the figure below, where the green network port in the red box represents a normally connected network port.

If the router has been set up, but you forget the network segment you set, cannot enter the router management interface, and do not want to restore the router to factory settings and reset it, you can view the default gateway through the connection details as shown in the figure below.

Now open the robot teaching pendant and check the robot's IP, as shown in the following figure (taking the UR robot as an example):

You can see that the robot's IP is in network segment 3, so use the router to set the camera, industrial computer, and robot to network segment 3 (if the robot's IP can be modified, you can also change the robot's IP to another network segment, so you can operate flexibly)

In the router management interface, open the LAN settings in the basic settings in the left toolbar, set the IP address in manual mode, for example, to set the 3rd network segment, enter 192.168.3.1, the subnet mask defaults to 255.255.255.0, click Set, and the setting is successful. As shown in the figure below:

Similarly, you need to set the IP address of the industrial computer to network segment 3. Click the following steps to set the industrial computer to network segment 3 (the 110 in 192.168.3.110 is an arbitrary value between 0 and 255, as long as it is not the same as the robot IP).

4. Robot hand-eye calibration

To achieve the coordinate conversion from the target point in the image to the grasping point on the actual object, it is necessary to have accurate information about the internal and external parameters of the camera. The internal parameters are the basic parameters inside the camera, including lens focal length, distortion , etc. Generally, the internal parameters of the camera have been calibrated and stored inside the camera when it leaves the factory.

The camera extrinsic parameters represent the posture conversion relationship between the robot and the camera (i.e. the hand-eye relationship, so the calibration of the camera extrinsic parameters is called robot hand-eye calibration). The relative posture of the robot and the camera is not fixed in different usage scenarios, and calibration is required at the work site to obtain the hand-eye relationship between the camera and the robot.

Since the robot hand-eye calibration uses the camera's intrinsic parameters, having accurate intrinsic parameters is a prerequisite for calibrating the extrinsic parameters.

There are different ways to classify robot hand-eye calibration. Depending on how the camera is mounted relative to the robot, hand-eye calibration is divided into two types:

1. The camera is fixed on the bracket independently of the robot, which is called ETH (Eye to hand) method.

2. The camera is fixed on the end flange of the robot, which is called EIH (Eye in hand) method.

At the same time, you can use Multiple random calibration plate poses or TCP point touch method to add calibration points. The main differences between the two are:

1. Multiple random calibration plate poses: Use the trajectory points automatically generated by the software or multiple poses added manually, take a photo at each pose and identify the corner points of the calibration plate, establish the relationship between the calibration plate, camera and robot, the process is simple and the calibration accuracy is high.

2. TCP cusp touch: After determining the calibration plate's position and posture using the three-point method, establish the relationship between the calibration plate, camera, and robot. This method is suitable for situations where the robot's activity space is limited, it cannot be controlled, and the calibration plate cannot be installed.

The classification method is shown in the figure below.

4.1 Basic principles of ETH calibration

The end of the robot is connected to a calibration plate of known size through a flange, and the coordinate A of each mark point on the calibration grid relative to the robot base coordinate Base can be obtained; the image of each dot on the calibration grid is obtained by taking a photo with a camera, and the coordinate B of the camera optical center relative to each mark point on the calibration grid can be obtained; the pose relationship X between the camera optical center and the robot base coordinate (Base) is the quantity to be determined. A, B and X form a closed loop to form an equation, and the unknown number X can be solved in the equation. By moving the robot and changing the pose of the calibration plate relative to the camera, multiple sets of equations can be obtained, and the values ​​of these equations are fitted and optimized to finally obtain the optimal value of X. The pose relationship is shown in the figure below.

When using the TCP touch method for calibration, the calibration plate is placed on the work plane, and a known TCP point is added to the end of the robot to touch the round point on the calibration plate. The principle is shown in the figure below, where A and B are known and the value of X is solved.

In actual operation, there are three ways to obtain coordinate A:

1. If the position relationship between the calibration plate and the flange end is known (calculated by the three-point method or the known size of the connector), then A can be directly calculated;

2. If the positional relationship between the calibration plate and the flange end is unknown, the positional relationship between the calibration plate and the flange end can be calculated by numerical method through a series of relative movements of the calibration plate during the calibration process, and then A can be calculated;

3. If the calibration plate and the robot end are not fixed, the value of A can be calculated by touching the calibration plate mark point with the sharp point of the known tcp coordinate. The above three methods correspond to three different ways of obtaining calibration data.

The ETH method calibrates the position relationship between the camera optical center and the robot base coordinates. If the robot base coordinates or the camera moves, the corresponding external parameters will change accordingly, and the hand-eye relationship needs to be recalibrated.

4.2 Basic principles of EIH calibration

The end of the robot fixes the camera through a fixed frame. At this time, the position between the center of the flange at the end of the robot and the optical center of the camera is relatively fixed, that is, the unknown variable X in the figure below; the position of the center of the flange at the end of the robot relative to the robot base coordinate system (Base) is a known quantity B; the camera takes pictures of the calibration grid to obtain the position relationship between the optical center of the camera and each point on the calibration grid, and the known quantity C can be obtained; the calibration grid is placed flat in the area where the camera can see, and its position relationship relative to the robot base coordinate is a fixed value A; in this way, variables A, B, C, and X form a closed-loop relationship. In the following equation, since A is a fixed value, the first two equations are combined, and only X is an unknown quantity in the new equation. Transform the end position of the robot to take pictures at different angles to obtain multiple sets of A, B, and C values, and use these values ​​for fitting calculation to obtain the optimal X value.

When using the TCP touch method for calibration, the calibration plate is placed on the work plane, and a TCP point of known size is installed at the end of the robot to touch the round point on the calibration plate. The principle is shown in the figure below. If A, B, and C are known, the value of X can also be obtained.

EIH calibrates the position relationship between the optical center of the camera and the center of the robot end flange. If the camera moves relative to the center coordinates of the robot end flange, the corresponding external parameters will change accordingly, and the external parameters need to be recalibrated.

Select a normal calibration plate (a normal calibration plate is one with clear dots, no obvious scratches, and no obvious bending or deformation).

In ETH mode, install the calibration flange on the robot's six-axis (if it is a four-axis robot, install it on the four-axis), and then install the calibration plate on the flange. The calibration flange can be installed in any direction at the end of the robot. Ensure that the calibration plate is installed firmly and parallel to the XY plane of the end of the robot.