Application of sensors in real-time obstacle avoidance of multi-joint robot system

1. Introduction

In order to work autonomously in unknown or time-varying environments, multi-joint robots should have the ability to sense the working environment and plan their own actions. To this end, it is necessary to improve the robot's ability to quickly understand and identify the current perceived environment and avoid obstacles in real time. Real-time obstacle avoidance is a key technology to achieve the autonomous working ability of intelligent robots. It is also a hot topic in the recent development of intelligent robots at home and abroad. Its notable feature is that it has sensor information feedback, which can achieve good intelligent behavior. This article mainly focuses on the research on real-time obstacle avoidance methods for multi-joint robots based on sensor information. The selection of sensors and sensor information fusion technology are introduced in detail.

2. The choice of sensors

One of the key issues in robot obstacle avoidance is how to use sensors to perceive the environment during movement. Any type of sensor has its own advantages and disadvantages. Various factors need to be carefully considered when selecting.

During the robot motion planning process, sensors mainly provide two types of information to the system:

(1) Information about the existence of obstacles near the robot.

(2) The distance between the obstacle and the robot. In recent years, sensors used in robot motion planning are generally divided into two categories: passive sensors and active sensors.

1. Passive Sensors

Passive sensors used in obstacle avoidance include tactile sensors and visual sensors.

(1) Tactile sensor

The robot's tactile system simulates the sensory function of human skin in contact with objects. It obtains information about the surrounding environment and is used to avoid obstacles. This enables the robot to have a tactile function, especially in the dark or when it is impossible to obtain information through vision due to the influence of obstacles.

A tactile sensor is a device that measures the parameters of the interaction between its own sensitive surface and external objects. A tactile sensor often contains many tactile sensitive elements, which are arranged in an array. These tactile sensitive elements contact the object to generate tactile images, which are then analyzed and processed. This working mode is called passive tactile. However, in practical applications, on the one hand, the spatial resolution of the tactile sensor is greatly improved.

The size of its working plane is much smaller than the object to be identified; on the other hand, the robot control needs to obtain the three-dimensional information of the object. Therefore, on the basis of passive touch, the tactile sensor is installed on the robot. As the robot continues to move, the sensor can obtain the three-dimensional tactile information of the identified object. Through further processing and identification, and reflecting it to the robot controller, the robot can obtain the surrounding environment information, identify the shape of the object, determine the spatial position of the object, etc., so as to achieve the purpose of intelligent control and obstacle avoidance. This working mode is called active touch. When installing tactile sensors, they are generally installed in the main operating parts such as the claws, feet, and joints.

The main drawbacks of using tactile sensors in the obstacle avoidance system of multi-joint robots are: signal lag, difficulty in achieving real-time obstacle avoidance, and the robot system is easily damaged during operation.

(2) Vision Sensor

The amount of information obtained by visual sensors is much greater than that obtained by other sensors, but at present, robot vision is still far from having the same functions as humans. Generally, the development of visual sensors is limited to the functions required to complete special tasks.

The visual sensor converts the optical image into an electrical signal, that is, converts the spatially distributed light intensity information incident on the photosensitive surface of the sensor into an electrical signal output in a time-series serial manner - a video signal, and the video signal can reproduce the incident light radiation image. There are three main types of solid-state visual sensors: one is a charge-coupled device (CCD); the second is a MOS image sensor, also known as a self-scanning photodiode array (SSPA); and the third is a charge injection device (CID). At present, the CCD camera is widely used in robot obstacle avoidance systems. It can be divided into two types: linear array and area array. Linear array CCD captures one-dimensional images, while area array CCD can capture two-dimensional plane images.

The image captured by the visual sensor is converted into a grayscale matrix after spatial sampling and analog-to-digital conversion, and is sent to the computer memory to form a digital image. In order to obtain the desired information from the image, it is necessary to use the computer image processing system to perform various processing on the digital image, and send the obtained control signal to each actuator, thereby reproducing the control of the multi-joint robot's obstacle avoidance process.

This type of sensor has three main defects in obstacle avoidance: first, it is limited by light conditions and working range; second, the driving circuit of this type of sensor is complex and expensive; third, the real-time performance is poor.

2. Active Sensors

Active sensors are divided into ultrasonic sensors, capacitive coupling sensors, eddy current sensors, and infrared sensors due to different intermediate transmission media.

(1) Ultrasonic sensor

Ultrasonic sensors rely on emitting sound wave signals of a certain frequency, and use the reflection and scattering of ultrasound on the interface of an object to detect the presence of an object. When ultrasound propagates in the air and encounters other media, it will be reflected due to the different acoustic impedances of the two media. Therefore, ultrasound is emitted to the object to be measured in the air, and the reflected wave is detected and analyzed to obtain information about the obstacle.

Ultrasonic sensors are widely used in robot ranging, positioning, and environment modeling tasks due to their simple, fast information processing and low price. However, they have certain limitations in the real-time obstacle avoidance system of multi-joint robots, which are mainly manifested in the following four aspects:

One reason is that the wavelength of ultrasound is relatively long, and it can produce mirror reflection for slightly larger flat obstacles. Since the sensor cannot receive the reflected signal, the obstacle cannot be detected.

Second, the blind area is large, because each ultrasonic transducer acts as both an ultrasonic transmitter and an ultrasonic receiver, so it cannot transmit and receive ultrasound at the same time. After transmitting ultrasound, it takes some time to process the returned sound waves. If the obstacle is too close (<30 or so), the sensor cannot receive the returned sound waves, so this type of sensor has a measurement blind area.

The third problem is that the detection beam angle is too large and the directivity is poor. It can only obtain the distance information of the target, but cannot accurately provide the boundary information of the target. The stability of a single sensor is not ideal. In practical applications, other sensors are often used to compensate, or multi-sensor fusion technology is used to improve detection accuracy.

Fourth, because ultrasound is affected by environmental conditions such as temperature and humidity, as well as the inherent wide beam angle of ultrasound, the error between the measured value and the actual value when the ultrasonic sensor is measuring distance is large.

(2) Capacitive coupling sensor

Capacitive coupling sensors are sensors that change capacitance when an object approaches the sensor. The change in capacitance can cause the oscillator to vibrate or produce a phase shift change, thereby detecting the presence of obstacles. This type of sensor is stable, reliable and durable. The disadvantage is that due to the low resolution of the sensor, the dimension of the object cannot be distinguished within its measurement range. The robot must assume that the obstacle is very large when handling it. For example, if the distance of the obstacle is 2cm, it is considered to be an object of 20 ∽30cm , which greatly limits the space for the robot arm to operate.

(3) Eddy current sensor

The eddy current sensor emits a high-frequency changing electromagnetic field to induce eddy currents in the surrounding targets. The size of the eddy current is related to the distance between the sensor and the target object. The magnetic field generated by the eddy current is opposite to the magnetic field of the sensor. The superposition of the two magnetic fields will reduce the inductance and impedance of the sensor. By using an appropriate circuit to convert the change in impedance into a change in voltage, the distance of the target object can be calculated.

Eddy current sensors are small in size, highly reliable, and inexpensive. They can be used as proximity sensors to detect the presence of obstacles and the distance of objects, and can also detect force, torque, or pressure using appropriate methods. The measurement accuracy is relatively high, and it can detect micro-displacements of 0.02 mm, and the measurement is also directional. However, the disadvantage of this sensor is that the effective distance is short (generally no more than 13 mm). In addition, this sensor is only suitable for detecting obstacles that are solid conductors.

(4) Infrared sensor

Infrared sensor is a relatively effective proximity sensor, which is often used by domestic and foreign scholars in the obstacle avoidance system of multi-joint robots. It is used to form a large-area robot "sensitive skin" and cover the surface of the robot arm. It can detect various objects in the operation process of the robot arm. The wavelength of the light emitted by the sensor is about a few hundred nanometers, which is a short-wavelength electromagnetic wave. Infrared sensors have the following characteristics: they are not affected by electromagnetic waves, are not noise sources, and can achieve non-contact measurement. In addition, infrared rays (referring to mid- and far-infrared rays) are not affected by the surrounding visible light, so they can be measured during the day and night.

Similar to sonar sensors, infrared sensors work in a transmit/receive state. This sensor emits infrared light from the same source and uses two photodetectors to measure the amount of light reflected back. Because these instruments measure light differently, they are very affected by the environment. The color of the object, the direction, and the surrounding light can cause measurement errors. However, because the emitted light is light and not sound, you can expect to get more infrared sensor measurements in a relatively short time. The ranging range is relatively short, roughly within 30cm.

3. Sensor selection strategy

The quality of sensor selection is directly related to the amount of information about the surrounding environment collected by the multi-joint robot. Therefore, there are currently two different methods for selecting the type and quantity of sensors in the robot obstacle avoidance system: a selection method based on the environment's optimization principle and a selection method based on the task.

(1) Selection method based on the optimization principle of the environment: pre-selection in the design stage and real-time selection suitable for changes in the environment and system status. The former gives the relationship between the appropriate number of sensors and the operating speed, which can determine the optimal arrangement of sensor units in the multi-sensor obstacle avoidance system. The latter uses the Bayesian method to use any prior object information to determine the positioning of the sensor, so as to minimize the uncertainty of the sensor's assumption about the obstacle object.

(2) Task-based selection method: The main idea of ​​this method is to divide the process of completing the obstacle avoidance task into several segments according to time and perception range, that is, to decompose the task and reasonably select the type and quantity of sensors based on the sensor information required at each stage.

3. Information fusion of sensors

In the intelligent robot obstacle avoidance system, because the function of any sensor is limited, when necessary, multiple sensors should be integrated together to fuse multiple sensor information, so that the characteristics of the external environment can be more accurately and comprehensively reflected, providing a correct basis for obstacle avoidance. Information fusion technology can increase the complementarity of various sensor information, adaptability to environmental changes, and improve the correctness of decision-making.

The basic purpose of multi-sensor data fusion is to obtain more information than each single sensor through comprehensive processing of multiple (kinds, types) of sensor data. It can also be understood as intelligent integration of the original information of multiple sensors to derive new and meaningful information. The value of this information is much higher than that obtained by a single sensor, and it is conducive to judgment and decision-making. Therefore, in recent years, multi-sensor information fusion technology systems have been increasingly used in robot obstacle avoidance systems, and good results can be achieved through experiments.

1. Sensor data fusion method

In the multi-sensor robot obstacle avoidance system, the environmental information provided by each information source has a certain degree of uncertainty. In addition, due to the large number of sensors and the fact that most of them are nonlinear, good global optimization and control are required, and the processing volume is large. Faced with the characteristics of large discrete data, large correlation, non-linear input information, and high reliability of fusion results, traditional data fusion methods (weighted average method, Bayesian estimation method, Dempster-Shafer evidence reasoning method, etc.) cannot meet the requirements well. For multi-joint robot obstacle avoidance systems, Kalman filtering method, production rules, and fuzzy logic artificial neural network methods are usually used to obtain a more reliable, unified, and accurate description of the environment, which is convenient for judgment and decision-making.

(1) Kalman filtering is used to fuse dynamic, low-level redundant multi-sensor data in real time. This method uses the statistical characteristics of the measurement model to recursively determine the statistically optimal fusion data estimate. Since the robot obstacle avoidance system has a linear dynamic model, and the system noise and sensor noise are white noise models with Gaussian distribution, Kalman filtering provides the only statistically optimal estimate for the fusion of multi-sensor data.

In the multi-sensor information processing applied to the robot obstacle avoidance system, domestic and foreign scholars often choose the joint Kalman filter method, the basic idea of ​​which is to use a set of filter modules running in parallel, each module only processes the information of a specific sensor. In addition, a "main filter" is used to fuse the information from all local filters. The obvious advantages of this structure are: the amount of calculation is evenly distributed in each parallel filter, and the calculation burden of the main filter is not large; it has a variety of redundant information, and can provide strong fault tolerance through appropriate reconstruction algorithm design.

(2) Production rules can be used to establish a natural scene expert system. Based on the detection data of multiple sensors, symbols are used to represent environmental characteristics. This can more comprehensively reflect the surrounding information of the obstacle avoidance system and prepare for the robot's path planning.

(3) The fuzzy logic method uses a model that simulates human thinking habits to systematically reflect the uncertainty of the multi-sensor data fusion process in the robot obstacle avoidance system, and completes the data fusion through fuzzy reasoning to achieve the expected effect.

(4) Artificial neural network is an information processing method that imitates the biological nervous system. It is a network learning method that is carried out through a teacher or unsupervised self-study algorithm. Once the learning is completed, the neural network can obtain a model structure for decision-making thinking based on the characteristic information stored in the form of network weight matrix and network topology structure. By integrating information from various different sensors in the system, accurate and reliable information that a single sensor cannot provide can be extracted. This is a very effective method for processing multi-sensor information in the presence of environmental interaction.

This method is applied to the multi-sensor information processing of the robot obstacle avoidance system. It mainly obtains environmental information at the operation site through sensors. The filtering and preprocessing modules correct and digitize the sensor information. After the safety mechanism is judged, it is used as the input source of the corresponding neural network fusion processor. The knowledge database is used as an auxiliary decision-making tool for the selection of the neural network fusion device and the knowledge source. The application receives the fusion result, adopts the corresponding control strategy, and sends the control command to the robot drive device. In this way, as much environmental information as possible at the actual operation site can be obtained quickly and accurately, thereby effectively completing the multi-sensor
information processing.

2. Sensor information processing

Since there are many types and quantities of sensors used in the robot obstacle avoidance system, the information processing is more complicated. The signal processing methods used in this system mainly include wavelet analysis, neural network, genetic algorithm, and immune algorithm.

(1) Wavelet analysis

The basic idea of ​​wavelet transform is to use a family of wavelet basis functions to represent or approximate a signal, which effectively solves the contradiction between time and frequency resolution and is suitable for local analysis of time-varying signals.

As a new signal processing method, wavelet transform has been applied to the real-time sensor signal detection and analysis of the robot obstacle avoidance system in recent years. By multi-scale decomposition of sensor signals, filtering out the noise components mixed in the measured sensor signals, and reconstructing the real signal, the reliability of the sampled data in the robot obstacle avoidance system can be effectively improved, thereby improving the control accuracy of the obstacle avoidance system. In addition, it also has a data compression function. Compressing a large number of sensor signals in this system can save storage space and increase computing speed.

(2) Neural network method

Neural network is a nonlinear function approximation method that does not require the selection of basis function systems. The robot obstacle avoidance system uses the highly nonlinear description ability of neural networks and uses this ability to model the multiple sensors of this system. Using the BP algorithm (error back propagation algorithm), the sensor output signal can be filtered, de-noised, and the sensor signal can be identified, so that the sensor output signal can more accurately reflect the external environment information and prepare for the robot's path planning algorithm.

The characteristics of this method are: it does not require detailed knowledge of the mechanism, avoiding the incompleteness of mathematical modeling; it uses software to process sensor signals, which is convenient and flexible, has strong applicability, and eliminates the need for hardware circuits.

(3) Genetic Algorithm

Genetic algorithm is a global optimization adaptive probability search algorithm proposed according to the law of "survival of the fittest" in nature. Genetic algorithm generates a new generation of population by applying a series of operations such as selection, hybridization, and mutation to the current population, and gradually evolves the population to the optimal solution state.

Genetic algorithms are applied to the sensor signal processing of the robot obstacle avoidance system. First, the actual sensor signal is uniformly sampled N times in a sampling cycle and sent to the computer, and several groups of data are randomly selected as the initial group. Then the three operations of selection, hybridization, and mutation are repeated until the given required voltage value is reached. In the robot obstacle avoidance system, the sensor signal can be accurately restored in the case of multiple sensor signals by using a simple amplification circuit and genetic algorithm software, thereby improving the measurement accuracy in sensor information processing.

(4) Immune algorithm

The immune algorithm is a computational method based on simulating organisms. The algorithm simulates the antibody-antigen interaction in the immune system and realizes digital signal processing through the system's recognition of antigens (input signals), adjustment of the affinity between antibodies (standard sample signals) and antigens, and elimination of antigens by antibodies.

In recent years, immune algorithms have also been applied to the sensor signal processing of robot obstacle avoidance systems. This method simulates the mechanism of action of the immune system and processes the complex and large number of sensor signals of this system. It can obtain a single set of sensor information that plays a decisive role in overlapping sensor signals. It runs fast, thereby reducing the time it takes for computers to process sensor information.

3. Sensor fault diagnosis

The implementation of sensor fault diagnosis can ensure that the diagnostic system obtains real-time and accurate information, avoids the negative effects caused by erroneous information, and ensures the correctness of the data. Therefore, sensor fault diagnosis is an important guarantee for the real-time obstacle avoidance of the system. The methods of sensor fault diagnosis applied in the robot obstacle avoidance system mainly include the following aspects:

(1) Fuzzy diagnosis method

The fuzzy diagnosis method is a fault diagnosis method that uses fuzzy mathematics as its theoretical basis and performs state recognition, reasoning and decision-making based on the fuzzy state of the system's sensors.

The advantage of fuzzy fault diagnosis method is that it can make full use of expert experience, taking into account the fuzziness of fault status and expert experience, making the diagnosis result more reasonable. At the same time, the amount of fuzzy diagnosis calculation is relatively small, the diagnosis speed is fast, the real-time performance is good, it is easy to apply on the computer, and the accuracy is also high. It is often applied by domestic and foreign scholars to robot obstacle avoidance system to diagnose sensor output results. However, fuzzy fault diagnosis method also has its imperfect aspects, such as the selection of membership function and the application of various diagnostic rules. There is no unified principle so far, and it often depends on the specific problem.

(2) Discrete wavelet network method

The discrete wavelet network method uses wavelet networks to diagnose sensor objects in obstacle avoidance systems. When the sensor object does not mutate, the difference between the output of the wavelet network and the output of the sensor object in the obstacle avoidance system is small. When the sensor mutates, the difference between the output of the wavelet network and the output of the sensor object in the obstacle avoidance system is large. Based on this, the variance can be used to detect faults. This method is highly flexible, has a strong ability to overcome noise, has low requirements for input signals, and does not require a mathematical model of the object. Disadvantages: At large scales, due to the large time domain width of the filter, there will be a certain delay in detection.

(3) Artificial neural network diagnosis method

In recent years, artificial neural network methods have been applied to the field of sensor fault diagnosis in robot obstacle avoidance systems. Artificial neural network is a network with parallel processing mechanism, and it can acquire external knowledge through learning. The knowledge is distributed and stored in the connection weights between each neuron. It can complete the complex mapping from input mode to output mode, and has the characteristics of strong fault tolerance and fast operation speed.

The method of using neural network method to diagnose the fault of robot obstacle avoidance system is as follows: ① select the output of key sensors in the system as the input variable of the neural network, and specify the output variable value of the network; ② select a neural network of appropriate type and structure; ③ train the network offline according to the historical data of the selected input and output signals to obtain the weight or threshold of the network; ④ apply the previously selected input and output data to the network online, and the network output can give the diagnosis result.

The advantage of this method is that it does not require an accurate mathematical model and can directly use process data to solve the robot obstacle avoidance system fault diagnosis problem. However, this method still has some problems, such as how to select the network structure. In addition, in the diagnosis process, self-learning and self-diagnosis are often used. Therefore, how to introduce the unsupervised training algorithm into the field of sensor fault diagnosis has also been explored.

IV. Conclusion

The real-time obstacle avoidance problem of intelligent multi-joint robots is a key and difficult problem in the current field of robot research. In the process of obstacle avoidance, we often face environments that cannot be known in advance, unpredictable or dynamically changing. The means by which robots perceive the environment are usually incomplete, the data provided by sensors are incomplete, discontinuous and unreliable, and there are still many problems with the algorithm of sensor information fusion. However, due to the rapid development of sensor technology and in-depth research in disciplines such as neural networks and fuzzy control theory, as well as the application of sensor information processing methods, it has provided the possibility for the final solution to the obstacle avoidance problem. However, for complex applications, it is still not satisfactory, so the existing problems are also the research direction of this field.

(1) Sensor fusion technology has been introduced into robot obstacle avoidance research in recent years and has achieved good results. However, it is difficult for some high-precision multi-joint robot obstacle avoidance systems to meet performance indicators using conventional sensors. Therefore, developing new sensors or constructing sensor arrays according to certain fusion strategies to compensate for the defects of a single sensor will be an important research direction.

(2) Artificial intelligence can make the robot obstacle avoidance system itself more flexible and understandable, and at the same time it can handle complex problems. Therefore, in future data fusion technology, using various methods of artificial intelligence to form multi-sensor data fusion based on knowledge will continue to be one of its research trends.

(3) In order to realize multi-sensor data fusion in the robot obstacle avoidance system, the processor structure will develop towards a parallel structure, including a parallel structure of sensor functions and a parallel structure of algorithm functions.

(4) In an intelligent system, the use of a single intelligent control method often cannot achieve satisfactory results. Conventional control methods and intelligent control methods should be used in combination to achieve good results. Neural networks and fuzzy reasoning are two important tools in obstacle avoidance research, but the research on the integrity of neural network sample sets has not yet made a breakthrough. It is obviously not advisable to use every point in the event space as a learning sample for the network; fuzzy logic reasoning focuses on the selection of fuzzy rules, but some rules are difficult to formally describe, or must be described by a large number of rules, which increases the amount of calculation, which deviates from the original intention of fuzzy logic application. Therefore, in recent years, a new method has been proposed based on multiple sets of sensor information, using neural network technology to achieve rapid recognition and classification of the current perceived environment by the robot, and then using fuzzy logic technology to achieve safe obstacle avoidance. It will be a potential research direction.

(5) When studying centralized multi-sensor systems, simulation technology and real-time control technology should be combined to establish an integrated development environment to process sensor signals. For distributed sensor systems, a communication-based implementation method should be sought to process sensor signals, which is one of the future development directions of sensor systems.

(6) The more advanced the robot's obstacle avoidance system is, the more sensors it has, the more complex the information processing is, and the more problems it will encounter with multi-rate sampling. However, the existing mature computer control theory involves single-rate sampling, that is, it is assumed that all A/D and D/A channels in the system work at the same sampling rate. In order to fill this gap, it is necessary to study the modeling, analysis and design methods of multi-rate sampling control systems. Therefore, the research on robot multi-sensor multi-rate sampling control systems is one of the future development directions of sensor systems.

(7) The obstacle avoidance system of a multi-joint robot is a complex intelligent system. Therefore, in practical applications, various functions must be considered comprehensively. This is an interdisciplinary topic involving multiple disciplines such as mechanics, electronics, computers, automation, and physics. The emergence of any new technology may bring breakthrough progress in the research of this field. Therefore, while studying robots, we must pay close attention to the development of related disciplines.