The first case in China! A 72-year-old high-level paraplegic patient eats fried dough sticks and plays mahjong with his mind. Zhejiang University's brain-computer interface has achieved many firsts
[Copy link]
"Hold it, great, move it towards your mouth, and move it back a little more, that's almost it, stop!" As Mr. Zhang took a sip of Coke, applause broke out in the ward.
Mr. Zhang, 72 years old, was completely paralyzed in his limbs due to a car accident two years ago, which caused a damage to the fourth cervical spinal cord. After systematic training, he can now not only shake hands, but also hold drinks, eat fried dough sticks, and play mahjong, but these actions are not done with his own hands, but he uses his "mind" to control external robotic arms and manipulators to complete them.
On January 16, Zhejiang University announced the important scientific research results of the "Dual Brain Project". This is the first clinical study of implantable brain-computer interface in China completed by the "Brain-Computer Interface" team of the Institute of Advanced Studies in cooperation with the Department of Neurosurgery of the Second Affiliated Hospital of Zhejiang University School of Medicine. Patients can fully utilize the motor cortex signals of the brain to accurately control external robotic arms and manipulators to achieve movement in three-dimensional space. At the same time, it is the first time that it is feasible for elderly patients to use implantable brain-computer interfaces to perform complex and effective motion control.
In addition to eating, drinking, socializing and entertainment, this latest achievement will help patients with limb paralysis to reconstruct their motor functions, thereby improving their quality of life. In the future, it will also have a positive impact on more areas such as assisted motor functions, functional reconstruction for the disabled, and functional enhancement for the elderly.
This is not the first time that the brain and the machine have telepathy, but it is very difficult.
In a ward of the Department of Functional Neurosurgery on the 16th floor of the Second Hospital of Zhejiang University, Mr. Zhang had just finished his lunch break. The nurse called him "Grandpa, Grandpa" and gently covered his legs with a blanket. Over there, the staff had already debugged the equipment. This was the beginning of the day's training.
The staff put a cup with fried dough sticks next to the robot arm. Mr. Zhang used his "mind" to make the robot arm align the position, spread his fingers, hold the cup, and move it back step by step. The process of moving was not always smooth. Sometimes it was a little to the left, sometimes a little to the right. Mr. Zhang had to "try hard" to think "right" or "left" to adjust the direction of the robot arm. After nearly half a minute of effort, the robot arm finally moved the cup to his mouth, and Mr. Zhang ate the fried dough sticks.
Grasping, holding, and moving are simple actions for ordinary people, but behind them are a series of complex processes such as signal sending, transmission, and decoding. Therefore, this process of "changing thoughts" is an impossible task for disabled people like Mr. Zhang who have spinal cord nerve damage and motor function loss.
The emergence of brain-computer interface technology in recent years has brought good news to such patients.
The so-called brain-computer interface is a channel that establishes a direct transmission of brain commands between the brain and external devices such as prostheses. Even if the spinal cord and motor nerve pathways are damaged but the cerebral cortex functions are still intact, the brain's signals can be interpreted by a computer and directly control external devices.
As early as 2012, the Zhejiang University team implanted microelectrode arrays in monkeys' brains and used computer information technology to successfully extract and decipher the neural signals of the monkeys' brains for four gestures: grasping, hooking, holding and pinching, so that the monkeys could directly control external robotic arms through their own "intentions". Furthermore, in 2014, the Zhejiang University team implanted cortical electroencephalogram microelectrodes in the human brain, achieving "intention-controlled" robotic arms to complete the difficult "rock, scissors, cloth" finger movements, creating the first in China at the time.
Compared with the previous two times, what is the difference between this latest achievement? Wang Yueming, a professor at the Qiushi Institute for Advanced Studies of Zhejiang University, said that the clinical application in 2014 was to "cover" the surface of the patient's cerebral cortex with an electrode sheet (cortical EEG electrode). The electrode itself was not inserted into the cerebral cortex. It was a semi-implanted operation of opening the skull but not inserting the cortex, and the discharge of a single neuron could not be detected. This time, the microelectrode array was directly inserted into the motor cortex of the brain. It is an implanted operation that can detect the discharge of a single neuron cell, and the signals obtained are more direct, stable and rich. "Compared with non-implanted research, for example, implantable research is equivalent to watching a football game in a stadium, and you can see with your own eyes whether the athlete is volleying or heading the ball, while non-implantable research is like 'listening' to the game outside the stadium, and you can only get a general idea through cheers or boos."
The 2012 study was also implantable, but the transition from monkey brains to human brains posed challenges to the decoding, encoding, calculation methods, and efficiency of the signals being studied. First, the former can obtain brain signals by actually moving their arms, while paralyzed patients only imagine movements, so there is no accurate movement information to build decoders, and the signal quality is less stable than the former; secondly, human brain activity is more affected by the environment, and the complexity of computer processing of these signals will also be greatly increased.
The volunteers of the implantable brain-computer interface research reported in the international community were all young and middle-aged, while Mr. Zhang is a typical elderly patient with relatively weak physical strength, attention, and emotional coordination. Zhang Jianmin, director of the Department of Neurosurgery at Zhejiang University Second Hospital, said: "This experiment requires a high degree of individualization. There is no previous experience to refer to. We need to continue to explore and innovate in perioperative management, surgical operation, electrode implantation accuracy, postoperative training mode, signal analysis, and medical care."
Robot-assisted surgery and nonlinear neural network algorithms are both new attempts
After making sufficient preoperative preparations, the research plan was approved by the hospital ethics committee in August last year and officially started after obtaining the informed consent of Mr. Zhang and his family.
The challenge begins with how to accurately implant the microelectrodes into the patient's brain while minimizing damage.
Zhang Jianmin said that the cerebral cortex neurons are divided into 6 layers, and the experiment requires the electrodes to be implanted in the 5th layer. Just as Song Yu of the Warring States Period said in "Dengtuzi's Lustful Fu", "If you add one inch, it will be too long, and if you subtract one inch, it will be too short", if the electrode is implanted too shallowly, it will not achieve the desired effect, and if it is too deep, it will damage other nerves, which is very difficult. "This is a brand new operation for us."
In the past, similar surgeries were all done with traditional manual implantation. Although the implantation effect and the quality of the subsequent EEG signals were generally acceptable, the accuracy was not ideal. Zhang Jianmin thought of a surgical robot. They used a surgical robot with a step size of 0.1 mm to accurately deliver two microelectrode arrays to the predetermined position, with an error control within 0.5 mm. This is also the world's first successful electrode implantation surgery using a surgical robot.
"There are 100 electrode pins on a 4 mm x 4 mm microelectrode array, and each pin can detect the discharge of one or more neuron cells. The other end of the electrode is connected to a computer, which can record the neural signals emitted by the brain in real time," said Wang Yueming.
The next key step is how to achieve "mind control". The team said that hundreds of billions of neurons in the human brain communicate with each other by sending tiny electrical pulses, thus giving orders to the human body's every move. To achieve mind control, it is necessary to collect and decode the human brain neural electrical signals within the electrode detection range in real time, and match different electrical signal characteristics with the movements of the robotic arm.
Since brain-computer interface technology relies on both the patient's EEG signal characteristics and machine algorithm design, there is currently no unified and standardized signal acquisition, decoding and other analysis methods, that is, the existing analysis methods cannot be directly used. In fact, the team also verified this during the research process. They initially used several sets of foreign linear algorithms, and the results were not very good. Later, Wang Yueming and his team members introduced nonlinear and neural network algorithms, and proposed a personalized solution for this elderly patient.
"Compared with young and middle-aged patients, the quality and stability of EEG signals of elderly patients are worse. The nonlinear decoder we designed can better 'read' the minds of the elderly and help patients better master how to control robotic arms and hands through feedback learning."
Of course, it is very difficult to achieve the goal of "integration of man and machine". The team adopted a step-by-step training method, first letting Mr. Zhang control the mouse on the computer screen to track and click the ball in two-dimensional motion and three-dimensional virtual reality motion, then practice commanding the robotic arm to complete nine directions of movement, such as up, down, left, and right, and finally simulate shaking hands, drinking water, eating, etc. The training took more than four months to achieve such exciting results.
Drinking water and eating are no longer difficult, and I can also play mahjong for entertainment, which makes me feel much better.
Mr. Zhang can eat when he is hungry and drink when he is thirsty. Through the brain-computer interface, Mr. Zhang can "do" some things by himself. He said that it feels great to have his dreams come true. Moreover, after the staff knew that Mr. Zhang liked to play mahjong, they specially designed a program so that he could play the computer mahjong game by controlling the mouse. "When he first came to us, Mr. Zhang was in a very low mood. He didn't respond to us when we talked to him. Now, you can see that he is much happier." said the head nurse.
Considering the patients is the starting point of all our work.
"The ultimate goal of any basic medical research is to apply it to the clinic and solve practical problems for patients. This is the so-called 'translational medicine'," said Zhang Jianmin. Patients with severe motor dysfunction such as high paraplegia, amyotrophic lateral sclerosis, and locked-in syndrome are expected to use implantable brain-computer interface technology and use external devices to reconstruct limb movement, language and other functions. Moreover, with the continuous development of brain science, the clinical application of this field will gradually expand from the existing functional reconstruction mainly based on motor function to more complex functional reconstruction such as language, sensation, and cognition. "As we all know, stroke is more common in the elderly. Although many patients with cerebrovascular diseases have been saved by our treatment, they often have sequelae such as hemiplegia and aphasia. Therefore, the successful translational research on brain-computer interface motor function reconstruction on elderly volunteers will have very important guiding significance for future clinical treatment and rehabilitation. "
Wang Yueming said that research in the field of brain-computer interface requires close cooperation among multiple disciplines such as neuroscience, information science, mechanical engineering and medicine, and Zhejiang University's characteristics as a comprehensive university provide a good soil for interdisciplinary research.
It took the team more than ten years to realize the "animal navigation system" by implanting electrodes in the brains of rats and apply brain-computer interfaces to the human brain. Today's research also means that Zhejiang University's brain-computer interface technology has reached the world's most advanced level.
This research was funded by the National Key R&D Program of China "Research and Application of Active Rehabilitation Technology for Cerebrovascular Disease Based on Brain-Computer Interface (2017YFC1308500)", the National Key R&D Program of China "Research on Key Technologies of Brain Information Cognition of Brain-Computer Fusion (2018YFA0701400)", and the National Natural Science Foundation of China Major Scientific Research Instrument Development Project "Development of Instruments for Real-time Analysis and Control of Complex Brain Neural Network Systems (31627802)".
| 
提升卡
变色卡
千斤顶
京公网安备 11010802033920号
