Design and implementation of intelligent home care robot based on MCU

    Core Tip : With the advent of an aging society, the issue of caring for elderly people living alone has received increasing attention. Therefore, researchers have used microcontrollers (MCUs) with high control and computing capabilities to develop intelligent care robots that can follow elderly people living alone at any time and have functions such as fall detection, emergency notification, and network interaction, in order to improve the quality of home care.

　　Taiwan is now gradually entering an aging society. Most children work away from home and cannot take care of the elderly wholeheartedly. Therefore, the care of the elderly who are alone at home is very important. Some people choose to hire a caregiver to take care of them at home, or decide to send them to a nursing home, but these methods are too expensive, and children cannot directly take care of their relatives, which seems less reassuring. In order to effectively solve this problem, electronic devices that can care for and care about the lives of the elderly have become increasingly important. Through the monitoring and interactive functions of the devices, not only can the elderly know the news of their relatives outside at home, but the children can also get health and life information of their parents at home at any time.

　　The elderly care system is designed with home monitoring functions and family interaction functions. Through Internet services, when the elderly need care, they can use the gesture recognition function to call the robot to their side; when they want to view the messages and pictures of relatives and friends on Facebook, they can use the Facebook dynamic service to get the latest dynamic information of relatives and friends in real time, and they can also have video chats with distant family members through Internet calls; at the same time, they can also monitor at any time, and when an accident occurs, they can immediately notify family members and medical staff to deal with it.

　　The system uses the main system chip to store and retrieve the values ​​sensed by Kinect, such as skeleton tracking, gesture recognition, and webcam video calls, and then uses serial transmission (USART) to send the values ​​to the main system chip, and make recognition modes such as distance to follow the elderly, control motors and interact with the elderly.

　　Kinect sensing helps expand innovative applications of care robots

　　The innovative features of the system include follow-up care mode – Kinect skeleton tracking and ultrasonic positioning, skeleton judgment – ​​fall detection, emergency notification – SMS sending, and interactive functions – gesture recognition and network video (Figure 1).

　　

　　Figure 1 Ideal interactive functions of care robots

　　In the follow-up care service, the system uses skeleton tracking and ultrasonic positioning. When the elderly move, the robot will follow behind to provide care. It can determine whether the elderly have fallen at any time, and it is very convenient to operate. Most existing fall detection systems rely on cameras or equipment worn by the user to identify falls. The fall detection of this system only needs to use Kinect skeleton tracking to determine whether a fall has occurred.

　　As for the emergency SMS transmission, when the elderly person is determined to have fallen, the robot will automatically send a SMS to the designated family member's mobile phone through the main system chip. After receiving the message, the family member can immediately respond or watch the video. As for the Facebook dynamic and interactive video service, the system is equipped with Facebook dynamic service and online video, so that the elderly can follow the pictures of their children who are away from home, and let relatives and friends use video interaction to see the elderly's situation at home, which can effectively shorten the distance between the elderly and their family.

　　．Follow-up care service

　　The main system of the care robot uses chips and Kinect to interact with the elderly and provide care services. The intelligent care robot can follow the elderly through Kinect to provide care at any time, and use Kinect's own skeleton recognition to determine the number and movement of skeletons in normal conditions. When the elderly accidentally fall, the skeleton will be judged by the instant drop or disappearance of the skeleton, and then the numerical signal will be sent back to the main system, which can be sent to the children's mobile phones for immediate notification through the emergency text message function. Family members can check the elderly's condition at home through online video to avoid missing the time to see a doctor.

　　为即时反应与传送数值来跟随与保障老人的安危，看护系统透过主系统芯片提供USART，即可直接利用USART而不须再以串列周边介面（SPI）进行转换动作。跟随方面，则运用产生脉衝宽度调变（PWM）脉波来控制HB-25马达驱动器来驱动直流马达，由于控制波形即可正反转，就能在主芯片应用与结合出让机器人前后左右的动作与功能。

　　The system also needs to use a PING ultrasonic distance sensor (Figure 2) to detect distance. In addition to applying PWM to transmit and receive sound waves, it also uses general input/output to receive numerical distance judgments and Kinect skeleton values.

　　

　　Figure 2 PING ultrasonic distance sensor

　　The robot uses chips to track, and ultrasound detects distance, allowing the main system to determine the location of the elderly and command the motor to move so that the robot can follow. It can stay by the elderly's side at all times, pay attention to any emergencies, and wave its hand to video call family members, etc.

　　PING超音波距离感应器用于测量与物体间的距离，範围介于3cm？3.3m，容易装置于机器人或自动设备上。超音波感测器利用脉波产生，透过声波喇叭进行一发一收的动作来得到讯号的时间长短，也就是感测器发送讯号至物体而反射后所经过的时间，在经过运算后即能得到精準的距离。为实现追踪功能，超音波定位追踪方法利用主系统芯片产生脉波与判断，使超音波能自动追随物体，也更加稳定照护功能。

　　In order to be able to carry daily necessities, a motor with greater torque (IG-30GM) is chosen as the driving force for the vehicle, and a high-current 12-volt (V) motor driver chip (HB-25) is used to control the DC motor. The control method is pulse width modulation to control the forward and reverse rotation, time difference, rotation speed and other parameters of the two motors. The chip plays an important role in motion control, such as the vehicle's forward, backward, and turning movements.

　．Kinect skeleton judgment

　　Kinect透过镜头即时撷取使用者的动作，之后并会接着解析相对应的指令。透过内建的红外线VGA镜头发出主动式雷射，在Kinect可扫描的範围内藉由雷射反射过程判断使用者位置，当使用者的身体部位识别完毕后，Kinect会将所撷取到的资料整理成一组骨架图，藉此对应于相关应用。

　　使用Kinect本身的骨架辨识功能，能模拟出老人的骨架，让主系统监控骨架与动作，判断骨架是否出现或是下降；判断特定动作，启动特定的功能，如图3为人体进入Kinect範围时会出现骨架识别，进而做出不同指令动作。

　　

　　Figure 3 Robot action flow chart

　　．Emergency SMS sending

　　When an elderly person falls, the main system receives the fall command value to determine whether the skeleton outlined by Kinect has dropped or disappeared, and immediately sends a text message to the children's mobile phones through the emergency text message function to notify them. After checking the text message, family members can immediately activate the Internet video service, so that they can understand the condition of the elderly at home at the first time and communicate with them.

　　In addition, the SMS function allows you to specify a mobile phone number, and it is very convenient to change emergency contacts.

　　．Facebook News/Interactive Video Service

　　When the elderly need a care robot to watch the status of their children who are away or to contact them, they can call the care robot to come to their side through gesture recognition. The robot can use gesture recognition to provide Facebook status services, watch their children check in or upload photos, and use webcams and online video functions to have video conversations with their children far away and keep up to date with each other's news at any time.

　Execute complete test project to make robots safer for elderly care

　　Figure 3 is the action flow chart of the system. The microcontroller (MCU) is used as the core control (Figure 4) to perform numerical calculations and instant responses. The high computing speed of the microcontroller is used to instantly complete all numerical reception and instructions, so that the robot will not move smoothly due to delays.

　　

　　Figure 4 Microcontroller action block diagram

　　The main system chip must also be explained with other hardware structures. Starting from the main system chip, all modules can be combined to achieve the application of intelligent care robots. During the implementation process, the projects that need to be carefully adjusted and tested include DC motor drive control, ultrasonic positioning angle tracking, Kinect command movement direction and robot following safety distance.

　　． DC motor drive control

　　为让本系统达到行动照护，所以直流马达是必定需要的，并使用马达驱动器，控制起来只须调整脉衝宽度调变（PWM），以及马达的正反转、时间差和转动速度等参数，就能让照护机器人行动起来更加顺畅（图5）。

　　

　　Figure 5 DC motor testing and construction

　　利用测试方法让马达得以进行正反转动作，再将完成的马达驱动置于该作品底座，裡头包含自行设计的电路板、配置电源及最重要的马达驱动器等。

　　．Ultrasonic positioning angle tracking

　　为能让机器人能自动追踪人体，自行简易设计出应用叁个超音波判断距离与方向，再与伺服器马达结合而形成的角度追踪，透过侦测叁种方向，马达就能精準追随并停止。

　　． Kinect command action direction

　　The system combines Kinect for skeleton and motion detection, and hand gestures are used to open the Facebook dynamic service and the online video function. In order to facilitate the opening of the function, different directions and movements of gestures are tested.

　　．Robot follows safe distance

　　为能让跟随老人并维持距离追踪，以及Kinect角度侦判範围，测试不同距离来决定能最精準且侦测到数值的车身与老人间距，分别测试50cm、80cm、100cm叁种不同距离。

　　图6为该系统的正面摆设，其中包含笔记型电脑的放置，用来观看视频及骨架侦测观察画面。Kinect置于上方处，方便撷取整个骨架，并结合超音波进行人体追踪，置放叁颗以扩大测量範围。

　　

　　Figure 6 Hardware configuration of the front side of the care system

　　The follow-up function can be used to achieve real-time monitoring, so that the elderly can reduce the risk of danger at home. The elderly can use their own lockers to store their daily necessities for instant access convenience, taking care of the elderly in the most humane and intelligent way. More functions will be added in the future to make the service more complete.