Design of remote monitoring system for smart home based on Android

Abstract: At present, remote video monitoring system has become an important part of smart home. The intelligent monitoring system is based on the Android operating system platform, with a tablet computer as the client, downloading monitoring images from IP cameras through the HTTP protocol, and using a variety of algorithms to intelligently monitor the situation in each room or different angles of the home.

　　In recent years, video surveillance systems have been widely used in manufacturing enterprises, property management, unmanned terminals, banks, transportation, warehouses and other fields. When there are objects and areas in the home that require special attention, the owner of the home can install cameras in his home environment and remotely observe their conditions through the network. When an unexpected situation occurs, the progress of the situation can also be monitored through the network. With the widespread popularity of smart mobile terminals, and due to their unique portability and easy access to the network, mobile terminals have become an ideal choice for dynamic monitoring and on-site monitoring.

　　The client device of the video surveillance system designed in this paper is an Android tablet computer. The client obtains real-time monitoring images from the IP camera, uses different algorithms to determine whether there are people entering or leaving the home according to different scenarios, marks the objects of interest and tracks them intelligently, identifies the lighting conditions of each room and the use of home appliances such as televisions and computers, and can realize 4-way intelligent monitoring at the same time.

　　1 Monitoring Platform Introduction

　　1.1 Client Platform Introduction

　　The client uses the Android smart operating system, which is the name of the open source mobile phone operating system based on the Linux platform released by Google on November 5, 2007. The platform consists of an operating system, middleware, user interface and application software. The underlying Linux kernel only provides basic functions, while other application software is developed by each company, and some programs are written in Java.

　　Data from early 2011 showed that the Android operating system, which had only been officially launched for two years, had surpassed the Symbian system, which had dominated for 10 years, and became the most popular smartphone platform in the world. Now, the Android system is not only used in smartphones, but is also rapidly expanding in the tablet market. At the same time, the Android smartphone operating system has five major advantages: openness, freedom from the constraints of operators, rich application selection, no restrictions on developers, and seamless integration of Google applications.

　　1.2 IP Camera Introduction

　　An IP camera is a device that can generate digital video streams and transmit them through a wired or wireless network. It does not require additional software or hardware to transmit live digital images directly to the Internet, allowing computers around the world to watch real-time live digital images through a browser. IP cameras support TCP/IP, SMTP E-mail, HTTP, and other Internet protocols, and support multiple operating systems such as Windows, Unix, Mac, and OS/2.

　　The design uses Hikvision's DS-2DM1-502 medium-speed network smart dome camera, which supports clients to download surveillance images and H.264 streams via the HTTP protocol.

　　2 Programming Principles and Implementation

　　2.1 Client Design Principles

　　This design uses an Android tablet computer as the client, and the client software flow is shown in Figure 1.

　　The specific implementation of the system is roughly divided into two parts: connecting with the IP camera to obtain the required video stream; making corresponding judgments and responses based on the monitoring scene.

　　2.1.1 Obtaining surveillance video

　　The client connects to the IP camera via WiFi. After logging into the IP camera and selecting the monitoring channel, it can simultaneously realize 4-way intelligent monitoring, that is, it can observe the basic situation at home: whether there are people entering or leaving the home, whether there is anyone at home, follow the objects that appear in the monitoring, identify the lighting conditions of each room, and the use of home appliances such as TVs and computers, etc. The client implementation method is to enable multi-threaded control of 4-way monitoring tasks, so that each monitoring task has relative independence, so as to change channels.

　　There are two specific implementation methods for multithreading, namely, inheriting the Thread class and implementing the Runnable interface. If the client adopts the method of inheriting the Thread class, it cannot inherit other classes except the thread class, and the interface sub-thread must inherit the View class, so this design adopts the second method. In the monitoring implementation, each screen is a separate thread and a complete view. It is a class object that implements the Runnable interface and inherits the View class. Therefore, each object is independent of each other, and each monitoring screen is independent of each other. The monitoring channel can be flexibly selected for background control and management.

　　In terms of interface implementation, the main interface is a 4-channel monitoring window. After initially selecting the monitoring channel, you can obtain the monitoring image sequence through the HTTP protocol, perform algorithm processing according to the scene, and play the intelligent monitoring screen.

　　2.1.2 Program Optimization

　　The client needs to call different intelligent monitoring algorithms according to different scenarios to judge and identify the events that occur. This process includes a large number of image processing operations, such as grayscale, background modeling difference, median filtering and morphological operations. The development language of the Android operating system is Java, but the running speed of pure Java-programmed image processing programs on Android phones is a big problem. Calling C program modules in Java programs through JNI (Java Native Interface) technology significantly improves the running speed of the application. The author intercepted 40 frames as observation objects. The average running time of each major operation step of the program is shown in Table 1, from which it can be seen that the running time has been shortened to 1/18~1/17 of the original.

　　Table 1 Comparison of runtimes of the main algorithms implemented in Java and JNI

　　2.2 Function and Implementation

　　The designed intelligent monitoring system has three major functions: managing all cameras in the home and flexibly selecting monitoring points; executing a variety of different intelligent algorithms; and being able to link with other networked devices to achieve smart home control.

　　The author tested the system using different practical scenarios. The functional description of the system and the experimental results are as follows.

1) Multi-channel monitoring

　　Get real-time monitoring pictures, and display up to 4 channels of intelligent monitoring pictures processed by the background algorithm on the same screen, as shown in Figure 2. If you need to change the monitoring channel, you can click the channel list on the right side of the monitoring interface to pop up the channel selection bar to change the channel, so as to monitor the situation at home more comprehensively in real time, as shown in Figure 3.

　　2) Intrusion Detection

　　As shown in Figure 4, the system uses motion object detection and recognition algorithms to detect intrusions into specific areas, thereby determining the entry and exit of people and triggering an alarm when a stranger intrudes. The alarm is in the form of a pop-up prompt message, and a flashing red border is added to the periphery of the monitoring channel window.

　　3) Monitoring of household appliance usage

　　The system can detect the lighting conditions in each room. If no one is active in a room for a long time or the room is well lit, a prompt message will pop up to remind the user to remotely control smart appliances and turn off unnecessary lighting devices, as shown in Figure 5.

　　The system can also determine whether the computer and TV are in use based on the brightness and shade changes of the computer display or TV screen and whether there are people in the room, and notify the user to take action, as shown in Figure 6.

　　4) Object Tracking

　　The user can select the object appearing in the monitoring screen as the target of interest according to the needs. The system will execute the motion detection program to circle the movement range of the object in real time to facilitate monitoring and management, as shown in Figure 7.

　　3 Conclusion

　　The intelligent monitoring system allows users to select multiple cameras for monitoring, so that they can remotely observe the situation at home in a comprehensive and timely manner, and can automatically issue an alarm when an abnormal situation occurs to remind users of danger in their home.

　　In addition, when no one is at home or in a room, the system can also remind users to turn off unnecessary appliances to save energy. This function reflects the integration of the monitoring system and the Internet of Things. This intelligent monitoring system has good portability and relatively complete functions.