A wireless image monitoring terminal with motion detection function

Wireless transmission has become an application trend in embedded systems that require data transmission. GPRS (General Packet Radio Service) has the advantages of always being online, fast login, and high-speed transmission [1], and is widely used in communication systems, wireless meter reading, and wireless monitoring equipment.
Traditional wireless image monitoring systems need to continuously transmit monitoring images to the monitoring center, but only a few of these monitoring images are useful, which wastes a lot of data traffic and increases costs. At the same time, monitoring personnel must always pay attention to the monitoring screen, which is not only boring, but also easy to ignore important monitoring screens. Therefore, it is of great significance to develop a monitoring terminal that can automatically filter out images of interest to users and send them to the monitoring center.
This wireless monitoring terminal uses Samsung's S3C2440A as the main control chip and is ported with the Linux operating system. The system performs motion detection on the images collected by the USB camera. If an object moves in the monitoring area, the monitoring image is transmitted to the monitoring center via GPRS. Since only the monitoring images that the user is interested in are sent, the data traffic in wireless data transmission is greatly reduced, avoiding the waste of data traffic caused by transmitting a large number of useless monitoring images in traditional wireless monitoring systems, and also reducing the workload of monitoring personnel due to long-term monitoring of monitoring images.
1 System composition
This system consists of three parts: hardware, underlying software and application software. The system structure is shown in Figure 1.

The microprocessor of the system hardware uses Samsung's S3C2440A processor, which contains an ARM920T core, a standard operating frequency of 400 MHz, rich embedded peripherals, and an external expansion of 64 MB Nandflash (model: K9F1208D0A) and 2 32 MB SDRAM (model HY57V561620); the camera uses the Oni S888 based on the ZC301 device, using a CMOS photosensitive device, 480,000 pixels, and a maximum resolution of 800×600. The driver needs to be installed separately. The GPRS module uses Huawei's GTM900 dual-band GSM/GPRS wireless module, which supports standard AT commands and enhanced AT commands, embedded TCP/IP protocol, and the operating frequency band is 800 MHz/900 MHz/1 800 MHz/850 MHz/1 900 MHz. This system uses the RS232 serial port to connect to the S3C2440, and the SIM card voltage is 3 V.
The underlying software consists of BootLoader (vivi), embedded Linux kernel (version 2.6.17), device driver, and file system. In the kernel configuration, it is necessary to add support for the video device programming interface. Select Device Drivers->Multimedia device->Video for linux in the configuration menu path.
The application software mainly includes three parts: data acquisition, motion detection, and image transmission.
2 Application software design
2.1 Basic process
The core modules of the application software include image acquisition module, motion detection module, and GPRS wireless data transmission module. First, the image acquisition module uses a USB camera to collect images with a resolution of 320×240, and then the motion detection module uses an improved frame difference method to perform motion detection on the collected images. If the motion detection module detects a moving object in the monitoring area, the image with the moving object is added to the image sending queue, and the GPRS data transmission thread is awakened to send the image in the queue to the monitoring center. When all the images in the image sending queue are sent out, the GPRS data transmission thread will be put into a dormant state.
2.2 Image Capture of Video4Linux
Video4Linux is the kernel driver for video devices in Linux. It provides a series of interface functions for application programming for video devices (common TV capture cards and USB cameras), and also provides data interfaces for radio communication and text broadcast decoding and vertical blanking [2]. This paper mainly designs programs for image acquisition for the USB camera device file /dev/video0. The main steps and some codes are introduced below:
(1) Open the video device
int fd; // File descriptor of the video device
fd = (open ("dev/video0", O_RDWR));
if (fd < 0) { // Open failed
printf ("No Camera found! n");
exit (-1);
}
(2) Read device information
The ioctl function stores the camera information in the structure videocap. The camera information mainly includes name, type, number of channels, image width, image height, etc.
struct video_capability videocap；
if(ioctl (fd，VIDIOCGCAP，&videocap) == -1){
printf(″Couldn′t get videodevice capability″);
exit(-1);
}
(3) Change the current device settings
Set the properties of the image frame and save the properties to be set in the structure videopict, which mainly include image brightness, color, contrast, palette parameters, etc.
if (ioctl (fd，VIDIOCSPICT，&videopict) < 0){
printf(″Couldnt set videopict params with VIDIOCSPICT″);
exit(-1);
}
(4) Image acquisition
There are two main ways to acquire images: read() direct reading and mmap() memory mapping. This system uses mmap() memory mapping.
vmmap.height=240；
vmmap.width=320；
vmmap.format=vd-> VIDEO_PALETTE_JPEG；
ioctl (vd->fd，VIDIOCMCAPTURE，&(vd->vmmap)；
ioctl (vd->fd，VIDIOCSYNC，0)；
pFramebuffer=(unsigned char*) mmap(0，vd->videombuf.size，
PROT_READ | PROT_WRITE，MAP_SHARED，vd->fd，0)；
The first ioctl() sets the size of the captured image to 320×240 and the image format to JPEG; the second ioctl() sets it to single frame capture, and then uses the mmap() function to map the image data to the memory pointed to by the pFramebuffer pointer.
2.3 Motion
Detection After the surveillance image is acquired, motion detection can be performed. In video image processing, the main methods for detecting moving objects are background difference method, frame difference method and optical flow method. This system adopts the frame difference method. The principle is: when there is an object moving in the surveillance area, the pixel value of the corresponding area in the acquired image frame will change significantly. At this time, it is only necessary to differentiate the two frames to obtain the absolute value of the brightness difference between the two frames. If the absolute value is greater than the set threshold, there is a moving object in the area; otherwise, there is no moving object. It can be expressed as follows [3]:

In practical applications, the selection of threshold T is critical. If the threshold is too low, the noise in the image cannot be effectively suppressed; otherwise, useful change information in the image will be suppressed. Usually, the selection of the threshold is related to specific external environmental conditions such as the lighting of the monitoring scene and the position of the camera. It is impossible to set the same threshold for different monitoring environments. This paper adopts the automatic calculation of thresholds for different monitoring environments. The method is to collect N background images before starting to collect images, and use the average value of the brightness components of these images as the threshold of the monitoring environment. The formula is as follows:

This system only needs to detect whether there is any object movement in the monitoring area, and does not need to extract the complete area of ​​the object. Therefore, it only needs to count the number of pixels greater than the threshold T during image difference. If the number of pixels exceeds the alarm threshold, it indicates that there is object movement; otherwise, it may be due to light changes or noise interference. This not only avoids the disadvantage of the frame difference method that cannot accurately calculate the moving area, but also reduces the computing pressure of the processor. The program flow is shown in Figure 2.

2.4 GPRS wireless data transmission
The GPRS module of this system is mainly used to transmit the monitoring images of moving objects sent into the image sending queue to the monitoring center. The data transmission flow chart is shown in Figure 3.

The main initialization process of the GPRS module GTM900B is: configure APN->enter TCPIP function->open a TCP link. Since the module has an embedded TCP/IP protocol, data transmission can be achieved by directly operating AT commands. The main initialization code is as follows:
SendData(″AT+CGDCONT=1，″IP″，″CMNET″″，13);
… //Check whether the configuration is successful
SendData(″AT%ETCPIP″，13);
… //Check whether tcpip is entered
SendData(″AT%IPOPEN=″TCP″，″202.206.1.26″，23″，13);
…
The function SendData() is to send data to the serial port device.
Two points should be noted during initialization:
(1) Use TCPIP related commands 20 s to 30 s after the module is powered on, because the module needs time to search the network and initialize related information.
(2) Use the AT%ETCPIP command. When the command has not returned, any input to the serial port will force the command to end. If %IPCLOSE:5 is returned, it means that the module exits the TCPIP function. At this time, AT%ETCPIP must be used again to open the TCPIP function.
When the GPRS module is initialized successfully, the module has been connected to the Internet. At this time, wireless data transmission can be performed. The main operation AT command is: AT%IPSEND. It should be noted that in the data transmission mode of setting AT%IOMODE=1, the data packet size cannot exceed 1024K. Therefore, an image must be divided into multiple data packets for transmission, and ASCII and HEX format conversion is required when sending and receiving data.
3 Test results
In the actual test, the threshold calculation value of the monitoring environment is 11, the alarm threshold is 1000, and the gray value conversion formula is gray=r×0.3+g×0.6+b×0.1. The calculated number of pixels greater than the threshold is 13562, which is greater than the alarm threshold. The GPRS module starts to transmit the image. The test image is shown in Figure 4.

The core operating frequency of S3C2440A is 406.425 MHz. When the GPRS module is idle, it takes about 55 ms from the acquisition of two 320×240 resolution images to the end of motion detection. When the GPRS is busy, it takes about 115 ms, which can basically realize the capture of monitoring images with moving objects. The GPRS module sends a captured JPEG image. Under normal network conditions, the system delay usually does not exceed 5 s.
This paper briefly introduces the system composition of the wireless monitoring terminal and elaborates on the implementation methods of the three core modules of the application software. The test operation results show that this system realizes the requirement of transmitting the monitoring images of moving objects to the monitoring center, and has the characteristics of low cost and simple structure. Compared with the traditional wireless monitoring system, this system can greatly reduce the data traffic in wireless data transmission, and also saves the monitoring personnel from the work of monitoring the monitoring images for a long time. In addition, the needs of different users are fully considered in the design of software and hardware. Only a slight expansion of the software and hardware can realize the transmission of monitoring images or alarm text messages to the user's MMS mobile phone, which has a good use and promotion value.