Design of touch screen driver based on Qt/Embeded in embedded Linux

Embedded Linux has attracted the attention of embedded system developers with its open source, kernel robustness and stability, scalability, and support and maintenance from professional commercial companies and the world's top free software developers. It has become the new favorite of embedded operating systems. Touch screens are widely used because of their friendly human-computer interaction, simple and flexible operation, and fast input speed, which greatly simplifies the input of embedded systems. This article introduces the design of touch screen driver based on Qt/Embedded on the embedded Linux system platform. This solution has been successfully used in engineering machinery safety instrument and power quality monitoring instrument projects, realizing the support of GUI (Graphical User Interface) interface for touch screens, and can be customized according to different touch screens.

1. Introduction to Qt/Embedded
Qt/Embedded is a development library for GUI and application development for embedded systems released by the famous Trolltech company. It is a comprehensive C++ graphical interface application development framework that inherits all standard APIs of Qt, provides a more compact window generation system than Xlib and XWindows systems, and directly operates FrameBuffer (see Figure 1). The fully modular design and efficient compilation system reduce memory consumption, making Qt/Embedded a powerful and comprehensive GUI development tool in embedded environments. Due to the powerful functions of Qt/Embedded, it is widely used in various fields, from consumer electronics (mobile phones, PDAs, set-top boxes) to industrial control (medical imaging equipment, mobile information systems).

2. Device driver basics under Linux
The Linux system mainly divides devices into three types: character devices, block devices, and network interfaces. Each module usually implements one of these types, and the corresponding modules can be divided into three types: character modules, block modules, and network modules. However, this classification method is not very strict. Programmers can construct a large module to implement different types of device drivers in it. In order to achieve good scalability and extensibility, it is usually necessary to create a different module for each function.

A character device is a device that can be accessed as a byte stream, and this feature is implemented by a character device driver. It usually needs to implement at least the open, close, read, and write system calls. Character devices can be accessed through file system nodes, such as character terminals (/dev/console) and serial ports (/dev/ttyS0). Block devices are also accessed through file system nodes under the /dev directory. Block devices can hold file systems. Linux allows applications to read and write block devices like character devices, and can pass any number of bytes of data at a time. Therefore, the difference between block devices and character devices is only the way the kernel manages data internally. That is, the interfaces between the kernel and the driver are different. In addition, the interface of the block device must support mounting the file system.

A network interface is a device that can exchange data with other hosts. It is driven by the network subsystem in the kernel and is responsible for sending and receiving data packets. It does not need to understand how each transaction is mapped to the actual transmitted data packets.

There are other types of driver modules in Linux that use the common services provided by the kernel to handle specific types of devices. So we can communicate with Universal Serial Bus (USB) modules, serial port modules, etc.

In this system, the controller converts the original voltage signal collected by the touch screen into coordinate data through a dedicated A/D, and transmits it to the embedded system through the RS-232 bus (see Figure 2). The Linux system uses the serial port module provided by the kernel to process the touch screen device, and stores the device in the /dev directory in the form of a file /dev/ttyS0, providing system calls such as open, read, write, and close. We only need to operate the serial port device like operating an ordinary data file, and send the coordinate data of the touch screen to the upper Qt/Embedded application layer.

3. Driver of touch screen under Qt
The signals related to user input events in Qt/Embedded are based on the interface call of the underlying input device, which is generally implemented by I/O reading and writing of device files. Most of these drivers have been encapsulated into the Qt library to form corresponding device driver interfaces, such as display card drivers, mice, keyboards, serial ports and parallel ports. The abstract base class of the mouse device is QWSMouse Handler, from which some specific implementation classes of mouse devices are derived. In the 3.3.4 version series of Qt/Embedded, the derived structure of mouse devices is shown in Figure 3.

Figure 3 Derived structure diagram of mouse devices (gray wireframe indicates that the class structure can be omitted)

The loading method of mouse devices is similar to that of KeyBoard devices. When the system constructs a QWSServer object, it calls the member function QWSServer:: openMouse. The program then calls QmouseDriverFactory::create() or QmouseDriverPlugin:: create() in the QWSServer:: openMouse function. This function obtains the device type and device node of the mouse device based on the Linux system environment variable QWS_MOUSE_PROTO. It opens and returns the base class pointer QWSMouseHandler of the corresponding device to the system. The system operates the specific subclass device pointer QWSCustomMouseHandler derived from the base class. [page]

The functions of touch screen and mouse devices are basically the same. Therefore, in the Qt library, the touch screen is generally simulated as a mouse device to implement the operation of the touch screen device. However, since the underlying interfaces of the touch screen and the mouse are different, the upper interface will be inconsistent. For example, the absolute position information is almost never obtained from the mouse driver interface, and only the relative movement amount is generally read. In addition, the acceleration of the mouse movement also needs to be taken into account, while the touch screen interface is almost entirely absolute position information and pressure information. In view of such differences, Qt/Embedded also distinguishes and abstracts the interface part of the same type of device, which is specifically implemented in the QmouseDriverInterface class.

In this system, the Linux system reads the coordinate data of the touch screen from the COM1 port, but because it is inconsistent with the underlying interface of the touch screen, it is necessary to add the QWSCustomMouseHandler program interface class to control the touch screen. Looking at the Qt/Embedded source code qwsmouselinuxtp_qws.cpp and qwsmousevr41xx_qws.cpp, it can be seen that Qt provides driver interface classes for linuxtp and vr41xx touch screens. If these two touch screen interfaces are used, you can directly add the configuration option -qt-mouse- when executing Qt's configure configuration. Since our touch screen is not one of the above two, a driver interface needs to be added.

From the derived structure of the mouse device driver class above, we can know that to add a driver interface, we must first generate the corresponding QWSCustomMouseHandler object according to the corresponding device name by calling the QmouseDriverFactory or QmouseDriverPlugin class. These specific device driver interface classes are derived from the QWSMouseHandler class and inherit the QWSMouseHandler class. Then the system calls QWSCustomMouseHandler:: readMouseData (), and the acquired touch screen positioning and status information are directly sent to the QWSMouseHandler class, the abstract layer of the mouse device driver class. Qt/Embedded uses the QWSMouseHandler class to operate the mouse device.

There are two ways to add device driver interface classes: one is to generate the corresponding QWSCustomMouseHandler by calling QmouseDriverFactory, and the other is to generate the corresponding QWSCustomMouseHandler by QmouseDriverPlugin. We adopt the first solution and modify the original interface qwsmouselinuxtp_qws.cpp to suit the new touch screen device driver interface.

First, we modify qwsmouselinuxtp_qws.cpp, first change the TS_EVENT structure to the data structure of the corresponding device, and then change the device file node opened in the QWSLinuxTPMouseHandlerPrivate function from /dev/ts to our own device file /dev/ttyS1. Then modify the readMouseData() function, read the device file according to our own data structure, and pass it to the QPoint class to locate the mouse or convert it to the mouse button state. The operation of this function can refer to the mouse driver source code under Windows.

In fact, after this modification, the touch screen device can be used normally, but in order to use the mouse and touch screen operations on this system platform at the same time, two steps must be completed: one is to correctly set the QWS_MOUSE_PROTO environment variable. Reading the QWSServer:: openMouse () function in qwindowsystem_qws.cpp, we can know that this environment variable can set multiple devices [: ] at the same time, and multiple devices are separated by spaces. Therefore, QWS_MOUSE_PROTO="Auto LinuxTP" can be set. Qt/Embedded generates the corresponding mouse and touch screen driver interfaces through this environment variable to operate the device. Then modify the source code of the mouse driver interface class of Qt/Embedded. Since the touch screen uses the system serial port, and Qt/Embedded automatically searches for the mouse interface and finds that the serial port is working, it will treat it as a mouse device for operation, which causes a device conflict. Therefore, we need to remove the sub-driver of the serial mouse in the qmousepc_qws.cpp file, find the code in the function QWSPcMouseHandlerPrivate::openDevices() as follows, and comment it out.

else if (driver=="Microsoft") {
QString dev=device.isEmpty()? QString("/dev/ttyS0") : device;
fd = open ( dev.latin1 (), O_RDWR | O_NDELAY );
if ( fd >= 0 )
sub[nsub++] = newQWSPcMouseSubHandler_ms(fd);
} else if (driver=="MouseSystems"){
QStringdev=device.isEmpty()?QString("/dev/ttyS0"). : device;
fd = open ( dev.latin1 (), O_RDWR | O_NDELAY );
if ( fd >= 0 ) sub[nsub++] = new QWSPcMouseSubHandler_mous esystems (fd);
}

Regarding the calibration of the touch screen, read the qwsmouselinuxtp_qws.h file (the code is as follows), and you can see that the coordinate calibration has been implemented in QWSMouseLinuxTPHandler. Generally, you can directly read the position and status of the coordinates.
class QWSLinuxTPMouseHandler:
public QWSCalibratedMouseHandler
{
};
Finally, you only need to add the option -qt-mouse- to the configure of Qt/Embedded installation and recompile. The application on the Qt/Embedded platform can provide support for the touch screen according to customized requirements.

4. Conclusion

This solution uses the same framework as the common mouse driver under Qt, with a simple design and clear organization. It has been successfully applied to the power quality monitor platform, and has stable operation, accurate positioning, and sensitive response. The friendly GUI interface and convenient human-computer interface make the power quality monitor more attractive. At the same time, the advantages of open source code are fully reflected in this solution. By reading a lot of source code, you can fully understand the software working mechanism and customize it according to user requirements to make a product that is truly suitable for users.