【i.MX6ULL】Driver Development 13——Capacitive Touch Driver Practice (Part 2)

DDZZ669

【i.MX6ULL】Driver Development 13——Capacitive Touch Driver Practice (Part 2) [Copy link]

The previous article introduced the writing of capacitive touch driver, including the modification of device tree and driver program (IIC driver + interrupt + input subsystem), and tested the touch function by printing out the touch coordinate values in real time.

In this article, we will first introduce the touch test library - tslib, which can be used to perform graphical touch tests . After that, we will analyze the principles of touch protocol reporting and the specific meaning of the data reported by the input subsystem .

1 Use of tslib

Tslib is an open source program that can provide functions such as filtering, de-jittering, and calibration for the samples obtained by the touch screen driver. It is usually used as an adaptation layer for the touch screen driver and provides a unified interface for upper-level applications .

1.1 tslib library transplantation

First download the source code of the tslib library: https://github.com/libts/tslib/tags

The latest version is 1.22, but this article will use version 1.21

1.1.1 Compile tslib on Ubuntu

Copy the downloaded source code to the Ubuntu virtual machine and decompress it:

tar xvf tslib-1.21.tar.bz2

When compiling tslib, you need to install some files in ubuntu first

sudo apt-get install autoconf 
sudo apt-get install automake 
sudo apt-get install libtool

Create a directory named "tslib" in Ubuntu to store the compilation results, and then execute the following instructions to compile:

cd tslib-1.21/ 
./autogen.sh 
./configure --host=arm-linux-gnueabihf --prefix=/home/xxpcb/myTest/imx6ull/otherlib/tslib/tslib/
make
make install

After the compilation is complete, make install will copy the compilation results to the specified tslib directory:

You can see that the final compilation generates 5 folders.

1.1.2 Configuring tslib on the development board

Copy all five compiled folders to the root file system of the development board:

sudo cp * -rf ~/myTest/nfs/rootfs/

Then open the /etc/ts.conf file of the board and find the following line:

module_raw input

If there is a "#" comment before this sentence, delete the "#". I don't have one by default, so I don't need to modify it.

Open the /etc/profile file of the board. My board does not have this file at this time, so I created a new one and added the following content to it:

export TSLIB_TSDEVICE=/dev/input/event2 
export TSLIB_CALIBFILE=/etc/pointercal 
export TSLIB_CONFFILE=/etc/ts.conf 
export TSLIB_PLUGINDIR=/lib/ts 
export TSLIB_CONSOLEDEVICE=none 
export TSLIB_FBDEVICE=/dev/fb0

TSLIB_TSDEVICE : Touch device file, set to /dev/input/event1 or event2 according to the specific situation (if the mouse and keyboard are connected, this number may change. For example, after I connected a wireless keyboard, touch became event again)
TSLIB_CALIBFILE : calibration file. This file may not exist. It will be automatically generated during calibration.
TSLIB_CONFFILE : Touch configuration file, which will be generated when porting tslib
TSLIB_PLUGINDIR : tslib plugin directory location
TSLIB_CONSOLEDEVICE : console settings, not set here, set to none
TSLIB_FBDEVICE : FB device, that is, the screen, should also be configured to /dev/fb0 or other according to the actual situation

1.2 tslib library test

1.2.1 Screen Calibration

Capacitive screens do not need to be calibrated, but you can also look at the calibration test case of tslib and enter the following command:

ts_calibrate

If you are not satisfied with the calibration, just delete the /etc/pointercal file.

1.2.2 Multi-touch drag test

Use the following command:

ts_test_mt

Then a touch test interface will appear. First test the Drag function. After your finger touches the screen and moves, the cross mark on the screen will move with it:

1.2.3 Multi-touch line test

Let’s test the Draw function again using the same command. When your finger touches the screen and moves, a sliding track will appear on the screen:

2 Multi-touch (MT) protocol explanation

The Multi-touch (MT) Protocol is introduced in the corresponding document in the Linux kernel source code, as shown below:

The multi-point capacitive touch protocol is divided into two types: TypeA and TypeB. Currently, TypeB protocol is basically used.

The TypeA protocol is applicable when touch points cannot be distinguished or tracked, and such devices report raw data.
The Type B protocol is suitable for touch devices that have hardware tracking and can distinguish touch points. This type of device updates the information of a certain touch point through a slot.

The touch point information is reported to the Linux kernel through a series of ABS_MT events. These events are defined in include/uapi/linux/input.h:

The more commonly used ones are:

ABS_MT_SLOT: Report touch point ID
ABS_MT_POSITION_X: Report the X coordinate information of the touch point
ABS_MT_POSITION_Y: Reports the Y coordinate information of the touch point
ABS_MT_TRACKING_ID: TypeB distinguishes touch points

The following describes the differences between the two protocols.

2.1 Type A protocol

The TypeA protocol is applicable when touch points cannot be distinguished or tracked, and such devices report raw data.

The timing of sending touch point information by TypeA protocol is as follows (taking 2 touch points as an example):

ABS_MT_POSITION_X x[0]
ABS_MT_POSITION_Y y[0]
SYN_MT_REPORT
ABS_MT_POSITION_X x[1]
ABS_MT_POSITION_Y y[1]
SYN_MT_REPORT
SYN_REPORT

First, report the x and y of each point
Then report a SYN_MT_REPORT
Report other points in sequence
After all points are reported, report a SYN_REPORT

When the first contact leaves , the reporting sequence is as follows (that is, only the remaining one is reported):

ABS_MT_POSITION_X x[1]
ABS_MT_POSITION_Y y[1]
SYN_MT_REPORT
SYN_REPORT

When the second contact is also removed , the reporting sequence is as follows (that is, reporting empty data):

SYN_MT_REPORT
SYN_REPORT

If the driver reports BTN_TOUCH or ABS_PRESSURE in addition to ABS_MT events, the last SYN_MT_REPORT event may be ignored. In addition, the last SYN_REPORT will be discarded by the input kernel, resulting in no empty touch events reaching the user layer.

2.2 Type B Protocol

The Type B protocol is suitable for touch devices that have hardware tracking and can distinguish touch points . This type of device updates the information of a certain touch point through a slot .

The timing of sending touch point information by TypeA protocol is as follows (taking 2 touch points as an example):

ABS_MT_SLOT 0
ABS_MT_TRACKING_ID 45
ABS_MT_POSITION_X x[0]
ABS_MT_POSITION_Y y[0]
ABS_MT_SLOT 1
ABS_MT_TRACKING_ID 46
ABS_MT_POSITION_X x[1]
ABS_MT_POSITION_Y y[1]
SYN_REPORT

Before each data point, an ABS_MT_SLOT event is reported with a touch point ID, which is provided by the touch IC.
TypeB requires that each SLOT must be associated with an ABS_MT_TRACKING_ID, which is automatically assigned by the Linux kernel.
Then report the x and y of a point
Report other points in sequence
After all points are reported, a SYN_REPORT is reported.

When the contact 45 moves in the X direction , the reporting sequence is as follows:

ABS_MT_SLOT 0
ABS_MT_POSITION_X x[0]
SYN_REPORT

When the contact in slot 0 leaves , the reporting sequence is as follows:

ABS_MT_TRACKING_ID -1
SYN_REPORT

This ABS_MT_SLOT is ignored because slot is changed to 0. This message removes the association between slot 0 and contact 45, thus destroying contact 45 and freeing up slot 0 for reuse by another contact.

When the second contact is removed , the reporting sequence is as follows:

ABS_MT_SLOT 1
ABS_MT_TRACKING_ID -1
SYN_REPORT

Summarize and compare the differences between the two touch protocols:

2.3 Multi-touch API functions

After understanding the two touch protocols, you need to use their corresponding API functions to report touch data during programming. The following are commonly used API functions.

2.3.1 input_mt_init_slots

This function is used to initialize the input slots of MT. Its function prototype is as follows:

/**
 * dev: MT设备对应的input_dev
 * num_slots: 设备要使用的slot的数量，也就是触摸点的数量
 * flags: 其他一些flags信息
 * return: 0-成功 负值-失败
 */
int input_mt_init_slots(struct input_dev *dev, 
              unsigned int num_slots, 
              unsigned int flags)

The third parameter includes the following flags:

#define INPUT_MT_POINTER     0x0001 /* pointer device, e.g. trackpad */ 
#define INPUT_MT_DIRECT      0x0002 /* direct device, e.g. touchscreen */ 
#define INPUT_MT_DROP_UNUSED 0x0004 /* drop contacts not seen in frame */ 
#define INPUT_MT_TRACK       0x0008 /* use in-kernel tracking */ 
#define INPUT_MT_SEMI_MT     0x0010 /* semi-mt device, finger count handled manually */

You can use the '|' operator to set multiple flags at the same time.

2.3.2 input_mt_slot

This function is used for Type B and is used to generate ABS_MT_SLOT events. Its function prototype is as follows:

/**
 * dev: MT设备对应的input_dev
 * slot: 当前发送的是哪个slot的坐标信息，也就是哪个触摸点
 * return: 无
 */
void input_mt_slot(struct input_dev *dev, int slot)

2.3.3 input_mt_report_slot_state

This function is used for Type B and is used to generate ABS_MT_TRACKING_ID and ABS_MT_TOOL_TYPE events. Its function prototype is as follows:

/**
 * dev: MT设备对应的input_dev
 * tool_type: 触摸类型
 * active: 触摸或抬起
 * return: 无
 */
void input_mt_report_slot_state(struct input_dev *dev, 
                                    unsigned int tool_type,  
                                            bool active)

The second parameter, tool_type, includes:

MT_TOOL_FINGER: Finger
MT_TOOL_PEN: Pen
MT_TOOL_PALM: Palm

The third parameter, active, includes:

true: continuous touch, the input subsystem core will automatically assign an ABS_MT_TRACKING_ID to the slot
false: The touch point is lifted, indicating that the touch point is invalid, and the input subsystem kernel will assign a -1 to the slot

2.3.4 input_report_abs

This function is used to report the touch point coordinates. Both TypeA and TypeB use this function to report the touch point coordinate information. The function prototype is as follows:

/**
 * dev: MT设备对应的input_dev
 * code: 要上报的是什么数据
 * value: 要上报的数据值
 * return: 无
 */
void input_report_abs(struct input_dev *dev,  
                          unsigned int code,  
                                   int value)

The second parameter, code, includes:

ABS_MT_POSITION_X
ABS_MT_POSITION_Y

2.3.5 input_mt_report_pointer_emulation

If the number of tracked touch points is greater than the number currently reported, the driver uses the BTN_TOOL_TAP event to notify the user space of the total number of touch points currently tracked, and then calls the input_mt_report_pointer_emulation function to set the use_count parameter to false, otherwise set the use_count parameter to true.

/**
 * dev: MT设备对应的input_dev
 * use_count: true-有效的触摸点数量 false-追踪到的触摸点数量多于当前上报的数量
 * return: 无
 */
void input_mt_report_pointer_emulation(struct input_dev *dev, bool use_count)

3 Explanation of the meaning of data reported by the input subsystem

3.1 Introduction to the input subsystem

In Linux, for input devices such as buttons, mice, keyboards, touch screens, etc., in order to facilitate unified management, the Linux kernel has specially created an input subsystem framework to handle input events.

Input means input, which is the subsystem that manages input. Like pinctrl and gpio subsystems, they are frameworks created by the Linux kernel for a certain type of device. The input subsystem framework diagram is as follows:

3.2 Input and Output Events

3.2.1 Event Type

evbit indicates the input event type. The optional event types are defined in the include/uapi/linux/input.h file. The event types are as follows:

The meaning of each is:

#define EV_SYN           0x00    /* 同步事件    */ 
#define EV_KEY           0x01    /* 按键事件    */ 
#define EV_REL           0x02    /* 相对坐标事件   */ 
#define EV_ABS           0x03    /* 绝对坐标事件   */ 
#define EV_MSC           0x04    /* 杂项(其他)事件  */ 
#define EV_SW            0x05    /* 开关事件    */ 
#define EV_LED           0x11    /* LED     */ 
#define EV_SND           0x12    /* sound(声音)   */ 
#define EV_REP           0x14    /* 重复事件    */ 
#define EV_FF            0x15    /* 压力事件    */ 
#define EV_PWR           0x16    /* 电源事件    */ 
#define EV_FF_STATUS     0x17    /* 压力状态事件   */

For example, if you want to use the input function of a key , you need to register the EV_KEY event. If you also want to use the continuous press function, you need to register the EV_REP event.

If you want to use the input function of the touch screen , you need to register the EV_KEY event.

3.2.2 Key value type

evbit, keybit, relbit, etc. are all used to store values corresponding to different events. The Linux kernel defines many key values:

#define KEY_RESERVED       0 
#define KEY_ESC            1 
#define KEY_1              2 
#define KEY_2              3 
#define KEY_3              4 
#define KEY_4              5 
//...... 
#define BTN_TOOL_QUINTTAP   0x148	/* Five fingers on trackpad */
#define BTN_TOUCH		    0x14a
#define BTN_STYLUS		    0x14b
//...... 

#define ABS_X			0x00
#define ABS_Y			0x01
#define ABS_Z			0x02
#define ABS_RX			0x03
#define ABS_RY			0x04
#define ABS_RZ			0x05

#define ABS_MT_SLOT		    0x2f	/* MT slot being modified */
#define ABS_MT_TOUCH_MAJOR	0x30	/* Major axis of touching ellipse */
#define ABS_MT_TOUCH_MINOR	0x31	/* Minor axis (omit if circular) */
#define ABS_MT_WIDTH_MAJOR	0x32	/* Major axis of approaching ellipse */
#define ABS_MT_WIDTH_MINOR	0x33	/* Minor axis (omit if circular) */
#define ABS_MT_ORIENTATION	0x34	/* Ellipse orientation */
#define ABS_MT_POSITION_X	0x35	/* Center X touch position */
#define ABS_MT_POSITION_Y	0x36	/* Center Y touch position */
#define ABS_MT_TOOL_TYPE	0x37	/* Type of touching device */
#define ABS_MT_BLOB_ID		0x38	/* Group a set of packets as a blob */
#define ABS_MT_TRACKING_ID	0x39	/* Unique ID of initiated contact */
#define ABS_MT_PRESSURE		0x3a	/* Pressure on contact area */
#define ABS_MT_DISTANCE		0x3b	/* Contact hover distance */
#define ABS_MT_TOOL_X		0x3c	/* Center X tool position */
#define ABS_MT_TOOL_Y		0x3d	/* Center Y tool position */

The specific definition is in the input.h file:

3.3 Analysis of touch data reporting examples

The previous article only printed the touch coordinates on the screen. In fact, when using a touch screen, the coordinate data needs to be reported to the application layer through the input subsystem. Now let's analyze the meaning of these data reported by the input subsystem in detail. For example, after pressing the touch key, the serial port will print as follows:

Extract the data content:

/*****************input_event 类型********************/ 
/*编号*/   /*tv_sec*/ /*tv_usec*/  /*type*/   /*code*/    /*value*/ 
0000000    00f6 0000   e539 0003     0003       0039      0000 0000
0000010    00f6 0000   e539 0003     0003       0035      009d 0000
0000020    00f6 0000   e539 0003     0003       0036      00c1 0000
0000030    00f6 0000   e539 0003     0001       014a      0001 0000
0000040    00f6 0000   e539 0003     0003       0000      009d 0000
0000050    00f6 0000   e539 0003     0003       0001      00c1 0000
0000060    00f6 0000   e539 0003     0000       0000      0000 0000
0000070    00f6 0000   11ad 0005     0003       0039      ffff ffff
0000080    00f6 0000   11ad 0005     0001       014a      0000 0000
0000090    00f6 0000   11ad 0005     0000       0000      0000 0000

type is the event type
- 0000: EV_SYN, synchronization event
- 0001: EV_KEY, key event
- 0003: EV_ABS, absolute coordinate event
code is the event code, that is, the key number
- 0000: ABS_X, single-point touch reports X coordinate value
- 0001: ABS_Y, single-point touch reports Y coordinate value
- 0035: ABS_MT_POSITION_X, multi-touch reporting of X coordinate value
- 0036: ABS_MT_POSITION_Y, multi-touch reporting Y coordinate value
- 0039: ABS_MT_TRACKING_ID, the track ID of the touch point
- 014a: BTN_TOUCH, touch button
value is the key value, 1 means pressed, 0 means released

Let's analyze the meaning of each line of output:

Line 1: Absolute coordinate event, track id of the touch point, id=0

Line 2: Absolute coordinate event, multi-touch X coordinate value, X=0x9d (157)

Line 3: Absolute coordinate event, multi-touch Y coordinate value, Y=0xc1 (193)

Line 4: key event, touch key, 1 means key is pressed

Line 5: Absolute coordinate event, single-point touch X coordinate value, X=0x9d (157)

Line 6: Absolute coordinate event, single-point touch Y coordinate value, Y=0xc1 (193)

Line 7: Synchronous event, reported by input_sync function

Line 8: Absolute coordinate event, track id of the touch point, id=0xffffffff=-1, which means the touch point has left the screen

Line 9: key event, touch key, 0 means no key

Line 10: Synchronous event, reported by input_sync function

Note: In the above printout, there are reports of multi-touch and single-touch. In fact, if multi-touch is used, the report of single-touch can be removed, as follows:

After removing it, test again and you can see that only multi-touch data is reported:

4 Compile the touch driver into the kernel

The touch driver you wrote yourself needs to be manually loaded before it can be used each time the system starts up, which is quite troublesome. Now you don't need to modify the driver file anymore, and you can compile your own driver directly into the kernel. The method is as follows:

Copy the touch screen driver file you wrote to the drivers/input/touchscreen/ directory of the Linux kernel:

cp gt911.c ../../kernel/nxp_kernel/linux-imx-rel_imx_4.1.15_2.1.0_ga/drivers/input/touchscreen/ -f

Modify the Makefile in the drivers/input/touchscreen directory and add the following line at the bottom:

obj-y += gt911.o

Then (using the compilation script written earlier) recompile the Linux kernel

Then copy the zImage to the board and restart the board.

Normally, the event number information of the touch driver is printed out when the kernel starts. I did print it here, but I kept getting IIC errors afterwards:

I can't figure out the reason for now. I just printed it out and it only comes here when the touch starts to read data. It feels like the interrupt is triggered as soon as the touch driver is loaded. But when reading the touch data through IIC in the interrupt, there is a problem again. . .

A temporary alternative is to load the touch driver in the startup file by adding the following statement in the /etc/init.d/rcS file:

cd /lib/modules/4.1.15
depmod
modprobe gt911.ko
cd /

5 Conclusion

This article first introduces the touch test library - tslib, which can be used to perform graphical touch tests . Then, it analyzes the reporting principle of the touch protocol and the specific meaning of the data reported by the input subsystem .