SensorTile realizes wireless touch screen Bluetooth HID

kangear

SensorTile realizes wireless touch screen Bluetooth HID [Copy link]

 

  This is based on the HID post by @littleshrimp. I have been looking up relevant information recently and finally figured it out. I have implemented a wireless touch screen based on SensorTile. Currently, it automatically slides from the upper left corner to the lower right corner. 1. Currently, single-point touch is implemented, and multi-point touch will be implemented later; 2. There are many reference documents, @littleshrimp moderator's air mouse post, Microsoft's descriptor article, Android article https://source.android.com/devices/input/touch-devices Finally, it was implemented. It was not easy, really not easy. I will post a picture to show off first, and then organize a detailed article later. I am more familiar with Android underlying development, and you can consult me if you have any related questions. Personal technical blog: https://blog.csdn.net/kangear Supplement: HID => linux kernel input subsystem => Android input subsystem As far as single-point touch is concerned, look back at the requirements for touch screens in Android, as shown in the figure below: report keys ABS_X and ABS_Y as well as BTN_TOUCH, and also require a configuration file to specify that the device type is a touch screen. (You can view which key values uploaded by a device through adb shell getevent -i) Corresponding to the Linux kernel (based on the 3.4 kernel), the HID descriptor needs to include User Page as DIGITIZER, including TIPSWITCH Here The descriptor can be slightly simplified. The final result is as follows:

1. 0x05, 0x0d, // USAGE_PAGE (Digitizers) 0x09, 0x04, // USAGE (Touch Screen) 0xa1, 0x01, // COLLECTION (Application) 0x85, 0x01, // REPORT_ID (Touch) (REPORTID_TOUCH is 1) 0x09, 0x20, // USAGE (Stylus ) 0xa1, 0x00, // COLLECTION (Physical) 0x09, 0x42, // USAGE (Tip Switch) 0x15, 0x00, // LOGICAL_MINIMUM (0) 0x25, 0x01, // LOGICAL_MAXIMUM (1) 0x75, 0x01, // REPORT_SIZE (1) 0x95, 0x01, // REPORT_COUNT (1) 0x81, 0x02, // INPUT (Data,Var,Abs) 0x95, 0x03, // REPORT_COUNT (3) 0x81, 0x03, // INPUT (Cnst,Ary,Abs) /* 0x09, 0x32, // USAGE (In Range) 0x09, 0x47, // USAGE ( Confidence) 0x95, 0x02, // REPORT_COUNT (2) 0x81, 0x02, // INPUT (Data,Var,Abs) 0x95, 0x0a, // REPORT_COUNT (10) 0x81, 0x03, // INPUT (Cnst,Ary,Abs) */ 0x05, 0x01, // USAGE_PAGE (Generic Desktop) 0x26, 0xff, 0x7f, // LOGICAL_MAXIMUM (32767) 0x75, 0x10, // REPORT_SIZE (16) 0x95, 0x01, // REPORT_COUNT (1) 0xa4, // PUSH 0x55, 0x0d, // UNIT_EXPONENT (-3) 0x 65,0x00,//     UNIT (None)
2.     0x09, 0x30,                         //     USAGE (X)
3.     0x35, 0x00,                         //     PHYSICAL_MINIMUM (0)
4.     0x46, 0x00, 0x00,                   //     PHYSICAL_MAXIMUM (0)
5.     0x81, 0x02,                         //     INPUT (Data,Var,Abs)
6.     0x09, 0x31,                         //     USAGE (Y)
7.     0x46, 0x00, 0x00,                   //     PHYSICAL_MAXIMUM (0)
8.     0x81, 0x02,                         //     INPUT (Data,Var,Abs)
9.     0xb4,                               //     POP
10.     /*
11.     0x05, 0x0d,                         //     USAGE PAGE (Digitizers)
12.     0x09, 0x48,                         //     USAGE (Width)
13.     0x09, 0x49,                         //     USAGE (Height)
14.     0x95, 0x02,                         //     REPORT_COUNT (2)
15.     0x81, 0x02,                         //     INPUT (Data,Var,Abs)
16.     0x95, 0x01,                         //     REPORT_COUNT (1)
17.     0x81, 0x03,                         //     INPUT (Cnst,Ary,Abs)
18.     */
19.     0xc0,                               //   END_COLLECTION
20.     0xc0,                               // END_COLLECTION

复制代码

然后在while循环里，模拟发出一个触摸滑动的事件

1.       static int8_t ipRepVal[REPORT_IP_LEN_0] = {0};
3.       /*
4.       static int8_t cnt = 0;
6.       if(cnt == 0) {
7.         ipRepVal[1] = MOUSE_OFFSET_X;
8.         ipRepVal[2] = MOUSE_OFFSET_Y;
9.         cnt++;
10.       } else {
11.         ipRepVal[1] = -MOUSE_OFFSET_X;
12.         ipRepVal[2] = -MOUSE_OFFSET_Y;
13.         cnt--;
14.       }
15.       */
16.       ipRepVal[1] = 0x01; // down
17.       if (abs_hor < 32760) {
18.         abs_hor += 20;
19.         abs_ver += 20;
20.       } else {
21.         ipRepVal[1] = 0x00; // up
22.       }
23.       ipRepVal[0] = 0x01; // Report ID
24.       ipRepVal[2] = abs_hor%256;
25.       ipRepVal[3] = abs_hor/256;
26.       ipRepVal[4] = abs_ver%256;
27.       ipRepVal[5] = abs_ver/256;
29.       HidDevice_Update_Input_Report(0,0,REPORT_IP_LEN_0,ipRepVal);
30.       if (ipRepVal[1] == 0x00) {
31.         abs_hor = 0;
32.         abs_ver = 0;
33.       }

复制代码

Android系统中的放一个配置文件，路径:/system/usr/idc/HID.idc

1. # Copyright (C) 2010 The Android Open Source Project
2. #
3. # Licensed under the Apache License, Version 2.0 (the "License");
4. # you may not use this file except in compliance with the License.
5. # You may obtain a copy of the License at
6. #
7. #      [url=http://www.apache.org/licenses/LICENSE-2.0]http://www.apache.org/licenses/LICENSE-2.0[/url]
8. #
9. # Unless required by applicable law or agreed to in writing, software
10. # distributed under the License is distributed on an "AS IS" BASIS,
11. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12. # See the License for the specific language governing permissions and
13. # limitations under the License.
15. #
16. # Input Device Calibration File for the touch screen.
17. #
19. # Basic Parameters
20. touch.deviceType = touchScreen
21. #touch.orientationAware = 1
23. # Size
24. # Based on empirical measurements, we estimate the size of the contact
25. # using size = sqrt(area) * 43 + 0.
26. #touch.size.calibration = area #touch.size.scale = 6 #touch.size.bias = 0 #touch.size.isSummed = 0 # Pressure # Driver reports signal strength as pressure. # # A normal thumb touch typically registers about 80 signal strength # units although we don't expect these values to be accurate. #touch.pressure.calibration = amplitude #touch.pressure.scale = 0.0125 # # Orientation #touch.orientation.calibration = none

复制代码

In this way, after connecting to the device, you can automatically slide from the upper left corner to the lower right corner as shown in the figure below. Here is the git diff file for detailed view:

链接已隐藏，如需查看请登录或者注册

lcn1992

Awesome, the host is almighty.

kangear

After the wireless touch screen is realized, I think there is still a lot of room for imagination. Physical plug-ins can be realized, and mobile games can be converted into somatosensory games (for example, eating chicken can be converted into a physical toy gun, and when the trigger is pulled, the phone will be sent to press the shooting button). It's very interesting to think about it.

littleshrimp

kangear posted on 2018-7-20 10:09 After the wireless touch screen is realized, I think there is still a lot of room for imagination. Physical plug-ins can be realized, and mobile games can be converted into somatosensory games (for example, eating chicken can be...

The idea of somatosensory eating chicken is good

kangear

littleshrimp posted on 2018-7-20 10:16 The idea of eating chicken with somatosensory is good

Multi-touch is ok

kangear

littleshrimp posted on 2018-7-20 10:16 The idea of eating chicken with physical sense is good

littleshrimp

kangear posted on 2018-7-23 19:22

Have you tried multi-touch on a normal phone? Does it work? The one I made will show the mouse pointer, but multi-touch should not show multiple pointers

kangear

littleshrimp posted on 2018-7-24 08:13 Have you tried multi-touch on a regular phone and found any effect? The one I used will display the mouse pointer, but multi-touch should not display multiple pointers

There will be a little problem on a regular phone, because the multi-touch driver in the Android kernel sets a VID PID whitelist for touch devices similar to the USB whitelist. Only those in this list are multi-touch screens, and the system always gets 0x00 0x00 from the SensorTile. So there is no way to adapt it to regular phones for now. I simply modified the kernel of this Android tablet to use it. I am looking for how to set VendorId and ProductId in SensorTile. Here is an article: Do Bluetooth devices also have VendorId and ProductId?

kangear

littleshrimp posted on 2018-7-24 08:13 Have you tried multi-touch on a normal phone? Does it work? The one I made will show the mouse pointer, but multi-touch should not show multiple pointers

The posts or replies I make are always reviewed. How inefficient!

littleshrimp

kangear posted on 2018-7-24 10:51 There will be a little problem on ordinary mobile phones, because the multi-touch driver in the Android kernel sets a VID PID whitelist for touch devices similar to the USB...

If the PID and VID are set to the values in the whitelist, ordinary mobile phones will not display the mouse pointer, right?

littleshrimp

kangear posted on 2018-7-24 11:05 The posts or replies I post are always being reviewed. The efficiency is so low

There is a link in the reply. There is nothing I can do.

kangear

littleshrimp posted on 2018-7-24 12:55 If PID and VID are set to the values in the whitelist, ordinary mobile phones will not display the mouse pointer, right?

Yes, then the pointer will not be displayed. You can also make it not display the pointer by putting a configuration file in /system/usr/idc/HID.idc with the content: touch.deviceType = touchScreen, and chmod 777 /system/usr/idc/HID.idc, so that the pointer will not be displayed. Android has pointer type and touch screen for input devices, and the default is pointer type. Because there is no configuration file for the one you implemented, the system defaults to pointer type and displays the mouse pointer for you.

littleshrimp

kangear posted on 2018-7-24 15:34 Well, then the pointer will not be displayed. You can also make it not display the pointer by putting a configuration file in /system/usr/idc/HID.idc, ...

See the result after you configure VID and PID

kangear

littleshrimp posted on 2018-7-24 20:21 Let's see the result after you configure VID and PID

It can be used on iOS now, and iOS does not distinguish between vid and pid. This is a surprise.

littleshrimp

kangear posted on 2018-7-25 17:40 It can be used on iOS now. iOS does not distinguish between vid and pid. This is a surprise.

Why does my touch screen not respond on iOS?

kangear

littleshrimp posted on 2018-7-25 18:44 Why is there no touch response on my iOS?

How can iOS have single-point touch? It is multi-point, and can and only can have multiple points.

kangear

littleshrimp posted on 2018-7-25 18:44 Why is my touch screen not responding under iOS?

There is no need to track the VID PID problem anymore. Linux kernel 3.5 and above are the same as iOS. They identify whether it is a multi-touch device based on the descriptor (based on whether the descriptor contains HID_DG_CONTACTID. If it contains, it is, otherwise it is not. My development board and Nexus 5 phone only use version 3.4 of the kernel. New phones generally have kernels greater than 3.5). The line for finding VID PID is paused.

kangear

I usually add my own theoretical basis when I post, but if posting a link will cause the post to be reviewed, I will not post the link here for the time being. My next plan is to use the button on the SensorTile to trigger it to send a press of a certain coordinate, as the saying goes, press the shooting button for me in PUBG.

littleshrimp

kangear posted on 2018-7-26 15:43 There is no need to track the VID PID problem anymore. Linux kernel 3.5 and above are the same as iOS. They identify whether it is a multi-touch device based on the descriptor (depending on...

Can the HID_DG_CONTACTID problem be solved?

littleshrimp

kangear posted on 2018-7-26 15:46 I usually add my own theoretical basis when I post, but if posting a link will cause the post to be reviewed, I will not post the link here for the time being. ...

As long as the link does not violate the rules, it will be approved sooner or later