In-depth analysis: pros and cons of three human-computer interaction methods for smart TVs and factors for change

The Internet function of TV is constantly strengthened and gradually enters the category of smart TV, which has led to the continuous development of new human-machine interface and remote control. NPD DisplaySearch believes that the human-machine interface of TV is expected to undergo major changes from this year to the next few years. The reason is quite simple, that is, the increasing number of services with Internet functions will make the selection list more complicated. The traditional menu of up/down/left/right/OK keys is no longer sufficient, but making it too complicated makes the remote control operation very cumbersome and slow. At the same time, how to let consumers experience the convenience through actual operation during the sales process to stimulate the motivation to buy smart TVs is a new topic. The changes in the human-machine interface of TV are testing the intelligence of the entire TV supply chain.

Many current smart TVs are designed to simply bring website browsers from the Internet to TVs, but this also creates new problems. Most websites are designed with mouse or pointer memory, and also include additional functions such as click and drag, such as drag and click, zoom in and out on Google Street View and Maps, etc. However, it is currently quite difficult to control these web page functions on TVs.

Sony launched Google TV at the end of 2010, which can be regarded as a bold attempt at the human-machine interface of smart TVs and an example of using a complex remote control for menu control. The numerous buttons of the Google TV remote control (as shown in the attached picture) require two hands to operate, which faces the change of consumer usage habits. On the other hand, it will face many difficulties to control the screen by similar touch input on the other side of the room.

Figure 1. Sony's first Google TV has a complex controller.

Currently, with the efforts of many TV manufacturers and TV brands, three common human-machine interface solutions have emerged: speech recognition, gesture recognition and motion sensing. Each method has its advantages and disadvantages, which are briefly listed below.

Table 1. New TV human-machine interface methods

At the 2012 Consumer Electronics Show (CES), prototypes of voice recognition and gesture recognition were first exhibited; but in fact, gesture recognition functions have been on the market for some time, but they have not been built into TVs, such as Microsoft Kinect gesture recognition for Xbox game consoles.

It's not clear yet which of these three approaches will emerge as a clear mainstream winner. Each has its pros and cons, so some TVs are taking a mixed approach rather than betting on any one technology. That seems like a sensible solution.

Table 2. Implementation status of new TV human-machine interfaces Source: Quarterly TV Design and Features Report

Basically, NPD DisplaySearch believes that speech recognition has several levels of complexity:

First, the content of the design must be limited to a fixed set of commands, such as "turn up the volume" or "next channel", and these fixed command sets must have a lot of flexibility to distinguish different tones.

Speech recognition can identify "single words" and "phrases", but generally speaking, the system also needs to develop the function of recognizing the voice characteristics of a specific person.

Voice input can generate text directly, but in order to avoid confusion caused by homophones or words with similar pronunciations (such as "there" and "their"), more complex computer calculation functions are sometimes required. At the same time, smart TVs must also have the function of self-learning and training experience.

NPD DisplaySearch said in its Quarterly TV Design and Features Report that adding voice recognition will have a significant impact on the diversity of TV products. North American TVs require three languages ​​(English, French, and Spanish), but in Western Europe, at least 16 languages ​​are required. In addition, there are many complex dialects and local accents in Asia. In addition, many countries have legal provisions to support minority languages, and eventually a TV may have up to 80 languages ​​built in.

In terms of gesture recognition, game consoles have proven that gesture recognition can be marketed. Its development needs must focus on ease of use and how to be more accurate on the screen. In NPD DisplaySearch's research, we found that many web pages are not suitable for gesture control, especially some web pages that require fine mouse movement. In addition, gesture recognition is not considered a good ergonomic design, and fatigue may affect the ability and posture of the arm to extend. In addition, consumers usually do not want to change their sitting posture to control the TV. Assuming a consumer holds a drink in his hand, it is a great challenge to wave his hand without spilling the drink.

Motion capture or remote point is a fast-growing technology. Since a computer mouse, touch pad, or Wii-style controller is widely accepted and familiar to consumers, the barrier to using it on a TV is minimal. However, these motion capture devices are much more expensive than ordinary remote controls, and the high cost becomes a major barrier.

As mentioned earlier, there is no single approach that can meet all TV HMI needs, so it is a relatively sensible approach to combine these approaches. In addition, installing a digital camera on the TV can add facial recognition, which is also the latest technology to build user profiles and facial recognition into the TV; and facial recognition can be used together with gesture control and video calls, as well as voice recognition, etc.

However, NPD DisplaySearch believes that consumers are actually the biggest variable in the human-machine interface of smart TVs. It is not yet clear how users will respond, wave their hands, or talk to control their TVs. And the meaning of gestures can be very different in different cultures and countries. In Italy, it is often necessary to gesture when speaking, but it is considered impolite in Japan. In Europe, consumers are very sensitive about privacy, so they may not want a digital camera lens on the TV they watch every day, especially a TV that can connect to the Internet.

In addition, another human-machine interface is to interact with or control the TV through other devices, such as smartphones or tablets. Currently, most brands of smartphones and tablets can be installed with APP functions that can remotely control the TV. These functions include mouse movement and handheld touch screen operations. In addition, more complex APP applications are gradually emerging. These applications include more information control, TV program schedule management, viewing program recommendations, TV function control, etc., and the breadth and depth of watching TV programs can be improved under the condition of concentrating on a handheld device.

With smartphone shipments expected to reach 1 billion units per year in the future, the era of one smartphone per person has arrived, so all smart computers with Internet access will have more than one smartphone in the same space or room. Smartphones are already personalized, and software and application functions can be built according to the usage habits of each user. Therefore, under the logical development prediction, a personalized TV should be easier to control with a smartphone, and consumers will also have a high degree of acceptance. After all, consumers already use their phones every day, so using their phones to control the TV they watch every day is the most customary. NPD DisplaySearch believes that what will truly revolutionize the human-machine interface of TVs will not only be voice recognition, gesture recognition, and motion sensing, etc., but the combination of smartphones and smart TVs will be one of the main trends.