A look at the evolution of domestic video chips

Video images are closely related to people's production and life, and are the main source of information for humans to obtain and exchange. According to statistics, more than 80% of human information comes from video images. In the process of human history, visual perception has played a major role in learning, production and various knowledge inheritance. Video technology is also constantly developing. From 576i to the current 4k, and the future 8k, the resolution has expanded dozens of times, constantly meeting people's demand for high-definition video. High-definition video is increasingly used in various fields, 3D technology is becoming more and more mature, and more and more applications require complex processing of massive video data. This trend also forces the video processing chip industry to move forward. Looking at the development of China's video processing industry in the past 10 years, it can be said to be a miniature version of the development history of semiconductor ICs.

The complete video processing flow includes four parts: video acquisition, video processing, video transmission and video display. Each link is closely linked. If one link lags behind, it will cause problems in the entire video ecosystem. However, among these four links, the most important one is video processing. The quality of the video processing IC directly determines the image effect. Video processing includes video encoding and decoding, video image deinterlacing, video format conversion, etc.

Video codec chips used to be dominated by foreign manufacturers

Video codecs, H.264 and H.265 are currently the most popular video codecs for transmission on the Internet. H.264/MPEG-4 AVC (H.264) is the latest and most promising video compression standard since the release of the MPEG-2 video compression standard in 1995. H.264 is the latest international video coding standard developed by a joint development group of ITU-T and ISO/IEC. With this standard, the compression efficiency at the same image quality is more than doubled compared to previous standards. Therefore, H.264 has been widely regarded as the most influential industry standard in recent years.

Figure 1 H.264 encoding and decoding process

The H.264 standard uses several key technologies, namely intra-frame prediction coding, inter-frame prediction coding, integer transform, quantization and entropy coding. Intra-frame coding is used to reduce the spatial redundancy of the image. Inter-frame prediction coding uses the temporal redundancy in consecutive frames to perform motion estimation and compensation. The prediction uses an integer transform similar to DCT based on 4×4 pixel blocks. Quantization, H.264 has 32 different quantization step sizes, which is very similar to the 31 quantization step sizes in H.263, but in H.264, the step size is progressive at a compound rate of 12.5% ​​instead of a fixed constant. The last step of the H.264 video encoding process is entropy coding. Two different entropy coding methods are used in H.264: Universal Variable Length Coding (UVLC) and Text-based Adaptive Binary Arithmetic Coding (CABAC). The encoding and decoding process of H.264 is shown in Figure 1.

Since H.264 has high compression and is suitable for transmission in and out of the network, with the continuous development of the Internet, various semiconductor manufacturers have also seen this gold mine. Since 2005, a large number of national semiconductor companies or electronic companies such as TI, Ambarella, NXP in the United States, TEWELL in Taiwan, SONY, SHARP in Japan, etc. have focused on H.264 encoding chips, tailored video codec chips suitable for different regions, and vigorously promoted the development of the entire market.

It can be said that before 2010, domestically developed chips were basically in the prototype or self-use stage, and the market acceptance was very low. According to rough data statistics, see Figure 2 below, domestic chips account for only about 1%, foreign chips account for more than 95%, and TI semiconductor giant occupies almost half of the Chinese market. The reason is that foreign manufacturers have rich experience accumulation, have been deeply involved in this field for many years, are familiar with the characteristics of the industry, and the stability of domestic chips needs to be improved. At that time, manufacturers in the application video industry basically only considered foreign chips for chip selection, and it was rare to choose domestic chips.

Figure 2: Domestic shipment share of codec chips in 2006

The emergence of domestic manufacturers, Huawei HiSilicon's performance is eye-catching

For the booming video codec market, upstream chip manufacturers such as TI and Ambarella have never stopped competing. The technology and accumulation of foreign manufacturers have always been in a leading position. However, in recent years, domestic manufacturers such as Huawei HiSilicon have also continued to work hard in the upstream of the industry, and have achieved a certain leading position in certain technical fields. They have built market fortresses through patents and technological breakthroughs and formed competitive advantages.

Since 2006, Vimicro has started the R&D and design of IP video surveillance systems, and has continued to invest in network camera chips, decoding terminals and video surveillance platforms, achieving brilliant results. In 2014, Vimicro released new products, the 718 and 736 chips: the 718 is the second-generation SVAC/H.264 core chip, and the 736 is a mass-market DVR core chip. The chip supports image processing functions such as automatic exposure, automatic white balance, and automatic gain; video encoding supports SVAC and H.264 standards, making security monitoring systems more reliable and flexible. As of 2017, Vimicro has been listed on the Nasdaq in the United States, and has expanded from chip design business to video surveillance equipment manufacturers, gradually playing a major role in the international market.

The real deterrent to foreign competitors is Huawei HiSilicon. Relying on the booming security market in China over the past decade, HiSilicon has teamed up with Hikvision and Dahua to target foreign semiconductor giants.

Under the premise of laying out H.264 product chips, Huawei HiSilicon has secured a firm foothold in the video codec market. At the same time, Huawei also vigorously promotes H.265 and is the main technical standard contributor for H.265. As security monitoring continues to develop towards high resolution, high compression rate, and low bandwidth occupancy, compared with H.264 encoding, H.265 video encoding compression technology reduces the demand for bandwidth by about 50% while ensuring the same image quality, and the storage space is also reduced by half accordingly, solving the difficulties of video in bandwidth, storage, and image quality under the trend of high definition and networking. Video applications based on H.265 technology are gradually becoming popular. The figure below is a comparison table of H.265 and H.264 bit rates. H.265 can not only reduce the bit rate of current mainstream high-definition formats such as 1080p (1080p only requires 1~2Mbps bit rate), but is also suitable for ultra-high-definition videos such as 4K and 8K. The H.265 standard allows network video to keep up with the "high resolution" of display screens.

Figure 3 H.265 and H.264 bit rate comparison table

Huawei HiSilicon's video codec chip system is very comprehensive. From the 65nm 3518A to the latest 28nm 3519, the chip products cover the consumer market, commercial market and industry market; the resolution ranges from D1 to the latest 4k, and the frame rate is up to 60fps. These comprehensive layouts and high reliability have won the favor of video application manufacturers, leaving TI's DM8168 far behind. At present, Huawei HiSilicon, Ambarella of the United States, and Socionext of Japan have formed a three-way situation in this field. TI has not launched any new codec chips since DM8168.

According to rough statistics, see Figure 4 below, the proportion of domestic chips (mainly Huawei HiSilicon) in the Chinese market has increased to about 60%, 60 times that of ten years ago, while the proportion of foreign chips has dropped to about 35%, with HiSilicon occupying more than half of the domestic market. Although Socionext is a newly established company, its background is Sony and Fujitsu. It entered the Chinese market in 2016 and has established its technological leadership in 8k.

Figure 4: Domestic shipment share of codec chips in 2006

As domestic chip manufacturers participate in the video codec chip design army, the industry chip prices have been driven down. The author draws a price comparison and trend chart of various chip manufacturers based on historical procurement data and the price of single-channel codec, with 1 as a unit price (specific price data is not directly provided by each manufacturer), as shown in Figure 5. From the figure, we can see that chip prices have been falling. At the same time, domestic manufacturer HiSilicon has a great price advantage due to its cost control, which forces national giants to lower their shipping prices. Benefiting from the price competition of chip manufacturers, chip codec application manufacturers such as Hikvision and Dahua have developed rapidly. As the two oligopolies in the security industry, Dahua and Hikvision have grown at an extremely strong rate with the help of the industry, and have already fought with application manufacturers in the international market. In 2015, both companies were promoted to the "Top 50 Global Installations", with Hikvision ranking second and Dahua ranking fifth. Judging from the annual revenue data, Hikvision's operating income increased from less than 800 million yuan in 2006 to 32 billion yuan in 2016. Dahua Technology's turnover increased nearly 100 times in 10 years, and its main operating income reached 13.3 billion yuan in 2016.

Figure 5 Codec chip price trend from 2006 to 2016

Image deinterlacing on a single chip, the glory days are gone

In the early stages of television development, the "interlaced scanning" technology was generally used to reduce the amount of data transmitted for the image. This technology divides a frame of image into two fields, the odd field for odd lines and the even field for even lines. The data of the odd field is sent first, followed by the data of the even lines, and the amount of video data sent will be reduced by half. In interlaced scanning, a frame of image is divided into two fields, the interval between fields is half of the interval between fields. That is, the scanning raster scans one line from left to right in the odd field and one line from top to bottom until the last line, and the same scanning method is used in the even field. In order to achieve better image effects, a deinterlacing chip is generally used on the display end to process the video image. There are many methods for deinterlacing, including inter-field interpolation, intra-field interpolation, motion compensation and other methods.

In the past, the main semiconductor manufacturers of single-chip deinterlacing chips were TI in the United States, i-Chips in Japan, and Macro Image in South Korea. There were basically no manufacturers involved in the domestic single-chip deinterlacing. From a global perspective, there are a lot of monitors in consumer terminals. In the VIDOE era, the use of deinterlacing chips is very huge. With the popularization of 1080p resolution, the application of TVBS signals has gradually decreased, replaced by 3G SDI. At the same time, DP and DVI signal protocols no longer include i-signals. In the field of professional video applications, the demand for single deinterlacing chips is decreasing.

At present, in the field of professional video processing, video processing manufacturers such as Barco in the United States, Vision Video in China, Chunzhong, etc., still have certain single-chip deinterlacing needs in order to pursue the best video display quality and maintain old products, but other small manufacturers basically give up the processing of interlaced video. Single-chip deinterlacing has gradually been transformed into a module of chip video processing. According to the latest survey data from the statistical agency WitsView, the total global shipments of LCD TVs in 2016 were 219 million units. Based on 80% calculation, the annual demand for chips with deinterlacing functions is about 176 million pieces. However, in my opinion, with the popularization of high-definition video and the upgrade of transmission bandwidth to households, the deinterlacing function module will gradually be phased out of history.

Video format conversion chips, the stage for international semiconductor manufacturers

Video format conversion technology is one of the research hotspots in today's information field. Many kinds of chips have been launched successively, mainly by American and Taiwanese companies, such as Genesis Microelectronics, Gennum Corporation, Trident, Trumpion (Taiwen), Etron Technology, Macronix International, Philips Semiconductor, Sunplus Technology, i-chips, etc. China is still in its infancy in this regard.

The application of format conversion technology requires it to have high real-time performance and low cost. Therefore, format conversion research has paid great attention to the ASICization of algorithms from the beginning. In the early days when VLSI integration was relatively small, single-frame linear interpolation was mainly studied. Later, with the improvement of silicon chip integration, adaptive algorithms of multi-frame algorithms between frames were gradually adopted. In the late 1990s, when motion estimation technology could be integrated on a single chip, format conversion of motion estimation flourished. Today, the integration of silicon chips is still increasing. With the possibility of System On Chip, format conversion will develop more complex and efficient technologies; the possible direction is to use more frame information for more accurate and complex motion compensation.

With the launch of 4k and 8K LCD TVs, video format conversion, especially scaling technology, is particularly important. At present, many 4k TVs are still not up to date in video transmission, so most of them use video magnification. This is especially true for the immature 8k. Here we have to mention the Japanese video scaling company i-chips. Putting aside patriotic feelings, technical practitioners in the video industry have awe for i-chips' products. In terms of scaling, i-chips can be said to have been making this product for decades, and has made it to the extreme with the spirit of craftsmanship, such as i-chips732. There is also Japan's Sharp, which has recently developed the X8-MEP image processing engine. Whether it is 4K image content, TV signal or BD image, the engine will correct the details according to the content of the image material, and reproduce the high-definition image with 8K clarity beyond the current.

Video format conversion chips are an indispensable key component in display systems. my country is a major producer of televisions and various types of displays. However, most of the key chips in these products are currently controlled by chip manufacturers in the United States or Japan. There are only a few domestic video chips, such as Huawei HiSilicon and Zhuhai Allwinner, which have not yet become popular.

Future trends in video processing

Lower power consumption. The ideal situation is that the power consumption of the HD terminal itself is reduced to the level where it can work stably through natural heat dissipation. As chip technology continues to mature, low power consumption is possible. A green earth is the general trend.

Higher compression capability: As the network market's requirements for high-definition video products continue to increase, the general 720P resolution is already a bit stretched, and the demand for 5 million, 8 million, and even 12 million pixels is constantly emerging, and the demand for different formats of bit streams has also become a standard configuration. These have posed greater challenges to the compression capabilities of high-definition video chips.

Intelligent video technologies such as intelligent analysis integration, face recognition, and big data analysis are booming and are increasingly being popularized in practical applications. With the advent of the high-definition 4K era and the exponential increase in information volume, the integration of intelligent video algorithms into video chips has become unstoppable.

Video can be configured from technology. FPGA giant Xilinx reVISION supports the fastest response visual system. Compared with the most competitive computing embedded GPU and typical SoC, it increases the unit power consumption image capture speed of machine learning inference by 6 times, increases the unit power consumption frame speed of computer vision processing by 42 times, and reduces the latency to 1/5. It can be foreseen that the integration of reVision technology in future video chips is a must.

Looking back at the past 10 years of domestic video chips, HiSilicon Semiconductor has played a leading role in the IP codec market with the help of H.264/265 codecs, in collaboration with domestic video application manufacturers such as Hikvision, Dahua, and Vision. However, in other video processing chip markets, domestic semiconductor manufacturers still need to seize the trend of video processing and catch up. The author predicts that future video chips will have the above four trends. If domestic semiconductor manufacturers can firmly grasp them, they will surely become "pigs standing on the vent of the wind."