Review and Prospect of Digital Video Coding and Decoding Technology

With the rapid development of technologies such as digitization, networking, and cloud computing, and the popularization of diversified personal consumer electronic multimedia products, movies and television no longer rely solely on traditional screens. Motion visual information can be obtained anytime and anywhere, and motion visual perception information is presented in a variety of media forms. People also rely more on motion visual perception to obtain information. (The term "motion visual perception" is used because the meanings of movies, television, and video cannot be fully covered.) The progress of motion visual perception technology is a reflection of the progress of information technology, and its premise is the research and development of digital video encoding and decoding technology for motion visual images. The quality of a motion visual perception effect directly corresponds to the basic theoretical research results of digital video codecs.

　　At present, digital movies, digital television, interactive personal intelligent terminals, etc. are inseparable from digital video coding and decoding technology. Digital video coding and decoding technology is not only widely used in many professional technical fields such as film and television, communications, and networks, but is also easily implemented on personal consumer electronic product terminals, and these devices may simultaneously implement the coding and decoding applications of multiple digital video signals.

　　The characteristics of the digital video codec technology field are that there are many types, multiple types coexist, new and old coexist, and research and development results change with each passing day. At present, there is no digital video codec technology that can replace all other digital video codec technologies. A systematic understanding of digital video codec technology is a good way to understand the current status and future of motion visual perception technology. We understand the common digital video codec technologies by the chronological order of digital video codec technology development and the classification of research and development institutions.

　　1. The H.261 standard was developed by the Video Coding Experts Group (VCEG) of the predecessor of ITU-T (ITU Telecommunication Standardization Sector) CCITT (International consultative committee on telecommunications and Telegraph) in 1988 and formulated by ITU-T (VCEG) in 1990. H.261 is mainly used in old video conferencing and video phone products. H.261 is the first international standard for digital video compression. The H.261 standard uses a hybrid coding framework, including inter-frame prediction based on motion compensation. It uses the common YCbCr color space, 4:2:0 chroma sampling format, 8-bit sampling accuracy, 16x16 macroblocks, block motion compensation, discrete cosine transform in 8x8 blocks, quantization, zig-zag scanning of quantization coefficients, run-level sign mapping, and Huffman coding. H.261 only supports progressive scan video input. Although H.261 is rarely used now, it is the originator of the video codec field, and all subsequent standard video codecs are designed based on it.

　　2. MPEG-1 standard is the first lossy compression standard for video and audio developed by the MPEG group (moving pictures experts group) under ISO/IEC (International Organization for Standards / International Electro-Technical Commission). MPEG-1 (Part 2) video compression algorithm, ISO / IEC (MPEG) completed the definition of MPEG-1 video coding standard in 1990. At the end of 1992, MPEG-1 (Part 2) was officially established as an international standard. Its original main goal was to record images on audio CD (Compact Disc) discs. The MPEG-1 (Part 2) video compression standard is the technical core of VCD (Video Compact Disc) discs. Some online videos also use the MPEG-1 (Part 2) format. The quality of the MPEG-1 (Part 2) codec is roughly equivalent to the original VHS videotape. VCD application stipulates that the resolution of MPEG-1 (Part 2) is about 352×240, and the digital video signal encoding uses a fixed bit rate (1.15Mbps). Although MPEG-1 (Part2) can achieve larger picture size and higher visual perception quality of motion as long as the quality of the input video source is good enough and the encoding bit rate is high enough. However, considering that all commercial VCD players need to have a unified technical standard and hardware processing capacity, it is stipulated that video bit rates higher than 1.15Mb/s or video resolutions higher than 352x288 are not used by single VCD players (including some DVD players). As a result, when playing fast-moving videos, the macro blocks cannot be fully adjusted during compression due to insufficient data volume, and blurred blocks appear on the video screen. The MPEG-1 (Part2) video compression algorithm is not effective in visual perception of motion when used in VCD, which may be the reason why VCD has not been successful in developed countries. MPEG-1 Layer 3 is the currently widely used mp3 audio compression technology. If universality is considered, the video/audio codec of MPEG-1 can be said to be the most universal codec. Almost all computers in the world can play files in MPEG-1 format. Almost all DVD players also support the playback of VCD. Technically speaking, compared with the H.261 standard, MPEG-1 adds half-pixel motion compensation and bidirectional motion prediction frames. Like H.261, MPEG-1 only supports progressive scan video input.

　　3. H.262 video compression standard is a video compression standard developed by ITU-T (VCEG) in 1994 after upgrading H.261. It is the same as the video compression standard MPEG-2 (ISO/IEC13818-2) developed by ISO/IEC (MPEG) and is used in DVD, SVCD and most digital video broadcasting systems and cable distribution systems. When used on standard DVD, it supports higher image quality and wide screen; when used on SVCD, its quality is not as good as DVD but much higher than VCD. MPEG-2 (ISO/IEC13818-2) is also used in the new generation of DVD standards, HD-DVD and Blu-ray. Technically speaking, the biggest improvement of MPEG-2 (ISO/IEC13818-2) over MPEG-1 is the added support for interlaced video. Although MPEG-2 (ISO/IEC13818-2) is a relatively old video encoding, it has great popularity and market acceptance. ISO/IEC (MPEG) originally planned to develop four versions, MPEG-1, MPEG-2, MPEG-3 and MPEG-4, to meet different bandwidth and digital image quality requirements. The MPEG-3 coding and compression standard, which was planned to be developed after MPEG-2, was originally developed for HDTV. However, because MPEG-2 can be applied to HDTV, MPEG-3, which was originally designed for HDTV, was abandoned before it was born.

　　4. H.263 video compression standard, established in 1995, is mainly used in video conferencing, video telephony and network video. In terms of compressing progressive scan video sources, H.263 has greatly improved the performance of previous video coding standards. Especially at the low bit rate end, it can greatly save bit rate while ensuring a certain quality, and has better support for network transmission. Compared with the previous international video coding standards (H.261, MPEG-1 and H.262/MPEG-2), H.263's performance has been revolutionary. In 1998, the second version H.263+, or H.263v2, which added new functions, significantly improved the coding efficiency and provided some other capabilities compared with the initial version. The third version H.263++, or H.263v3, was completed in 2000. It adds more new functions on the basis of H.263+.

　　5. MPEG-4 (ISO/IEC 14496-2) officially became an international standard in early 1999. Sometimes it is also called "ASP". They can be used in network transmission, broadcasting and media storage. Compared with MPEG-2 and the first version of H.263, its compression performance is improved. MPEG4 MPEG-4 (ISO/IEC 14496-2) It is a solution suitable for low transmission rate applications. The main difference from previous video coding standards is that MPEG-4 (ISO/IEC 14496-2) pays more attention to the interactivity and flexibility of multimedia systems. MPEG-4 (ISO/IEC 14496-2) is the first dynamic image standard with interactivity. Another feature of it is its comprehensiveness. From the root, MPEG-4 (ISO/IEC 14496-2) can blend natural objects with artificial objects in the visual perception of motion. The design goals of MPEG-4 (ISO/IEC 14496-2) are also wider adaptability and more flexible extensibility. It introduces H.263 technology and 1/4 pixel motion compensation technology. Like MPEG-2, it supports both progressive scanning and interlaced scanning.

　　6. The H.264 video compression standard is the same standard as MPEG-4 (ISO/IEC 14496-10), which is sometimes also called MPEG-4 AVC, or "AVC" or "JVT". H.264/MPEG-4 AVC was developed in 2003 and is a high-performance video coding standard jointly developed by ISO/IEC (MPEG) and ITU-T (VCEG), and has been widely used. The standard introduces a series of new technologies that can greatly improve compression performance and can greatly surpass previous standards at both high and low bit rates. Products that have used H.264 technology include Sony's PSP, Nero's Nero Digital product suite, Apple's Mac OS X v10.4, and the new generation of DVD standards HD-DVD and Blu-ray discs. MPEG-4|/H264 and AVC are currently the mainstream video compression standards in different fields such as communications, computers, broadcasting and television. The new patent licensing policy of MPEG-4|/H264 and AVC is considered too harsh, which makes industrial promotion difficult. This has prompted related companies and departments to compete to develop their own independent video compression standards. Since MPEG-4|/H264 and AVC are open platforms, companies and institutions can develop different formats based on MPEG-4|/H264 and AVC standards, which has led to the emergence of many video encoding standards. In the past, many video formats based on MPEG-4 (ISO/IEC 14496-2) technology have appeared on the market, such as WMV 9, Quick Time, DivX, Xvid, 3ivx, etc. Some files ending with *.avi, *.mp4, *.ogm or *.mkv also use MPEG-4 (ISO/IEC 14496-2) video codecs.

　　7. HEVC, also informally known as H.265, H.NGVC, and MPEG-H (Part 2), is a video compression standard draft. The HEVC standard is developed based on the H.264 standard and is currently being jointly developed by ISO/IEC (MPEG) and ITU-T (VCEG). ISO/IEC (MPEG) and ITU-T (VCEG) have formed a joint collaborative team JCT-VC (Joint Collaborative Team on Video Coding in Chinese: Joint Collaborative Team on Video Coding) to jointly develop the HEVC standard. In June 2012, MPEG LA announced the start of HEVC patent licenses. In July 2012, HEVC submitted a draft international standard, and in January 2013, the final draft of HEVC is expected to be approved as a new generation of international standards. To facilitate the compression of high-resolution video. HEVC expands the size of macroblocks from 16x16 in H.264 to 64x64, and uses the flexible block structure RQT (Residual Quad-tree Transform) and sample adaptive offset (Sample Adaptive Offset) to improve performance. Although it increases the difficulty of the algorithm, it reduces distortion, improves compression rate, and reduces bit rate. HEVC doubles the data compression ratio of H.264 and has higher video quality. HEVC can support up to 7680×4320 resolution. HEVC is the highest video compression standard today and for some time to come, and is the foundation for the development of ultra-high-definition television.

　　The two major international organizations that formulate video coding standards, ITU-T (VCEG) and ISO/IEC (MPEG), have formulated international standards for related video compression standard technologies at different times. The standards of ITU-T (VCEG) include H.261, H.263, and H.264, which are mainly used in the field of real-time video communication; the MPEG series of standards are formulated by ISO/IEC (MPEG), and are mainly used in video storage (DVD), broadcasting and television, streaming media on the Internet or wireless networks, etc. (Note: MPEG-1, MPEG-, MPEG-4 and other series of standards are a large outline formulated by the MPEG organization. Under each outline name, there are technical standards in multiple related fields. Therefore, when we discuss video compression standards, we must be specific to the content in the brackets at the end) ITU-T (VCEG) and ISO/IEC (MPEG) organizations have also jointly formulated some standards. The H.262 standard is equivalent to the video coding standard of MPEG-2 (ISO/IEC 14496-2), and the H.264 standard is included in MPEG-4 (ISO/IEC 14496-10).

　　8. AVS (Audio Video coding Standard in Chinese) is an audio and video compression coding standard developed by the Digital Audio and Video Coding and Decoding Technology Standard Working Group of the Ministry of Information Industry of China. It includes more than just video coding standards. AVS (GB/T 20090.2) is a complete standard system including system, video, audio, and media copyright management. It mainly adopts a patent authorization method different from MPEG-4 AVC/H.264, develops a national standard for digital video coding and decoding technology with independent intellectual property rights in China, proposes an on-demand vertical algorithm combination, a simple and efficient hybrid coding technology system, and is accepted as one of the three major video coding standards in the world. Technically, the technology used in the video coding part of AVS is very similar to H.264 and AVC. Among them, the patent group of low-complexity coding algorithms composed of more than ten independent innovative technologies makes the coding efficiency under equivalent conditions only 30%-70% of the international standard of the same period, and the flexible code stream structure representation method, error-resistant and content protection technology suitable for a variety of applications, and also proposes a number of patent technologies suitable for network TV applications. While the standard was being formulated, the design and implementation of the AVS high-definition decoding chip was completed simultaneously, which promoted the industrial application of the AVS standard. The AVS+ (Advanced Audio and Video Codec for Broadcasting and Television Part 1: Video) industry standard released in July 2012 will meet the needs of the development of new broadcast, film and television services such as high-definition television and 3D television.

　　9. WMV (Windows Media Video) is a family of video codecs from Microsoft, including WMV 7, WMV 8, WMV 9, and WPV 10. This family of codecs can be applied to narrowband video from dial-up Internet access to broadband video for high-definition television (HDTV). Using Windows Media Video, users can also burn video files to CDs, DVDs, or other devices. It is also suitable for use as a media server. WMV can be seen as an enhanced version of MPEG-4. The WMV-9 version is the VC-1 standard officially adopted by SMPTE (Society of Motion Picture and Television Engineers) in 2006. Starting with the seventh version (WMV1), Microsoft began to use its own non-standard MPEG-4 Part 2. However, since the ninth version of WMV is already an independent standard of SMPTE (421M, also known as VC-1), the development of WMV is no longer like MPEG-4. It is already its own proprietary codec technology. (It was called VC-9 when it was first launched, and was later renamed VC-1 by SMPTE). Technically, VC-1 and H.264 have many similarities. VC-1 compression technology integrates the advantages of MPEG and H.264, using Biliner and Bicubic methods, and the sub-pixel can be as small as 1/4 pixel. VC-1 only has 4 types of motion composition, and the compression ratio cannot beat H.264. The complexity of the compression algorithm is only about 50% of that of H.264, but it has outstanding performance for special effects movies. It is worth mentioning that the SMPTE organization is an avant-garde organization that explores the motion visual perception technology of digital movies and digital television from the film field. Most of the international standards for motion visual perception technology formulated by it are based on high-definition film. The latest ultra-high-definition television technology development in the contemporary film and television field can be seen everywhere. The UHDTV-2036 standard first proposed by SMPTE in 2007 has evolved into UHDTV 2036-1 and UHDTV 2036-2, which correspond to the digital televisions we currently call 4K and 8K. UHDTV was officially accepted as the international standard for ultra-high-definition television on August 23, 2012, and the ITU-T defined format is ITU-R BT.2020.

　　10. Real Video is a video codec developed by Real Networks. It was developed in 1997 and has reached Real Player 15. From the beginning of its development, it was positioned as a format for video playback on the Internet. It supports multiple playback platforms, including Windows, Mac, Linux, Solaris and some mobile phones. Compared with other video codecs, Real Video can usually compress video data to a smaller size. Therefore, it can achieve uninterrupted video playback under the condition of dial-up Internet access with a 56Kbps MODEM. The general file extensions are .AM, RM, RAM. The RMVB format, which is Real Video with a dynamic encoding rate, is now widely popular. In addition to being able to play in the form of ordinary video files, Real Video can also cooperate with Real Server servers to download and play video images during data transmission, instead of having to download first before playing like most video files. It is currently commonly seen in online playback on the Internet.

　　11. Sorenson Video is a codec developed by Sorenson Media Inc. in 1998 in conjunction with Apple's Quick Time. Many Quick Time videos on the Internet use this codec compression technology. It is also targeted at Adobe Flash applications. Earlier versions of Adobe Flash Player all use Sorenson Video codec technology support. Subsequent series such as Sorenson Video 2, Sorenson Video 3, and Sorenson Spark were launched, but the Sorenson Spark version was separated from Apple's Quick Time application. Sorenson Spark became the built-in motion video codec for Macromedia's Flash MX. Sorenson Spark's efficient inter-frame compression is what makes it different from other compression technologies. Compared with other codecs, it requires a very low data rate to produce high-quality video. It is used to play the widely used Flash format video. Sorenson Spark is considered an incomplete H.263 application. Currently, H.264/MPEG-4 AVC has been incorporated into the new generation of Quick Time. The video player Adobe Flash Player has also switched to using the True Motion VP6 video codec launched by On2 Technologies. After the launch of the True Motion VP6 video codec, the VP7 and VP8 series were further launched. Google acquired On2 Technologies in 2010, and VP8 may become Google's proprietary video compression solution.

　　12. Indeo Video is a high-performance, pure software video compression codec developed by Intel for PC video applications. It has the characteristics of high compression ratio and low loss of audio and video quality, and can be used in Internet/Intranet and multimedia applications. Indeo Video is widely used in the fields of building multimedia websites, dynamic effect speeches, game cutscenes, non-linear material storage, etc. At the same time, Indeo Video 5.1 is a software compression codec that comes with the two PC operating systems Windows ME/2000. Therefore, Indeo Video 5.1 became the most popular AVI codec at the time.

　　13. SDTV (Standard Definition TV), EDTV (Enhanced Definition TV), and HDTV (High Definition TV) are international digital television signal coding standards established by ATSC (Advanced Television Systems Committee). They include a variety of picture formats used by digital television and are the most common coding standards in the current broadcast digital television industry. SDTV 480i standard definition television is equivalent to the highest picture quality of analog television systems, EDTV 480p enhanced definition television is equivalent to the picture quality of DVD movies, and HDTV includes two picture resolutions, 1920×1080 and 1280×720. The standards established by ATSC are based on the MPEG-2 encoding format and support a variety of display resolutions, aspect ratios, and frame rates. The following table lists common parameter combinations.

　　(Note: UHDTV 4K and UHDTV 8K are standards under development. UHDTV 4K comes from the 4K concept in the film industry. It is used by Hollywood film companies in the process of converting film into digital images. The name 4K comes from its horizontal resolution of about 4000 pixels. The current 4K standard adopted by the film industry is 4096×2160 pixel resolution. The industry also commonly uses Full Aperture 4K (4096 x 3112), Academy 4K (3656 × 2664) and other standards. In addition, for 4K digital movies, there are also 2K digital movie formats with a resolution of 2048×1080, 1.3K digital movie formats with a resolution of 1280×1024, and 0.8K digital movie formats with a resolution of 1024×768 pixels.)

　　summary

　　The quality of digital video signals processed by various codecs is ultimately measured by human visual perception. Resolution is an important parameter that affects human perception, and image resolution is directly related to image clarity. By introducing common video codec names and the correlation between different video codec names and motion visual perception, we can intuitively experience the current and future video codec technologies. Of course, if we have a deeper understanding of video codec technology, we will know that the brightness, contrast, color, color gamut, color saturation, aspect ratio of dynamic images, motion picture frame rate, and dynamic delay response will all affect our motion visual perception.