GPU codec technology demonstrates its power and leads the 4K×2K era

The 4K×2K era will be dominated by multi-core GPUs. UHD mobile devices have become an important battlefield for major consumer electronics brands. With the launch of a new wave of UHD image displays, the demand for highly efficient compressed images has emerged. The industry's decoding technology can provide ideal image quality without sacrificing frame rate, while offloading multimedia processing to the GPU, reducing CPU workload and overall mobile device power consumption.

　　Ultra HD (UHD) 4K×2K resolution was a common theme across all CES 2013. Companies such as LG, Samsung, Sony, Toshiba and Hisense were the first to launch UHD smart TVs, with different screen sizes or display technologies. But UHD specifications are not only used in TVs, for example, Panasonic demonstrated a tablet device based on UHD lateral electric field effect (IPS) screen technology at the 2013 Consumer Electronics Show (CES), and several manufacturers produce UHD monitors for the professional market.

　　This article will review the latest Internet Protocol (IP) multimedia processors capable of processing UHD images, the application market, and how they can be effectively used to enrich playback content on mobile and embedded devices.

　　Market overview of UHD content creation

　　First, if we look at what is driving ultra-high definition and how the broadcast industry is creating content using UHD, we can immediately see that most of the major film studios, such as Paramount, Warner Bros., 20th Century Fox and Sony Pictures, have already begun using UHD technology.

　　UHD TVs have been released in 2013.

　　Movies such as The Amazing Spider-Man, The Girl with the Dragon Tattoo and The Hobbit were all shot in UHD, so UHD content is starting to be released in cinemas. YouTube users have been able to upload UHD videos since 2010, and Sony has stated that "opinions on 3D seem to be split, but opinions on UHD are generally positive".

　　Canon is also pushing the UHD specification hard, launching a number of new professional and DSLR cameras to market. Canon is using these events to promote a series of short films from world-renowned and independent directors to showcase the potential of this new technology.

　　However, since UHD movies must be introduced to the market with a new wave of imaging devices, there are many factors that will affect the speed of UHD adoption. Movie theaters are the ideal place to use UHD content initially, but the infrastructure needs more updating because older projectors are not compatible with UHD. Using multiple 2K projectors in parallel does not produce UHD effects, just like using a multi-projector large-screen projection system to show ultra-high resolution and ultra-large screen movies (IMAX) is not feasible.

　　Of course, it will take some time for the new technology to be widely adopted. The broadcasting industry and the content delivery chain must assemble the necessary equipment and infrastructure to support UHD playback; original equipment manufacturers (OEMs) must manufacture UHD TVs, set-top boxes (STBs) and media players that can decode and display playback content. Consumers must also be prepared to accept another "digital technology conversion" (Digital Switchover) when upgrading their equipment.

　　When observing the market acceptance of 3D technology, it can be seen that cinemas are the main market driver. After watching a 3D movie, some moviegoers are keen to recreate the same experience at home, and soon 3D TVs begin to appear in the consumer market. UHD should be in a similar situation, especially if image compression technology continues to improve and the concept of large organic light-emitting diode (OLED) screens becomes a reality.

　　This process is similar to the experience of going from standard definition (SD) to high definition (HD). If you can take full advantage of the hardware power to encode and decode the video data as it is transmitted, these videos can be of amazing quality. If you use graphics processing coupled with image decoding technology, you can add a lot of other advantages, such as image stabilization, pixel level brightness/contrast correction, and other post-production processing options.

　　The industry's PowerVR imaging processor and PowerVR graphics processor IP cores are already capable of processing high-definition content at a stable and sustained high frame rate and creating breathtaking images.

HEVC--UHD/Wireless Display Promoter

　　Video coding technology needs to be improved to deliver UHD content in a cost-effective manner. As expected, the new video coding and decoding H.265 standard (also known as HEVC) has gained market traction. Many manufacturers have demonstrated HEVC UHD content at some exhibitions.

　　Mobile video transcoding is also becoming increasingly popular. For example, Intel's Atom Z2420--Lexington smartphone processor and Atom Z2760--Clover Trail system-on-chip (SoC) both include PowerVR graphics and image processors, showing Intel's commitment to providing complete multimedia solutions for the mobile and embedded markets.

　　UHD resolutions increase the need for efficient compression, whether it is broadcast or bandwidth-constrained channels on the Internet. The same principle applies to PowerVR decoding technology, which can provide UHD @60fps, operating at the high bitrates required by wireless displays. Even at the extreme bitrates of 230Mbit/s required for 4,096×1,744 (2.35:1), the technology still provides ideal image quality without sacrificing frame rate.

　　HEVC technology also integrates advanced error handling and concealment algorithms to optimize solutions, especially for screen display applications that are susceptible to transmission problems. In addition, this rapidly growing market also requires processor cores that can support multiple standards and multi-stream capabilities, because device manufacturers always want to integrate the latest and greatest multimedia options into their products.

　　Recently, silicon IP companies have launched image-related product lines, such as the D4500MP and E4500MP, which not only maintain quality throughout the entire processing flow, but also support 4:4:4 and 4:2:2 color resolutions, which are very important for high-quality wireless displays, because most of them need to be converted from three primary colors (RGB) to YCbCr. For example, simply mixing images and graphics processing, and adjusting the display function (On-screen Display, OSD) without 4:4:4 encoding will result in poor display effects. Therefore, increasing the color depth and resolution of the image processor core will not only provide customers with a way to differentiate their products, but also further increase computing efficiency, because most of the image processing in the core is based on 10 bits.

　　For example, the recently announced UHD TV standard, ITU-R BT.2020, has more color options and increased bit depth, so the picture displayed on the TV is obviously better. Although BT.2020 is still under development, the advantages it brings will influence the faster and more widespread adoption of 10-bit color in UHD TVs and HEVC.

　　Most of the above advantages are related to picture quality, but the biggest benefit of using the industry's processor core is overall system power consumption. The host central processing unit (CPU) offloads more intensive multimedia processing tasks to the image decoding/encoding core, and can refocus on other tasks or even enter standby mode, thereby reducing power consumption, maximizing battery life, and increasing the responsiveness of the computing platform.

　　Equipped with high-performance GPU, ultra-high-definition smart TV has diversified applications

　　Modern graphics processing cores support a wide range of graphics and computing application programming interfaces (APIs) that can process images and enhance image decoding operations performed by the graphics processing unit (GPU), thereby offloading the main CPU work, reducing power consumption and increasing overall system performance.

　　At the heart of the "smart" revolution in consumer electronics, high-performance, ultra-low power graphics IP cores have played an increasingly important role, and these GPUs are becoming the standard for SoC designs for smart TVs and set-top boxes. For example, LG's H13 processor is the first UHD-capable platform based on the PowerVR Series6 graphics core; this platform is targeted at high-end smart TVs, providing more than 100 billion floating point operations per second (GFLOPS) of computing performance while providing the right fill rate for ultra-high-definition resolution.

　　In the future, smart TV interfaces will gradually integrate APIs and high-performance hardware to bring photo-realistic effects to games, improve the user navigation process, and provide an excellent user interface (UI).

　　Here are just some of the enhanced capabilities that are available on smart TVs, using graphics and compute APIs on the same GPU:

　　．Advanced gaming features

　　As long as tablets, smartphones, and even game consoles have built-in high-performance, low-power graphics processors, all electronic games can be easily transferred and used between these devices.

　　． Treat videos and photos as stickers

　　Easily edit photos, vividly display channel changes, user-defined custom UI and picture-in-picture effects, etc.

　　． Gesture recognition and control input processing (voice control

　　．Facial Recognition

　　The main applications are to identify users in social media, video conferencing or electronic program guide (EPG) personalization and recommendation; parents can even filter content that is not suitable for children.

　　． Post-processing of user-generated content

　　For example, image and video stabilization.

　　． Convert video formats for the visually impaired

　　For example, converting images to high-contrast colors.

　　Companies can develop advanced versions of voice-activated technology. By using the computing power of GPUs to perform speech processing, smart TVs can recognize natural language syntax and provide consistent search options that can simultaneously filter results from video-on-demand (VoD) services, search engines, external devices, and other devices.

　　By providing OpenGL ES functionality beyond the basic 3D graphics API, high-performance GPUs will be able to support advanced visual operations such as displacement and lighting mapping, anisotropic lighting, deformation effects (eliminating UI pages), frame bending, opacity and translucency ("backsight" and "see-through" effects) and Fresnel reflections (the effect of light reflecting on the surface of liquid), providing visually stunning and feature-rich experiences for consumer products.

　　In general, 2013 is expected to be the year when UHD specifications will bear fruit, and everything from content to playback will be in UHD format. Therefore, system-on-chip solutions equipped with HEVC image decoding and encoding, plus advanced graphics functions, become very important.

　　In fact, PowerVR Series5XT and Series6 GPUs can already handle UHD formats, while PowerVR Video Series4 IP already supports the industry's widely used standard codes H.264, VP8, MPEG-4 and many other standards; future PowerVR image processors will also add support for HEVC.

　　Customers only need to select the appropriate IP to develop UHD solutions based on the optimization principles of low power consumption and high efficiency. Of course, the development of UHD products also requires software, including optimized drivers, intelligent algorithms and developer technical support, which are all important factors in accelerating the market.