Seven Samsung experts jointly explain the latest mobile phone SoC Exynos 2200 (Part 1)

Samsung Newsroom shared the importance of the Graphics Processing Unit (GPU) and Image Signal Processor (ISP) in Exynos mobile processors (SoC). Samsung Newsroom first met with Mingoo Kim, SoC Development Head and Executive Vice President in charge of System-on-Chip (SoC) design for System LSI Business, before meeting with seven IP development heads. (This article is the first in a series of articles, and will continue to focus on other IP blocks.)

Kim began by explaining the fundamental reasons behind the concept of SoC, in which various functions are integrated into one chip. "When chips are segmented, it is difficult to fully manage their power consumption," Kim said. "When each function consumes power separately, battery efficiency also decreases. In addition, bandwidth limitations and transmission time delays affect data transmission between chips, resulting in performance degradation." Consuming the least amount of power is very important for smartphones because, unlike desktop computers, smartphones are not continuously powered.

“The SoC is highly efficient due to comprehensive power control. And as a single chip, it also takes up less space in a smartphone,” Kim explained. “When all functions are performed by a single chip, performance improves significantly.” Modern mobile phones have evolved beyond simply sending and receiving calls and text messages to perform a variety of advanced functions, including shooting videos, mobile gaming, and financial services. SoCs smaller than a thumbnail play a major role in making this possible.

Since SoC is the epitome of all major types of information technology that exist today, Kim describes it as the "flower of system semiconductors." "SoC is not an easy field to work in, but it is an optimistic field that any engineer aspires to be a part of," Kim said. "The role of SoC in future industries, such as the metaverse, autonomous driving, and 6G, will be endless."

Samsung will continue to focus on developing proprietary IP, including GPU, NPU, ISP, modem, RF, etc. Samsung strives to go one step further in chip design and become a platform solution company. "With our competitiveness in SoC, our goal is to make Exynos recognized as the best mobile processor brand," said Kim. "Through this special series, we hope to share with more people the role and importance of SoC, the characteristics and advantages of Exynos, and its development direction."

Expanding the possibilities of mobile gaming: GPUs provide more powerful graphics computing capabilities

Generally speaking, graphics processing requires large-scale calculations, which are faster and more efficient if processed in parallel. However, the structure of the central processing unit (CPU) is specially designed for fast serial processing. Therefore, due to this constant computing performance of the CPU, using the CPU for graphics processing will cause computing delays. For example, when playing games, your character may not be able to avoid enemy attacks due to the delay in touch input caused by rendering graphics on the display.

To solve these problems, GPU came into being. Before the GPU, the CPU was responsible for everything. GPU was conceived to improve efficiency as an accelerator for frequently used image calculations. In short, the CPU is like a general-purpose calculator, while the GPU is a large-scale parallel calculator specialized in graphics processing. This is where the GPU comes in, which is one of the key components of graphics processing. It receives commands from the CPU and displays the shape, position, color, and texture of the object on the monitor.

The Exynos 2200 is equipped with the Xclipse 920, the first mobile GPU jointly developed by Samsung and AMD, a US-based global company that specializes in semiconductor products such as GPUs for PCs and game consoles. The name "Xclipse" is a combination of the "X" in Exynos and the word "eclipse". The name represents Samsung's goal to surpass the limits of mobile gaming and bring performance to the level of console gaming, thus ushering in a new era - the gaming era.

To this end, Samsung worked with AMD to develop a power-efficient, console-class GPU for mobile devices. AMD’s GPUs are designed for PCs or game consoles, which means they had to be redesigned for the mobile environment. Specifically, in addition to managing heat dissipation, it was also redesigned to accommodate the relatively limited mobile memory bandwidth. “Based on our extensive low-power design knowledge gained from developing mobile SoCs, we were able to successfully achieve power efficiency and miniaturization in our first generation product,” said Sungboem Park, vice president and expert in mobile processor design, who is in charge of the GPU. “We focused on technologies to minimize heat since mobile devices don’t have fans like game consoles, while also maintaining performance so that frames don’t lag.”

The main role of the GPU in Exynos is to display objects in a 3D virtual space on a 2D smartphone screen, which is especially important when playing graphically demanding games on mobile devices. In particular, the Xclipse 920 is the first mobile GPU to support hardware-accelerated ray tracing (RT). Ray tracing is a technology that produces realistic lighting effects by simulating light reflected from 3D objects. By accelerating in hardware rather than software, the Xclipse 920 is able to perform real-time calculations faster. In addition, variable rate shading (VRS) technology adjusts the amount of GPU calculations based on changes in the color, shadows, motion and other variables of objects on the screen to reduce the load on the GPU.

▲Xclipse 920 comes with hardware support for ray tracing (RT) technology, which produces realistic lighting effects by simulating light reflected from 3D objects.

As the number of gaming users grows and graphics become more refined, the direction of GPU development becomes increasingly important. This includes increasing performance levels to that of high-performance consoles while reducing battery consumption. By improving these two key areas, users will be able to experience high-fidelity graphics on mobile devices similar to those on consoles. "In general, mobile devices tend to lag behind consoles by about five years in terms of graphics technology, but by working with AMD, we are able to quickly integrate the latest console technology in the Exynos 2200 mobile processor," said Park. "The SoC will be used in the Galaxy S22, and we plan to continue to implement other features in the RDNA series by working closely with AMD."

When asked about the future direction of mobile GPU development, Park responded, "As smartphone performance increases overall, it is indeed difficult to create a performance advantage large enough for consumers to notice. From now on, the reason why flagship smartphone users buy the latest smartphone is likely to be determined by gaming performance. This means that the possibility of further development of GPUs that determine smartphone gaming performance is high."

He also predicts that mobile GPUs will become more important in the AR and VR fields as well. “For AR, GPUs need to be equipped in lightweight devices such as glasses, so low-power design is very important,” Park said. “The performance requirements in the VR field are much higher because the entire visible virtual world needs to be rendered at once. Therefore, we need to meet various development requirements while also generating more realistic images faster than is currently possible. Therefore, the development of mobile GPUs is crucial and will have unlimited applications,” he emphasized.

Improve photo performance satisfaction: ISP provides more natural and vivid images

The ISP corrects the raw data obtained from the image sensor and creates photos or videos in the form that the user likes. The ISP also corrects the potential physical limitations of the camera module - consisting of the optical system and image sensor - interpolates red, green and blue (R/G/B) and removes noise. In addition, it also performs post-processing, such as adjusting the brightness of the video and emphasizing detailed areas. In short, through fine-tuning and post-processing, the ISP generates the pictures or videos that users want most.

Mobile phone camera ISP processing full process

In early smartphone models, the ISP was equipped as a separate chip. However, due to market demand, integrated ISPs soon became the norm. "Initially, we worked with overseas research institutions to develop a high-performance ISP that could be used in digital cameras," said Jongseong Choi, Project Leader (PL) of the Multimedia Development Team, who has been working in the field of video processing. "As a result, our first embedded ISP solution was used in the main camera of the Galaxy S4." Since 2012, Exynos mobile processors have contributed to a significant improvement in the video image quality of smartphone cameras by internalizing the ISP, achieving performance at the level of digital single-lens reflex (DSLR) cameras. "

Jongseong Choi, project leader of the Multimedia Development Group, has been working in the field of video processing for more than 20 years.

With a high-performance ISP, consumers can enjoy higher video image quality and faster processing speeds. "It's hard to explain how good a photo is with specific numbers because taking photos is subjective, but we are conducting various research, such as video evaluation using deep learning and ISP adjustment technology to create natural and clear photos and videos," Choi explained. In addition, the ISP also determines the speed in continuous shooting and is responsible for the fast processing of high-resolution photos and videos.

The high-performance ISP equipped with the latest Exynos can process up to 200 million pixels. It supports up to seven image sensors and can process videos and images from up to four image sensors at the same time. It also includes the ability to apply different parameters to each element, such as sky, bushes, skin, etc., by combining semantic segmentation technology with the help of the NPU to identify the scene being captured. The AI ​​function can detect and tag faces, and adjust the brightness, focus and color of the video based on the face coordinates.