Xiaomi Mi 12 series imaging system decrypted: rewrite the entire camera architecture, heterogeneous/parallel operation

     Today, Xiaomi officially published an article to explain in detail the imaging system of the Xiaomi Mi 12 series. The Xiaomi Mi 12 series focuses on solving the problem of not taking photos fast enough in the past. At the same time, the Xiaomi Mi 12 Pro is the first to launch the Sony IMX707 sensor, which has better night scene shooting capabilities.

　　Xiaomi said that it took two years to rewrite the entire camera architecture and integrate and rebuild software and hardware capabilities to create a new Xiaomi imaging brain. The Xiaomi imaging brain equipped on the Xiaomi 12 series consists of four parts: intent recognition unit, computing unit, acceleration engine, and ecological engine.

• The purpose of the intention recognition unit is to determine the user's intention and then dynamically adjust the intensity of the algorithm, including intelligent dynamic allocation of memory, etc. When the user keeps pressing the shutter, the Xiaomi imaging brain will think that the user's intention is to record more pictures, and then it will prioritize the speed of taking pictures.

• The acceleration engine is powered by the Snapdragon 8 platform, integrating and fully scheduling the computing power of multiple modules such as CPU, GPU, DSP, ISP, etc., allowing the entire process to work in parallel and collaboratively, thereby achieving an optimization effect of significantly increasing the speed.

• The computing unit uses powerful computational photography capabilities to call on advanced algorithms to adjust the image quality, such as multi-frame synthesis, noise reduction algorithms, false color removal algorithms, white balance calibration, etc., to make the picture clearer and more natural, thus achieving an effect close to or even exceeding that of traditional optical photography.

• The ecological engine opens up the pipeline from hardware to third-party applications, and opens the capabilities of Xiaomi Imaging Brain to third parties through Xiaomi's own SDK. In this way, third-party applications will be able to call on the capabilities of Xiaomi Imaging Brain in the future, thereby achieving the effects of image quality optimization and speed improvement.

　　Officials said that the Xiaomi Mi 12 series camera startup speed is 21.2% faster, the photo taking speed is 53.7% faster, the continuous photo taking speed is 209% faster, and the night scene photo taking speed is also 39.9% faster. Among them, the heterogeneous computing and algorithm parallel processing of Xiaomi's imaging brain played a key role.

　　In order to optimize the shooting process, Xiaomi has adopted a new "thread parallelization technology" to move the processing algorithm upward and process multiple photos in parallel. In addition, the original hardware HAL layer is abstracted and managed as Xiaomi's own HAL layer, so that the hardware algorithm can be combined and changed, and even managed as a plug-in. Coupled with algorithm preloading optimization and algorithm concurrency optimization, the ability to process multiple continuous shots is greatly improved. Heterogeneous computing can reduce unnecessary algorithm library loading, thereby achieving the effect of reducing memory and speeding up.

　　IT Home learned that the Sony IMX707 CMOS sensor first used in Xiaomi 12 Pro is 1/1.28 inches in size, has 50 million pixels, a single pixel size of 1.22μm, and supports four-in-one pixels. The photosensitivity of this sensor is 49% higher than that of IMX766. Combined with Xiaomi's self-developed Night Owl 2.0 algorithm, it can achieve pure night scene shooting effects.

　　Not only that, the enhanced photosensitivity also enables the IMX707 sensor to achieve faster shooting speeds and better capture capabilities.