"Linglong" ISP: Three major advantages solve the industry's pain points and win customers with "performance + service"

With the increasing variety of application scenarios such as AIoT, intelligent monitoring, face recognition and autonomous driving, the market has also put forward new requirements for the performance of image processors (ISPs).

At the end of 2020, ARM China launched the first product in the "Linglong" multimedia product line - the LINLON i3/i5 ISP processor, which has excellent performance in flexible configurability, wide dynamic range, low-light noise reduction, etc., effectively solving the pain points that have always existed in the traditional ISP field and is generally welcomed by customers.

The "Linglong" multimedia product line is the fourth self-developed IP product line of ARM China after the "Zhouyi" AIPU, "Xingchen" processor and "Shanhai" IoT information security solution. While building the imaging ecosystem, it will also continue to improve the current situation of "lack of core and soul" in my country's integrated circuit industry and further enable local independent innovation.

Three highlights to solve industry pain points

ISP is responsible for receiving the original signal data of the photosensitive element (Sensor) in the entire imaging process. It can be understood as the first step in the processing flow of taking photos and videos, and plays a very important role in image quality.

Nowadays, smartphones have quad-camera and penta-camera configurations, and the number of cameras used in smart cars has also greatly increased. Application scenarios such as security, education, and medical care also have diverse requirements for imaging performance. Taking the security monitoring field as an example, IHS research points out that after 2021, there will be more than 1 billion surveillance cameras in the world, of which more than 50% will be in China.

As both “quality” and “quantity” are improving, the traditional ISP sector faces many challenges.

First, the diversity of application scenarios makes it difficult for a general ISP to cover everything. The ISP field presents more customized features and requires flexible and configurable solutions. Second, ordinary linear image sensors can only provide a dynamic range effect of 72dB, while the dynamic range of the human eye is about 100dB. Therefore, imaging in some high dynamic range scenes often cannot present all the details well. Third, noise reduction is an eternal theme, especially the noise reduction performance under harsh conditions such as low light. The performance of most products on the market is still not ideal.

Therefore, from the beginning of the development of the "Linglong" ISP product, we have focused on the three industry "pain points" of configuration flexibility, wide dynamic effect, and low-light noise reduction. The name "Linglong" also has a profound meaning. It not only means small and exquisite, but also expresses the meaning of being visually exquisite, which is very consistent with the characteristics of Arm China's multimedia product line.

In terms of flexible configuration, the "Linglong" ISP product gives customers a certain amount of choice in various fields, and there are even some customized interfaces, which can help customers achieve two goals: one is to make the product more differentiated; the other is to target different scenarios, different sensors, different modules, different light and surrounding environments, so that customers can have greater expandability and customization space in specific fields to make distinctive products.

In terms of wide dynamic range, the "Linglong" ISP provides pixel-level dynamic range enhancement through a combination of wide dynamic and digital wide dynamic technologies. It can support DOL2/3, stagger2/3 processing, stitching and splicing between different frames, and support traditional HDR and digital HDR processing technologies.

In terms of low-light noise reduction, the "Linglong" ISP product adopts a variety of 2D and 3D noise reduction technologies, including brightness noise reduction, chrominance noise reduction and time domain noise reduction. In particular, it achieves excellent noise reduction effects through the multi-frame image fusion of 3D noise reduction, and can provide customers with multi-scale and multi-level noise reduction effects.

The currently released "Linglong" i3 ISP is mainly aimed at low-power lightweight application scenarios such as smart home and AIoT. It supports 2K video processing and single-channel video signal access processing. The clock comprehensive frequency is about 400MHz, and can achieve real-time processing effects of 5 million pixels and 60fps.

The Linglong i5 ISP is mainly used in mid-to-high-end market applications such as security and automotive applications. It supports 4K video and multi-channel video signal access processing, and can achieve 8 million pixels and 60fps real-time processing speed. The comprehensive clock frequency can reach 600MHz at the TSMC 16nm process level. Customers can realize various application configurations based on these parameter characteristics and performance characteristics.

Startup team of established company

Full understanding of the market is the basis for the "Linglong" ISP product to hit the pain points of the industry and form distinctive advantages. To be able to lead the industry in product performance and win the reputation of customers, it is inseparable from the excellent R&D team of Arm China's "Linglong" product line.

According to Jiwei.com, ARM China conducted extensive discussions with industry partners at the beginning of defining the "Linglong" ISP, which involved a lot of market research, customer tracking, and demand analysis.

For example, the products of different CIS manufacturers have biases, which are related to their own technology and manufacturing process technology. For ISP research and development, it is necessary to judge whether the products of front-end manufacturers can effectively solve customer needs and whether they can become mainstream technology.

"On the one hand, this requires the ability to make judgments based on years of industry experience, and on the other hand, it also requires confirming new product directions downstream, which requires know-how in different fields." Li Liang, product manager of ARM China's "Linglong" ISP, told reporters.

"ARM China's ISP team is positioned as a 'startup team of an established company', with the goal of building one of the best teams in the country. The resumes of team members cover many aspects such as ISP IP design. When recruiting, we require that the personnel have strong technical skills and fit the corporate culture of ARM China. They must meet the company's requirements for domestically developed IP and product quality, and also have the spirit to fight tough battles and endure hardships." said Xie Liang.

Although it is not fast to develop an ISP product in two years, the process involved setting up product lines, recruiting personnel, and even rewriting algorithms and setting up RTL architecture, which is equivalent to a process from 0 to 1.

Liu Yuxuan, director of ARM China ISP Engineering Department, introduced that, for example, in order to address the two pain points of wide dynamic range and low-light noise reduction, which are also the algorithm modules with the highest market demand, the ARM China ISP algorithm team concentrated its superior forces at the beginning of the design and carried out a lot of optimization work on these two algorithms.

In Liu Yuxuan's view, the biggest challenge comes from the final customer evaluation and product introduction stage. For image technology, some are measured by objective standards, such as clarity, signal-to-noise ratio, etc., and a considerable part is subjective evaluation standards. It should be said that different opinions vary from person to person. Different customers have different styles, which puts higher demands on the software and support team. "Our team has strong adaptability, so we have done a good job in responding to different customer needs; in the future, we will also be able to quickly iterate according to customer needs."

"Performance + Service" wins customers

According to Jiwei.com, the "Linglong" ISP series products have been delivered to customers and have received a lot of positive feedback from customers. It is expected that related products will be released in the first quarter of this year.

In addition to the above-mentioned advantages of the "Linglong" ISP product in terms of performance and flexibility, the support strength of Arm China's local team, the speed of feedback to new customer needs, and the professionalism of the support team are also the reasons why this product has been favored by customers since its launch.

It is understood that in order to make the "Linglong" ISP product achieve more professional results, Arm China established professional imaging and optical laboratories in its offices in Shenzhen and Shanghai. It did not slow down the R&D progress during the epidemic last year, and overcame difficulties to import equipment worth millions to help customers with debugging.

Liu Yuxuan said that in addition to helping customers solve image problems, the Arm China team can also provide quick and professional answers and responses to many issues such as product, system level, and integration. This is one of the important reasons why customers can confidently choose "Linglong" IPS products.

得益于“玲珑”ISP灵活的系统软硬件架构，安谋中国ISP团队能够快速实现客户定制化的改动，这是传统ISP设计无法达到的。比如在差异化需求非常显著的安防领域，客户会定制一些符合面积功耗要求的特殊功能，安谋中国ISP团队可以在短时间内提供专业化的方案。

Han Xiaomin, general manager of Jiwei Consulting, pointed out that products like the "Linglong" ISP have already demonstrated semi-customized characteristics. IP manufacturers work closely with customers, focus on the broad needs of customers, and continuously promote the evolution and iteration of solutions. They have the ability to respond quickly and provide comprehensive technical support, which is also a testament to Arm China's "deep empowerment" of innovation for Chinese semiconductor companies.