Expert opinion丨Intelligent vision helps AI large models reach a new turning point Arm’s innovative ecosystem serves the local market
Throughout the entire history of innovation and development in the digital economy, every disruptive innovation is often accompanied by the emergence of a new paradigm. There is an ancient saying that "it is better to travel thousands of miles than to read thousands of books", which accurately explains the importance of personal experience and practice: human beings receive external information through visual, auditory and other main perception methods, think about it and then take action. Understand, change and create the world. Back to the present, with the continuous improvement of computing power, the development of AI has also ushered in an era of acceleration due to breakthroughs in large models. Today, new paradigms represented by technologies such as robots and autonomous driving will redefine the model of human-computer interaction. We will also witness the gradual transfer of human perception, thinking and action capabilities to machines.
In a new era where models and actions are everywhere, intelligent vision will become essential
70% of the information humans obtain comes from vision. Currently, intelligent vision has penetrated into smart homes, smart cities, smart retail and smart manufacturing. In a new era where models and actions are ubiquitous, intelligent vision will also become necessary. Indispensable. Whether it is chat robots or autonomous driving technology, it is necessary to use perception technologies such as intelligent vision to collect real data from real life and form more accurate models to provide key safety and obstacle avoidance capabilities, or to better adapt to diverse needs. and behavior.
Five major technology trends in intelligent vision systems
With the popularity of intelligent vision applications, its systems are also developing towards five major technology trends:
Cloud-edge-device collaboration: High-definition image and video data, huge amounts of data, and real-time requirements for applications such as autonomous driving determine that intelligent vision systems need to be able to distribute computing and decision-making tasks on the cloud, edge, and terminal devices, and Realize load mobilization and coordination. The cloud can undertake computing-intensive tasks such as training deep learning models and processing large-scale image and video data sets. Edge devices can perform real-time image processing, simple feature extraction and preliminary decision-making; terminal devices can receive intelligent decision-making results from the cloud or the edge, and perform display, user interaction, etc., or perform terminal processing under ultra-low latency and offline requirements. Lateral visual data processing and decision-making.
AI blessing: The application of deep learning and neural networks in the field of computer vision has achieved great success. Future intelligent vision systems will continue to rely on deep learning models and neural networks to improve the analysis and recognition capabilities of image and video data through training on large-scale data sets, and use reinforcement learning and autonomous and adaptive learning to enable the system to Continuously learn and improve within the environment and adapt to new scenarios and tasks.
Image processing: Intelligent vision systems require accurate and high-quality image processing to support more efficient feature extraction, target detection and tracking, image analysis and understanding, and image generation and synthesis.
Computing power support: Due to complex models and algorithms, large-scale image data processing, real-time, high precision, high concurrency and other requirements, intelligent vision systems require powerful computing resources and computing power support.
Privacy and security: The large amount of data collection and wide application of intelligent vision systems have also raised concerns about privacy and security. The future trend is to design intelligent vision systems that are more privacy-protecting, safe and reliable, using encryption, anonymization and other technical means to protect user privacy, while strengthening system security to prevent malicious attacks and abuse.
Smart vision applications spur the evolution of smart vision chip architecture
At the chip level, in order to meet the growing demand for AI support, computing power, energy efficiency and real-time performance, the smart vision chip architecture has also undergone a series of evolutions - from DSPs and ASICs with fixed functions and difficult to program, to universal Powerful and easy-to-program CPU architecture, up to today’s advanced heterogeneous SoC integrating CPU, ISP, NPU AI accelerator, VPU, and GPU. Intelligent vision chips have gradually achieved the characteristics of low power consumption, high performance and high integration. It is suitable for resource-constrained edge devices, such as smartphones, cameras, and IoT devices, laying the foundation for intelligent vision to become a universal capability.
Supporting the efficient and agile development of visual products and solutions is a real need in the industry. Arm's rich product portfolio, including Arm Cortex-A CPU, Mali GPU, Mali-C55 ISP, Ethos ML accelerator, Corstone subsystem, etc., can bring different combinations of high quality, high reliability, high performance and high energy efficiency to meet the needs of intelligent The needs of various visual market segments help accelerate the innovation of intelligent vision.
Arm’s new smart vision reference design integrates third-party IP for the first time
Quickly create product differentiation for enterprises
为了助力智能物联和视觉领域的创新企业降本增效，使更多的新商机和新想法可以更快地转化成量产的产品。Arm 处理器和系统 IP 也在向预先集成验证的子系统的方式转变。这种新的设计方式和产品形态，通过选用 Arm 广泛应用于智能视觉领域的处理器和系统 IP，并根据客户的特定应用引入第三方 IP，构建出相对标准化的 IP 组合，集成后 IP 组合必须经过预先验证。此外，该 IP 组合还将搭配对应的 Arm 虚拟硬件（Arm Virtual Hardware）与参考软件栈，形成坚实的基础技术平台。如此一来，不仅给予客户足够的灵活性和选择，使企业可以专注于产品的差异化，加快产品上市时间，并且帮助客户显著降低芯片开发成本和风险，大幅缩短研发周期。
The first example of such a product form is Arm's newly released smart vision reference design, which combines Arm IP with ARM Technology's IP for the first time, and is pre-integrated and pre-verified by ARM Technology. The solution's hardware reference design includes CPU, ISP, NPU and VPU processor options, as well as interconnect and system IP to glue the subsystems together. This visual reference design has passed Arm SystemReady certification and PSA security certification, ensuring support for mainstream operating systems and platform security. Its Arm virtual hardware can provide virtual models for ecological partners to realize software and hardware co-design, further shortening the system design cycle.
Arm releases “new smart vision reference design” to meet the strong growth demand for vision application equipment in the Chinese market
The new smart vision reference design has indeed received positive feedback from customers. As a new chip solution startup company focusing on intelligent sensing (including machine vision), Yugan Microelectronics needs to focus resources on product differentiation and quickly promote product launches in order to demonstrate competitiveness. According to feedback from Yugan Microelectronics CEO Jiang Hong: Arm smart vision reference design can well lay the company's underlying technology foundation, significantly reducing the effort they spend on the underlying technology architecture, and investing more time and resources in The development of its technological advantages creates innovation and differentiation to seize market opportunities.
Create an innovative ecology and go to the visual future together
As smart vision reaches new nodes, it will be more commonly used in all walks of life, creating more market opportunities. Software and hardware providers in the industry, as well as system integrators, all need a platform for resource gathering. Quickly realize ideas and seize business opportunities. The "Arm Intelligent Vision Partner Program" came into being, bringing together ecological partners such as AI, vision, chip design, software, algorithms, and system integration to collaborate together. Through this plan, chip design service providers will be able to access and be familiar with Arm's reference designs, and quickly tape out based on the reference designs, helping customers launch chips that meet SystemReady and PSA security standards; on the other hand, for software providers Specifically, the Arm smart vision reference design includes a cloud-native open source vision software stack that can adapt to different application scenarios for its customers, customize software for different device manufacturers, and integrate, test and tune it to achieve mass production. level; in addition, system integrators usually have the comprehensive capabilities of chip design, firmware software development, creation of FPGAs and even development boards to provide one-stop services. They can optimize products and improve developer experience in this plan.
继成为 Arm 首家在华图像信号处理器的认证调式夥伴后，诚迈科技也成为首批加入 Arm 智能视觉伙伴项目的成员。 诚迈科技计算机视觉事业部总经理韩薇明确表达：通过 Arm 先进的处理器与 Arm 智能视觉参考设计，结合诚迈科技在软件方面的专业积累与机器视觉的全套解决方案，加入这项计划将能助力诚迈科技加速智能视觉在各领域的落地应用，为客户带来更高效率、性能与创新优势。
The Arm Intelligent Vision Partner Program has received huge support from the industry since its launch. Currently, more than ten ecological partners have joined
With the development of AI and large models, intelligent vision technology will be everywhere. With vision becoming a ubiquitous capability, pre-integrated, pre-verified standardized subsystems will provide a solid foundation for accelerating vision product design and innovation. Realizing the vision of continuous innovation in the visual field requires the collaborative cooperation of the entire ecosystem. I believe that Arm and its ecological partners will jointly create the future of intelligent vision!
Author of this article: Ma Jian, Vice President of Business Development of Arm IoT Division
Featured Posts
京公网安备 11010802033920号