Qualcomm continues to promote the transformation of generative AI on the device side and launches Qualcomm AI Hub to empower developers

Latest update time：2024-02-26

Reads：

Main points

—

• Qualcomm now enables large-scale commercial use of on-device AI in next-generation PCs, smartphones, software-defined cars, XR devices, and the Internet of Things, making intelligent computing ubiquitous.

• Qualcomm AI Hub provides more than 75 optimized AI models for Snapdragon and Qualcomm platforms, helping developers shorten product time to market and allowing the many advantages of terminal-side AI to be used in applications. The models will also be available on Hugging Face and GitHub. Developers can run these models themselves on cloud-hosted terminals powered by Qualcomm platforms with just a few lines of code.

• Qualcomm AI Research demonstrated running multi-modal large models and custom large visual models on Android smartphones and Windows PCs.

Today, Qualcomm Technologies released the company’s latest progress in the field of AI at the Mobile World Congress (MWC) in Barcelona. From the new Qualcomm® AI Hub to cutting-edge research breakthroughs and AI-powered commercial device demonstrations, Qualcomm Technologies is empowering developers and transforming the user experience across a wide range of device categories supported by Snapdragon® ^and^Qualcomm platforms.

Dejia Ma, senior vice president and general manager of technology planning and edge solutions business at Qualcomm Technologies, said :

With the launch of the third-generation Snapdragon 8 for smartphones and Snapdragon X Elite for PCs, we have started the large-scale commercial use of terminal-side AI. Now, with Qualcomm AI Hub, we will empower developers to fully realize the potential of these cutting-edge technologies and create attractive AI-enabled applications. Qualcomm AI Hub provides developers with a comprehensive AI model library, allowing them to easily and quickly integrate pre-optimized AI models into applications to create a faster, more reliable and more private user experience.

Qualcomm AI Hub unlocks excellent device-side AI performance for developers

The new Qualcomm AI Hub contains a pre-optimized AI model library that supports seamless deployment on terminals equipped with Snapdragon and Qualcomm platforms. The model library provides developers with more than 75 mainstream AI and generative AI models, such as Whisper, ControlNet, Stable Diffusion and Baichuan-7B, which can be packaged in different execution environments (runtime) and can achieve excellence in different forms of terminals. Improve terminal-side AI performance, reduce memory usage and improve energy efficiency. All models are optimized to take full advantage of the hardware acceleration capabilities of all cores (NPU, CPU, and GPU) within Qualcomm AI Engine, resulting in 4x faster inference. The AI model library can automatically handle model conversion from source frameworks to mainstream execution environments, work directly with Qualcomm AI Engine Direct SDK, and apply hardware-aware optimizations. Developers can seamlessly integrate these models into applications, shorten product time to market, and take advantage of the many advantages of on-device AI deployment, such as immediacy, reliability, privacy, personalization, and cost advantages.

These optimized models are now available on Qualcomm AI Hub, GitHub and Hugging Face. Qualcomm AI Hub will continue to add new models and support more platforms and operating systems. Developers can now register and log in to run their own models on cloud-hosted terminals equipped with Qualcomm Technologies' platform, and obtain new features and AI models in advance through Qualcomm AI Hub.

Clement Delangue, co-founder and CEO of Hugging Face, said :

We are excited to host Qualcomm Technologies’ AI models on Hugging Face. These mainstream AI models are optimized for device-side machine learning and can be used on Snapdragon and Qualcomm platforms. They will empower the next generation of mobile developers and edge AI applications, making AI further benefit everyone.

Cutting-edge AI research progress

Qualcomm AI Research demonstrates the first Large Language and Visual Assistant (LLaVA) running on Android smartphones, a large multi-modal language model (LMM) with over 7 billion parameters that accepts text and images Multiple types of data input, including the ability to generate multiple rounds of conversations about images with the AI assistant. This LMM can generate tokens at the speed of real-time response on the terminal side, thus enhancing privacy, reliability, personalization and cost advantages. LMM with language understanding and visual understanding capabilities can enable many use cases, such as identifying and discussing complex visual patterns, objects and scenes.
Qualcomm AI Research will also demonstrate Qualcomm’s first LoRA model running on Android smartphones. By running Stable Diffusion with LoRA support, users can create high-quality custom images based on personal or artistic preferences. LoRA reduces the number of trainable parameters of the AI model and enables more efficient, scalable, and customized terminal-side generative AI use cases. In addition to being able to fine-tune language visual models (LVM) for different artistic styles, LoRA is also widely used in customized AI models (such as large language models) to create tailor-made personal assistants, improve language translation, etc.
On Windows PC, Qualcomm AI Research will demonstrate the world's first large-scale multi-modal language model (LMM) with more than 7 billion parameters running on the terminal side, which can accept text and audio input (such as music, traffic environment audio, etc.), and Generate multiple rounds of dialogue based on audio content.

Demonstrating cross-terminal category empowerment generative AI at MWC Barcelona

Smartphones: Qualcomm Technologies will showcase a series of commercial flagship AI smartphones powered by the third-generation Snapdragon® 8 mobile platform, including Honor Magic6 Pro, OPPO Find X7 Ultra and Xiaomi 14 Pro. Each terminal integrates exciting new generative AI features, such as image expansion (Xiaomi), smart film and schedule (Honor), and AI elimination (OPPO).
PC: The new Snapdragon X Elite and its 45 TOPS NPU are specially designed for terminal-side AI and will change the way users interact with PCs. Qualcomm Technologies will use the popular free image editor GIMP to integrate the Stable Diffusion plug-in for demonstration: users can enter the image they want, and generative AI will generate the image in 7 seconds, which is 3 times faster than x86 competitors.
Automotive: Leveraging industry-leading AI hardware and software solutions, Qualcomm Technologies will also demonstrate traditional AI and generative AI capabilities supported by the Snapdragon® Digital Chassis™ platform, aiming to provide drivers and passengers with more powerful, efficient, private, ^and A safe and personalized experience.
Consumer-grade Internet of Things: The company will demonstrate the Humane AI Pin running on the Snapdragon platform, allowing users to use AI anytime, anywhere in a new, conversational and screenless terminal form.
Connection: The newly launched Snapdragon X80 modem and radio frequency system integrates the second-generation 5G AI processor to help improve cellular performance, expand coverage, reduce latency and improve energy efficiency. Qualcomm also launched the first AI-optimized Wi-Fi 7 system, the Qualcomm FastConnect™ 7900 mobile connection system, which uses AI to set a new benchmark for adaptive, high-performance, low-latency and low-power local wireless connections.
5G Infrastructure: Qualcomm Technologies will demonstrate three breakthrough AI-assisted network management enhancements, including a generative AI assistant that helps radio access network (RAN) engineers simplify network and slice management tasks, and an AI assistant that reduces network energy consumption. Open RAN applications (rApps), and AI-assisted 5G network slicing lifecycle management suite.

To learn more about Qualcomm Technologies’ AI innovations and demonstrations, please click [Read the original text] or visit the Qualcomm booth at MWC Barcelona (Fira Gran Via Hall 3, No. 3E10) to experience it.

* Qualcomm, Snapdragon, Digital Chassis and FastConnect are trademarks or registered trademarks of Qualcomm Incorporated.

Latest articles about

■The new Snapdragon Automotive Platform Ultimate Edition: Opening a new era of automotive intelligence

■Qualcomm 7, the "full attendance" at the CIIE, attended the CIIE, showing a new chapter in intelligent computing and ecological cooperation

■Honor Magic 7 series released: Enjoy the magic and AI

■Snapdragon 8 Extreme Edition Mobile Platform: Comprehensively Reconstructing the Flagship Mobile Experience

■Qualcomm Launches New Snapdragon Cockpit Extreme Platform and Snapdragon Ride Extreme Platform to Lead the Evolution of Software-Defined Cars

■Qualcomm announces multi-year strategic collaboration with Google to provide generative AI digital cockpit solutions

■Qualcomm launches Snapdragon 8 Extreme Edition, integrating the world's fastest mobile CPU

■NetEase and Qualcomm collaborate to create an innovative mobile gaming experience for Naraka: Bladepoint based on the Snapdragon 8 Extreme Edition mobile platform

■Qualcomm and Zhipu collaborated to adapt and optimize the GLM-4V end-side visual model based on the Snapdragon 8 Extreme Edition mobile platform to accelerate the end-side deployment of multi-modal generative AI experience

■Qualcomm and Tencent Hunyuan cooperate to jointly promote the terminal-side deployment of Tencent Hunyuan large model based on Snapdragon 8 Extreme Edition