Android is the first to run through multi-modal large models, and the terminal can view images and generate text locally! Qualcomm: WiFi will become AI

z Mingmin comes from Ao Fei Si
Qubit | Official account QbitAI

Multi-modal large models are deployed locally on Android phones for the first time !

Now, you can have a conversation with your mobile phone's AI assistant by inputting photos and voice, and all functions are completely run on the terminal side .

On the first day of MWC 2024, Qualcomm made a big move and focused on terminal-side AI.

In addition to being able to run multi-modal large models on mobile phones, the world's first audio inference large multi-modal model demonstration running on Windows PC is also here.

It can understand audio and perform reasoning, enabling multiple rounds of dialogue with voice input.

Finally, Qualcomm also released the AI ​​Hub for developers , which makes it easier for developers to develop large model applications on mobile phones and PCs. It currently supports more than 75 large models.

And even WiFi systems are being enhanced by AI. Qualcomm’s latest generation WiFi 7 solution, FastConnect 7900 , is also facing the hybrid AI era and is the world’s first WiFi system that uses AI enhancement.

Qualcomm’s set of punches only proves one thing: the terminal-side AI trend has now arrived.

Terminal-side AI has arrived

Qualcomm flexed its muscles this time and demonstrated new breakthroughs in generative AI running on mobile phones and PCs.

Moreover, in the official information, it has been emphasized many times that "all functions are completely run on the terminal side" - but it is not yet clear whether they are all implemented without the Internet .

But all in all, the terminal side can run more large AI models, which is definitely big news for terminal changes and user value changes.

For specific details, please see the official demo.

On Android phones, Qualcomm has raised the local operation of generative AI to a multi-modal level.

The first large-scale language and visual assistant model (LLaVA) running on Android smartphones can accept multiple types of data input including text and images, and can conduct multiple rounds of conversations based on the input content.

Now, users can take a photo and ask the AI ​​assistant questions:

What are these ingredients? What kind of food can it cook? How many calories are in each dish?

The AI ​​assistant can give answers based on photo information. All functions are completely run on the terminal , allowing multiple rounds of conversations to ensure response speed.

In addition, Qualcomm also released an instance that can run LoRA on Android phones.

LoRA can adjust or customize the generated content of the model without changing the underlying model. By using an adapter that is only 2% of the size of the model, the generative AI model can be personalized and customized.

For example, Stable Diffusion can be customized through LoRA. Large language models can also be customized as personal assistants through LoRA to improve translation capabilities, etc.

The PC aspect also emphasizes the local deployment of multi-modal capabilities.

Now, you can run large audio inference multi-modal models on Windows PC, enabling multiple rounds of dialogue to be completed by voice.

Windows PCs equipped with Snapdragon X Elite will be able to understand birdsong, music, or various sounds. For example, it can listen to songs and make similar recommendations.

At the same time, Qualcomm also "translated" what a true AI PC is .

The NPU computing power of Snapdragon X Elite is as high as 45TOPS . The two devices simultaneously run GIMP (a popular image editor) integrated with the Stable Diffusion plug-in for AI image generation. Snapdragon X Elite can generate an image in just 7.25 seconds, which is three times faster than the X86 competitor (22.26 seconds) .

For developers, Qualcomm launched a new AI Hub .

It supports more than 75 models, both traditional AI models and generative AI models, such as ControlNet, Stable Diffusion, Baichuan-7B, etc., and can be deployed on Snapdragon and Qualcomm platforms.

Developers select the required model, the framework used, and determine the target platform (such as a specific model of mobile phone or chip). Qualcomm AI Hub can provide developers with optimized models for designated applications and designated platforms. It only takes a few lines of code to get the model and integrate it into your application.

Qualcomm said that each model supported by AI Hub has been optimized. The Qualcomm-based AI engine can achieve 4 times inference acceleration while occupying less memory bandwidth and storage space.

These optimized models are available on Qualcomm AI Hub, Hugging Face and GitHub.

The first AI-enhanced WiFi 7 system

Why use AI to enhance WiFi?

Because Qualcomm believes that the future of AI is hybrid AI, which means it needs to span clouds, terminals and edge clouds.

This also places higher demands on connections.

At MWC 2024, Qualcomm brought a new generation of WiFi 7 solutions: Qualcomm FastConnect 7900 system.

This is also the world’s first AI-enhanced WiFi system that integrates proximity sensing capabilities.

In addition, this is also the first time Qualcomm has integrated Bluetooth, WiFi and ultra-bandwidth on a 6nm chip, achieving the effect of "one can beat three".

Compared with the previous generation, the 7900 adopts a new RF front-end module and architecture, which reduces system power consumption by 40% while improving energy efficiency; the system also helps reduce the board area by 25%, leaving more battery space for Improve battery life.

At this year's MWC, Qualcomm not only released a series of terminal AI technologies, but also unveiled a series of flagship phones equipped with Snapdragon 8 Gen 3, such as Honor Magic6 Pro, OPPO X7 Ultra, Xiaomi 14 Pro, etc.

They bring AI image expansion (Xiaomi), AI creation schedule (Honor), AI image elimination (OPPO) and other functions.

There may be some debate as to whether AI-powered mobile phones are the first of their kind, but the implementation of AI on the terminal side is moving towards Everywhere and Everyone, driven by Qualcomm’s underlying efforts...

Reference link:
https://www.qualcomm.com/news/media-center/press-kits/mwc-barcelona-2024

Click here ???? Follow me and remember to mark it with a star