Source: Synced

Nowadays, as the process of squeezing large models into smartphones accelerates, ordinary users can also experience the charm of generative AI themselves.

Image Source: Generated by Wujie AI

As 2023 draws to a close, large models and generative AI have become the mainstream trend in the AI community. Since the emergence of ChatGPT, all kinds of general and professional AI large models have emerged one after another, and the momentum is unstoppable. With the advancement of model compression technologies such as quantization, network pruning, and knowledge distillation, the AI computing power of terminal devices such as smartphones continues to increase, and large models are moving from cloud deployment to landing on terminals.

For this already unfolding "terminal-side revolution," chip giants and mobile phone manufacturers have become the main force, constantly attempting to deploy and run generative AI large models on terminal devices such as smartphones, occasionally giving people a little surprise and shock.

In February of this year, we saw for the first time the Stable Diffusion large model with over 1 billion parameters running on a smartphone, which was powered by the second-generation Qualcomm Snapdragon 8 mobile platform. In less than 15 seconds, it executed 20 steps of inference and generated a 512×512 pixel image. This also set the record for the fastest inference speed on a smartphone at that time.

Do you think this is the speed limit for running Stable Diffusion on a smartphone? Obviously not. At the 2023 Snapdragon Technology Summit held last month, the generation of the image was shortened by tens of times in less than 15 seconds. The latest flagship mobile platform, the third-generation Snapdragon 8 (referred to as Snapdragon 8 Gen 3), made this speed increase possible.

At the summit, Qualcomm demonstrated the same Stable Diffusion large model running on a smartphone equipped with the Snapdragon 8 Gen 3, which generated an image in less than 1 second (0.6 seconds) locally. From 15 seconds to 0.6 seconds, the evolution speed of terminal-side generative AI large models has left us amazed.

This is just a small test of Qualcomm's demonstration of its terminal-side AI capabilities at this year's Snapdragon Technology Summit. In addition to a qualitative leap in generation speed (from 15 seconds to 0.6 seconds), the Snapdragon 8 Gen 3 will support a tenfold increase in the scale of large model parameters that can be run (from 1 billion to tens of billions).

The large model empowerment platform is no longer limited to smartphones. Qualcomm's next-generation AI PC chip, the Snapdragon X Elite, has made its debut, "squeezing" large models with tens of billions of parameters into PC platforms, greatly expanding the breadth of generative AI applications.

Thus, Qualcomm's leadership and technological superiority in the field of terminal-side AI have been further strengthened, providing more imagination and possibilities for terminal AI landing for manufacturers of smartphones, PCs, and even headphones.

Dual Platforms Advance, Making Generative AI More Accessible

Since February of this year, Qualcomm has successively run Stable Diffusion and ControlNet on Android smartphones powered by the second-generation Snapdragon 8 mobile platform, making it a reality to run large models on smartphones. The release of the Snapdragon 8 Gen 3 at the Snapdragon Summit has raised the potential for terminal-side landing of large models to new heights, continuing to lead the trend in this field.

It is understood that the Snapdragon 8 Gen 3 uses the powerful Hexagon NPU tailored for mobile terminals in Qualcomm's AI engine, integrating upgraded hardware acceleration units, microslice inference units, enhanced tensor, scalar, and vector units, all sharing a large-capacity shared memory with twice the bandwidth. It also supports mixed precision of INT8+INT16 and all precisions of INT4, INT8, INT16, and FP16. Its performance has increased by 98%, and its energy efficiency has improved by 40%.

The integrated Qualcomm sensor hub in the Snapdragon 8 Gen 3 is also stronger, with 2 always-on sensing ISPs, 2 micro NPUs, and 1 DPS, a 30% increase in memory, and support for INT4 precision, resulting in a 3.5x increase in AI performance.

It can be said that the Snapdragon 8 Gen 3 has endowed smartphones with unprecedented powerful terminal-side AI performance, becoming the first mobile platform specially designed by Qualcomm for generative AI. So, is this flagship mobile platform worthy of its name? This is the question we are most concerned about.

On this platform, for the first time, the Qualcomm AI engine supports multimodal generative AI, including various large language models and visual language models. Especially at the terminal side, the Snapdragon 8 Gen 3 is the first to support the operation of models with 10 billion parameters, entering the hundred billion level in scale. At the same time, it has reduced the time for Stable Diffusion to generate images to less than 1 second, once again setting the record for the fastest speed on a smartphone.

When running the Meta large language model Llama2-7B, the Snapdragon 8 Gen 3 can generate 20 tokens per second, also one of the fastest on the smartphone terminal.

Ziad Asghar, Senior Vice President of Product Management at Qualcomm

Now, the Snapdragon platform already supports the operation of large models from companies or institutions such as OpenAI, Meta, Microsoft, Android, stability.ai, BLOOM, Baidu, Zhipu, Baichuan Intelligence, and Youdao, providing an opportunity for these manufacturers to "downshift" their own large model capabilities to the terminal side and offering users a rich selection of generative AI experiences.

As Qualcomm further acts to empower large models to land on terminals, smartphone manufacturers using the Snapdragon 8 Gen 3 mobile platform have quickly followed suit, accelerating the integration of large models into smartphones.

Taking the example of the new Xiaomi 14 series, the first to be equipped with the Snapdragon 8 Gen 3 mobile platform, AI large model technology has been integrated into Xiaomi's new Penglai OS and various terminal-side applications have been launched. With the support of large model capabilities, the Xiaomi 14 series supports multiple functions such as AI drawing, AI image search, AI portrait, and AI image expansion, bringing new vitality to input methods, WPS, and photo albums.

The iQOO 12 series under the vivo brand also uses the Snapdragon 8 Gen 3 mobile platform, and the new Qualcomm AI engine significantly enhances the imaging experience, maximizing photo texture and detail. At the same time, the series of smartphones pre-installed with the OriginOS 4 system introduces generative AI functions, and the AI assistant, Blue Heart Xiao V, can perform super semantic search, super Q&A, super writing, super image creation, and super intelligent interaction, significantly enhancing the level of intelligence.

At the Snapdragon Summit earlier, Honor CEO Zhao Ming announced that the new flagship smartphone, Honor Magic 6, will be equipped with the Snapdragon 8 Gen 3 mobile platform, already supporting terminal-side AI large models with 7 billion parameters, providing users with a more intelligent experience. He demonstrated on-site the intelligent operations such as photo search and video generation performed by terminal-side AI, with impressive results.

The integration of large model capabilities into smartphone terminals has given birth to a large number of generative AI applications that are either fun or helpful in improving productivity. In the future, a more diverse range of generative AI applications will allow more users to experience the charm of AI firsthand and enhance their interactive experiences, which is also one of the important driving forces for Qualcomm to create generative AI capabilities on the terminal side.

In addition to continuously using AI to empower mobile platforms, Qualcomm is also seeking to take advantage of and seize the development opportunities of large models and generative AI to reshape the PC industry and expand its leading advantage in the field of AI.

Generative AI Has a "Second Battlefield"

Qualcomm has released the all-new processor Snapdragon X Elite for the PC platform, which uses a 4nm process technology and integrates a custom Qualcomm Oryon CPU. Its running speed is twice that of Intel's 12-core processor, with 68% lower power consumption than Intel's competing products, and its peak running speed is 50% faster than Apple's M2.

If these parameters still don't convince you of the strength of the Snapdragon X Elite, then let's run a test. The Snapdragon X Elite has two versions: the 23W version focusing on battery life and the 80W version focusing on performance. The single-core and multi-core scores under the Geekbench 6 test are as shown in the figure below, with the single-core performance being particularly outstanding.

Image Source: fonearena

Similarly, the Snapdragon X Elite is designed specifically for AI. It features the industry-leading Qualcomm AI engine with integrated Hexagon NPU, heterogeneous computing power of up to 75TOPS, and Hexagon NPU computing power of 45TOPS, providing the computational foundation for transformative experiences in creative applications, video conferencing, and productivity assistants. The Qualcomm AI engine also supports a wide range of accelerated applications and experiences, creating endless possibilities.

The powerful generative AI capability has also become a major feature of the Snapdragon X Elite. As of its release, it already supports the operation of generative AI models with over 13 billion parameters on the terminal side, generating 30 tokens per second for models with 7 billion parameters, with AI processing speed 4.5 times that of competitors. The image below shows the PC-side AI drawing capability.

From smartphones to PCs, diverse terminal platforms provide more channels for unleashing generative AI capabilities. This also indicates that Qualcomm continues to "step up" in the race to land generative AI on the terminal side, which is not only based on its technical accumulation but also a continuation of its own AI development strategy in the era of large models.

Mobile Phones and PCs Accelerate into a New AI Era

In today's era of large models, the path of deploying generative AI on the terminal side is becoming increasingly clear. Based on this, Qualcomm has always adhered to the concept and strategy that "hybrid AI is the future of AI," emphasizing the coordinated development of the cloud and the terminal, and believing that terminal-side AI is the key to expanding the reach of generative AI to a wider global audience.

Following this forward-looking understanding, Qualcomm has gradually built a strong and comprehensive terminal-side AI capability in its own layout, leading the innovation and progress of terminal-side large models and generative AI. There is no doubt that the release of the Snapdragon 8 Gen 3 and Snapdragon X Elite at the Snapdragon Summit will further solidify Qualcomm's position as a leader in terminal-side AI.

_

_

Qualcomm's full-stack AI optimization solution

If large models continue to land at the current pace, we will be able to experience more generative AI applications on smart terminals such as smartphones, PCs, XR wearable devices, and smart connected cars, making human-machine interaction more natural and personalized. In turn, these generative AI applications and the transformative experiences they bring will have a profound impact on these terminal devices, even changing their core driving forces and redefining them.

Taking smartphones, which are closely related to people's work and life, as an example, in addition to pursuing absolute performance, people are increasingly focusing on creative applications. This requires AI to show its prowess, and generative AI performance has become the core competitive advantage in the next generation of smartphone processors.

Therefore, it is not surprising that Qualcomm has been focusing on this in recent years. From last year's Snapdragon 8 Gen 2 to the Snapdragon 8 Gen 3 at this year's Snapdragon Summit, AI has already become ubiquitous, and generative AI has leaped to become the "new gem on the AI crown," attracting widespread attention and favor.

As the range and maturity of large models and generative AI capabilities landing on smartphone terminals continue to expand, we may see traditional smartphones being replaced by "AI phones" with generative AI capabilities. The new flagship smartphones released by manufacturers such as Xiaomi are already showing signs of this, using generative AI to enhance the user experience and efficiency in certain scenarios, laying the groundwork for a qualitative change in the future of smartphones.

At the same time, there is a similar trend of leaning towards generative AI capabilities in the innovation of PC terminals. In this regard, Microsoft's Copilot integrated into Windows is a good example. Of course, there are many more scenarios and ways in which generative AI is revolutionizing human interaction with PCs, enabling more efficient intelligent collaboration, simplified workflows, enhanced productivity, and customized user experiences.

The Snapdragon X Elite, targeting generative AI from the start, is bound to inject new vitality into the current PC market. At the 6th Hongqiao International Economic Forum's sub-forum on "Intelligent Technology and Future Industrial Development," Meng Pu, Chairman of Qualcomm China, delivered a keynote speech, stating that the next generation of PCs will enter a new "AI PC" era characterized by intelligence, high performance, and efficiency, which will be of great significance for office efficiency and other aspects. It is believed that in the future, Qualcomm will continue to strengthen its basic capabilities in large models and generative AI based on mobile platforms such as the Snapdragon 8 Gen 3 and PC platforms such as the Snapdragon X Elite, and more manufacturers launching new flagship smartphones and PC products with these platforms will continue to enhance their autonomy and R&D capabilities, allowing us to look forward to the appearance of a more diverse range of generative AI applications.

However, before the era of "AI phones" and "AI PCs" truly arrives, it remains to be seen whether generative AI capabilities can create a strong perception for users, accurately meet their needs, create dependencies on related applications, and drive their desire to upgrade their devices. These are the questions that Qualcomm and smartphone manufacturers need to consider now and in the future.

Conclusion

Since its release, the powerful generative AI capabilities demonstrated by the Snapdragon 8 Gen 3 and Snapdragon X Elite have left people with endless imagination. As Qualcomm President and CEO Amon said, "We are entering a generative AI era that will change user experiences and create a new cycle for the mobile and computing industries."

In this process, as a leader in terminal-side technology, Qualcomm is assuming the role it deserves in the era. On the one hand, it is gathering software and hardware from its powerful full-stack AI optimization capabilities to help the rapid development of the terminal-side AI ecosystem; on the other hand, the new cross-platform technology "Snapdragon Seamless" achieves seamless integration of Android, Windows, and other operating systems, and this multi-terminal experience is expected to further expand the landing of AI on smartphones, PCs, and other terminals.

It can be foreseen that the AI experience based on the Snapdragon ecosystem will spread very quickly, and the speed of application landing will also greatly accelerate. In the future, Qualcomm will continue to maintain its technological advantage in terminal-side AI, making the deployment of large models and generative AI more convenient, diverse, and attractive. At the same time, together with major manufacturers, it will innovate generative AI applications based on large models, lowering the threshold for ordinary users to use generative AI.

As the new era of generative AI opens, whether Qualcomm can continue to meet people's expectations, I think the Snapdragon Summit has already given us the answer.

Reference link: https://www.qualcomm.cn/news/releases/2023/10/releases-2023-10-24-3

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。