Source: Titanium Media

Author: Deng Jianyun

Image source: Generated by Wujie AI

When "Generative Artificial Intelligence" becomes a hot topic in the industry, while focusing on various emerging applications, "computing power" as the underlying support for applications has also become one of the focuses.

For traditional high-computing power terminals, the "computing power gap" required for large model training and AI generation can be filled by adding additional GPU hardware. However, for mobile intelligent terminals, limited by integration and power consumption, it is more difficult to break free from the need for networking and achieve edge deployment of generative artificial intelligence.

Flagship chip's advance in the AI field, and the difference in the layout path of mid-range

In the past two months, MediaTek and Qualcomm have successively released new generation flagship SoC chips, both of which have mentioned the improvement of AI deployment capability at the edge.

Taking MediaTek as an example, the previously released Dimensity 9300 chip internally integrates MediaTek's seventh-generation AI processor APU 790, designed for generative AI. The performance of integer and floating-point operations is twice that of the previous generation, and power consumption has been reduced by 45%. APU 790 has a built-in hardware-level generative AI engine, which can achieve faster and more secure edge AI computing, and is deeply adapted to the Transformer model for sub-acceleration.

According to official data, its processing speed is 8 times that of the previous generation, and it can generate images in 1 second. At the same time, based on the characteristics of billion-level parameter large language models, MediaTek has developed mixed-precision INT4 quantization technology, combined with memory hardware compression technology NeuroPilot Compression, which can more efficiently utilize memory bandwidth, reduce the occupation of terminal memory by AI large models, and support terminal operation of AI large language models with up to 33 billion parameters.

The release of the new generation of smartphone chips has provided hardware support for generative artificial intelligence. However, products equipped with flagship SoC chips are generally limited to the high-end market. According to IDC's data, in the third quarter of 2023, the shipment share of the high-end smartphone market (above 3500 yuan) in China was 33%.

In other words, even if all high-end smartphones begin to support edge generative AI, only 3 out of 10 users have the opportunity to access this feature at the user level. The lower-positioned mid-range smartphone market is temporarily unable to enjoy the experience upgrade brought by large models. It is worth noting that the market share of smartphones in the 2500-3499 yuan range has also reached 19% and is in a trend of continuous growth.

How to expand the application landing scope and make generative artificial intelligence more than just a "tied product" of high-end products is the primary issue facing various terminal manufacturers and upstream chip suppliers. Qualcomm's third-generation Snapdragon 7, launched for the next generation of mid-to-high-end products, did not mention the advancement of AI functions, and the main improvement is still focused on CPU, GPU performance, and power consumption for traditional usage scenarios.

In contrast, since the release of the Dimensity 8000 series, MediaTek has made mid-to-high-end models outside of flagship phones the "basic disk" for accelerating the penetration of mobile chips in the market. The iteration speed of the chip is faster, and it has also deepened cooperation with terminal smartphone brands. The latest Dimensity 8300 is also the first to support generative AI among products of the same level, supporting up to 100 billion parameter AI large language models.

This chip integrates MediaTek's AI processor APU 780, equipped with a generative AI engine, and the performance of integer and floating-point operations is twice that of the previous generation. It supports Transformer operator acceleration and mixed-precision INT4 quantization technology. The AI comprehensive performance is 3.3 times that of the previous generation, enabling smooth operation of innovative applications of edge generative AI.

According to Li Yanji, Deputy General Manager of MediaTek's Wireless Communication Business Unit, Xiaomi has become the first mobile phone manufacturer to realize edge generative AI applications through its chip capabilities.

Xiaomi Group President and Redmi Brand General Manager Lu Weibing stated, "Redmi phones will be the first to be equipped with the Dimensity 8300-Ultra chip globally. This phone will apply the new generation of generative AI technology in performance scheduling, system applications, and hardware underlying capabilities." The recently released Redmi K70E is the first smartphone product that Lu Weibing mentioned, supporting edge generative artificial intelligence.

From the cloud side to the edge side, what can we expect from the era of mobile large models?

Although the products and applications are available, when it comes to large model technology, it seems to be far from ordinary users. The reason lies in the scarcity of specific scenario landing. For the general consumers, they cannot perceive the experiential differences brought by large models.

It is fortunate that the application of large models on mobile devices is accelerating. Currently, smartphone manufacturers such as Huawei, Xiaomi, and vivo have put the application of large models on mobile devices on the agenda. Many combinations of voice assistants and large models have entered the internal testing or even public testing stages.

An industry insider told Titanium Media App, "The addition of large models can allow intelligent voice assistants to break out of the existing algorithmic 'mechanical Q&A' mode. The understanding of user commands and problem-solving capabilities will be enhanced, and this is just the first step. After the landing of edge generative artificial intelligence, advanced applications such as 'image-to-text,' 'text-to-image,' and 'image-to-image,' which are currently niche skills, are expected to be realized on mobile devices, allowing voice assistants to show creativity."

In the long run, large models and edge generative artificial intelligence do have the potential to bring more interesting or practical experiences to terminal hardware such as smartphones. However, after experiencing the current stage of large model applications on mobile devices, Titanium Media App can only say that it is "promising for the future" rather than "immediately innovative."

Frankly speaking, the current large model applications on mobile devices are mostly just gimmicks, and the perceivable "intelligent upgrade" is very limited. Although the voice assistant can understand and answer more questions, and even have some simple logical thinking with connections, it still lags far behind the leading Chat GPT within the field.

In terms of AI drawing applications, there are also many challenges for mobile devices. In terms of computing power, the mainstream Stable Diffusion in the industry has been tested, and at least an RTX 3070 or higher-level graphics card is required to obtain a relatively smooth application experience. The learning curve is also not low because AI drawing often requires the selection of learning models and continuous adjustments and modifications to the images to obtain a fairly good picture.

Even for tools like Stable Diffusion, which has been continuously improved since the rise of AI drawing applications, the deployment on mobile devices will be a bigger problem. If it can only generate images through text descriptions, for ordinary consumers, it is at most a novelty. Once the novelty wears off, it will immediately become a niche skill, and large model applications will follow the same old path as intelligent voice assistants did in the past, becoming popular and then gradually cooling down in the market.

Therefore, edge large model applications also need to find high-frequency scenarios for adaptation. Li Yanji believes that "performance scheduling" will be a good starting point. "Large models can not only learn traditional text and image data, but also learn user habits, and then have the adaptive switching ability for application scenarios," he mentioned.

Specifically, it means breaking away from the traditional fixed algorithm scheduling logic and allowing AI to participate more. For example, when users use their smartphones in daily life, their demands for smartphone performance when processing social information, playing games, and watching short videos are different. In the past, the smartphone system often relied on applications to do reverse adaptation, such as calling high-frequency large cores to improve performance for gaming, and using small cores for text chatting and standby to maintain battery life. This scheduling capability is very rigid.

After smartphones have edge AI capabilities, they can make performance scheduling more proactive, and can optimize in real-time according to the current scenario demands, rather than relying on software and strategy-level adaptation.

Titanium Media App analysis believes that compared to seemingly innovative application experiences such as chatting and AI drawing, the key to creating experiential value for edge large models in the short term lies in precise empowerment of existing high-frequency scenarios. Whether it is optimization in terms of imaging, performance, or real-time intelligent adjustment of hardware such as screens and speakers, these are dimensions that can bring a perceptible upgrade to user experience.

Suppose one day, through the combination of edge large models and learning from user habits and application perception data, and using comprehensive scheduling of software and hardware, intelligent smartphones can improve battery life by another 20% or even 50%. Who would still say it is just a "gimmick"?

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。