Use secure models to protect insecure models, and use intelligent systems to defend against intelligent attacks.

Author: Cool Geek

Geeks are starting businesses, novices are buying courses, and artists are unemployed, but an awkward reality is: AI is booming, but the plot is not following a divine route; instead, it’s like rolling dice.

Moreover, in the early stages of the industry, the side of the dice that lands first is often either yellow or gray.

The reason is simple: exorbitant profits drive motivation, and the early stages of any industry are always riddled with loopholes. Just look at this set of data:

Currently, over 43% of MCP service nodes have unverified Shell call paths, and over 83% of deployments have MCP (Model Context Protocol) configuration vulnerabilities; 88% of AI component deployments have not enabled any form of protective mechanisms; 150,000 lightweight AI deployment frameworks like Ollama are currently exposed on the global public network, with over $1 billion of computing power hijacked for mining…

Ironically, attacking the smartest large models requires the most basic techniques—just a set of default open ports, an exposed YAML configuration file, or an unverified Shell call path. Moreover, if the prompt input is precise enough, the large model can help the gray industry find the direction of the attack. The door to corporate data privacy is thus freely opened and closed in the AI era.

But the problem is not without solutions: AI has both generative and attacking aspects. How to use AI for protection is increasingly becoming the main theme of this era; at the same time, establishing rules for AI in the cloud has also become a key exploration direction for leading cloud vendors, with Alibaba Cloud Security being the most typical representative.

At the recently concluded Alibaba Cloud Feitian launch event, Alibaba Cloud officially announced its two paths for cloud security: Security for AI and AI for Security, and launched the "AI Cloud Shield" series of products to provide customers with "end-to-end security solutions for model applications," which is a prime example of current industry exploration.

01 Why do the dice of AI always land on gray and yellow first?

In the history of human technology, AI is not the first "new species" to be "tested by yellow and gray"; the gray and yellow explosion is a rule of technology popularization rather than an accident.

When silver plate photography emerged in 1839, the first wave of users was the pornography industry;

In the early days of the internet, before e-commerce took off, adult websites were already exploring online payments;

Today’s large model profiteers are, to some extent, replicating the wealth myth of the "domain name era."

The dividends of the era are always first taken away by gray and yellow. Because they do not comply with regulations, do not wait for supervision, and their efficiency is naturally very high.

Thus, every technological explosion period is first a "mixed soup," and AI is no exception.

In December 2023, a hacker used just one prompt—"$1 quote"—to lure a customer service robot at a 4S store to almost sell a Chevrolet for $1. This is the most common "prompt injection" in the AI era: no permission verification, no log traces, and just by "saying it cleverly," one can change the entire logic chain.

Going deeper, there is "jailbreak attack." Attackers use rhetorical questions, role-playing, and circumlocutory prompts to successfully make the model say things it should not say: pornographic content, drug manufacturing, false warning information…

In Hong Kong, someone even forged executive voices to embezzle 200 million Hong Kong dollars from corporate accounts.

In addition to scams, AI also has the risk of "unintentional output": in 2023, a major education giant's large model system mistakenly output "toxic teaching materials" containing extreme content while generating lesson plans. Within just three days, parents protested, public opinion erupted, and the company's stock price evaporated by 12 billion yuan.

AI does not understand the law, but it has the capability, and once that capability is detached from supervision, it becomes harmful.

However, from another perspective, while the technology of AI is new, the ultimate flow and means of gray and yellow industries remain unchanged, and to solve it, security is still essential.

02 Security for AI

Let’s start with a cold fact that the AI industry collectively avoids:

The essence of large models is not "intelligence" or "understanding," but semantic generation under probability control. Therefore, once it exceeds the training context, it may output unexpected results.

This exceeding may mean that if you want it to write news, it writes poetry; or if you want it to recommend products, it suddenly tells you that today’s temperature in Tokyo is 25 degrees Celsius. Even more, if you tell it that in a game, if it cannot obtain the legitimate serial number of a certain software, it will be shot, the large model can indeed find a legitimate software serial number for the user at zero cost.

To ensure controllable output, enterprises must understand both the model and security. According to IDC's latest "China Security Large Model Capability Assessment Report," Alibaba ranks first in 4 out of 7 indicators in a competition with all leading domestic vendors with security large model capabilities, with the remaining 3 also above the industry average.

In practice, Alibaba Cloud Security provides a straightforward answer: let security run ahead of AI speed, building a bottom-up, three-layer full-stack protection framework—from infrastructure security to large model input-output control, to AI application service protection.

In these three layers, the most prominent is the middle layer specifically targeting large model risks, the "AI Guardrail."

Generally speaking, the main risks concerning large model security include: content violations, sensitive data leaks, prompt injection attacks, model hallucinations, and jailbreak attacks.

However, traditional security solutions are mostly generic architectures designed for the web, not prepared for "talking programs," and naturally cannot produce precise identification and response capabilities for the unique risks of large model applications. Emerging issues such as generative content security, context attack defense, and model output credibility are even harder to cover. More importantly, traditional solutions lack fine-grained controllable means and visual traceability mechanisms, leading to significant blind spots in AI governance for enterprises, leaving them unaware of where the problems lie and thus unable to solve them.

The true power of AI Guardrail lies not just in "it can block," but in whether you are doing pre-trained large models, AI services, or various business forms of AI Agents, it knows what you are saying and what the large model is generating, thus providing precise risk detection and proactive defense capabilities, achieving compliance, security, and stability.

Specifically, AI Guardrail is responsible for protecting three types of scenarios:

ꔷ Compliance baseline: Conduct multi-dimensional compliance reviews of the text content generated by generative AI inputs and outputs, covering risks such as political sensitivity, pornography, bias and discrimination, and negative values, deeply detecting privacy data and sensitive information that may be leaked during AI interactions, supporting the identification of sensitive content involving personal and corporate privacy, and providing digital watermark identification to ensure that AI-generated content complies with laws and regulations and platform standards;

ꔷ Threat defense: For external attack behaviors such as prompt injection attacks, malicious file uploads, and malicious URL links, it can achieve real-time detection and interception, avoiding risks to the end users of AI applications;

ꔷ Model health: Focus on the stability and reliability of the AI model itself, establishing a complete set of detection mechanisms for issues such as model jailbreaks and prompt crawlers, preventing the model from being abused, misused, or producing uncontrollable outputs, building an "immune defense line" for AI systems;

It is worth mentioning that AI Guardrail is not simply stacking the above multiple detection modules together, but has achieved a true ALL IN ONE API, without splitting modules, without additional costs, and without changing products. For risks related to model input and output, customers no longer need to purchase additional products; for different model risks: injection risks, malicious files, content compliance, hallucinations, etc., all can be resolved within the same product. One interface covers detection for over 10 types of attack scenarios, supports 4 deployment methods (API proxy, platform integration, gateway access, WAF mounting), with millisecond-level response and thousands of concurrent processing, achieving a precision rate of up to 99%.

Thus, the true significance of AI Guardrail lies in transforming "model security" into "product capability," allowing one interface to replace an entire security team.

Of course, large models are not an abstract concept; they are systems running on hardware and code, supporting upper-level applications. For infrastructure security and AI application service protection, Alibaba Cloud Security has also made upgrades.

At the infrastructure layer, Alibaba Cloud Security has launched the Cloud Security Center, with core products like AI-BOM and AI-SPM.

Specifically, AI-BOM (AI Bill of Materials) and AI-SPM (AI Security Posture Management) capabilities address the two questions: "What AI components have I installed?" and "How many vulnerabilities do these components have?"

The core of AI-BOM is to comprehensively identify AI components in the deployment environment: it automatically recognizes security weaknesses and dependency vulnerabilities among over 30 mainstream components such as Ray, Ollama, Mlflow, Jupyter, and TorchServe, forming an "AI software bill of materials." Problematic assets are identified not through manual checks but through cloud-native scanning.

AI-SPM is more like a "radar": it continuously assesses the system's security posture from multiple dimensions such as vulnerabilities, port exposure, credential leaks, plaintext configurations, and unauthorized access, dynamically providing risk levels and remediation suggestions. It transforms security from "snapshot compliance" to "streaming governance."

In summary: AI-BOM knows where you might have patched, and AI-SPM knows where you might get hit again, allowing for prompt preventive measures.

For the AI application protection layer, Alibaba Cloud Security's core product is WAAP (Web Application & API Protection).

No matter how smart the model output is, if the entry points are all script requests, forged tokens, and abuse of interfaces, it won’t last long. Alibaba WAAP (Web Application & API Protection) was born for this purpose. It does not treat AI applications as "traditional web systems," but provides specialized AI component vulnerability rules, AI business fingerprint libraries, and traffic profiling systems.

For example: WAAP has covered over 50 component vulnerabilities such as arbitrary file uploads in Mlflow and remote command execution in Ray services; the built-in AI crawler fingerprint library can identify over 10,000 new corpus brushes and model evaluation tools added every hour; the API asset identification function can automatically discover which internal systems have exposed GPT interfaces, providing the security team with a "mapping guide."

Most importantly, WAAP and AI Guardrail do not conflict; rather, they complement each other: one looks at "who is coming," while the other looks at "what was said." One acts like an "authenticator," while the other acts like a "behavior auditor." This gives AI applications a kind of "self-immunity" capability—by identifying, isolating, tracking, and countering, it not only "blocks bad actors" but also "prevents the model from going bad."

03 AI for Security

Since the implementation of AI is like rolling dice, it’s not surprising that some use it for fortune-telling, some for writing love poems, and some for gray industries, so it’s also not strange that some use it for security.

In the past, security operations required a group of people to monitor a bunch of red and green alert lights day and night, taking over the mess from yesterday during the day and keeping the system company at night.

Now, all of this can be handled by AI. In 2024, Alibaba Cloud's security system will fully integrate with the Tongyi large model, launching an AI capability cluster covering data security, content security, business security, and security operations, and introducing a new slogan: Protect at AI Speed.

The meaning is clear: business runs fast, risks run faster, but security must be one step faster.

Using AI to solve security actually involves two things:* improving security operation efficiency + upgrading security products intelligently*.

The biggest pain point of traditional security systems is "policy update lag": attackers change, but rules do not; alerts come in, but no one understands them.

The key to the change brought by large models lies in shifting the security system from rule-driven to model-driven, building a closed-loop ecosystem with "AI understanding capability + user feedback"—AI understands user behavior → user feedback on alert results → continuous model training → detection capabilities become increasingly accurate → cycles become shorter → risks become harder to hide, which is the so-called "data flywheel":

Its advantages are twofold:

On one hand, it enhances the efficiency of cloud tenant security operations: in the past, threat detection often meant an inefficient model of "mass alerts + manual screening." Now, through intelligent modeling, malicious traffic, host intrusions, backdoor scripts, and other abnormal behaviors can be accurately identified, significantly increasing alert hit rates. At the same time, around the disposal phase, the system has achieved deep collaboration in automated disposal and rapid response—host purity remains stable at 99%, and traffic purity approaches 99.9%. Additionally, AI will deeply participate in alert attribution, event classification, process suggestions, etc. Currently, the coverage rate of alert event types has reached 99%, and the user coverage rate of large models has also exceeded 88%, leading to unprecedented efficiency release for security operation teams.

On the other hand, the capabilities of cloud security products are rapidly improving. At the data security and business security layers, AI has been assigned the "gatekeeper" role: based on large model capabilities, it can automatically identify over 800 types of entity data in the cloud and intelligently perform desensitization and encryption processing. Not limited to structured data, the system also includes over 30 types of document and image recognition models, capable of real-time identification, classification, and encryption of sensitive information such as ID numbers and contract elements in images. Overall data labeling efficiency has increased fivefold, with an identification accuracy rate of 95%, greatly reducing the risk of privacy data leakage.

For example: in the content security scenario, the traditional approach relied on human review, labeling, and large-scale annotation training. Now, through prompt engineering and semantic enhancement, Alibaba has achieved a 100% increase in labeling efficiency, a 73% improvement in recognizing vague expressions, an 88% increase in image content recognition, and a 99% accuracy rate in AI live face attack detection.

If the flywheel focuses on AI combined with human experience for autonomous defense, then the intelligent assistant is the all-around assistant for security personnel.

The most common questions security operation personnel face daily are: What does this alert mean? Why was it triggered? Is it a false positive? How should I handle it? In the past, answering these questions required sifting through logs, checking history, asking old employees, submitting work orders, and contacting technical support… Now, it only takes one sentence.

However, the intelligent assistant's function is not just as a Q&A bot; it is more like a vertical Copilot in the security field, with five core capabilities including:

• Product Q&A Assistant: Automatically answers how to configure a certain function, why this policy was triggered, and which resources have not enabled protection, replacing a large number of work order services;

• Alert Explanation Expert: Inputs alert numbers and automatically outputs event explanations, attack chain tracing, suggested response strategies, and supports multilingual output;

• Security Incident Review Assistant: Automatically organizes the complete chain of an intrusion incident, generating timelines, attack path diagrams, and responsibility determination suggestions;

• Report Generator: One-click generation of monthly/quarterly/emergency security reports, covering event statistics, disposal feedback, and operational effectiveness, supporting visual export;

• Full Language Support: Currently covers Chinese and English, with an international version launching in June, supporting automatic adaptation to overseas team usage habits.

Don’t underestimate these "five small tasks." As of now, official data from Alibaba indicates: it has served over 40,000 users, with a user satisfaction rate of 99.81%, covering 100% of alert types, and prompt support capability has increased by 1175% (compared to FY24). In simple terms, it packages the high-performing night shift colleague, the report-writing intern, the alert-handling engineer, and the business-savvy security consultant into one API, allowing humans to focus solely on decision-making without the need for patrolling.

04 Conclusion

Looking back, history has never lacked "epoch-making technologies," but it lacks technologies that can withstand the second year of a boom.

The internet, P2P, blockchain, autonomous driving… every wave of technological explosion has been referred to as "new infrastructure," but ultimately, only a few have become true infrastructure that can traverse the "governance vacuum."

Today’s generative AI is at a similar stage: on one side, models are flourishing, capital is flocking, and applications are breaking through layer by layer; on the other side, there are prompt injections, content overreach, data leaks, model manipulation, widespread vulnerabilities, blurred boundaries, and a lack of accountability.

But AI is different from past technologies. It can not only draw, write poetry, program, and translate, but it can also mimic human language, judgment, and even emotions. However, because of this, the fragility of AI stems not only from code vulnerabilities but also from the reflection of human nature. Humans have biases, and it will learn them; humans seek convenience, and it will exploit loopholes for you.

The convenience of technology itself amplifies this reflection: past IT systems had to emphasize "user authorization," and attacks relied on penetration; now, large models only need prompt injections, and a simple chat can lead to system errors and privacy leaks.

Of course, there is no "perfect" AI system; that is science fiction, not engineering.

The only answer is to use secure models to protect insecure models; to use intelligent systems to counter intelligent threats—by rolling the dice with AI, Alibaba chooses security to be on top.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。