Author: Xiao Bai
This article is an original submission from the author, and the views only represent the author's personal understanding. ETHPanda has edited and organized the content.
The debate around AI is often simplified into two camps: one advocating for continued acceleration and the other urging for an immediate pause.
Vitalik pointed out in response to "AI 2040: Plan A" that the real divergence in this debate is not about policy choices but rather about fundamentally different judgments people have about the future.
One judgment believes that AI is still a "normal technology." It will change employment, industries, and social structures but will ultimately still be constrained by markets, laws, energy, capital, and organizational capabilities.
The other judgment believes that once AI begins to participate on a large scale in AI development, its capability growth may enter a rapid cycle. At that point, what humanity faces is not just job replacement but a potential gradual loss of actual control over research, economy, military, and institutions.
If the first judgment is valid, establishing a global chip tracking, computing power approval, and data center monitoring system would clearly be a severe overreaction.
However, if the second judgment holds true, maintaining a default competition and believing the market will eventually resolve everything is similarly a high-risk gamble.
Thus, what AI governance really needs to answer is not just "should we pause," but:
How can we retain the ability to slow down without permanently handing power to a few individuals when the future is uncertain as to which type of world it will be?
The core of "AI 2040" is not just "slowing down"
"AI 2040" proposes a relatively idealized transition roadmap to superintelligence.
It envisions major countries and AI companies slowing the development speed of cutting-edge AI through international coordination, increasing research transparency, enabling more participants to catch up to near the frontiers, and then continuing to advance after safety and governance issues are addressed.
I believe the biggest problem with this plan lies not in the goals but in the execution.
As long as one party secretly trains stronger models, other participants will worry about being left behind, and the agreement will quickly become ineffective. Therefore, to make the slowdown credible, it is crucial to track high-end chips, monitor large data centers, and assess whether computing power is used for training or other purposes.
This means that to prevent superintelligence from forming a new power center, humanity may first need to establish a global computing power control center.
This is precisely the most challenging contradiction in the AI pause proposal:
The mechanisms to prevent technology from going out of control could themselves become tools of power going out of control.
"AI risks are significant" does not automatically prove that a certain government, international organization, or large laboratory should gain unlimited power.
Similarly, "regulation may be abused" cannot conversely prove that AI risks do not exist.
What really needs to be scrutinized is the institutional structure:
Who defines danger, who holds the data, who has the authority to initiate restrictions, who can receive exemptions, who supervises the enforcers, and whether this power can be revoked after an error in judgment.
If these questions do not have answers, "AI safety" can easily become a timeless technical license.
Between the pause button and the throne lies a single exemption
"Pause AI" sounds like a simple action, but it could represent two completely different systems.
The first is selective pausing.
Those in power can decide who is qualified to train models and who must stop; who belongs to trustworthy institutions and who is regarded as dangerous participants. Others are restricted while the ones in power retain control over models, chips, and computing power.
This mechanism does not truly end the competition; it merely pre-determines the winner.
It is not a pause button but more like a throne.
The second is symmetrical pausing.
When publicly stated risk conditions are triggered, all major participants are restricted, including the party responsible for implementing the pause. No secret exemptions can be granted, and no one can demand a competitor to halt while allowing their own laboratory to proceed.
This mechanism is closer to a real circuit breaker.
Vitalik clearly affirmed the inclusion of "mutual assured destruction of computing power" in Plan A. Compared to giving a few participants the right to selectively restrict others while exempting themselves, this mechanism is at least more symmetrical.
However, symmetry does not automatically resolve issues.
A system capable of halting global advanced computing power is itself a powerful tool. It could be triggered by false intelligence, exploited by political conflict, manipulated by insiders, or could cause significant losses due to technical failures.
Therefore, a credible pause mechanism must meet several conditions:
The trigger conditions must be public, evidence cannot be monopolized by a single institution, restrictions should be as symmetrical as possible, the pause must have a clear deadline, and the rules for restarting must be clearly defined in advance.
In particular, it should not only design "how to enter a state of emergency" but also "how to exit a state of emergency."
In reality, once institutions gain power, they often find it easier to prove that danger still exists than to actively relinquish power.
Therefore, the pause mechanism for AI must also have a pause button.
The true concentration of power does not only come from companies and governments
In discussions today about AI power, people usually fear two things.
One is the monopoly of the strongest models by a few large companies.
Two is the government controlling chips, data, and developers in the name of security.
But Vitalik reminds us of a third, deeper risk:
Superintelligence itself may also become the greatest concentrator of power.
The reason humanity has bargaining power today is that humans possess some scarce resources: labor, knowledge, organizational capabilities, capital, political power, and military force. Different groups must negotiate with one another because no one can completely bypass the others.
If AI is faster, cheaper, and more effective than humans in research, programming, management, finance, communication, cyberattacks, and even military planning, these abilities that originally belonged to humans will rapidly depreciate.
By that time, even with ten AI companies competing in the market, it does not mean humans still hold the ultimate decision-making power.
Market competition can limit a company's profits but does not necessarily ensure ordinary people continue to have bargaining power.
So, "let trustworthy people control the strongest AI" is not a complete answer.
It simply shifts the question from "will AI control humanity" to "which small group should first gain the power to control the strongest AI."
d/acc is a foundation but not a panacea
The d/acc that Vitalik has long advocated can be understood as a priority to develop defensive, decentralized technologies.
Such as safer software, formal verification, cryptography, open hardware, cyber defense, biological safety, and information verification tools.
The value of these directions lies in that they do not depend on a specific superintelligence timeline.
If AI ultimately remains a normal technology, safer software, more reliable hardware, and stronger public defense capabilities are still worth building.
If AI rapidly strengthens, these technologies can also enhance society's ability to withstand cyberattacks, biological risks, hardware backdoors, and information manipulation.
This is a low-regret strategy: when uncertain about which type of world the future will belong to, first build the security infrastructure that is worth having regardless.
But d/acc is not a cure-all.
If AI capabilities indeed leap forward in a short time, defensive technologies may not be deployed in time. Safer systems can reduce risks but cannot prove that all competitions should continue.
Thus, d/acc is better suited as a long-term foundation, while the pause mechanism serves as insurance in extreme cases.
The foundation needs continuous construction; insurance should not be easily activated but cannot wait until the system is already out of control to start design.
More important than predicting dates is agreeing on trigger conditions in advance
The most practically significant suggestion in Vitalik's lengthy article is not to immediately pause AI but to agree in advance:
What evidence must emerge for all parties to change their judgments.
Today, asking everyone to agree that "superintelligence will appear in 2030" or "will not appear in the next twenty years" is nearly impossible.
But different camps can still agree on some risk signals in advance.
For example:
Can AI independently complete a sustained research and development task for several weeks;
Can it significantly accelerate the development of the next generation of AI;
Does it possess the ability to discover and exploit network vulnerabilities on a large scale;
Has it started to autonomously replicate, acquire resources, or circumvent oversight;
Is it being integrated into weapons, financial systems, and critical infrastructure;
Has it caused sustained, large-scale societal harm.
The key is not to find a perfect metric but to shift the debate from "do you believe in superintelligence" to "what real evidence would prompt you to update your judgment."
Those who believe AI risks are exaggerated may accept this, as they believe those scenarios will not occur.
Those worried about superintelligence may also accept it since they think these signals could appear soon.
The two sides do not need to align their worldviews first; they just need to promise in advance:
After reality changes, policies must also change.
Intervention should not only have two settings: continue or shut down
Truly executable AI governance should not directly switch between "completely free development" and "global shutdown."
A more reasonable approach is to establish graded interventions.
When risks are low, increase capability disclosures, accident reports, and third-party testing.
As risks continue to rise, restrict model access to critical infrastructure, limit high-risk permissions such as autonomous execution, fund transfers, and weapon controls.
If AI has clearly automated AI research and development or multiple severe signals appear simultaneously, then restrict new cutting-edge training and large-scale computing power expansion.
Only when lower-intensity measures cannot control risks should a broader and shorter pause be initiated.
This graded design is not intended to weaken regulation but to match the strength of policies with the strength of evidence.
Truly dangerous institutions are often not incapable of taking hard measures but can bypass all intermediate steps to obtain unlimited power directly under the pretext of "state of emergency."
Conclusion
The real difficulty in AI governance is not the lack of positions but the absence of an institutional framework that accommodates erroneous judgments.
Continuing competition is not a neutral choice.
A global pause is also not a neutral choice.
Open models are not inherently safe, and closed models are not inherently responsible.
A credible governance mechanism must at least answer four questions:
On what evidence does it act?
Does it equally constrain those in power?
Can it be externally verified?
After erroneous judgments, can power be revoked?
In the era of AI, perhaps we do indeed need a pause button.
But the real challenge of the pause button has never been whether it can be created.
But rather:
Who has the authority to press it, who has the authority to restart it, and how can we ensure that no one can turn this button into their throne.
免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。
