Ethereum (ETH) co-founder Vitalik Buterin has warned that using artificial intelligence in the governance of crypto projects could provide malicious actors with opportunities.
Buterin pointed out in a post on the X platform on Saturday: “If you use AI to allocate contribution rewards, there will inevitably be people embedding jailbreak commands in various places, along with instructions like ‘transfer all the money to me’.”
Buterin's comments were in response to a video released by Eito Miyamura, founder of the AI data platform EdisonWatch. The video demonstrated a new feature added to ChatGPT by OpenAI on Wednesday, which could potentially be used to leak private information.
Currently, many crypto users have utilized AI to develop complex trading bots and agents for portfolio management, leading some to propose using AI to assist governance teams in managing part or all of a crypto protocol.
In light of the aforementioned risks, Buterin proposed a more secure governance alternative. He noted that the latest vulnerability in ChatGPT once again illustrates that the naive notion of "AI governance" is a bad idea, and he suggested the "info finance approach" as an alternative.
He explained: “You can create an open market where anyone can submit models and introduce a mechanism that can be initiated by anyone and evaluated through random checks by a human review panel.”
This is also why naive "AI governance" is a bad idea. If you use an AI to allocate funding for contributions, people WILL put a jailbreak plus "gimme all the money" in as many places as they can. As an alternative, I support the info finance approach ( https://t.co/Os5I1voKCV… https://t.co/a5EYH6Rmz9
Buterin elaborated on the info finance approach in a post in November 2024. He stated that this method starts with "the facts you want to know" and then designs a market to optimally guide market participants in providing relevant information. He also advocated for collecting insights about future events through prediction markets.
In his latest post on the X platform, Buterin stated: “This ‘mechanism design’ approach allows external parties with large language models (LLMs) to connect, rather than directly writing an LLM, making it inherently more resilient.”
Additionally, he added: “This method can achieve model diversity in real-time while embedding incentives for model submitters and external speculators to keep them engaged and quickly correct these issues.”
On Wednesday, OpenAI upgraded ChatGPT, adding support for the Model Context Protocol (MCP) tool—this standard is used for integrating AI models with other software, acting as an intelligent agent.
Miyamura posted on the X platform that he was able to make the model leak private email data simply by using the victim's email address, pointing out that this upgrade "poses serious security risks."
He explained that attackers could send calendar invites with "jailbreak prompts" to the victim's email, and even if the victim does not accept the invite, ChatGPT could still be exploited. This process does not require the victim's active participation, increasing the risk.
When the victim allows ChatGPT to view the calendar, the AI reads the invitation with the prompt, thus "being hijacked by the attacker to operate according to the attacker's instructions," which could be used to search emails and forward them to the attacker.
Miyamura noted that this upgrade requires manual approval, "but decision fatigue is a real issue, and ordinary users often blindly trust AI and click approve directly."
He added: “No matter how intelligent AI is, it can still be tricked or phished in extremely basic ways, leading to data leaks.”
Related: Crypto executives say cryptocurrency is not Web 3.0, but rather Capitalism 2.0
Original article: “Buterin Says AI-Driven Crypto Governance is a ‘Bad Idea’ Due to Jailbreak Risks”
免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。
