Anthropic CEO's extensive interview: After AI becomes a super weapon, how to find a balance between business and security?

CN
PANews
Follow
2 hours ago

Source: Bloomberg

Compiled by: Felix, PANews

Bloomberg recently interviewed Anthropic CEO Dario Amodei, discussing his entrepreneurial journey in San Francisco, the competition with OpenAI, and the ultimate goal of AI.

PANews has compiled the highlights of the interview.

Host: You are currently at the center of the AI universe. What does that feel like?

Dario: Throughout my career, especially during this period at Anthropic, I have experienced a smooth exponential curve. This experience of a smooth exponential curve is: nothing happens, nothing happens, a little bit happens, and then "whoosh," it grows wildly. This is the experience of the world and also the experience of a company’s scale compared to others and the world. I have been observing this chart for a long time. I once said, "We will probably become the AI company with the highest revenue and valuation at this time," and this has indeed happened. So on one hand, I am not surprised because the chart shows a smooth line; but on the other hand, when things actually happen, you see more details and colors, which is absolutely surprising. We must always remember those fundamental questions: How do you train good models? How do you put them into good products? How do you ensure safety? How do you help people while managing the societal risks that technology brings? All these are the same questions, just looked at under a larger microscope.

Host: You grew up in San Francisco. What was it like as a child? I know your father was a leather craftsman and your mother worked in a library. How did that affect you?

Dario: The first internet revolution was happening around me at that time, but I was completely uninterested in it. I was only interested in doing math problems, doodling, understanding the universe, and science fiction. That was my overall environment, and I was curious about the world.

Host: You grew up in this tech-centric city, and now it is the center of AI. Did this city influence your worldview?

Dario: The pervasive nonconformity, individualism, and the spirit of "it's okay to be crazy" here have indeed influenced me. If you go to Europe or even to other parts of this country, you will find that thinking differently or having some crazy ideas is not encouraged and is even considered strange. Although I have many criticisms of Silicon Valley, I think there is one good thing: it doesn't matter if all the experts oppose you; if you have a coherent vision of the world, you should pursue it. It may completely fail, but if successful, it will create a long-tail effect, and you may discover a huge goldmine in certain veins. I think this spirit is very important.

Host: Your decision to leave OpenAI has become legend in Silicon Valley. What exactly happened? Besides the rumors from the outside, what were the real issues? Where did you have disagreements?

Dario: To put it simply. When you are developing powerful technology, you face many difficult issues. Anthropic faces these dilemmas every day, and we do not always know whether our decisions are right or wrong. So there are many reasonable disagreements on safety issues, and we indeed have some such disagreements with them, but that alone is not enough reason to leave. Even at Anthropic, there are disagreements among people with me and among people themselves. However, when you feel that you cannot trust someone, when you think their values are not as they claim, when you feel they are not honest and are not acting out of the motivations they profess, when you see disturbing patterns of deception, it is hard to continue to cooperate with and trust the company. Ultimately, if your vision is different from others and there is mutual distrust, why should you argue? The solution is to go your separate ways. I can completely accept that we do things our way, and they do theirs. Let's see who wins in the market and who wins in public opinion. This is more compelling than any dramatic gossip about who left.

Host: At the AI summit in India, you and Sam Altman seemed to refuse to hold hands on stage. What happened there?

Dario: The reality is that the summit was extremely disorganized. We were all called to the stage at the last minute, they changed our standing order suddenly, took a photo, and then suddenly ordered all of us to hold hands. If you have attended international summits with heads of state, you will know they are often super chaotic (this is not aimed specifically at India).

Host: But everyone else was holding hands.

Dario: I don't know how to explain it to you. Narendra Modi suddenly called everyone to hold hands on stage.

Host: Well, Sam and Elon are suing each other. You don’t like Sam. If the most important technology builders in the world can’t hold hands on stage, how can we believe you will cooperate on risks that concern human survival?

Dario: What I want to tell you is that the people building this technology differ greatly in quality and credibility. The notion that "no one trusts anyone" is simply wrong. I have known Demis Hassabis, who developed the Gemini model, for 15 years; they are competitors of the Claude model, but we have collaborated on many issues. We buy computing power from Google and often exchange ideas about safety. So I think, first, some participants are indeed more trustworthy than others; second, there are participants outside of Anthropic whom I trust. What needs to happen is for trustworthy participants to unite and put untrustworthy participants in a position where they have to adopt the same standards. In my experience, some people won't take the initiative to do the right thing, but if the majority in the industry are doing the right thing, others have little choice. This has two aspects: the positive side is mutual inspiration (i.e., upward competition), such as how Demis’s Alpha Fold inspired us; our work in biological research and interpretability research has also inspired them, and this is not zero-sum competition. This is the positive "carrot" aspect of the game; while the "stick" aspect is that if you don't do the right thing, you'll look terrible. They may be very reluctant to do the right thing while pretending to be doing something else and even feel we have some ulterior, sinister motives, which is to be expected, but this is precisely how the industry can unite and collaborate.

Host: Early on, others focused on fun, attention-grabbing consumer applications while you put your bets on coding and enterprise applications. Claude Code and Claude Cowork have been very successful. Why make this bet? Was it a value-driven decision or a business decision?

Dario: When we founded Anthropic, the fundamental point was that we wanted to do things the right way. But to fund the extremely costly creation of models, you need a business model. If your business model fundamentally conflicts with your values, you will have a hard time: either betray your values or become irrelevant. Choosing a business model that is compatible with your values is much better. We have seen the social media and consumer world, which often encourages user engagement or even addiction (like the mess we see with AI video models), just to maximize user retention for ad revenue. In contrast, on the enterprise side, we want to use AI to cure diseases that could not be cured before (in collaboration with biotech, pharmaceutical, and academic institutions), make energy cheaper and more efficient, help education and nonprofit organizations, and promote economic growth. These are essentially enterprise applications. Another factor is that enterprises value trust and long-term relationships highly. The consumer side can sometimes be gimmicky, while the enterprise side requires you to build trust over years and deliver what you promise. This aligns very well with our goal of deploying models in a positive and safe manner.

Host: A developer can switch from Claude to ChatGPT or Gemini in an afternoon. Is it really possible to maintain long-term superiority in this industry? How long would it take a powerful competitor to replicate what you have built?

Dario: Model quality is the most important thing. We are currently far ahead in model quality. Although there is some switching inertia, I (and Anthropic) have never relied on the "stickiness" of the product to prevent people from switching. You need to have a better model, a better product. We see that the growth rate has not turned down at all. At least at the time of this interview recording, if there is any change, it is that it is still rising. So I tend to think that model quality is the most crucial aspect.

Host: Shortly after the release of Claude Cowork, $285 billion in market value evaporated overnight, with traders calling it the "SaaSpocalypse." If AI continues to evolve at this speed, how much traditional software will be replaced, and how fast?

Dario: It’s hard to predict accurately in advance. All traditional software companies have their respective moats. Some of these moats will disappear, while others will remain. The moat of "quickly writing complex software" will certainly disappear; you can no longer rely on that defense. However, customer relationships, knowledge of industry operations, and unique domain expertise are still important. I advise these companies: don’t get complacent, list all your moats, and see which will disappear and which will strengthen because new moats will also emerge. Those who adapt flexibly will do well, while those who deceive themselves and are satisfied with past experiences will have a hard time. Overall, I believe the software industry will become larger rather than smaller, although there will definitely be some big losers.

Host: Explain this.

Dario: Because the "cake" is getting bigger. If AI raises the ceiling of the industry tenfold, existing businesses can easily achieve 1.5 times growth, even though their relative share within the larger cake shrinks. However, those who do not adapt and bury their heads in the sand will experience hard times or even go out of business.

Host: Your biggest supporters are companies like Amazon, Google, Microsoft, and Nvidia. They all have their own agendas and are both partners and competitors. Your huge business milestones are tied to financing—who is really in control?

Dario: In many cases, we speak candidly about our true thoughts. We are all adults, and we can seek common ground while having differences on certain issues.

Host: Reports have suggested your valuation is higher than OpenAI. For a startup established five years ago, a valuation nearing a trillion dollars is astonishing. How do you interpret this number? If you’re more disciplined with computing power and have a faster path to profitability, why do you need so much money?

Dario: The demand for computing power is growing rapidly. The business fundamentals are good, but you may need three to four times the computing power within a year; the growth is extremely fast. We fully expect revenue growth to reach or even exceed this increase in computing power. Raising funds is to cushion the impact of this uncertainty, which is very rational, and the dilution of equity for the company is quite small; this does not indicate any issue with the business fundamentals.

Host: Does it feel good to surpass your main competitors?

Dario: We still have many tough challenges ahead. We are promoting the idea of "competing upwards," attempting to pull other companies alongside us. Although sometimes they do not acknowledge it and attack us while copying us, this pull effect is incredibly valuable. The value of being a leading company lies not merely in defeating opponents but in the ability to drive the entire ecosystem forward.

Host: But winning must feel somewhat rewarding.

Dario: We are certainly always striving for success and don’t want to fail. I don’t think we should shut down this technology; we compete within a free enterprise system, we just need to mitigate the risks of the models. We are always looking for a balance between the extremes.

Host: Your product development speed is astonishing. How are you achieving this?

Dario: Two reasons: first, we have a unified culture and organizational efficiency, and everyone is on the same page; second, it is Claude itself. We are currently using Claude to help develop our models and products, making them more efficient. This creates a lot of increasingly reliable acceleration effects.

Host: What is the craziest thing you have seen AI do?

Dario: It has all been about biology and medicine. I have seen Claude diagnose medical issues that several senior doctors missed. In biology, the model has begun to become astonishingly excellent in drug design or computational chemistry. As a former biologist, I know it requires a high level of specialized training. This is the immense positive aspect of AI; it will greatly enhance the quality of human life. Imagine a hundred years of scientific progress from 1900 to now, then imagine what another hundred years of progress would look like. If we can successfully navigate the challenges we face now, we will have a much better world.

Host: I know you enjoy writing. Do you use Claude to help you write?

Dario: Yes, but I haven’t let it directly write text for me yet, as I am somewhat particular about specific styles. I mainly use Claude for brainstorming, clarifying topics, or providing references. It currently plays a supportive role; although it’s not yet at the level of writing better than me, that day will come.

Host: Writing helps people organize their thoughts and involves a lot of critical thinking. If we let Claude take over, will we lose that capacity for thought?

Dario: I do worry about that a bit. I insist on writing myself, partly for the readers, but also to clarify my thought process. If I were to simply ask it to "write an article about AI risks," it would not capture what I truly want to express, and it would also deprive me of the benefits of that thought process. In the future, as the models become better, we may need to find a subtle way to use them that allows for more direct utility while retaining the cognitive benefits.

Host: You have openly discussed unemployment, predicting that AI could eliminate half of entry-level white-collar jobs within 1 to 5 years. That was a year ago; is that figure still 50% or higher now?

Dario: If you look at the original full video, my true statement was: I don’t know what will happen, but that represents a scale of how crazy things may get. However, people’s minds tend to clip that three-second "the end is near," while ignoring all the solutions I explored at the same time (like token taxes, corporate adjustments, macroeconomic policies, etc.). My message is by no means "the end is near," but rather that we need to see, pay attention to, and actively respond to it in advance. In the short term, AI will lead to higher efficiency for people, but this is just a transitional stage. Just as we saw in the history of automation, AI might ultimately take over this part of the work. Taking Anthropic’s software engineers as an example, AI now helps them write most of the code, making them more efficient, but there also begin to be situations where it’s directly better to have AI carry out specific tasks. On the other hand, this also spurs new demand, such as for "application AI solution architects (frontend deployment engineers)" who have both technical and customer communication skills. Job destruction and adjustment will happen simultaneously.

Host: You released a chart showing potential job destruction, which jobs would disappear, and which would emerge?

Dario: It is hard to predict accurately, like predicting the stock market. But overall, the possibility of entry-level white-collar positions (in banking, finance, etc.) being replaced by AI is very high. When we communicate with business clients, we find that they face a choice: do they lay off workers to save costs or utilize the same resources to do more new things? We always try to push them towards a "positive-sum game": using efficiency gains to do more rather than laying off workers. As the "cake" expands, people may find new places, which is a matter of finding matching speed. The scale of destruction will be large, which is why I try to warn people.

Host: Isn’t such a high-scale unemployment rate a reason for revolution to break out?

Dario: This is definitely a result we want to avoid. I believe there will be opportunities preserved in several areas: one is the physical world (as the development of robots lags behind the AI information processing speed, manufacturing and construction still require a lot of manpower); the second is all human-centered jobs. Even if AI is more accurate than doctors in diagnosing or performs customer service better, people will still wish and need to talk and connect with other humans on important matters. Humans will also retain the role of guiding AI's values and intentions.

Host: Many people have opposed your views, such as Jensen Huang stating that you confuse "task" and "job," and others claim this is a "doomsday marketing" strategy beneficial to Anthropic.

Dario: I want to explicitly refute this. We elaborated for five entire pages in our reports and papers on the differences between tasks and jobs, explained why this time is different, and proposed six solutions covering both private and government sectors. But I find that the shortcomings of Silicon Valley and social media are laziness; people irresponsibly comment after only watching that three-second clip from a year ago. Calling serious intellectual labor "cheap marketing" is itself a cheap marketing strategy that rejects serious discussion and is extremely lazy.

Host: As a leading global AI company, you are deeply involved in many areas of U.S. national security. You have had a long-standing anti-war stance since your time at Caltech, yet you were one of the first AI companies to sign classified network operation contracts with the Department of Defense. Please explain.

Dario: The reason is that the world is changing. When I saw Russia invade Ukraine, I was worried about the resurgence of authoritarian groups. We need to defend ourselves. We absolutely do not want our adversaries to use AI to analyze intelligence, attack Ukraine, while we cannot defend ourselves. So we are providing support across government agencies. By the way, we are not doing this for money; operating on government networks is actually a hassle, and the profits are low. The reason we are doing this is out of "care," so there must be clear limits and red lines on the use of technology: no large-scale surveillance and no development of fully autonomous weapons. If we were to abandon our democratic values to win competition, it would not be worth it. This is the balance and stance we uphold.

Host: You will start a partnership with Palantir in 2024; their technology is used by U.S. Immigration and Customs Enforcement, police departments, and even in Gaza. Is Claude being used for surveillance?

Dario: We do not cooperate with ICE and CBP, either directly or through Palantir. As far as I know, we are not conducting business in Gaza either. We are very cautious to limit our participation to within the values we endorse.

Host: So you have drawn red lines, the president has suspended some of your federal government licenses, and the Pentagon has labeled you as a supply chain risk, while OpenAI has stepped in and signed contracts you were unwilling to sign. What does it feel like to win this struggle?

Dario: I don’t think this is a struggle of "winning or losing"; it’s a social debate on how the government should rationally use AI. We are striving to establish precedents in concerning scenarios and have raised attention on the issue. We have seen bipartisan efforts in Congress attempting to prohibit high-risk applications and set up guardrails. If you must say winning means prompting our country to think more cautiously about the appropriate boundaries for new technologies, that is victory.

Host: Some describe you as an ideological fanatic who should not hold such power, or a left-wing zealot. Does that bother you?

Dario: I often get criticized, and it is much worse than that. What others call me and Anthropic does not matter; I only care about two things: the success of the company and that we uphold our values. In fact, this simplifies my life; I have always been clear about my stance.

Host: Is the AI war more likely to prevent World War III or trigger it?

Dario: Overall, it is more likely to prevent it. However, without usage limitations, it could trigger wars. If there is inadequate regulation, both sides could easily misinterpret and escalate into conflict. If we have an overwhelming advantage in intelligence gathering and response capabilities, adversaries will think twice before taking action. Superior intelligence can effectively deter conflict.

Host: Anthropic makes headlines nearly every week, most recently regarding the Mythos model. You have said that Mythos is too powerful to be released to the public. What surprised you the most about it?

Dario: What surprised me the most is that it can not only discover vulnerabilities but also convert those vulnerabilities into tangible attack tools, marking a significant leap in capability. Some companies testing it early even requested that we not release it and referred to it as a "superweapon," claiming it would require a gun permit to use. Our decision not to release it to the public is not to lock it away forever but because the current network defense mechanisms are still easily "jailbreakable." We will wait until the defense mechanisms are confident enough to handle it, and then release it along with strong protective measures.

Host: But there is a lot of opposition from the outside: some researchers claim it can be replicated with cheaper open-source models, while others say this is entirely grandstanding and a marketing ploy. How do you respond?

Dario: The assertion that it can be replicated with open-source models is extremely ridiculous. The Mythos model discovered 271 new vulnerabilities in the Firefox browser and thousands of new vulnerabilities in many private companies that cannot be publicly disclosed, which previous models could not identify at all. The person on Twitter who claimed it could be replicated with open-source models simply repointed the line of code we identified as problematic after we pointed it out; this is not the same thing at all.

As for the claims of "marketing," not releasing this super powerful model has caused us significant commercial losses; it could have greatly accelerated both internal and external R&D. If this helps defenders, it equally helps attackers. So our current strategy is to first give it to defenders to patch system vulnerabilities. Once all the "holes" are patched up, the future internet ecology will become extremely safe. Those engaging in sniping and reverse operations on X (including certain competitors), failing to seriously consider the societal risk trade-offs, reflect an extremely immature mindset.

Host: Have you made any trade-offs that you are not fully satisfied with?

Dario: The entire history of Anthropic is filled with trade-offs. In an ideal world, you might spend years researching every single concern about chatbots before releasing them. While we did delay the initial version of Claude for several months, everything is still a trade-off. Now that we are in a leading commercial position, Daniela and I are doing everything we can to tilt the balance further towards "proceeding with caution." This is the intent behind limiting the release of Mythos; if you are not a leading player, it is hard to make such a decision.

Host: Since this technology is so powerful, why doesn't the government take you over directly?

Dario: This is a very serious concern. Throughout history, all unprecedentedly powerful technologies (nuclear weapons, the internet, GPS, mobile phones) were initially born in government and federal laboratories. AI is the first technology fully built in the private sector with government involvement coming late; this situation is actually dangerous and unstable. But the risks of the government not engaging are indeed greater than those posed by the private sector. Therefore, we need checks and balances: we have set up a "long-term benefit trust," which has the power to remove most board members and even fire me, introducing elements of public governance.

At the government level, we need legislative and judicial branches to implement mandatory pre-release testing and red line regulation. I fear both that companies will dominate it and that governments will monopolize it; both sides must balance each other out. It is quite ridiculous that there is a group of people in Silicon Valley who, from extremely opposing any regulatory or transparency requirements (claiming it stifles innovation), instantly swing to declaring "the government should nationalize it" when they see real danger for the first time. This reaction of swinging between extreme poles is very immature; what we need is a rational middle ground.

Host: There is frequent talk in your field about the moment when AI becomes good enough for "self-improvement." Do your researchers think that moment is approaching, and if so, how far away is it?

Dario: I think it is not a specific "moment," but a continuously accelerating exponential process. We have already seen AI starting to assist in proposing architectures for the next generation of AI. A year ago, AI increased total factor productivity by 10-15%; now it could be 20-30%. It will not suddenly spiral out of control at a certain point, rather, on this smooth exponential curve, we need to evaluate at each point whether we should slow down and impose more controls. The rational response is to smoothly and gradually upgrade our countermeasures as the power of technology grows, rather than panicking.

Host: You once said that there is a 10-25% chance of human civilization collapsing. That is not trivial. Is there a possibility that something manufactured by Anthropic could lead to that collapse?

Dario: I certainly hope not. This risk probability stems from the inherent unpredictability of technology and the complexity of the world situation. The various measures we are taking are precisely to reduce, not increase, this probability of collapse. Just like in aviation, even if your planes are ten times safer than others, you cannot absolutely guarantee to others that they "will never crash."

Related Reading: OpenAI and Anthropic collectively backtrack: Is the doomsday theory of AI employment no longer popular?

免责声明:本文章仅代表作者个人观点,不代表本平台的立场和观点。本文章仅供信息分享,不构成对任何人的任何投资建议。用户与作者之间的任何争议,与本平台无关。如网页中刊载的文章或图片涉及侵权,请提供相关的权利证明和身份证明发送邮件到support@aicoin.com,本平台相关工作人员将会进行核查。

Share To
APP

X

Telegram

Facebook

Reddit

CopyLink