A 27-year-old AI researcher resigned from Anthropic because he is worried that AI could lead to human extinction.

CN
PANews
Follow
1 hour ago

Author: Digital Life Kazi Ke

Yesterday, I saw a post on X that had already surpassed 100 million views.

ImageTo be honest, I haven't seen such a post with this kind of traffic on X for a long, long time.

The person who posted is named Jacob Coxon.

He just resigned from Anthropic.

He said that in the past three years, he worked on large model pre-training at OpenAI and Anthropic.

And now, he has quit.

The reason is also quite absurd.

Because he feels that OpenAI and Anthropic are racing towards a self-improving superintelligence.

And betting with everyone's life.

He even said that many of the people actually involved in creating these cutting-edge AIs privately believe:

If things go out of control, humanity could go extinct before this decade ends.

Image

My first reaction was actually:

Who is this guy? Is he another marketing account…

Then I verified it and found out that he is indeed real.

This guy is 27 years old this year, British, with a background in mathematics.

In 2016 and 2017, he consecutively entered the UK International Mathematical Olympiad team and later studied mathematics at Cambridge University's Trinity College.

In 2021, he participated in a medical statistics paper published in eLife, using Bayesian methods to study the relationship between genotypes and survival rates in patients with tuberculous meningitis.

Around 2023, he joined OpenAI, and Jacob Coxon was on the official list of contributors to GPT-4o.

By the time of GPT-4.5, OpenAI even directly listed him as a core contributor.

And then this year.

He went from OpenAI to Anthropic, working on core pre-training.

However, after working there for a short time, feeling that it was not sustainable, he quit and then posted these warnings on X.

Even the Wall Street Journal reported that he mentioned wanting to leave the AI industry for good.

After this post was made, the alignment lead at Anthropic directly came out to support Jacob.

He said the young man is right, and the probability of it happening within ten years is greater than 10%.

Another person from Anthropic, who works in safety research, also came out to support Jacob “as an individual.”

According to his observation:

In AI companies, the higher the position, the more people tend to worry about this issue.

Even Joe Benton, who just left Anthropic and previously led scalable supervision, said Jacob's description of the industry situation is indeed accurate.

Of course, these are still just their judgments about future risks.

No one knows if 10% is right or not.

However, after reading the content they published and many comments, I suddenly felt a sense of absurdity.

This absurdity comes from an article I read 11 years ago.

And until today, the impression is extremely deep; in fact, I still read this article once every year, it feels fresh every time.

This is a long, mind-boggling article written by Tim Urban in January 2015:

The AI Revolution.

It has two parts, the first part is titled:

The Path to Superintelligence.

Image

The second part is titled:

Our Immortality or Extinction.

I still remember, in 2015, as a student, after reading the translated versions of these two long articles on Zhihu, the massive impact they had on my youthful mind and superficial understanding.

I was deeply shocked for a very long time.

Because that article ultimately made an extremely simple, extremely brutal, yet extremely grand judgment:

If humanity ultimately creates a superintelligence far beyond itself.

It will only push human civilization forcefully towards two directions:

One is extinction.

The other is immortality.

There is no other solution.

Then yesterday, after Jacob posted that post, I pulled up this article again and read it.

I suddenly felt.

It seems we really are standing on the timeline highlighted in that article.

There’s even a feeling that everything is destined.

Tim Urban first talked about a particularly classic thing.

That is, the human species has a hard time truly understanding exponential growth.

Because our brains naturally prefer linear reasoning.

Today is a bit more than yesterday.

Tomorrow probably a bit more than today.

But the development of civilization is often completely not the case; this is often the picture of civilization.

Image

Standing at that turning point may seem stimulating.

But you must know, people standing on the timeline cannot see what is on the right side.

So this is the true feeling of standing here:

Image

What a cruel reality.

Tim Urban also gave a particularly interesting example.

Imagine you time travel back to the year 1750, randomly grab a person, and bring him to 2015.

Let him see airplanes, highways, smartphones, the internet, satellites, nuclear weapons, space stations, and other high-tech things.

This person might think he directly entered a fairyland and might be so shocked that he would wet himself.

But if the person from 1750 also time travels the same 265 years forward.

And grabs a person from 1485 back.

At this point, the shock experienced by the person from 1485 might be much smaller.

Going further back, to experience the same civilization disparity, it might take thousands of years.

Tim Urban gave a fun name to this kind of civilization progress that is enough to cause ancient people’s brains to crash:

DPU.

Die Progress Unit.

The general meaning is, “a technological advancement significant enough to shock ancient people to death.”

Before, accumulating one might take 10,000 years.

Later only a few hundred years.

Even later, multiple decades, and then, increasingly faster.

When I first read it in 2015, I didn't feel very deeply.

But looking back today in 2026.

Emmmmmmm.

It seems our generation has already begun to personally experience this.

In 2022, ChatGPT came out, and everyone found that AI could actually chat like a person.

In 2023, GPT-4 came out, and we discovered this is called multimodal large models.

Then there were reasoning models represented by o1.

Claude Code.

Codex.

Agent.

One after another.

Overwhelming.

By last week, GPT-6 Astra could operate a computer like a human, open Blender, Houdini, Unity, Excel, find buttons in unfamiliar software, trial and error, and keep working for several hours.

Let alone in academia.

In May, an OpenAI model directly overturned an Erdős discrete geometry conjecture studied for 80 years.

On September 4, Anthropic took Claude 11 days to convert Fermat's Last Theorem into a formal proof of 13 million lines, which humans initially estimated might take ten years.

And the recently controversial, OpenAI's limit-solved millennium problem in 88 hours.

This world seems to become increasingly sci-fi.

Of course, the core is still another thing, which is also what Jacob worries about the most: "self-improving superintelligence."

In 2015.

Tim Urban asked:

What happens if AI someday starts helping humans research stronger AI?

In February 2026.

When OpenAI released GPT-5.3 Codex, they said:

This is their first model that plays a key role in its own creation process.

Nowadays, almost any slightly more powerful model will involve AI participation in the training phase.

This feeling is truly fascinating.

It's as if eleven years ago someone shot a virtual line through the fog.

At that time, no one knew where that line would lead.

Eleven years later, we look down.

We are actually walking on it.

There’s also something from that article that I still remember today.

The Intelligence Ladder.

Tim Urban said.

Humans tend to underestimate superintelligence.

Because we habitually understand "smarter" as:

Ordinary people, smart people, geniuses, Einstein, super AI, and so on.

It seems everyone is within a very narrow range.

But if you change the reference point.

Things suddenly become very terrifying.

For example, chimpanzees and humans.

From the whole biological intelligence scale, chimpanzees and humans are actually only separated by a little bit.

Image

But just that little bit.

One side can only poke termite mounds with sticks or scratch trees.

While the other has invented calculus, quantum mechanics, the internet, etc., and even can fly out of Earth.

So if there is something above humans, which has one or two cognitive levels higher than us, it may already be very difficult to comprehend.

But what if it goes even further?

Tim Urban wrote back then.

A superintelligence of sufficiently high level looking at us.

Would probably be like how we look at ants.

Image

Even if it wants to explain to us a very simple concept, we might still lack the cognitive structure needed to understand that concept.

This is also why that article will certainly lead to:

Intelligence Explosion.

Intelligence explosion.

If one day, AI really reaches the level of top AI researchers, then one of its most valuable tasks.

Quite naturally would be:

To continue researching AI and help humans make the next generation of models smarter.

After the next generation is smarter, it will be better at researching the next generation.

Thus.

Intelligence becomes stronger.

The ability to create stronger intelligence also becomes stronger.

Then it forms a positive feedback.

This thing in 2015 seemed.

Really very sci-fi.

But in 2026, I can no longer purely regard it as sci-fi like ten years ago.

Because in various model factories, the most primitive feedback loop has already formed.

Of course, today we are still far from true recursive self-improvement.

GPT-6 will not create GPT-7 by itself, and then GPT-7 will not wake up in the middle of the night to train GPT-8.

But the earliest few links of that logical chain can already be seen.

As a result, I feel a very strange sense of fate.

It seems that as long as humanity really plants the seed of “intelligence,” the subsequent developments naturally acquire a tremendous inertia.

Smarter AI can create greater economic value.

Greater economic value attracts more capital.

More capital buys more GPUs.

More GPUs train stronger AI.

Stronger AI begins to help humans create even stronger AI.

Companies are still competing.

Countries are still competing.

If you don’t do it, others are doing it.

If you don’t move forward, it can’t be helped; others move forward, can you not run?

In the end, the entire civilization flows like a river that becomes increasingly steep.

Everyone knows the water flow is getting faster.

But no single person or group can truly steer the course of the entire world.

And what has truly stuck with me from Tim Urban's article for eleven years.

Is that picture at the end of the water flow.

He cited a concept from Nick Bostrom.

He said, throughout Earth’s history, countless species have appeared and almost all have disappeared.

Extinction seems to be an attractor state; once you fall into it, you can never return.

But superintelligence might be the first to open another kind of attractor for life.

Species Immortality.

Species immortality.

Tim Urban drew a balance beam.

Humans have been walking on it; as long as you keep walking, someday.

A meteor, a plague, war, a certain technology we created ourselves, etc.

Could potentially knock us off this balance beam.

Image

And after superintelligence appears.

We might experience the most severe impact.

One side is the end familiar to all species.

Extinction.

The other side.

However, is the place that Earth's life has never reached before.

Immortality.

This is the part I find most captivating about that article from 2015.

It compresses almost all questions regarding artificial intelligence down to the scale of civilization.

Today we discuss every day.

Will AI take programmers' jobs.

What about designers.

Which company may win, and so on.

But if we extend the time sufficiently long, can this greatest gamble of humanity, AI, allow our species for the first time to free life from the ultimate rule of death?

But if we lose the gamble, there is only one outcome.

Extinction, even accelerating extinction.

So seeing this, you might understand why in the beginning, Jacob Coxon, after studying AI for so long, would resign and post such a message on X.

Image

Of course, I can also understand.

Why so many people, despite their worries, still choose to remain in the lab to continue researching.

Because the other side of this thing is indeed too tempting.

AI has already begun to participate in mathematical discoveries.

Started entering drug development, started analyzing the human genome, started aggressively researching new materials, started operating experiments and software, began accelerating everything in the human world.

If this curve keeps going forward.

Give it 3 years, 5 years.

What will happen?

I don't know.

Perhaps no one knows.

But sometimes I feel that humanity's creation of AI.

Has slowly exceeded the scope of an ordinary technological product.

It resembles a species that attempts for the first time to create something smarter than itself.

Is this the greatest step in human civilization?

Or is it the last black ball of humanity that we drew from the jar?

I don’t know.

I am just a leaf drifting in this era.

I use AI every day, feeling the convenience brought by AI, thinking that AI is just too fun.

And then occasionally, like yesterday, I reopen an article from eleven years ago.

Only then do I suddenly raise my head up from the daily trivialities of Benchmark, Token, Agent.

To see where this river is really flowing.

This question is too big.

But if I had to choose.

Of course, I hope that Tim Urban's balance beam.

Ultimately knocks us to the other side.

I hope cancer truly disappears one day.

I hope aging becomes a disease that can be treated.

I hope the people I care about do not have to leave.

I hope humanity can live for a thousand years.

I hope we can leave the solar system and head to the distant Alpha Centauri system.

I hope someday.

Those articles today about AI destroying humanity.

All become historical blame undue alarm.

If in the end.

AI can genuinely help humanity cross that balance beam that no species has crossed for millions of years.

Allowing life to finally break free from the constraints of disease, aging, random disasters, and death.

Then it will undoubtedly become:

The greatest thing in all of human history.

Without exception.

免责声明:本文章仅代表作者个人观点,不代表本平台的立场和观点。本文章仅供信息分享,不构成对任何人的任何投资建议。用户与作者之间的任何争议,与本平台无关。如网页中刊载的文章或图片涉及侵权,请提供相关的权利证明和身份证明发送邮件到support@aicoin.com,本平台相关工作人员将会进行核查。

Share To
APP

X

Telegram

Facebook

Reddit

CopyLink