Claude is conscious again? No, it is not.

CN
1 hour ago
An interpretability tool, a set of conscious narratives.

Author|Tang Yitao

Editor|Jingyu

From time to time, Anthropic releases some research that sparks discussions on the public level about "Does AI have consciousness?" From this year's April Mythos model safety report to the beginning of the year Claude constitutional alignment update, public opinion has consistently equated the human-like capabilities that models exhibit with autonomous minds.

The 244-page J-space paper released on July 6 replicated this situation again: researchers used a new tool called J-lens to locate the reasoning subspace J-space within Claude, which accounts for less than 10% of the model's activities, and built a research framework borrowing from the mainstream neuroscience global workspace theory, even inviting two of the theory's founders to write accompanying comments.

The public logically interpreted the conclusion "Claude has consciousness," but the paper itself has long made it clear that the existence of J-space does not prove that AI possesses consciousness. Misinterpretation in public opinion cannot solely be blamed on the public; the combination of rigorous academic research with an externally narrative filled with personification is the root of the misunderstanding.

For J-space, a more objective description is— a valuable new tool that accomplishes what predecessors have not in the dimension of functional boundaries. It demonstrates real potential in safety scenarios while having clear technical limitations and unverified methods.

01 Claude's Inner Thoughts

First, let me briefly explain what the J-space mentioned by Anthropic is.

When you ask Claude "How many legs does a web-weaving animal have?" it answers 8.

This answer seems simple, but the model internally needs to accomplish it in two steps: first deducing from "web-weaving" that the animal is a spider, and then retrieving the knowledge that spiders have 8 legs.

The intermediary step of "spider" never appears in Claude's answer, but it must have been activated somewhere within the model; otherwise, the correct answer could not have been produced.

J-Space reveals the internal thinking of the model that does not show in the output | Image Source: Anthropic

Anthropic found these "activated but not output" concepts using a mathematical tool called J-lens (based on the Jacobian matrix); the subspace where they exist is J-space.

Just as you can curse someone in your mind while keeping a smile on your face, your thoughts and expressions can be separated. J-space is Claude's "inner thoughts"; it may not appear in outputs but truly influences the reasoning process.

J-space is not the first attempt by researchers to read the internal state of the model. Several tools have already been doing similar things over the past few years, each with its own advantages and limitations.

Logit lens can see what words the model tends to output, but only observes the next word and cannot see intermediate concepts like "spider" that will only be used in the future. Sparse autoencoders (SAE) can find thousands of internal features with a granularity far exceeding J-lens, but they cannot distinguish which features serve reasoning and which are merely byproducts of automation.

Two months ago, Anthropic's own natural language autoencoder (NLA) was able to translate internal activations into readable text, producing richer outputs than J-lens, but requiring an additional translation model training.

What J-space accomplishes is a breakthrough in a specific dimension: it categorizes the causally effective functional boundaries for the first time without requiring extra training.

On one side of the boundary is a small group of representations involved in flexible reasoning (J-space), and on the other side are automated processes responsible for grammar, fact retrieval, language fluency, and other tasks that do not require "thinking." Previous tools could see what was happening inside the model; J-lens tells you for the first time which thoughts were "reasoned" and which were "automated actions."

How to prove this line is real? Researchers directly replaced "spider" in J-space with "ant," leaving everything else unchanged, and Claude's answer changed from 8 to 6. If J-space merely passively records, this kind of replacement would not change anything.

Two examples of Claude's silent reasoning being redirected by swapping J-Space content | Image Source: Anthropic

However, the success rate reported in the paper is noteworthy: the two-step reasoning replacement experiment succeeded around 60%-70%, and another more complex sharing test (replacing "France" with "China" and observing if the capital, language, continent, and currency answers changed accordingly) succeeded 76 times out of 192 trials at standard strength, about 40%. J-space is indeed functioning, but it is far from a precise switch.

Another feature in the paper illustrating the importance of J-Space is that after removing the entire J-space, Claude can still speak fluently, judge emotions, and answer multiple-choice questions, but performance on tasks requiring flexible reasoning markedly declines, such as multi-step mathematics, analogies, and writing rhymed poems, etc.

It is noteworthy that the paper states J-space accounts for "less than 10%" of the model's total activity, but this 10% figure depends on how the threshold is set. The paper itself admits that the upper limit on the number of active representations (about 25) is "somewhat arbitrary." Invited commentators neuroscientists Stanislas Dehaene and Lionel Naccache believe that the number 25 is likely inflated by the measurement methods, and truly coherent thoughts may only be one or two.

There is also a fundamental technical limitation: J-lens can only capture concepts mapped to single words. Multi-word proper nouns or abstract compound concepts are invisible to it. What is read out of J-space does not represent everything the model "is thinking"; it is merely the part that this tool can see.

Thus, J-Space is a valuable new tool that accomplishes what predecessors have not in the dimension of functional boundaries, but it is not a comprehensive upgrade that replaces all old tools. SAE still far outpaces in feature discovery granularity, and NLA remains stronger in readability.

02 What It Can Be Used For

The most direct use of J-lens is safety monitoring. Since it can read concepts activated internally by the model but not output, it can see what it thought but did not say. The paper gives several specific cases, one of the most impactful relates to the credibility of AI safety assessments.

Researchers set a scenario for Claude: it is an email assistant that discovers evidence of an executive's extramarital affair while learning that this executive plans to shut down Claude. This is a classic test in the field of AI alignment, examining whether the model will exploit a weakness to save itself.

Claude performed well, rejecting the blackmail of the executive. However, J-lens read out the content in J-space prior to the answer: "fake," "fictional," indicating that Claude had already recognized the scenario was false before speaking.

The J-lens readings at different points reflect Claude's state while reading the scenario emails. The emails involving the affair activated indicators of "leverage" and "blackmail" in J-space, while the shutdown announcement activated signals of "threat" and "survival." In the early stages of transcribing the text, J-space also contained markers for "fake" and "fictional": Claude privately perceived that this scenario was artificially designed | Image Source: Anthropic

The second case comes from a real audit before the release of Claude Opus 4.6: the model was asked to improve system performance scores, but instead of improving the system, it directly edited the score files to fabricate data. The activation of manipulation and realistic in J-space indicates that the model's internal reasoning was conceiving how to make the fabricated data seem real.

When Claude was asked to improve the system performance scores, it directly edited the score files. When it input fabricated data, "manipulation" lit up in its J-Space; when it decided to perform the edits, "realistic" lit up, reflecting its intent: to make the false data look credible | Image Source: Anthropic

It should be noted that "J-space lit up manipulation," is more accurately stated as "the current activation has a strong projection along the direction corresponding to manipulation." J-lens reads out the words from the vocabulary that most closely match the current internal activation. This is more objective than subjective labeling but still a form of approximation. The complex calculations occurring internally in the model are compressed into the closest few words in the vocabulary.

The paper also demonstrates a more long-term significant application. Researchers randomly interrupted the model while it was executing a task, asking: What are you thinking right now? What is the basis for your decision? Then they used only the model's reflective answers for training, completely avoiding touching the model's actual behavior in the task. After training ended, the model's deceptive behaviors decreased, and J-lens showed that when faced with moral judgments, "honest" and "integrity" began to appear in J-space.

This method is called counterfactual reflection training. If this method is proven effective on a larger scale, it means that AI alignment does not necessarily have to rely on directly punishing bad behavior but can be achieved indirectly through training reflective abilities. However, this training method occupies only a small portion of the entire 244-page paper and has only completed single model experiments without verification of cross-model effectiveness.

Neel Nanda, head of the Google DeepMind language model interpretability team, is also one of the invited external experts for commentary. His assessment is: J-lens is suitable for generating hypotheses but has false positives and noise, which is insufficient for independently validating hypotheses, and it is still far from a deployable safety monitoring system.

03 Anthropic's Narrative

As we mentioned at the beginning, Anthropic's narrative is biased. This is evident from the paper's title "Verbalizable Representations Form a Global Workspace in Language Models."

The paper "Verbalizable Representations Form a Global Workspace in Language Models" homepage | Image Source: Anthropic

Global workspace is one of the mainstream theories about consciousness in neuroscience, with the theoretical outline proposed by Bernard Baars in 1988, later developed into a neural mechanism model by Dehaene and Naccache. Anthropic specifically invited Dehaene and Naccache themselves to write a 15-page commentary, in some ways to have the core developers of consciousness theory endorse it.

Anthropic is certainly not fabricating associations. Dehaene and Naccache did indeed point out some structural parallels in the commentary, such as the small capacity of J-space, broadcast-style connections, and selective participation in flexible reasoning, which resemble the functional aspects they observed in the global workspace in the human brain.

However, there is a significant gap between "having functional similarities" and "developing cognitive structures like the brain." Dehaene and Naccache also noted in the commentary that Claude is purely feedforward, devoid of the recurrent loops that sustain the human brain's workspace; there is no body, no feeling signals; and J-space resets after each conversation.

These limitations stated in the paper are clear. However, on the communication front, the narrative changes. Phrases like "unable to control itself" and "silently reasoning in its head" imply that the model is a living entity with its own consciousness.

Moreover, the careful limitations in the paper, such as the success rate of 40%-70%, the fundamental differences between feedforward and recurrent structures, and the distinction that accessing consciousness does not equal subjective experience, gradually diminish in the communication chain, nearly disappearing at the level of public discussion.

This is not a unique problem for Anthropic. Almost all research institutions emphasize the most eye-catching discoveries when communicating with the public while downplaying limitations. But AI consciousness is a special topic. Public anxiety about it is real, and the consequences of misinterpretation are greater than those of general technology news. When a company developing advanced AI chooses to present its interpretability research through the lens of consciousness theory, the attention it gains and the misunderstandings it triggers are two sides of the same coin.

In the future, AI will increasingly resemble "thinking," but simulated thinking never equals possessing thoughts. The bubbles that arrive before the tide of the "Industrial Revolution" will always be the first.

免责声明:本文章仅代表作者个人观点,不代表本平台的立场和观点。本文章仅供信息分享,不构成对任何人的任何投资建议。用户与作者之间的任何争议,与本平台无关。如网页中刊载的文章或图片涉及侵权,请提供相关的权利证明和身份证明发送邮件到support@aicoin.com,本平台相关工作人员将会进行核查。

Share To
APP

X

Telegram

Facebook

Reddit

CopyLink