Oh no, it looks like domestic large models might really not be able to keep learning from GPT anymore. Yesterday, OpenAI released an internal safety report on their Agent, which revealed the raw Chain of Thought (CoT). A lot of the English words were directly strung together without any spaces or grammar, mixed with code variables, abbreviations, and machine-adaptive judgments. If OpenAI hadn’t specifically included a human-readable version with added spaces in the report, it would look like gibberish to humans. Karpathy also shared and commented on it, pulling up one of his earlier tweets where he said: when a model stops using human-readable English in its chain of thought, it means reinforcement learning (RL) has been done really well. And now, it seems OpenAI’s RL has truly gone deep. What do you guys think—can domestic models still keep distilling in the future?