Scaling Law is being re scaled ---Interpretation of the latest paper "Learning Beyond Gradients" by OpenAI core researchers In the past few years, the AI industry has almost defaulted to larger parameters, more data, longer training, and stronger GPUs, which means stronger models and scaling laws. In the past few months, the industry has begun to believe that more reasoning and more agents can complete longer and higher value tasks, which is stronger intelligence. This constitutes the industry's understanding of scaling law, and as long as the scaling law continues to hold, the model will continue to approach AGI. A recent paper by OpenAI core researcher Weng Jiayi titled "Learning Beyond Gradients" proposes a new scaling dimension: AI can not only learn through gradient descent, but also continuously modify its behavioral system through heuristic, policy, workflow, strategy, and code generation. This is the latest important development in the AI industry that may be transitioning from a "Scaling Model" to a "Scaling System" phase, following the development of agentic and harness. The ability flywheel of AI in the past was essentially: more data → larger models → stronger capabilities → more users → more data. But now, what the paper wants to tell us is that the new capability flywheel: stronger model → stronger heuristic generation → stronger runtime system → stronger agent capability → more real-world feedback → stronger runtime evolution → in turn, enhances model performance. The industry is accelerating from intelligence=weights. Transition to: Intelligence=weights+runtime system. LLM is essentially input → Transformer → output. After the model training is completed, the ability is basically frozen. Learning mainly occurs in gradient descent, backpropagation, and weight updates. That is to say, learning=modifying parameters. LLM is like the human brain, parameters are like brain cells. But the vast amount of complex capabilities in the real world actually do not come entirely from parameters. Just like where human civilization is truly powerful, it's not just the brain itself. What truly caused civilization to explode are language, writing, tools, mathematics workflow、 Software system, organizational structure, scientific methods. These are essentially 'external heuristic systems'. 《Learning Beyond Gradients》， The innovation lies in its attempt to liberate "learning" from the parameter space. In the past, it was: reward → grade → weights. Now it starts to become: feedback → heuristic modification → runtime evolution. Learning begins in the program space, not the parameter space. heuristic， It is somewhat similar to an expert system, but greatly enhances its capabilities: in the past, expert systems had rules written by humans; Now, rules are being automatically generated by LLM. This is a shift from quantity to quality in terms of efficiency. The failure of traditional expert systems is not entirely due to the wrong direction of "rules", but rather because humans are unable to maintain large-scale dynamic rule systems. In the past, writing rules was too slow, modifying rules was too expensive, rules were prone to conflicts, long tail cases would explode, and system complexity would spiral out of control, so expert systems were eventually replaced by deep learning. But the emergence of LLM changed this constraint. The cost of generating rules is now close to zero. The model can not only generate rules, modify rules, delete rules, debug rules, but also start to automatically generate them workflow、tool graph、planner、memory strategy， Even fixing agent behavior. This means that AI is beginning to be able to modify its own runtime system. So, more and more capabilities began to spill over from the "model itself" to system structures such as memory, planner, search, tool use, verifier, and runtime orchestration. A larger model=stronger AI, becoming: stronger model x stronger runtime system=stronger AI. This will form a new ability flywheel. In the past, AI only had 'model scaling'. In the future, AI will begin to emerge: Model Scaling x System Scaling x Runtime Self Improvement. We are likely transitioning from a scaling law at the end of last year to a heuristic driven scaling law that combines agent and harness squared. More importantly, the growth of the runtime system has only just begun. Many agent systems are still very early today. Memory is weak, planner is weak, workflow persistence is weak, and long-range task capability is weak, essentially still in the 'DOS era'. But in the future, the actual capabilities of the same basic model may differ by tens of times under different Harnesses. Because the bottleneck of many complex tasks is no longer whether the model can perform, but whether the system can continue to organize behavior. That's also why the most important competition in the future may no longer be just about "who has the most parameters", but "who first forms a closed loop: model+memory+tool ecosystem+heuristic runtime+self improving harness". In a sense, Transformer is becoming more and more like a 'cognitive kernel'. The true AGI may be built around the Transformer runtime civilization、heuristic ecosystem、agent society、memory graph、self-improvement loop The combination. The most exciting aspect of 'Learning Beyond Gradients' for me is not actually' Beyond Gradients'. But it started to try: turning the Scaling Law itself into a system that can be further scaled.