Speaker: Ted Xiao (Founding Technical Member of Prometheus, Former Researcher at Google DeepMind)

Compiled by: Haozhen　⁠

If AGI and general robots succeed, will a world rich in resources, benefiting all humanity, naturally emerge?

Ted Xiao (Founding Technical Member of Prometheus, Former Researcher at Google DeepMind) in his speech at the Humanoid Hub Conference analyzed the process from the conception to mass production of new technologies based on this hypothesis: even if intelligence and labor capabilities become abundant, what stages will still slow down R&D and production? What research and business opportunities are worth investing in ahead of time?

In his view:

•According to Amdahl's law, if half the time is spent on stages that cannot be accelerated, even if the rest of the work is fast, the total time can at most be reduced by half.

•Verification determines the upper limit of model performance post-training.

•If the verifier has vulnerabilities, the model may learn to exploit those vulnerabilities to achieve high scores, but actual performance may not improve.

•Verifiers also need to be tested; when faced with new designs not covered by historical data, their judgments still need to be validated with actual results.

•The development team needs to clarify on-site what qualifies as acceptable, design the verifier accordingly, and then test and improve it with limited actual measurements.

We compiled Ted Xiao's speech. The verification bottleneck he discusses is one of the reasons we are optimistic about AI for Science, robotics, and hardware manufacturing. As AI capabilities enhance, although models can propose solutions more quickly, determining whether the solutions are valid often still requires real experiments and engineering tests, and the strengthening of models does not easily accelerate these tests.

⁠/⁠

01.

What is still needed after achieving AGI and general robots?

Current research and investment broadly focus on two directions: one end is bits, and the other end is atoms. This trend can be seen in Silicon Valley and even globally, with rapid technological advancements in both directions.

•The bits side includes AGI supporters, who hope to create a "nation of geniuses" in data centers, first producing AGI, and then letting AGI solve other problems.

•The atom side focuses on robot learning, dexterous manipulation when in contact with objects, and the proliferation of general robots.

If we assume that AGI and general robots can succeed, then when preparing for a post-AGI society, we still need to consider what is lacking between them: how can digital superintelligence truly improve quality of life and provide people with a richer material existence?

To answer this question, we can first look at how material life has been improved today.

Technological progress has continually created wealth, promoted social prosperity, and improved human life, with a significant part being the invention of new physical world technologies and their actual implementation. The comfortable lives we enjoy today are the result of generations of work by millions of engineers, scientists, designers, as well as white-collar and blue-collar workers worldwide.

Specifically, turning an idea into a product that can be mass-produced and used requires design, engineering development, prototyping, pre-production testing and certification, production, gradually increasing output, and scaling up production.

Judging what AGI and robots are still lacking also requires placing them within this concrete workflow. But which segments of this chain involve design, analysis, and other intellectual work, physical labor in actual operations, and capital investment, and to what extent can these be delegated to AGI, embodied intelligence, or require other methods to solve?

According to Ted Xiao's classification, the blue areas in the diagram represent abstract, digitized work, where AI models that operate solely in a digital environment have already demonstrated the capability to handle such tasks. The pink areas involve interacting with objects and dexterous manipulation, tasks that general robots can undertake.

However, even if both digital AGI and the dexterity of robots reach high levels, there is still a lot of work in the engineering chain that cannot be directly handed over to them. Whether assessing GDP, labor shares, industry profit margins, or job numbers in various fields, this point becomes evident. Even simply listing these tasks reveals that achieving human-level operational capability does not automatically mean that the remaining engineering problems will resolve themselves.

02.

Amdahl's Law: Unaccelerated Stages Determine Overall Acceleration Limits

To enable everyone to enjoy the rich material life that some currently have, and to improve the quality of life for each person, it is essential to address the issues in this chain one by one. You cannot simply accelerate one end and assume the other stages will automatically follow.

The unresolved stages will affect the speed at which a technology or product completes development, validation, and achieves scaled production. As pointed out by Amdahl’s law, even if a program has many operations that can be executed in parallel, overall speed is still constrained by those operations that must be completed sequentially and cannot be handled simultaneously; innovation and invention face similar limitations.

As illustrated in the diagram, let p represent the proportion of the total time originally spent on accelerated stages, and s represent the speedup factor for that segment of work. We have seen that when some segments achieve a very large s, the required time significantly shortens, but no matter how many times faster a segment becomes, the overall speedup limit still depends on p, which reflects how much time those segments originally occupied.

The gray, orange, and blue curves in the diagram correspond to different proportions of accelerable segments. If the work that can be accelerated constitutes 50% of the original total time (which covers a broader range than shown earlier in the engineering chain), even if this part becomes infinitely fast, the total time for the entire process can at most be halved, meaning the overall speed doubles.

When p approaches 99%, 99.9%, or even higher, the total time required for research, repeated improvement, and deployment may be reduced to one-tenth, one-hundredth, or even less of the original time.

The faster the local acceleration, the easier the non-accelerated portions become a bottleneck in the overall R&D and production process.

In fact, the time required for many tasks is difficult to compress further, and the limitations mainly stem from two aspects:

One type of limitation arises from the time required for testing itself.

•When testing fast-charging batteries, load testing and charge and discharge cycle testing are required, and the entire process often exceeds one and a half years.

•Drug development, from initial conception, research, and reporting to phase three clinical trials, can sometimes take over ten years. Considering that human-controlled trials must assess the safety, efficacy, and side effects of drugs, the time required for these experiments cannot be eliminated.

•Even after lengthy testing, experimental results may still be unstable and difficult to replicate. The research team at Amgen attempted to replicate 53 influential preclinical cancer studies but ultimately succeeded in replicating only 6, about 11%. Therefore, even previously published, seemingly valuable research may not provide stable and reliable feedback.

An additional issue is that engineering knowledge is dispersed across different companies, systems, and job positions. To accelerate more stages, it is essential to understand how these tasks are conducted. However, some of the know-how belongs to corporate intellectual property, while others remain in practitioners' experiences and are hard to access directly.

It is worth noting that software development is relatively unique in this regard: code and version histories can be preserved comprehensively, and whether a program runs as intended often has clear rules and tools for checking. The physical economy has also accumulated substantial engineering knowledge, but when opening GitHub, we cannot find a complete set of materials explaining how humanity built the world as it is today.

03.

Verification Determines the Upper Limit of Model Performance Post-Training

To further assess which stages can still be accelerated, we should first look at how long verification takes. The previous examples of battery and drug development both touch on the same question: how long do we have to wait to obtain trustworthy results that guide the next round of experiments?

In this context, the answers vary greatly across different tasks, ranging from a few milliseconds to several decades.

•Some results can be verified using models or simulations, and investing more computing power often leads to quicker feedback. Thus, increasing GPU, CPU, and funding is useful for such tasks. In this case, although implementation may still be challenging, we at least know where we can start to accelerate. Physical simulations, using Lean to check formal mathematical proofs, code unit testing, and certain design tasks fall into this category of tasks that can be accelerated with computing power.

•However, some results must be verified through actual measurements. At this stage, validation can take hours or even years.

The time required for verification is crucial because models need to continue learning based on verification results. Experience from frontier model research suggests that as long as we can clearly and reliably judge whether the results meet requirements, we can use these judgments as rewards to train models through RLVR (Reinforcement Learning with Verifiable Rewards), improving their performance on corresponding tasks.

Microsoft AI once described its model as a "mountain-climbing machine," which is actually a fitting metaphor: as long as we can reliably determine whether the results meet standards, the model can continuously optimize towards the goal.

Therefore, verification determines the level that frontier models can achieve during the post-training phase, a fact evident in fields such as mathematics, coding, etc.

However, the rewards and verification results used for training models may also have issues: the programs or models responsible for checking results and providing feedback, known as verifiers, may have vulnerabilities that the models could exploit to score higher. The more comprehensive the training, the greater the impact of these flaws could be.

04.

Robot Training: Relying on Simulation or Real Data?

In the physical world, the ability to obtain quick and reliable feedback also influences research methods.

Robot training is one example: over the past decade, motion control, full-body control, and manipulation tasks have formed different research paths, with varying methods of obtaining training feedback and evaluating results.

•One approach is sim-to-real, obtaining rapid feedback through physical simulations and model-based learning.

•The other primarily uses human demonstration data collected in real environments: first training models with these demonstrations, and then evaluating the performance of strategies on real machines without relying on simulation for feedback.

For example, with motion control, if you take a Unitree G1 today, download the code from GitHub, and use a GPU and a small MLP network, you can see results in just 15 minutes of training. The training is conducted entirely in simulation; with a GPU-optimized physical simulation environment, thousands or even millions of robot action executions can be simulated in parallel, without needing to collect real data from a humanoid robot or remotely operate it to teach walking.

MLP (Multi-Layer Perceptron): a basic neural network consisting of multiple layers of interconnected neurons. Here, it is used to calculate corresponding control actions based on the robot's state.

The trained control strategy (policy), which determines how the robot acts, can be directly applied to the real robot to perform actions like walking, backflips, or even more agile movements and full-body control. Such effects were hard to imagine just a few years ago.

In contrast, robot operation research over the past decade has primarily focused on large-scale real-world data collection, and this data relies heavily on human expertise and demonstrations. Whether using first-person data (egocentric data), remote operation of robots, or devices like data gloves and UMI, researchers have been leveraging these methods to expand offline expert datasets, allowing robots to learn to perform tasks requiring dexterity.

Teams like Physical Intelligence, Generalist, Dyna, and Google DeepMind, after training their models, still need to conduct real A/B evaluations in laboratories, often requiring hundreds of hours to release a new model.

For example, when Google published the RT-1 and RT-2 papers, it enlisted dozens of contractors, dozens of robots, and hundreds of hours of manual labor to compare which checkpoint performed better.

Due to the slow feedback from real machines, research teams find it difficult to collect enough data with each model update. Ted Xiao suggests selecting data based on training purposes; for pre-training, data collected in the past can be used as long as the demonstrations approximate ideal performance; for collecting correction data, the current version should execute tasks first, recording where it needs intervention and the actions following corrections.

However, many operational skills that humans have mastered have not been documented as data for robots to learn from. The physical economy faces similar issues.

•In the field of robotics, humans have evolved to master a vast array of physical skills, necessitating researchers to gather data on how humans deploy these skills for robots to learn from. Currently, the scale of such data is far from comparable to that available on the internet. Therefore, collecting this data and developing algorithms capable of learning from limited data and expanding as data grows has been a continuous pursuit.

•The physical economy has also accumulated a large amount of such knowledge and data; otherwise, we wouldn't have been able to build the society we have today. However, these materials have not been systematically recorded, stored, or organized, making them unavailable for direct use like the contents on Wikipedia or Reddit.

More importantly, in the physical economy, humanity has not exhausted all possible invented technologies; engineering innovation still requires ongoing iteration. In contrast, for many dexterous operations we hope robots will master, existing human capabilities can serve as benchmarks, although there are also operations and certain industrial scenarios that need to exceed human abilities.

Therefore, if someone builds a robot that can achieve human-level performance across various tasks, it will represent a tremendous success. However, if a set of engineering systems can only assist humans in reinventing technologies from the mid-20th century, its value is quite limited.

Thus, engineering innovation requires both the acquisition of existing data and the continuous advancement of technology. We need to understand the current level of technology, how the best teams work, how a new idea transitions through design, experimentation, and engineering processes into reality, and how these processes can be improved further.

05.

Engineering R&D: After Computing Acceleration, Waiting for Experiments

Looking further, engineering testing can also be ranked by feedback speed: manual calculations, software analysis, and automated checks generally provide results relatively quickly, whereas experiments involving real equipment, materials, and environments may take weeks, months, years, or even decades.

Testing is also limited by resources and the number of times it can be performed; for example, when F1 teams design new cars each year, the number of wind tunnel tests and the computing power available for CFD simulations are constrained by regulations. These testing and simulation quotas are also part of the competitive R&D efforts of the teams.

Besides limited testing resources, the applicability of verification through computing or simulation is also an issue. The same design might pass the computational or simulation checks in the blue sections of the diagram but reveal problems in the actual tests represented in the orange sections. It is necessary to clarify in what circumstances these verification methods are reliable and when they fail to reflect actual performance, as this is crucial for engineering practice.

Ted Xiao noted that teams like Anthropic, OpenAI, and Gemini have started training models for some digitized design and engineering tasks. In these tasks that can be checked quickly and at low cost, model performance has already shown significant improvements. The engineering design demos showcased by these teams left a strong impression on Ted Xiao.

However, integrating these designs into actual production engineering processes does not guarantee satisfactory results; sometimes even the fundamental checks performed by engineers cannot pass. Therefore, at least for now, relying solely on feedback from computational or simulation checks does not seem sufficient to ensure that models meet actual engineering requirements.

This situation is not limited to CAD. Many fields face similar challenges: progress in areas where computations can replace actual measurements is rapid, but parts that still must wait for real test results face significant difficulties in synchronizing acceleration.

Automated laboratories are attempting to improve these actual measurement stages. Teams from Periodic Labs, OpenAI, Anthropic, and others are exploring ways to leverage feedback from real experiments in fields such as materials science and drug discovery, also including wet lab experiments involving actual samples and reagents. They are both increasing experimental efficiency and attempting to enable models to interpret experimental results or check experimental plans.

For instance, Periodic Labs has announced progress: their automated laboratory can conduct approximately 100 real experiments involving X-ray diffraction daily, and the new laboratory may increase that number to 1,000 experiments per day. Yet, even so, this scale is still far smaller than the millions of attempts that can be made in coding model training and humanoid robot simulation training.

As a result, Periodic Labs is also attempting to train an LLM judge, allowing the LLM to interpret X-ray diffraction patterns. Its agreement rate with that of PhD experts reached 77%. They hope to refine the LLM judge using real experimental feedback, and then have the judge provide reward scores to train the models.

However, the effectiveness of such methods varies across different teams. Compared to coding, engineering calculations, and mathematics fields, advances in automated laboratories are prone to stagnation or redundancy, and many facets still need to await actual experiments.

The research by Anthropic on protein complexes and CRISPR has also demonstrated these limitations: scientists collaborate with Anthropic's internal models to repeatedly propose and validate experimental designs in computers, and then deliver the selected experiments to human-operated wet laboratories for execution. With the collaboration of scientists and models, more experimental designs and initial validations can be completed in computers, reducing previously required months or weeks of work to just weeks or days. However, the wet experiments conducted by humans still need weeks, and the acceleration of design and analysis does not shorten the waiting time for the experiments themselves.

In summary, experiments can proceed in parallel, but if the subsequent round of experiments is to be designed based on the results of the previous one, it must wait until the first round is completed. Even if more experiments are conducted in parallel at once, it merely increases the information obtained in that round; researchers still need to see results to propose new hypotheses and arrange the next round of experiments. Therefore, parallel experiments can enhance the exploration conducted in each round but cannot eliminate the waiting time between rounds.

06.

What Are the Limitations of Replacing Actual Measurements with Models?

To shorten the time required for verification, many fields are attempting to use computations to replace a portion of real testing. Measurements that originally take a long time, are cumbersome to perform, are easily influenced by external factors, and are costly may find opportunities to leverage computing power to accelerate verification, if they can be reliably executed through models or simulations. For real tests that remain necessary, we need to improve testing efficiency or find ways to avoid having to wait for actual test results with every design iteration.

If we follow the ideas of Periodic Labs, we can design a value function, that is, train a critic capable of evaluating plans based on real feedback. If the model is reliable enough, it can replace some costly real measurements with low-cost computations.

However, there is an issue: the verifier is trained on historical data; it may only be reliable under the tasks and conditions covered by that data. Yet we often need to evaluate new designs that have not been seen before.

While models may indeed learn to handle situations not encountered during training, such as AlphaGo Zero, Go can determine results based on rules without incurring high costs for every verification through real experimental means. The physical world lacks such conditions: how proteins fold and bind, whether new materials and crystals can be discovered and synthesized will ultimately undergo scrutiny by nature's laws. And the real observations we can use to train verifiers, reward functions, and value functions remain quite limited.

Models may be able to handle some new situations, but this ability has limits. The greater the difference between new designs and historical data cases, the harder it becomes to acquire reliable and useful feedback.

Consequently, while models may execute inventions or designs that follow established lines much faster, surpassing existing results still presents significant challenges. Learning value functions is worth a try, but it does not necessarily solve the problem of exploring the unknown directly.

07.

How Should Verification Be Improved Next?

Ted Xiao combines his experiences in robot learning and frontier model research, along with advancements among peers, to assert that the next crucial issue is to establish methods that are both trustworthy and capable of scaling verification, confirming their reliability under actual use conditions and in new rounds of R&D tasks.

Swipe left and right to see more charts

Returning to the initial hypothesis, firstly, the intelligence provided by AGI and the labor capacity provided by robots are both necessary conditions for achieving material abundance. However, even if both succeed in the future, the remaining issues in engineering and invention will not disappear automatically.

Secondly, directions that can be reliably verified tend to make progress more easily. When verification methods are trustworthy and can be used on a large scale, we can continue learning and developing technology based on feedback, gradually approaching the limits of capability in these tasks.

Finally, evaluation metrics must reflect the actual results we desire. As scores improve, does model performance in reality also enhance? After integrating AI models, robots, and engineering processes, has the entire system completed the expected work? These need to be measured directly.

Three Research Directions

Specifically in research, Ted Xiao believes there are three main tasks worth pursuing.

The first is to establish more verifiers, including the commonly cited RL environments. Many teams now wish to increase training environments and invest more computing power for RL. But aside from increasing quantity and coverage, we also need to delve into a field, understand how work is conducted within, what standards of practical application must be met, and not just look at academic benchmarks.

The second is to verify whether verifiers are trustworthy. We need to be present in the actual environments where models operate to understand what tasks and operational conditions they will encounter and then check whether the verifiers' judgments are reliable under these conditions.

The third is to arrange real testing effectively. Ultimately, both the verifier and the complete systems must undergo real-world validation, but these tests are costly, time-consuming, and require a substantial capital investment. The number of attempts that can be made is limited and some waiting times cannot be compressed. Therefore, for parts that can be simulated reliably, synthetic data can be used for supplemental training; for mandatory real tests, it is essential to clarify what needs to be verified first, and then prioritize those experiments whose results can best guide the next rounds of design or training, ensuring each result contributes to improving the verifier and technology.

What Should Be Done for Actual Progress?

Over the past year and a half, Ted Xiao has been considering specific steps to take.

•For costly and limited actual measurements in the final stages, capital investment can be increased to secure more experimental resources and expand testing scales through broader collaborations.

•However, offline data remains scarce and dispersed, making deployment crucial: Ted Xiao believes that unlike the digital world, where OpenAI can train models for years in laboratories using internet data before launching ChatGPT, ready data is rare and scattered in the physical world. We need to deploy systems for actual work, continuously collect data, improve verifiers and models, and gain insights into the problems that need addressing.

Taking metal cutting as an example, Ted Xiao suggests that software teams should first go on-site to observe how operators complete procedures and listen to their insights on potential issues, then articulate these problems clearly using a shared language.

•Focusing on these tasks is also necessary. There are many opportunities, and researchers and entrepreneurs easily chase multiple directions simultaneously, but simply improving a single segment is insufficient to significantly shorten the total time for technology development and production. Local successes may bring wealth, reputation, and research influence, but may not accelerate the entire process to the same extent.

•Ultimately, what matters is whether the output of the physical economy has improved, if the research and development of new technologies has accelerated, and whether industrial output has increased. High scores on several benchmarks or attention on social media can never replace these actual results.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到[email protected]，本平台相关工作人员将会进行核查。