Original text: Training a Humanoid Robot for Hard Work

Authors: Alberto Rodriguez, Director of Robot Behavior at Atlas, Research Engineers Shane Rozen-Levy and Vinay Kamidi

Translation: Felix, PANews

This humanoid robot is completely different from any robot you have seen before. In the latest video, some things are obvious: the Atlas robot rotates its torso 180 degrees, squats down to lift a mini refrigerator, and then delivers it to an engineer who is taking a break. But there are also some less obvious details, such as how the robot utilizes its arms, legs, and torso to perform tasks that are difficult for humans, and some details that cannot be fully showcased in the video, such as the speed of the robot's development and the precision of its behavior.

This is indeed refreshing, but why do it?

Atlas moves a mini refrigerator

Boston Dynamics' other robots are designed to automate the most physically demanding tasks. The Stretch robot is able to autonomously unload boxes weighing up to 23 kilograms from trucks in extremely hot environments. The Spot robot conducts the same measurements along the same inspection route at exactly the same time every day, detecting early signs of problems in factory workshops. While these tasks may be monotonous, they require a high level of attention to detail, and Stretch and Spot are consistently providing this kind of task.

The goal of Atlas is to achieve a very wide range of functions in scenarios such as factories, warehouses, or construction sites that require high strength, endurance, and agility. Boston Dynamics is working to make Atlas a versatile tool for physical labor. Achieving the performance and reliability required for real-world environments necessitates significant improvements in both hardware and behavioral control.

The following is a series of carefully designed experiments that demonstrate important advances in hardware and behavior. Within just weeks after Atlas was unveiled in January, it showcased the performance of the humanoid robot in strength, agility, and full-body control.

Physical intelligence for the real world

In recent years, the market has witnessed a fundamental shift in behavioral architectures, driven by demonstration data and exhibiting increasingly enhanced generalization capabilities. This is a key element in fulfilling the promise of humanoid robots: they should be adaptive, quick to learn, and easy to reassign tasks. These architectures can drive behavior not only for desktop robotic arms but also for fully-fledged humanoid robots to perform real-world tasks.

Although contemporary state-of-the-art mainstream methods can produce excellent behaviors, they also have limitations: they over-rely on continuous camera feedback, using it not just to understand the world but also to guide control loops; their environmental interactions are limited to very few surfaces that robots can touch, often just fingertips; and they are almost entirely focused on lightweight tasks.

True work, especially that which involves heavy physical labor, requires a broader definition of "physical intelligence." When handling objects, teams utilize any part of the body to bear weight and adapt to the shape, mass, and stiffness of objects through tactile feedback.

You can't simply lift a refrigerator by observing and using your hands. You must be prepared to anticipate its weight, lean forward, adjust your body to its shape and weight, and judge whether you can lift it. Real work happens during the interaction. Humanoid robots should be able to pinch a box with their forearms and biceps, lift heavy objects from the ground to their thighs with their knees, and carry long and heavy items on their shoulders, just as they can easily pick up a refrigerator.

Atlas uses reinforcement learning (RL) to learn how to lift refrigerators by practicing numerous versions of the refrigerator-lifting motion in a simulated environment. The hardest part is not seeing the refrigerator or knowing how to lift it, but learning to adapt to any possible shape of refrigerator that Atlas might encounter in the real world. This is a problem that combines control and perception, where perception is implicitly completed through the body’s proprioception. The strategies that drive these behaviors have learned to adapt to various changes such as the refrigerator's position, weight, ground friction, and grip with the refrigerator or the configuration of its placement between the torso, arms, and hands. This level of adaptability is one of the fundamental building blocks of physical intelligence.

Robots carrying heavy loads

The hardware shown today is also unique. This generation of Atlas robots is designed not only to provide the flexibility and strength required for real work but also to incorporate the simplicity and reliability needed for mass production. While the humanoid design has its advantages, strategic breakthroughs can significantly enhance performance and efficiency.

Here are some highlights that may not be immediately obvious:

• Minimalist actuators: Only two types of actuators are used on the robot's body. This allows for a focus on manufacturing more efficient and powerful actuators at a larger scale, ultimately reducing costs. All of these are rotary actuators, which are easier to accurately represent in simulations, critical for the high-performance reinforcement learning using proprioceptive feedback mentioned above.

• Highly repetitive components: The same subcomponents are reused as much as possible on the body. Both legs and arms are identical. The structure of shoulders to shoulders and pelvis to pelvis is also exactly the same.

• Unlimited rotation joints: These actuators can rotate indefinitely. This is achieved by eliminating all cables between the joints, thus removing the major cause of actuator hardware failures. Conversely, this reduces Atlas' customer costs and gives Atlas a uniquely efficient way to move.

• Symmetrical feet: Because Atlas has equally excellent forward and backward capabilities, its two feet are symmetrical.

• Easy maintenance: Arms, legs, hands, and head are field-replaceable units that can be swapped out in just a few minutes.

The movement of the mini refrigerator showcases strength, full-body coordination, and the utilization of proprioceptive feedback. This sets a benchmark for industrial work: handling heavy items in manufacturing environments that typically require two people to lift.

However, some impractical tasks are also meaningful. For instance, a 90-kilogram robot being able to perform handstands and backflips is due to its excellent thermal management system, meaning Atlas can work in hot environments. Furthermore, these actions can train other transferable skills: for example, how to move with agile balance, how to operate fully in constrained environments, and how to recover from slips and falls.

Training process

As a product and a research platform, one of Atlas' goals is to train and deploy new behaviors within a day. While this demonstration did not achieve that speed, Atlas' ability to stably perform the refrigerator movement task already far exceeds expectations.

The following is the method for training robots:

• Reference trajectories: Training new behaviors requires reference trajectories, that is, data that tells the strategy what it should do. This could be remote operation demonstrations, animated trajectories, or descriptions of a more abstract goal. For the refrigerator moving task, a simple animation was first used to fully leverage Atlas' superhuman range of motion.

• Incentives: Then, a goal is set for the robot to follow the animated trajectory to complete the task as closely as possible. A reward mechanism is established to reinforce the desired behavior (keeping the heavy object in Atlas' gripper and maintaining the same position and orientation), while also applying pull and push disturbances to the robot and the refrigerator, enabling them to focus on the main task even when disturbed.

• Simulation: Atlas runs simulation programs in parallel on graphics processing units (GPUs), conducting millions of hours of motion practice. Through vast amounts of simulation experience, Atlas has learned to adjust its behaviors based on variations in the refrigerator.

• Real robot: After good results from simulations, tests are conducted on the hardware. Simulations can only help to a certain extent, while hardware testing is fundamental for continuous improvement.

• Iteration: Once real data on the strategy's performance on the real robot is obtained, adjustments can be made back to the training phase to reinforce the behavior.

Narrowing the gap between simulation and reality

One of the most significant improvements in the enterprise version of Atlas is its high fidelity of the simulation environment. The gap between Atlas' simulation and reality is very small; training, testing, and rapid iteration can be conducted easily. Generally speaking, if a behavior looks good in simulation, it will also perform well on the robot.

The gap between simulation and reality refers to the difference in performance of a strategy in a simulated environment compared to its performance on real hardware. Assumptions and mathematical simplifications in simulation cannot capture the complexities of the real world. Subtle changes and variables like friction, latency, or sensor noise can accumulate and lead to failures in the physical world.

While it may never be possible to completely eliminate this gap, we are very close. The Atlas team has established a rigorous pipeline and system support for testing and development. Train a strategy today, tomorrow you can test it on the robot with a mature strategy and gather data to drive the next iteration and the development of new behaviors.

What makes the gap between simulation and reality so small?

High-fidelity hardware: Unlike previous platforms, this platform only uses two powerful, efficient actuators and is entirely symmetrical. This simplicity in design and structure, combined with the efficiency of the actuators, means the robot can be modeled in simulation with extremely high accuracy. Since the robot model is very close to the actual hardware, fidelity issues are reduced when deploying trained strategies. The simulation results match the actual outcomes perfectly.

Domain randomization: To make the strategy more robust, the robot was not trained in ideal conditions. By adopting a domain randomization approach, parameters such as refrigerator weight, floor friction, or motor power are adjusted throughout the training process. Small random variations during training allow for more resilient final behaviors when facing various uncertainties in the real world. For example, the strategy for moving the refrigerator was initially trained for loads between 50-70 pounds, but the robot successfully moved a refrigerator filled with items weighing over 100 pounds. The team also does not test under perfect conditions. They place various items in the refrigerator from the lab; weights are inconsistent, distributions are uneven, and there can be shifts during movement. Through a well-developed strategy, all these disturbances can be managed by Atlas rather than handled by engineers.

People and processes: Finally, processes and operations are designed to streamline training, testing, and experimentation. The team has established rigorous protocols, with numerous personnel working behind the scenes. The team collaborates closely with many responsible for the actual operation of the robot, including hardware design teams, maintenance technicians, and robot captains. The entire organization works together to make Atlas as reliable and efficient as possible, while continually pushing the limits of new capabilities.

Related articles: On the day of Yushu's IPO, Nvidia released a humanoid robot

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。