Researchers taught a quadruped to use its legs for manipulation

Listen to this article

Researchers from Carnegie Mellon University (CMU) and UC Berkeley want to give quadrupeds more capabilities similar to their biological counterparts. Just like real dogs can use their front legs for things other than walking and running, like digging and other manipulation tasks, the researchers think quadrupeds could someday do the same.
Currently, we see quadrupeds use their legs as just legs to navigate their surroundings. Some of them, like Boston Dynamics’ Spot, get around these limitations by adding a robotic arm to the quadruped’s back. This arm allows Spot to manipulate things, like opening doors and pressing buttons, while maintaining the flexibility that four legs give locomotion.
However, the researchers at CMU and UC Berkeley taught a Unitree Go1 quadruped, equipped with an Intel RealSense camera for perception, how to use its front legs to climb walls, press buttons, kick a soccer ball and perform other object interactions in the real world, on top of teaching it how to walk.
The team started this challenging task by decoupling the skill learning into two broad categories: locomotion, which involves movements like walking or climbing a wall, and manipulation, which involves using one leg to interact with objects while balancing on three legs. Decoupling these tasks help the quadruped to simultaneously move to stay balanced and manipulate objects with one leg.
By training in simulation, the team taught the quadruped these skills and transferred them to the real world with their proposed sim2real variant. This variant builds upon recent locomotion success.
All of these skills are combined into a robust long-term plan by teaching the quadruped a behavior tree that encodes a high-level task hierarchy from one clean expert demonstration. This allows the quadruped to move through the behavior tree and return to its last successful movement when it runs into problems with certain branches of the behavior tree.
For example, if a quadruped is tasked with pressing a button on a wall but fails to climb up the wall, it returns to the last task it did successfully, like approaching the wall, and starts there again.
The research team was made up of Xuxin Cheng, a Master’s student in robotics at CMU, Ashish Kumar, a graduate student at UC Berkeley, and Deepak Pathak, an assistant professor at CMU in Computer Science. You can read their technical paper “Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion” (PDF) to learn more. They said a limitation of their work is that they decoupled high-level decision making and low-level command tracking, but that a full end-to-end solution is “an exciting future direction.”