CMU researchers teach robots to do chores by watching humans

Listen to this article

Researchers at Carnegie Mellon University (CMU) have developed a learning method for robots that allows them to perform household tasks after watching a person do them just once. | Source: Carnegie Mellon University

A research team from Carnegie Mellon University’s (CMU) School of Computer Science has developed a new method for robots to learn called WHIRL, which stands for In-the-Wild Human Imitating Robot Learning. 

WHIRL is an efficient algorithm for one shot visual imitation. With WHIRL, a robot can learn to perform household tasks just watching a person perform them. 

The CMU team added a camera and its algorithm to an off-the-shelf robot to test the abilities of its software. When testing the robot, the team found that it was able to perform over 20 tasks after watching someone perform them just one time. 

The robot learned how to do things like opening and closing appliances, cabinets, doors and drawers, putting a lid on a pot, pushing in a chair and taking the trash bag out of the can, among other things. None of the items the robot interacted with, whether it be appliances or doors, were modified to suit the robot. 

While the robot’s first few attempts at most tasks failed, it was able to quickly latch onto how to perform the task correctly after a few successes. 

The robot often completed tasks using different movements than the humans who demonstrated them, but the Carnegie Mellon team isn’t concerned about that. WHIRL doesn’t aim to make a robot, equipped with different tools than a human, perform a task the same way a person would. Instead, the robot focuses on getting to the same end result. 

Robots typically learn to do tasks with one of two methods. The first, called imitation learning, involves humans manually operating a robot to teach it a task. The second, called reinforcement learning, requires robots to learn from millions of examples in simulation, then adapting that training to the real world.

Both methods involve repeatedly teaching the robot to perform a task, making it difficult to teach a robot to perform multiple tasks. With WHIRL, a robot can learn multiple tasks quickly, with a human only having to demonstrate it once. 

Shikhar Bahl, a PhD student at Carnegie Mellon’s Robotics Institute (RI), worked with Deepak Pathak and Abhinav Gupta, faculty members at the RI on the research. The team presented WHIRL at the Robotics: Science and Systems conference in New York earlier this month.