MIT helps robots retrieve objects buried in a pile

Listen to this article

Tara Boroushaki, Fadel Adib and Nazish Naeem (from left to right) working with FuseBot. | Source: James Day, MIT Media Lab

Researchers at the Massachusetts Institute of Technology (MIT) developed a system that can help robots retrieve target items from a pile of things as long as some of the items have RFID tags.

RFID tags reflect signals sent to them from an antenna. According to a recent market report from Accenture, more than 90% of U.S. retailers use RFID tags. However, these tags aren’t universal, and oftentimes in e-commerce warehouses employees are dealing with piles of objects that have a mix of tagged and untagged items.

The system created at MIT, called FuseBot, builds off of previous work from the school in which researchers demonstrated a robotic arm that combines visual information and radio frequency (RF) signals to find objects that were tagged with RFID tags. Now, FuseBot can find items even if the target item doesn’t have a tag, as long as some items within the pile do.

“What this paper shows, for the first time, is that the mere presence of an RFID-tagged item in the environment makes it much easier for you to achieve other tasks in a more efficient manner. We were able to do this because we added multimodal reasoning to the system — FuseBot can reason about both vision and RF to understand a pile of items,” Fadel Adib, associate professor in the Department of Electrical Engineering and director of the Signal Kinetics group in the MIT Media Lab, said.

The system uses a robotic arm that is equipped with a video camera and RF antenna. It uses the camera to scan a pile of objects and create a 3D model of the environment. At the same time, FuseBot sends signals from its antenna to locate the RFID tags in the pile. These radio waves can pass through most solid surfaces, so the robot is able to see into the pile.

FuseBot knows that its target item doesn’t have a tag, so it knows that the item won’t be at the exact location as any of the tags. It combines the information it gathered from its antenna with its current 3D model of the environment and then highlights areas where the target item could be located.

FuseBot uses this information to reason about the objects in the pile and the location of RFID tags to decide which items to remove to get to the targeted item. The team’s goal is for FuseBot to find the item in as few moves as possible.

This type of reasoning can be difficult for FuseBot, as the robot doesn’t know how the objects are oriented under the pile or how soft or hard the object is. Some objects on the bottom of the pile could even be deformed because of heavier objects pressing into them. To overcome these obstacles, the robot uses probabilistic reasoning. It uses the information it does have about the size and shape of an object and its nearest RFID tag location to create a model of the 3D space an object is likely to occupy.

After removing each object, FuseBot scans the pile again and reasons again about which object will be the next best to remove.

“If I give a human a pile of items to search, they will most likely remove the biggest item first to see what is underneath it. What the robot does is similar, but it also incorporates RFID information to make a more informed decision. It asks, ‘How much more will it understand about this pile if it removes this item from the surface?’” Tara Boroushaki, research assistant in the Signal Kinetics group, said.

The MIT team ran more than 180 experimental trials with FuseBot using piles of household items, like office supplies, stuffed animals and clothing. Each new pile had a random number of items and RFID-tagged items.

FuseBot was able to extract its target item 95% of the time, an increase from the 84% success rate you would see on similar robotic systems. It was able to get to the target item with 40% fewer moves than typical systems, allowing it to retrieve items more than twice as fast.

Moving forward, the MIT team hopes to incorporate more complex models into the FuseBot so that it does better with soft, deformable objects. The team is also interested in moving objects in different ways, such as a robotic arm that pushes items out of the way instead of grabbing them.