AI Helps Train Teams of Collaborative Robots and Drones

A team of researchers at University of Illinois Grainger College of Engineering has developed a new method to train multiple agents like robots and drones to work together with the use of artificial intelligence (AI). The agents rely on reinforcement learning, which is one of the main types of machine learning techniques.

Decentralized Technique

Huy Tran is an aerospace engineer at the university.

“It’s easier when agents can talk to each other,” said Huy Tran. “But we wanted to do this in a way that’s decentralized, meaning that they don’t talk to each other. We also focused on citations where it’s not obvious what the different roles or jobs for the agents should be.”

According to Tran, this scenario is more complex because it’s not clear what one agent should do against another agent.

“The interesting question is how do we learn to accomplish a task together over time,” he said.

Reinforcement Learning Technique

The team relied on the machine learning technique called reinforcement learning to get around this problem. It enabled them to create a utility function that tells the agent when it is doing something useful for the team.

“With team goals, it’s hard to know who contributed to the win,” Tran continued. “We developed a machine learning technique that allows us to identify when an individual agent contributed to the global team objective. If you look at it in terms of sports, one soccer player may score, but we also want to know about actions by other teammates that led to the goal, like assists. It’s hard to understand these delayed effects.”

The researchers’ algorithms also identify when an agent or robot is doing something that goes against, or doesn’t contribute to the goal.

“It’s not so much the robot chose to do something wrong, just something that isn’t useful to the end goal,” he said.

The algorithms were tested using simulated games, such as StarCraft.

“StarCraft can be a little bit more unpredictable — we were excited to see our method work well in this environment too.”

This type of algorithm is applicable to various real-world situations, the team says. Some of the potential applications include military surveillance, robots in a warehouse, traffic signal control, autonomous vehicles coordinating deliveries, and controlling an electric power grid.

The team carrying out this breakthrough research included Seung Hyun Kim, Neale Van Stralen, and Girish Chowdhary. It was presented at the Autonomous Agents and Multi-Agent Systems peer-reviewed conference.