Institute For The Future

Teaching Robots How to Work Together

A conversation with swarm robotics pioneer Nora Ayanian

Interview by Mike Liebhold, Mark Frauenfelder, David Pescovitz

In a video created by the Automatic Coordination of Teams Lab at the University of Southern California, a set of 49 drones line the floor of an empty studio. They lift off in synch, never breaking the shape of a perfect square, and proceed to soar, dive, and twirl like synchronized swimmers. Then, suddenly, they break formation and rearrange into the letters U-S-C. This demonstration appears, on the surface, similar to the one put on at this year’s Super Bowl. But the small size of the robots, the precision of their coordinated movements, and their ability to navigate indoors represent substantial breakthroughs. And the future applications of this technology extend far beyond entertaining spectacle. Multi-robot coordination could revolutionize everything from farming to disaster response. IFTF caught up with the Lab’s director, Nora Ayanian, recently named one of MIT’s top 35 visionaries under 35, to get her perspective on where this could all go in the next decade.

Your drones fly in beautifully coordinated, synchronized ways. How you are able to achieve this?

We were able to get 49 small [drones] flying together, which is, to my knowledge, the first time that’s been done indoors, basically by using an external motion capture system to determine where the robots actually are and then relay that info back to the individual drones to help them navigate. A majority of the computation is actually done on board. The drones get information about their location from the server that is running the motion capture system, but the updates are not frequent enough for stable flight since there are so many drones and so few radios. So we fuse information from their on-board accelerometers with the updates from the motion capture system.

What about the robots’ behavior? It seems reminiscent of a school of fish or birds.

In terms of the large groups of robots moving in synch, that may be. And there are people in robotics that study the movement of birds and fish. But those animals don’t do the things that I want my systems to be able to do. I work normally on smaller groups of robots where we have specific tasks that we want them to accomplish and they are not all doing the same thing …I was thinking about how these robots could be better and asked, “Why is nobody studying how people interact with each other?” If you’ve ever worked in a group with others, we usually allocate tasks based on people’s skills. Tactics like this can be used by robots as well. And the nice thing about humans is that you can ask them, “How did you determine that you should do this and the other person should do that?” And they’ll actually answer. That’s how I got to one of the problems that I’m working on now: How do you learn from human coordination in order to get robots to coordinate better? For example, we want to learn how diversity helps humans solve complicated problems, so that we can allow robots to take advantage of diversity in teams as well. We are also interested in how humans solve problems that robots encounter, with only the sensor information that robots would have. By working on these projects, we hope to get teams of robots out of the lab and into the real world, doing things like environmental monitoring and precision agriculture.

 

Are there practical applications of multi-robot coordination that are mainstream today?

Absolutely. It’s been used in manufacturing for a long time and in warehousing. That’s how you get your packages in two days or less. They’ve basically turned the warehousing paradigm upside down. Instead of people going and picking the packages off of the shelves, the shelves actually come to the people—each of the shelves is carried over by a robot. Multi-robot systems are also working in mining, so there’s lots of these systems at work today; it’s just that not everybody knows about all of them.

Where is this going? What does a future of abundant computing power mean for your work?

It means a robot can operate without external influence. It can make decisions, take large stacks of data and figure out what they mean, and decide what to do. There is certainly a limitation now—anything that requires a large amount of data analysis is impossible to do on board for some of the robots, especially the flying ones. The ability to do more on board means that they can be more self-sufficient and rely on the cloud less or use the cloud in a more interesting way—for example, instead of using it to crunch data a robot needs on its own, the cloud can be used to share information across robots, to get information that’s available online, such as maps and geophysical data, or to exchange information with other connected devices, like mobile phones.

What else do you see as changing in meaningful ways?

I think that having a unified way of interacting with these devices might be the most interesting change. Right now, everything has its own specific mode of interaction, communication, and operation. Over time those will all, potentially, merge into some standard, where we all interact with these devices in the same ways. And that will lead to a lot more innovation in what these devices can do. We really only touch the tip of the iceberg in what multi-robot systems can do today.

What would this world look like? What could you imagine in the next decade?

I imagine a world that’s like The Jetsons, where we have robots all around us and they are not necessarily part of one multi-robot system, but instead, there are multiple different systems of robots. There are big problems that we can make a huge impact on—like saving lives by containing or putting out forest fires or search and rescue. But they will also just make our lives a lot easier. One day having Rosie the Robot will be kind of normal.