Meet the artist training Spot robots to make their own art

Listen to this article

Agnieszka Pilat, an artist finding innovative ways to fuse art and technology, plans to spend the next four months training three Boston Dynamics Spot quadrupeds to create art autonomously. 

Starting Dec. 3, Pilat’s show, called Heterobata, is on display at the National Gallery of Victoria’s Triennial Show in Melbourne, Australia. The robots are programmed to understand a range of commands, and they will act autonomously to execute them in whatever order desired. 

“When people come in, they will see three Boston Dynamics quadruped robots in a space that’s very white and open. We call this space ‘Heterobata,’ which is a place that does not exist in the real world,” Pilat told The Robot Report. “And they will see them making art and engaging with the audience a little bit, and just living as kind of a sample of the new species or rehearsing the future.” 

The term “Heterobata” is a hybrid, according to Pilat. To come up with it, she drew from the word utopia, which is also an ideal place that doesn’t exist in the real world. She also took inspiration from Jean Baudrillard’s Simulacrum, which the French philosopher defined as something that replaces reality with representation. 

---

Submit your nominations for innovation awards in the 2024 RBR50 awards.

---

Basia, Omuzana, and Bunny have distinct ‘personalities’

Each of Pilat’s robots is programmed a different personality and a different role to play in the exhibit. She described them as a “nascent moment in technology,” with emerging personalities mimic the way in which organisms become specialized over time. 

The first robot is called Basia, which lives with Pilat in her New York City home. Basia is the painter of the group and will autonomously create works of art using strokes Pilat taught it.

“She’s a painter, she’s an artist,” Pilat said. “And that’s what she’ll be doing, but the other ones are not really doing that at all. They’re not trying to paint.” 

The next robot, Omuzana, will mostly be strutting around the exhibit, according to Pilat. Omuzana observes the other robots and keeps watch over them. 

“She has the most dignified personality, and she kind of walks between the rooms that we have built her,” Pilat said. “I think of her [as] a protector of robots, so that’s her personality.” 

The last, and silliest, robot is Bunny. The only quadruped in the room without an arm, Bunny is physically lighter and faster than the other two robots.

Pilat said she thinks of Bunny as the youngest of the group who loves to have its picture taken. This robot has a special selfie area that Pilat created so Bunny can pose with attendees. 

“The main difference between Bunny and the other robots is that I think of her as observing the moment, or mimicking maybe more than even observing moments,” Pilat said. “And mimicking behavior of what’s happening in humanity right now, and in human culture.”

Those who come to the exhibit have an opportunity to observe the three robots interacting, creating art, and resting throughout the day. Pilat predicted that Bunny will quickly become the most photographed of the three, because of the robot’s love of posing for selfies. 

Artist robots evolve from an extension of the arm to creating on their own

While this is one of the first times the public can watch Pilat’s quadrupeds at work in person, the artist has been working, and painting, with them for years. And her process has changed drastically with the rise of generative AI. 

In her residencies at Boston Dynamics and Agility Robotics, Pilat composed the paintings herself and instructed the robots in carrying out her vision. 

“So, when I first was working with these robots with Boston Dynamics, I always operated the robot manually. So I was always in the room when the work was happening,” Pilat said. “And throughout the process, I developed kind of the aesthetics of how the robots paint, and it truly comes from my ability to use the robot a certain way.”

When Pilat was working with robots in this way, she said they worked as an extension of her arm. During this time, Pilat learned how to work with the robots’ strengths and weaknesses when it comes to the challenging coordination needed to paint. For example, Pilat struggled to draw curves when painting with the robot, so many of her paintings only feature straight lines. 

The robots also struggled working with artistic mediums. Pilat worked with very soft oil sticks, and she said the robots would often smash or push too much with the sticks because they struggled to understand how much pressure to apply. 

“Over time, I decided I started loving these mistakes and problems more and more,” Pilat said. “Materials create lots of mistakes in art, and I realized over time that this is what the arts have been. Every challenge creates beauty and uniqueness for the machine paintings.”  

In this latest exhibit, the robots will be composing paintings autonomously. They do this using 16 unique characters designed by Pilat. The robots can use these different characters in any way they want and to create anything they want. 

“This is really a hybrid of software, robotics, machine learning, and generative AI,” Pilat said. “I’ve been mindful of not using pure generative AI with my robots, because they’re already controversial in the art world.” 

To Pilat, this show is a kindergarten for the robots and the AI systems that run them. In the future, the artist said she wants to put together a purely automated exhibit. In such a show, the robots would go directly online, use an generative AI like DALL·E, to create an image, and then recreate it in the physical world with paint. 

“The next show might be more pictorial in the sense that I’m totally automated out of the system,” Pilat said. “I’m not giving them a language that I designed.”

Pilat predicts the future of art and technology

Since working more and more with robots, Pilat said she has found that she’s automated herself out of being an artist. 

“So this whole year, I painted very little. My work really revolved around working with the robots, building the exhibition here at the National Gallery,” Pilat said. “So [art] became very conceptual for me, in the sense that I didn’t use my hands anymore.” 

“If we think of technology as the legacy of humanity, to see that technology follows what we cherish the most as human beings, which is creativity, and that we’re able to infuse it into the machine, that is extremely exciting,” Pilat said. 

Pilat considers herself very lucky to be able to work on the cutting edge of art and technology, and she understands that many artists don’t have the same opportunities. But this doesn’t mean they don’t have a stake in the future of technology and art. 

“We all have agency; we just need to realize it,” the artist said. “AI is large language models that are being trained around our behavior every time we go online. We are training AI, so I think it’s both a great responsibility and a great privilege. So being aware of it is so important.”

Pilat asserted that artists, and anyone else who spends time online should be aware that AI is studying our behavior. She called it a “collective child of humanity,” and says that we should all be more aware of how we behave online. 

“I think that technology always augments our abilities as humans, so I’m excited to see what that technology will augment,” Pilat said. “It’s hard to predict what kind of creativity AI will augment, but I’m sure it’s going to augment our current abilities, and when we look back it’s going to seem so natural.”