Sereact’s PickGPT lets users instruct robots with natural language

Listen to this article

Sereact, a German-based AI and robotics developer, has announced PickGPT, a robotics transformer that combines large language models (LLMs) with the company’s computer vision technology. 

PickGPT allows robot operators to instruct robots using natural language by enabling robots to process natural language and visual information and correlate multimode data. PickGPT allows robots to perceive their environment with unprecedented intelligence and accuracy and understand and execute instructions in natural language. 

“PickGPT enables straightforward communication between humans and robots via voice commands and intuitive interfaces. This simplifies the link with other warehouse systems and enables employees to instruct the robot in a simple way – for example, to define pick points or to blacklist certain items,” Ralf Gulde, Co-Founder and CEO of Sereact, said.

The robotics developer is a software-based robotics system that doesn’t require programming or training. Because of this, PickGPT reduces the time needed to set up and customize robots and allows employees to spend that time focusing on other essential tasks. It also expands the range of people who can work with robots as they don’t need a coding background to work with the system.

Along with faster setup, PickGPT offers more flexibility and quality control. With the robotics transformer, robots can respond quickly and easily to new tasks or changing environmental conditions, which are common in warehouse settings. 

PickGPT can also recognize and identify objects and products in real time based on their visual characteristics. The transformer automatically analyzes product images and matches them with items in any real-world scenario. 

It can also understand and interpret product titles, descriptions, and other textual information to automatically identify and assign related products to regions in an image. This means that PickGPT has a high potential for quality control processes. 

The robotics transformer can also simplify the return process for e-commerce companies. PickGPT has the ability to distinguish between packaging material and products in the first step of the return process, and it can then sort the items automatically based on a description or product image, meaning there is no need for barcode scanning. 

Sereact was founded in 2021 by Marc Tuscher and Ralf Gulde, who both researched the combination of robotics and AI at the University of Stuttgart. The company is based in Stuttgart-Vaihingen, Germany. 

If you’re interested in learning more about LLMs and robotics, there will be a keynote session titled “Generative AI’s Impact on Robotics” at RoboBusiness in Santa Clara, CA on October 18-19. This keynote panel of robotics industry leaders will discuss the applications of LLMs and text generation applications to robotics. It will also explore fundamental ways generative AI can be applied to robotics design, model training, simulation, control algorithms and product commercialization. Speakers at the talk include Amit Goel, the Director of Product Management at NVIDIA, Ken Goldberg, the William S. Floyd Jr Distinguished Chair in Engineering at UC Berkeley, Ted Larson, the CEO of OLogic, Jeff Linnell, founder and CEO of Formant, and Pras Velegapudi, the VP of innovation at Agility Robotics.