‘Smart’ Walking Stick Helps Visually Impaired Grocery Shop

A team of engineers at the University of Colorado Boulder are using artificial intelligence (AI) to develop a ‘smart’ walking stick for the blind or visually impaired.

The research was published in IEEE.

According to the team, the smart walking stick could eventually help blind people navigate various tasks, such as shopping at the grocery store or finding a place to sit.

Shivendra Agrawal is a doctoral student in the Department of Computer Science.

“I really enjoy grocery shopping and spend a significant amount of time in the store,” Agrawal said. “A lot of people can’t do that, however, and it can be really restrictive. We think this is a solvable problem.”

Designing the Smart Walking Stick

The walking stick resembles a cane, but it is a bit different. For one, it has a camera and uses computer vision technology, helping it map and catalog its environment. It can then guide users by using vibrations in the handle and with spoken directions.

The team says the device isn’t a substitute for making places more accessible, but the prototype can help millions of individuals become more independent.

“AI and computer vision are improving, and people are using them to build self-driving cars and similar inventions,” Agrawal said. “But these technologies also have the potential to improve quality of life for many people.”

“Imagine you’re in a cafe,” Agrawal continued. “You don’t want to sit just anywhere. You usually take a seat close to the walls to preserve your privacy, and you usually don’t like to sit face-to-face with a stranger.”

Experimental Cafe and Grocery Store

To better understand the decision-making needs of people with blindness or impaired vision, the researchers created an experimental cafe in their lab. Examining how a smart walking stick could be used to facilitate navigation amid obstacles and furniture, patrons were asked to traverse the maze-like environment and update on their experience.

Subjects equipped with a laptop-laden backpack and state-of-the art walking stick scanned the room using an attached camera. Advanced algorithms inside their laptops assessed features within the environment, then mapped out the optimal route to select seating– much like automated vehicle navigation systems.

The study demonstrated successful and encouraging outcomes, revealing that experimental subjects were able to locate a chair with accurate precision on 10 occasions out of 12 trials. The test participants had their eyesight obscured for the duration of the experiment. The team is now aiming to build upon their findings by engaging individuals who have diminished vision or blindness.

Agrawal and his team then developed the tool to tackle the challenge of finding an item amongst seemingly identical options in store shelves. The device can help people easily identify their desired product from the vast array available.

In their lab, the team engineered a lifelike grocery shelf – stocked with different brands of cereal. They produced an expansive database of product photos to feed into dedicated software, allowing study subjects to scan and select products using only a walking stick.

“It assigns a score to the objects present, selecting what is the most likely product,” Agrawal said. “Then the system issues commands like ‘move a little bit to your left.’”

Agrawal says that it will still be a while before the walking stick is used in the real world. The team wants to first make it more compact, designing it in a way that works with a standard smartphone attached to the cane.

“Our aim is to make this technology mature but also attract other researchers into this field of assistive robotics,” Agrawal said. “We think assistive robotics has the potential to change the world.”