Tangram Vision packs AI skills onto HiFi 3D sensor

Listen to this article

If you’ve ever talked to the folks at Tangram Vision, chances are you came away more knowledgeable about the state of perception for robotics. The team at the Boulder, Colo.-based startup has 60-plus years of combined experience with perception and sensors.

Founded in 2020, it has developed a toolset to help roboticists solve perception problems such as multimodal sensor calibration, sensor fusion and sensor uptime. And now it’s making its first foray into robotic perception hardware with a 3D depth sensor called HiFi that packs a powerful AI punch as well.

HiFi launched on Kickstarter with early backers able to get a HiFi for $349, which is 36% off the $549 retail price. Tangram surpassed its $25,000 goal four hours after launching it; at press time, the campaign had raised more than $40,000. Tangram expects to ship the first HiFi units in April 2024.

The Tangram Vision team came out of Occipital, the spatial computing company that launched the Structure Sensor 3D sensor with a $1.3 million campaign on Kickstarter in 2013.

“Tools for robotics and AI have come a long way in the past few years, but sensors haven’t kept pace,” said Brandon Minor, CEO, Tangram Vision. “After speaking with hundreds of robotics and AI engineers, we knew they wanted to spend less time struggling with sensors and more time building cool products. HiFi makes it dead simple to add AI-enhanced 3D data to robots and more.”

A HiFi 3D sensor mounted onto a cobot arm from Universal Robots. Source: Tangram Vision

HiFi’s capabilities

HiFi uses ams Mira220 cameras. It combines a 136° field of view 2-megapixel global shutter IR stereo pair with a 136° field of view 2-megapixel RGB camera and dual active projectors. A 65mm baseline delivers depth from 0.3m to 5m with 30-60 frames per second. Because of its field of view, Tangram said roboticists can replace two depth sensors with a single HiFi in many applications.

HiFi includes a host of software, drivers and onboard tools for roboticists. For example, it is designed to natively work with ROS 2. It comes with 8 TOPS of machine learning processing power that enables on-sensor fusion of AI and 3D data. It also has 8GB of onboard memory that Tangram said enables neural networks to be run onboard HiFi.

It features a dedicated depth processor on the Texas Instruments (TI) Jacinto chip. Tangram said the Jacinto processor lets HiFi offer compatibility with the TI’s Deep Learning Library that comes with ready-to-deploy models for scene segmentation, pose estimation, object identification, object tracking and more. HiFi also comes equipped with Tangram’s sensor software tools like self-calibration, time sync, and uptime management.

Submit a session abstract now to be an event speaker. Submission Deadline: December 15, 2023

Robotics applications

Tangram told The Robot Report it never had any intention of developing a 3D sensor. But it said it was compelled to do so after so many of its robotics customers expressed disappointment with the state of 3D sensing.

“Right now, robotics engineers must choose between 3D data quality, AI capabilities, or reliability. But they really want all three,” said Minor.

In a testimonial for the HiFi campaign, Nathan Brooks, CTO of PickNik, seemingly agreed. He said his company has “been talking with Tangram for years and sharing some of our pains in deploying robotic applications in the real world and unstructured environments.”

Rodnitzky said mobile robotics has historically been Tangram’s bread and butter. But he said HiFi will enable the company to work with different types of robots and applications.

“Bin-picking applications didn’t make sense for us before, but now they do,” he said. “This could allow us to also dip our toes into industrial automation where Zivid, IDS and Cognex have dominated with more expensive sensors.”

Market opportunity
To understand the opportunity Tangram envisions for HiFi, just look at the volatility of the depth sensor market in recent years. The Kinect was once a solid option, but that died when Apple acquired PrimeSense for $360 million in 2013. Google shut down Project Tango in late 2017 after a promising three years. And Intel nearly shuttered its RealSense division before deciding to continue introducing new depth sensors.
Tangram doesn’t have the scale of those aforementioned companies. But Minor doesn’t think that’s an issue.
“I don’t know if folks trust those larger players. There’s risk there as well,” he said. “We don’t have as many mouths to feed, and this could be game-changing for the industry and our company.”
Minor also said selling HiFi is an opportunity for Tangram to introduce its calibration services, and its recurring revenue model, to companies that have never heard of it before.
“We launched Tangram to lower the barrier to entry into computer vision,” said Minor. “That’s why we started with calibration – it’s an impossible task that’s very hard to do. HiFi is one more piece of that puzzle. Let us build a sensor that meets depth data quality standards and comes with amazing AI processing capabilities to do sophisticated processes. And roboticists can focus on applications and what they do well.”