Meta shares AI model that can detect objects it hasn’t seen before

AI normally needs to be trained on existing material to detect objects, but Meta has a way for the technology to spot items without help. The social media giant has published a “Segment Anything” AI model that can detect objects in pictures and videos even if they weren’t part of the training set. You can select items by clicking them or using free-form text prompts. As Reuters explains, you can type the word “cat” and watch the AI highlight all the felines in a given photo.

The model can also work in tandem with other models. It can help reconstruct an object in 3D using a single image, or draw from views from a mixed reality headset. Effectively, Segment Anything can limit the need for additional AI training.

Both the AI model and a dataset will be downloadable with a non-commercial license. That is, creators can’t use it for products. This is primarily for research and expanding access to the technology. Right now, Meta uses somewhat similar tech to moderate banned content, recommend posts and tag photos.

The developers acknowledge that the existing model is flawed. It might miss finer details, and isn’t as accurate at detecting the boundaries as some models. And while Segment Anything can handle prompts in real-time, it bogs down when demanding image processing is involved. Some more specialized AI tools are likely to outperform this model in their respective fields, Meta says.

You aren’t about to see this AI in robots or other devices where fast, accurate object detection is (usually) vital. However, models like this may still help in situations where it’s impractical to rely exclusively on training data. A social network could use the tech to keep up with a rapidly growing volume of content. If nothing else, this shows that Meta wants to generalize computer vision.

Meta is no stranger to sharing AI breakthroughs, such a translator for unwritten languages. With that said, there’s pressure on the company to show that it’s as much of a powerhouse in the category as tech heavyweights like Google and Microsoft. It’s already planning generative AI “personas” for its social apps, and inventions like Segment Anything show that it has a few advantages of its own.