The Future of AR Glasses Is AI-Enabled

Advancements in artificial intelligence are dominating the mainstream conversation. Applications of AI seem limitless – and it’s no surprise that it will transform our world and how we interact with it. Along with many other industries, that especially applies to augmented reality.

AI models are already being used to build immersive AR experiences – especially on mobile device apps. The fields of augmented reality and artificial intelligence are working together to create intuitive, unique experiences that more thoroughly blend the real and digital worlds. The next frontier in AI and AR development is implementing advanced AI capabilities into consumer AR glasses to expand the possibilities of immersive engagement.

Generative AI will accelerate the creation of 3D worlds, and create limitless possibilities for wearers to create their own realities

Generative AI features advances in algorithms, language models and the increased processing power available to run the necessary calculations to map and interact with the physical world.

To date, there’s limitations on 3D models in AR glasses because they’re essentially manual. However, Generative AI would create these very quickly and autonomously.  Generative AI in AR glasses will help build 3D models more quickly, unlocking the full potential of AR. This creation of the digital world, to overlay the physical world, will be faster, more complete and immersive without intensive manual labor needed.

Generative AI will also transform the user-experience and how we interact with physical space. With Generative AI-enabled AR glasses, wearers can literally transpose their imaginations into the real world. Without needing to code, wearers can use voice recognition to ‘speak’ their images and 3D objects through their AR glasses exactly how they want it. They could say: “imagine there is a dolphin swimming through the room” – and it would appear in front of them. The opportunities for this immersion are virtually limitless for entertainment, work, and beyond.

AR glasses that employ Generative AI will also change the gaming world to create a far more immersive and personalized gaming experience. For example, with ChatGPT, it will be easier to create more realistic characters and add new quests or gaming worlds. It could also be used to improve the gaming experience by analyzing player behavior and making the game easier or more challenging for the player in real time, customizing the experience autonomously.

AI-enabled translation of speech + written text will reduce language barriers

Automatic Speech Recognition (ASR) uses neural network audiovisual speech recognition (an algorithm relying on image processing to extract text). This can translate written text – like that on a menu in a foreign country –  into your native language in real time.

When applied in AR glasses, it can provide real-time subtitles of your native language while someone is speaking in another language – all within the frames of your AR glasses. This eliminates the frustration of tourists and business people trying to communicate in the local language and thus foster a more communicative and collaborative world.

Text recognition and translation combine AI Optical Character Recognition (OCR) techniques with text-to-text translation engines such as DeepL. AI engines like Stable Diffusion can also augment one’s communication with animations or other visual aids that can help convey complex or detailed concepts. This deepens the user’s engagement: a pair of AR glasses employing this AI can showcase a corresponding image, or video in real-time that is relevant to what the user is saying in front of them at a panel or presentation. Google recently teased developing AR glasses with this functionality.

It also enables the deaf community to engage in everyday conversations without the need to lipread or make eye contact by instantly turning audio into captions that are displayed in front of the wearer’s eyes.

AI is making its presence felt in healthcare, education and numerous other fields. Soon we will have smart AR glass that, like popular science fiction, will transport people into augmented or virtual reality environments where AI quickly maps the room and position of the speaker to make virtual communication seamless and less cumbersome no matter where either party is located.

AI helps AR glasses easily detect and label objects in the real world – deepening engagement.

Convolutional neural network (CNN) algorithms in object detection are currently used in mobile devices to estimate the position and extent of objects within a scene. Once it detects an object, AR software can overlay text onto it or generate another object into the physical world, and create an interaction between the two. Objects that are transposed into the real world have many applications including instruction, navigation, diet and nutrition, and many others.

When wearing a pair of AR glasses with these AI capabilities, for example, a user can walk the streets of any city and learn about any landmark in real-time upon viewing it. The AR glasses can identify, label, and provide information about the city and its landmarks – all through the wearer’s frames. As object recognition technology improves, nutritional data such as calories, protein, fat, and cholesterol of any food and serving size will be available. In the meantime, simple QR codes on products will conjure up the nutritional details for users.

Outside of purely object detection, facial recognition software is also becoming commonplace for people detection. Already, facial recognition is taking off in the airline industry as more flights use the technology to confirm a passenger’s identity – adding an additional security layer and speeding up the boarding process. Facial recognition, when employed in AR glasses, could give the power of recognition to wearers everywhere. For example, in the near future with AR glasses, you may be able to meet with others on social media and receive their background information instantly before deciding if you want to ‘friend’ or connect with them.

AI-enabled AR glasses are changing our lives and their visuals and capabilities will continue to improve. In our increasingly connected world, they are simplifying tasks and breaking down barriers that only a few years ago were thought to be impenetrable. Artificial intelligence advancements are occurring so rapidly that over the next 10 years, AI will make more progress than in the fifty before it. Whether it’s government, business, or a personal environment, artificial intelligence will soon merge with AR glasses to blend our physical and digital environments.