tinyML’s Role in Enabling Computer Vision at the Edge – Thought Leaders

By: Davis Sawyer, Co-Founder & Chief Product Officer, Deeplite

Computer vision has great potential to improve our everyday lives – and there are many applications and uses for it. A few examples include:

• Smart doorbells for home security help prevent “porch pirates” and break-ins. According to research by I.H.S. Markit (published in SecurityInfoWatch) the number of global surveillance cameras worldwide was expected to reach one billion in 2021. In the U.S. alone, the number of cameras were expected to reach 85 million;
• In parking lots, AI-enabled cameras automate the tracking of available and occupied parking spots to let consumers know where open spaces are;
• Dashboard cameras on trucks are now reading speed limit signs and dynamically reducing the truck’s speed to improve safety;
• And drones with connected cameras are monitoring remote and hard-to-reach areas, and they can process images and make decisions in real-time.

All of these applications use intelligent video analytics, driven by AI and Machine Learning (ML), to watch video, use intelligence to make decisions, and then take action.

Computer Vision Needs More Resources at the Edge

However, like many AI-driven applications, computer vision needs bursts of computing power, memory, and energy to do its complex analysis and make decisions. While this is fine in a data center with a lot of computer power, it can prevent the move of AI to the edge. Specifically, small devices that are located far from corporate data centers and operate on small batteries need a new breed of AI that is smaller, faster and “lighter” than traditional approaches. And existing devices will need to be upgraded with new AI + ML (computer vision) functionality to remain viable and competitive.

New Advancements Boost Deep Neural Networks

Today, new advancements in AI are making Deep Neural Networks (DNNs) faster, smaller and more energy efficient – and helping move AI from the cloud and data centers to edge devices and battery-powered sensors. When it comes to AI model training, the staggering carbon footprint has been documented and discussed (i.e. training one AI language model emits as much CO2 as 5 cars over their lifetimes). However, we need to understand what the environmental impact of AI model Inference is and how to reduce this footprint. This is where model optimization can have tremendous benefits through reducing the economic and environmental cost of DNNs.

TinyML Enables AI on Small Devices

One such advancement is tinyML, a powerful new trend to enable smaller, battery-powered devices to use advanced ML to deliver computer vision and other perception tasks. It facilitates ML inference on small, resource-constrained devices typically on the edge of the cloud, and helps enable edge applications closer to the user.

For example, a server GPU like an NVIDIA A100 has over 40GB of available memory, which is suitable to run complex AI like computer vision and natural language processing. However, when we talk about edge devices and tinyML, a common microcontroller (MCU) may have only 256KB of on-chip memory, which is over 100,000x less memory than the cloud! In addition, unlike data centers and the cloud, edge device hardware cannot easily be updated in the field. This means we must “fit” our AI into the available hardware, which can take months to years of trial and error for developers to achieve, if at all. This is where tinyML, in particular automated machine learning (also called AutoML) can play a major role in breaking barriers to adopting AI in the real world.

And tinyML’s influence is growing. With over 10,000 members, the tinyML Foundation is growing the ecosystem to support the development and deployment of ultra-low power machine learning solutions at the edge. The Foundation unites a global community of hardware, software, machine learning, data scientists, systems engineers, designers, product, and businesspeople.

A World of Opportunities

In all, there are billions of small, connected devices everywhere that can benefit from advanced intelligence. The challenge is that they have very limited resources, so how can we add intelligence to them? tinyML can play a key role in bringing AI and ML to more computer vision-based, real-world applications, at the edge on small devices. And this can unlock a world of benefits to people and companies across a range of products, services and industries, helping us push into new frontiers for AI.