Low Resource Embedded and Edge AI

Summary

Low-resource (e.g. memory, computation, power) and edge connected sensor systems are becoming more and more prevalent in the digital world, providing the ability to sense and control our physical world. Whilst historically these systems have been "dumb" with the majority of the intelligence delegated to high-resource cloud provision, increasingly there is a need to run AI models directly on these low-resource devices for reasons of latency, bandwidth and privacy. This module investigates techniques such as pruning, compression, distillation and splitting that allow these low-resource devices to play a fully-fledged role in the AI world.

Objectives

Successful participants will:

• Introduce the properties and challenges of low-resource AI;
• Provide examples of how to engineer a low-resource machine learning model that satisfies the domain requirements;
• Reason about the relative merits of different approaches in terms of accuracy, latency, memory etc
• Discuss how to balance the tradeoffs of running models fully on the edge, on the cloud, or a combination of the two
• Suggest how models can be distributed across multiple low-resource devices

Contents

Introduction to Low-Resource AI

• Techniques for low-resource machine learning models: Model choice; Compression; Quantization; Pruning; Distillation
• Early-exiting and Split cloud-edge approaches

Distributed Machine Learning

• Low-resource agentic models

Real-world case-studies

Requirements

A core AI/ML module to provide context on models.