Introduction to Edge AI and Embedded Inference. Overview of Edge AI – definitions, applications, and market trends. Comparison with cloud-based inference. Characteristics of embedded platforms for AI: constraints and design trade-offs. Overview of typical edge AI workflows – data collection, model training, deployment. Introduction to neural network models: CNN, DNN, quantization, pruning, and compression for edge inference. Overview of popular model formats – ONNX, TFLite, and CoreML. Embedded AI Toolchains and Deployment Pipelines. Introduction to hardware accelerators: NPU, DSP, GPU in embedded SoCs. Overview of AI toolchains: NXP eIQ (for i.MX), TI TIDL (for Sitara, Jacinto SoCs), Coral Edge TPU, and NVIDIA Jetson. Model optimization: post-training quantization, compilation, and deployment. Use of ONNX Runtime, TensorFlow Lite, and vendor SDKs. Layer mapping and performance profiling. Deployment flow: preprocessing, inference, and post-processing on the device. Applications, Security, and Performance Evaluation. Application case studies: real-time object detection, smart camera, human presence detection, gesture control, predictive maintenance. Integrating edge AI with peripherals (camera, microphone, sensors). Performance metrics: latency, throughput, power, and memory. Challenges in model accuracy vs efficiency. Introduction to secure model deployment – model integrity, secure boot, and encrypted weights. Future directions: federated learning, Edge-to-Cloud AI, TinyML.
Suggested Lab Sessions:
· Overview of microcontrollers and embedded systems
· Implementation of neural networks using edge computing
· Implement AI toolchains for model conversion, optimization and deployment
· Implement security and integration aspects for AI-enabled embedded systems.