Publication

Abstract : Edge device workloads based upon Digital Signal Processing (DSP) and Artificial Intelligence (AI) are being implemented in growing numbers of edge computing environments where they must function in time-constrained and power-constrained conditions. However, a majority of microcontrollers use cache-based memory hierarchies to manage their memory and these introduce unpredictable amounts of latency because the cache behaves unpredictably and is generally very small. Therefore, we have developed SpongePad-AI, a simple and lightweight C/C++ runtime to provide predictable, low-variation execution for real-time edge workloads by replacing cache usage with programmer controlled accesses to programmable Scratchpad Memory (SPM), and a DMA-ping-pong buffering technique that will allow continuous overlap of computation and data transfer. SpongePad-AI includes two adaptive control techniques; a TinyML heuristic controller that utilizes online feedback to adjust tile size and DMA queue length, and a Tiny Neural Network (TinyNN) controller that uses historical latency information to predict optimal parameters. Both of the control mechanisms can be used as-is across a wide range of platforms, require minimal resources, and do not rely on hardware specific DMA capabilities. The performance of SpongePad-AI was evaluated by utilizing the standard DSP/AI operations: GEMM, Convolution, FIR, and FFT. The runtime achieved an average throughput increase of 8.6x over a cache-only baseline, reduced the variation in latency from 70-80%, increased the predictability of the 99th percentile of the system's behavior, increased the overall energy efficiency of the system, and provided consistent APIs regardless of platform. The results show that SpongePad-AI provides deterministic, realtime inference and DSP execution capabilities in modern edge computing systems.