


























Abstract:This paper introduces a unified, hardware-independent baremetal runtime architecture designed to enable high-performance machine learning (ML) inference on heterogeneous accelerators, such as AI Engine (AIE) arrays, without the overhead of an underlying real-time or general-purpose operating system. Existing edge-deployment frameworks, such as TinyML, often rely on real-time operating systems (RTOS), which introduce unnecessary complexity and performance bottlenecks. To address this, our solution fundamentally decouples the runtime from hardware specifics by flattening complex control logic into linear, executable Runtime Control Blocks (RCBs). This "Control as Data" paradigm allows high-level models, including Adaptive Data Flow (ADF) graphs, to be executed by a generic engine through a minimal Runtime Hardware Abstraction Layer (RHAL). We further integrate Runtime Platform Management (RTPM) to handle system-level orchestration (including a lightweight network stack) and a Runtime In-Memory File System (RIMFS) to manage data in OS-free environments. We demonstrate the framework's efficacy with a ResNet-18 image classification implementation. Experimental results show 9.2$\times$ higher compute efficiency (throughput per AIE tile) compared to Linux-based Vitis AI deployment, 3--7$\times$ reduction in data movement overhead, and near-zero latency variance (CV~$=0.03\%$). The system achieves 68.78\% Top-1 accuracy on ImageNet using only 28 AIE tiles compared to Vitis AI's 304 tiles, validating both the efficiency and correctness of this unified bare-metal architecture.
| Comments: | 9 Pages, 3 Figures, 3 Tables, target to Computer Frontiers 26 |
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI) |
| MSC classes: | 68Q10 |
| Cite as: | arXiv:2604.09565 [cs.DC] |
| (or arXiv:2604.09565v1 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2604.09565 arXiv-issued DOI via DataCite |
From: Hua Jiang [view email]
[v1]
Sun, 15 Feb 2026 22:12:45 UTC (1,167 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。