Abstract:Deploying large deep neural networks on memory-constrained mobile devices is a central challenge in edge ML. While compression, pruning, and quantization reduce per-parameter cost, transformer-based models remain too large for the 3.3-7.4 GB RAM envelope of commodity Android handsets. We present the DNN pipeline scheduling subsystem of CROWDio, which achieves practical ONNX inference across resource-constrained Android workers without model modification, by distributing memory pressure across devices via five mechanisms: JIT deferred partition loading, a single-partition-resident constraint, a 4-tier affinity scheduler, a zlib-compressed tensor transport, and a streaming 1:1 dependency model. Evaluated on DistilBERT (Sanh et al., 2019) (approximately 67 M parameters, SST-2) across five Android handsets over ten runs, our system holds peak per-device RSS to 43+-2 MB and limits battery draw to 50+-3 mAh per run, while streaming concurrency cuts batch latency 34% below barrier synchronisation.
| Comments: | 6 pages, 3 figures, 4 tables. Accepted at the ICML 2026 Workshop on Machine Learning for the Global South |
| Subjects: | Machine Learning (cs.LG) |
| ACM classes: | I.2.6; C.2.4 |
| Cite as: | arXiv:2605.20723 [cs.LG] |
| (or arXiv:2605.20723v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.20723 arXiv-issued DOI via DataCite (pending registration) |
From: Lakshani Manamperi [view email]
[v1]
Wed, 20 May 2026 05:21:54 UTC (415 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。