Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
5-2024
Abstract
Canvas-based attention scheduling was recently pro-posed to improve the efficiency of real-time machine perception systems. This framework introduces a notion of focus locales, referring to those areas where the attention of the inference system should “allocate its attention”. Data from these locales (e.g., parts of the input video frames containing objects of interest) are packed together into a smaller canvas frame which is processed by the downstream machine learning algorithm. Compared with processing the entire input data frame, this practice saves resources while maintaining inference quality. Previous work was limited to a simplified solution where the focus locales are quantized to a small set of allowed sizes for the ease of packing into the canvas in a best-effort manner. In this paper, we remove this limiting constraint thus obviating quantization, and derive the first spatiotemporal schedulability bound for objects of arbitrary sizes in a canvas-based attention scheduling framework. We further allow object resizing and design a set of scheduling algorithms to adapt to varying workloads dynamically. Experiments on a representative AI-powered embedded platform with a real-world video dataset demonstrate the improvements in performance and inform the design and capacity planning of modern real-time machine perception pipelines.
Keywords
Quantization (signal), Limiting, Scheduling algorithms, Pipelines, Streaming media, Real-time systems, Scheduling
Discipline
Artificial Intelligence and Robotics | Theory and Algorithms
Research Areas
Data Science and Engineering
Areas of Excellence
Digital transformation
Publication
2024 IEEE 30th Real-Time and Embedded Technology and Applications Symposium (RTAS): Hong Kong, May 13-16: Proceedings
First Page
348
Last Page
359
ISBN
9798350358414
Identifier
10.1109/RTAS61025.2024.00035
Publisher
IEEE
City or Country
Piscataway, NJ
Embargo Period
8-26-2024
Citation
HU, Yigong; GOKARN, Ila; LIU, Shengzhong; MISRA, Archan; and ADBELZAHER, Tarek.
Algorithms for canvas-based attention scheduling with resizing. (2024). 2024 IEEE 30th Real-Time and Embedded Technology and Applications Symposium (RTAS): Hong Kong, May 13-16: Proceedings. 348-359.
Available at: https://ink.library.smu.edu.sg/sis_research/9226
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/RTAS61025.2024.00035