Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

7-2006

Abstract

We present a novel autonomic control system for high performance stream processing systems. The system uses bandwidth controls on incoming or outgoing streams to achieve a desired resource utilization balance among a set of concurrently executing stream processing tasks. We show that CPU prioritization and allocation mechanisms in schedulers and virtual machine managers are not sufficient to control such I/O-centric applications, and present an autonomic bandwidth control system that adaptively adjusts incoming and outgoing traffic rates to achieve system management goals. The system dynamically learns the bandwidth rate necessary to meet the system management goals using stochastic nonlinear optimization, and detects changes in the stream processing applications that require bandwidth adjustment. Our prototype Linux implementation is lightweight, has low overhead, and is capable of effectively managing stream processing applications.

Discipline

Numerical Analysis and Scientific Computing

Research Areas

Intelligent Systems and Optimization

Areas of Excellence

Digital transformation

Publication

Proceedings of the 26th IEEE International Conference on Distributed Computing Systems (ICDCS'06), Lisboa, Portugal, July 4-7

Volume

2006

First Page

1

Last Page

7

ISBN

9780769525402

Identifier

10.1109/ICDCS.2006.20

Publisher

IEEE Computer Society

City or Country

Los Alamitos, CA

Additional URL

https://doi.org/10.1109/ICDCS.2006.20

Download

Share

COinS